Subject: RE: Http overhead

From: "Joris Dobbelsteen" <[email-masked]>
Date: Fri, 1 Dec 2000 15:28:21 +0100

Many apache servers send data chunked. Takes a couple bytes (average of
4) for every block transfered. Maybe a total overhead of an additional
20-40 bytes per transfer (maybe less). This is just a guess...

- Joris

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of John Stracke
> Sent: Thursday 30 November, 2000 16:45
> To: [email-masked]
> Subject: Re: Http overhead
> 
> 
> [email-masked] wrote:
> 
> >      The latest standard (HTTP 1.1) has provisions for 
> compression and "chunked"
> > transfers which change this, but I haven't seen these used 
> in any real-world
> > situations
> > yet.
> 
> Apache will recognize a file with a ".gz" extension as 
> gzipped, and send the
> Content-Encoding: x-gzip line.  Netscape will recognize 
> Content-Encoding: x-gzip,
> and uncompress the file.  Unfortunately, at least in my 
> installation (Apache
> 1.3.14, Red Hat 7), Apache doesn't look at the extension 
> before the ".gz" to get
> the content-type; "foo.txt.gz" gets marked as Content-Type: 
> application/x-gzip.
> 
> --
> /=================================================================\
> |John Stracke    | http://www.ecal.com |My opinions are my own.   |
> |Chief Scientist |================================================|
> |eCal Corp.      |But how do we know destroying the Van Allen belt|
> [email-masked]|will kill all life on Earth if we don't try it? |
> \=================================================================/
> 
> 
> 
>

---

From: John Stracke <[email-masked]>
Date: Fri, 01 Dec 2000 10:04:16 -0500

> 20-40 bytes per transfer (maybe less). This is just a guess...

Yes, but it's actually better than that: AFAIK, Apache uses chunked
transfer-encoding only for dynamic resources, where it can't predict the
content-length.  The alternative would be (a) buffer the output before
sending it down, or (b) defeat persistent connections.  Either (a) or (b)
would increase; (b) would actually cost extra bandwidth, and (a) would
cause bandwidth consumption to come in spikes.  So, most likely, the cost
of chunking is lower than the cost of not chunking; it's certainly lower
than the nominal overhead of the encoding.

(Sorry to those to whom this is obvious--probably including Joris--but I
didn't want to leave anybody thinking they could save bandwidth by turning
off chunking.  :-)

--
/=================================================================\
|John Stracke    | http://www.ecal.com |My opinions are my own.   |
|Chief Scientist |================================================|
|eCal Corp.      |In the country of the blind, the one-eyed man is|
[email-masked]|in therapy.                                     |
\=================================================================/

---

From: "Joris Dobbelsteen" <[email-masked]>
Date: Fri, 1 Dec 2000 22:24:25 +0100

You are right, I did know this.

Indeed that's why I favor chunked encoding for dynamic resources. MSN's
(search) server (MS-IIS) transfers the data by just closing the
connection at the end (not sending any indication about the size of the
document) what also decreases the transfer 'reliability' - or better,
you don't know wether you have all or not... HTML or JPEG/GIF you can
guess it, but with many other sources you don't (like TXT)....

Also MSN's server has thus more overhead for the server and waisted time
between the requests for the client...

- Joris

> Sent: Friday 01 December, 2000 16:04
> To: WWW WG (E-mail)