Subject: Re: Of HTTP/1.1 persistent connections and TCP Keepalive timers

From: Dave Kristol <[email-masked]>
Date: Thu, 2 Nov 2000 12:17:28 -0500 (EST)

[email-masked] wrote:
  > Thanks for all the info about how web servers are typically implemented. My 
  > apologies tho, I didn't really make my questions clear. What I'm really  
  > interested in is how *browsers* are typically implemented, because, say, I'm 
  > going to write my own HTTP/1.1-speaking gizmo for a very application-specific 
  > purpose, and I have reasons to want it (the server side) to treat 
  > client-initiated connections as persistent.

Actually, I think most responses recognized that you were interested in
browsers.  The problem is that the browser can only express its
preference.  But the server is in command, and it can close a
connection whenever it feels it must.  That's why there were all the
comments on server behavior.  (In truth, the browser may *also* close
the connection at will, but that's a less common case.)

  > 
  > So this causes me to be curious about how BROWSERS will typically behave in 
  > this context.
  > 
  > Assuming one writes one's own HTTP/1.1-speaking gizmo (according to RFC2616), 
  > then I have these questions...
  > 
  > 

I am not a browser implementer, so my answers are speculative, but....

  > Q1. Do the popular BROWSERS typically take the platform's OS's TCP defaults 
  > for the keepalive (IF such capability is provided by the TCP/IP stack, and IF 
  > it is actually used by the browser), or do they typically set this value to
  > something in particular?

They take the OS's defaults.

  > 
  > 
  > Q2. What typical assumptions are made on the BROWSERS' parts about an 
  > established connection to a HTTP/1.1-speaking server in the absence of user 
  > actions? If a browser opened a HTTP/1.1 connection and such a server is 
  > behaving as-specified by RFC2616, then it is up to the browser to close the 
  > connection. What do browsers typically do?

Browsers probably open connections with the hope of keeping them open as
long as possible.  But as I said earlier, that doesn't mean they will stay
open.  I believe the only way the client discovers the connection has been
closed is by getting an error response when it tries to write to a (now-closed)
connection.

[Rest deleted -- I believe the questions were answered earlier.]

Dave Kristol

---

From: [email-masked]
Date: Thu, 02 Nov 2000 09:45:39 -0800

> They [browsers] don't leave connections open and idle long enough for this
> to be relevant.  Browsers typically close idle connections in one or two
> minutes, even if the server does not.

Thanks, this is the sort of info I am interested in.

When I said "say I'm going to write my own HTTP/1.1-speaking gizmo", I should 
probably have made it clear that I am postulating a server-side gizmo, hence 
being interested in how the browsers out there in the wild will behave if this 
server-side gizmo doesn't behave just like typical existing web servers.

> Communicator doesn't do 1.1 yet.
>
> [...]
> 
> The default for direct connections in IE is 1.1; I believe that Opera
> also uses 1.1 by default now.

Innaresting. Does anyone have a pointer handy to any documentation for any of 
the popular browsers that indicates what all versions of what all protocols 
they do speak? An admittedly cursory search for such wrt Communicator didn't 
turn up anything.

thanks,

JeffH

---

From: "Fielding, Roy" <[email-masked]>
Date: Thu, 2 Nov 2000 10:08:27 -0800

The decision on when to close is left to either side.  A server will
close the connection based on its resource-consumption requirements
which may vary substantially based on the type of server and the
number of clients it is intended to serve.  A client will close the
connection if it is connection-limited and needs to open many other
connections, or if it just believes in being network friendly.

Unfortunately, none of the major browsers are network friendly,
so they typically ignore the connection (not even recognizing FIN
as an event) until they later attempt to use it again.  Most
general-purpose servers have a short activity time-out on
connections and will close the connection after that time-out
(typically under 10 seconds, though a high-activity server will
set this to one second or turn off persistent connections altogether).

Cheers,

Roy T. Fielding, Chief Scientist, eBuilt, Inc.            (www.ebuilt.com)
                 Chairman, The Apache Software Foundation (www.apache.org)

---

From: "Carl Kugler/Boulder/IBM" <[email-masked]>
Date: Thu, 2 Nov 2000 10:59:16 -0700

...
>
>For example, suppose the  server sends back an HTTP response to the client
>that does not have a Content-Length: header field and that it is not
>chunked.
>
>Then the only way the client knows that it has read the
>entire response off of the pipe is when the server closes the connection.
>When the server closes the connection the client will receive a
>zero-byte read which is socket layer's indication that the pipe
>is broken.
>
This is not good.  If this is the server's normal behavior, the client has
no way to distinguish a dropped connection from end of file.  So the client
can never be sure it received an entire message.

If this is the server's behavior for error conditions, this is still not
good unless the server either waits for the entire request (could be a
humongous POST and/or a very slow connection), or only closes one half of
the connection (which, BTW, is impossible in Java, except maybe in the
latest releases).  Otherwise, the client might get a RST while transmitting
the request, and will then never see the error response.

...
>
>I've done some snoops on browsers that are GETting HTML pages with
>lots of embedded links that are in the same realm as the original HTML
>page.
>
>Despite this obvious opportunity to take advantage of persistent connections
>the browser opens  a connection for each subsequent GET.
>
I was looking at IE's (version 5 somthing) traffic yesterday, and it seems to send two requests per connection.

     -Carl

---

From: "Wallace, William" <[email-masked]>
Date: Thu, 2 Nov 2000 13:19:46 -0500

What experimentation, if any, has been done with adaptive server timeouts? A
project I'm working on reduces timeouts based on current load. When a
certain point is reached connections are closed immediately until the load
drops down.

I haven't got as far as testing this on a live site so I'm curious whether
anyone has any real world experience.

> -----Original Message-----
> From: Fielding, Roy [mailto:[email-masked]]
> Sent: Thursday, November 02, 2000 1:08 PM
> To: [email-masked]'; [email-masked]
> Subject: RE: Of HTTP/1.1 persistent connections and TCP 
> Keepalive timers
> 
>

---

From: "Fred Bohle" <[email-masked]>
Date: Thu, 2 Nov 2000 12:27:24 -0600

Fred Bohle@NEON
11/02/2000 12:27 PM

     We seem to be diverging into TCP coding.  A read will return zero
length
when the other end has issued a normal close (and all the data has been
read).
A read will return -1 when the connection is ReSeT, or there is a
connection time-out
of any sort.  So the server can too tell the difference between end of data
and
a connection failure.

Fred

From: Carl Kugler/Boulder/IBM <[email-masked]> on 11/02/2000 11:59 AM

To:   James Lacey <[email-masked]>
cc:   http-wg <[email-masked]>

Subject:  Re: Of HTTP/1.1 persistent connections and TCP Keepalive timers

...

...
>the browser opens  a connection for each subsequent GET.
I was looking at IE's (version 5 somthing) traffic yesterday, and it seems
to send two requests per connection.

     -Carl

---

From: "Carl Kugler/Boulder/IBM" <[email-masked]>
Date: Thu, 2 Nov 2000 13:19:01 -0700

> From: Joris Dobbelsteen [mailto:joris.dobbelsteen@m...]
...
>
> The browser can leave the connection open for an infinite ammount of time
> (if not restricted by lower-level protocols, like TCP).
>
> However, idle time-outs when the connection is CONNECTED are, in the real
> world, handled by the server instead of the client. This is because the
> server is not interrested in HTTP connections that are idle for an
infinite
> ammount of time and just consuming bandwidth (the lower-level connection
> must be maintained)...
>
...

How does an idle connection consume bandwidth?

     -Carl