Subject: RE: HTTP and half close (WAS: HTTP client abort detection)

From: Miles Sabin <[email-masked]>
Date: Sat, 17 Mar 2001 09:37:30 -0000

Roy T. Fielding wrote,
> Using the typical socket API with TCP, an HTTP connection is 
> considered to be closed when it receives a fatal error or EOF 
> either:
>
>  1) when attempting to read an unfinished request and there 
>     are no pending responses being sent on the connection 
>     (this is easily determined because all HTTP requests are 
>     length-delimited), or
>
>  2) when writing a response.

So, in other words, the type 1 clients of my previous posting
(early half close) are correct, and the type B servers of my
previous posting (early EOF implies client abort) are incorrect.

That's all very well, but it's only one of the possible choices 
for the mapping of TCP/socket API behaviour on to HTTP behaviour.
It'd be nice to have some justification for why the choice should 
be made this way rather than some other, and some reassurance 
that this is the way the choice has been made in real 
implementations. 

It also leaves me puzzled about the question of delimiting
request entities. If a client TCP half close *isn't* considered 
as an HTTP connection close, then why can't we use that that as 
an alternative to Content-Length or chunking? Given what you've
said above, the parenthesis in,

   5.By the server closing the connection. (Closing the 
     connection cannot be used to indicate the end of a request 
     body, since that would leave no possibility for the server 
     to send back a response.)

simply doesn't apply. Why not allow EOF as a request entity
delimiter?

> Note that the above applies to the socket API.  There may be a 
> completely different algorithm for determining when the 
> connection is closed when using other API, particularly event-
> based ones.

Could you elaborate on this? It seems to imply that we might
have client and server HTTP over TCP implementations, one built 
on the socket API, one built on something else, each with
different mappings of TCP behaviour to HTTP behaviour, hence that 
the two might not be fully interoperable. I'm not sufficiently 
familiar with non-socket APIs to say whether this would be likely 
to be a problem in practice, but it certainly seems like a 
worrying prospect.

> The reason this isn't defined in the HTTP spec is because it is 
> not an interoperability issue for the application protocol -- 
> it is an implementation detail that is entirely dependent on 
> the nature of the lower-layer API used by the application.  It 
> is the stuff for a good book on network programming.

Perhaps it's not an issue for the abstract protocol, but it is
an issue for it's most common concrete realization. Isn't this
sort of interaction between the transport layer and the
application layer precisely the sort of thing which prompted
Jim Gettys and Alan Freiers "HTTP Connection Management" ID?

On the face of it, it'd be useful to have one or more transport
layer specific profiles as adjuncts to the abstract protocol
specification.

Cheers,

Miles

-- 
Miles Sabin                               InterX
Internet Systems Architect                5/6 Glenthorne Mews
+44 (0)20 8817 4030                       London, W6 0LJ, England
[email-masked]                         http://www.interx.com/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 17 Mar 2001 23:03:28 -0800

> that this is the way the choice has been made in real 

There are literally millions of theoretical possibilties that are
not mentioned in a protocol spec.  If they were all mentioned, then
nobody would have time to read the spec.

The reasoning in this case is very simple: be conservative in what you
send and liberal in what you accept.  If you follow that principle, there
is only one choice, and you will find it in every robust implementation of
HTTP over TCP.  Go ahead and check the source code at apache.org.

> simply doesn't apply. Why not allow EOF as a request entity

Because there is no value obtained by allowing it, and the cost would be
to add an HTTP requirement that is specific to those TCP API that provide
half-close as an option.  If there exists a platform for which the
application-layer cannot discern a half-close condition from an error
condition, then the server on that platform cannot know if it has
received the entire request or only a partial request, and since this
can impact the fidelity of carrying out that request, you have defined
a protocol which is less robust than what is defined in RFC 2616.
Therefore, it is not an option allowed by HTTP/1.1.

> to be a problem in practice, but it certainly seems like a 

They are interoperable right now precisely because HTTP does not
define things in terms of TCP.  There are many platforms that have
HTTP servers without any TCP -- the Web was created in a world where
TCP was not the universal standard for transport (not that it is today).
The intermediary boxes that translate from non-TCP server network to
TCP client network do not know anything about HTTP, and thus an HTTP
message can only be sent through that gateway if neither side makes
implementation assumptions about the other beyond those carried within
the HTTP message.

> layer specific profiles as adjuncts to the abstract protocol

There is absolutely no question that there is value in documenting
implementation profiles for protocols.  I am particularly fond of the
ones written by W. Richard Stevens.  But there is also a damn good
reason why those profiles are better written by individual authors
rather than within a standards organization, and that is because they
deal with how a standard should be implemented rather than what the
standard should be.

Cheers,

Roy T. Fielding, Chief Scientist, eBuilt, Inc.
                 2652 McGaw Avenue
                 Irvine, CA 92614-5840  fax:+1.949.609.0001
                 ([email-masked])  <http://www.eBuilt.com>