Subject: Can the response entity be transmitted before all the request entity has been read?

From: Jamie Lokier <[email-masked]>
Date: Fri, 12 Mar 2004 11:41:53 +0000

Dear all,

I have been implementing an HTTP/1.1 server which consumes the request
entity lazily, as need by the application.

If the application decides that it can generate some or all of the
response before it finishes reading the whole request entity, is that
allowed?

Can the application generate all the response without reading any of
the request entity and without sending 100 (Continue) - is that
allowed?

I've been studying RFC 2616 and don't see an answer.

RFC 2616, 8.2.2:

  8.2.2 Monitoring Connections for Error Status Messages

     An HTTP/1.1 (or later) client sending a message-body SHOULD monitor
     the network connection for an error status while it is transmitting
     the request. If the client sees an error status, it SHOULD
     immediately cease transmitting the body. If the body is being sent
     using a "chunked" encoding (section 3.6), a zero length chunk and
     empty trailer MAY be used to prematurely mark the end of the message.
     If the body was preceded by a Content-Length header, the client MUST
     close the connection.

I'm not entirely sure if this means _network_ errors (such as TCP
shutdown or reset), or HTTP Error Status codes 4xx and 5xx.  I guess
it means HTTP Error Status codes, because of the implication that the
connection might continue.

1. Regarding my first question, whether the response can be partly
transmitted before all of the request entity has been read.

If the above section means HTTP Error Status codes, then the answer is
no if the status is an error code.  In other words, if the server
wanted to generate an error response, but use data from the request
entity to do it, incrementally, it would have to read all of the
request entity that it needs before transmitting any of the response.
Is this right?

What is the answer if the status is not an error code.  Can the server
begin transmitting the response entity before all of the request
entity has been read?  (Assume that the server is intelligent enough
to avoid deadlock by always reading request data when it arrives).

One simple example would be a server which serves the file it is being
transmitted, or a transformation of it.  It would be quite nice to
upload a file, and have the transformed version generated
incrementally and downloaded as the upload proceeds.  (This is
obviously possible using two separate connections, but my question is
is it supported by HTTP/1.1 over a single connection?)

2. Regarding my second question, whether a non-error response can be
safely transmitted without reading the request entity.  Is it allowed?
Does the connection have to be made non-persistent; i.e. closed afterwards?

If 100 (Continue) isn't transmitted, some clients may start
transmitting their request entities anyway.  Therefore there's no way
for the server to determine whether data received from the client is
the request entity or the next request, if it wants to try avoiding
100 (Continue) to produce a non-error response without using the
request entity.

This comes up because the server will transmit 100 (Continue) only
when the application begins to read the request entity via the server,
naturally.  If the application generates a 200 OK response complete
with entity, and has not read the request entity, what must the server
do?  What should it do?

Does it have to read the request entity anyway if it wants to keep the
connection persistent?  (If so, it might be worth doing that if the
request entity is known to be very small).

Or can it avoid reading the request entity completely, by not sending
100 (Continue) and somehow determining when the next request arrives
on that connection.  Is that possible?

Thanks for your answers :)
-- Jamie

---

From: Scott Lawrence <[email-masked]>
Date: Fri, 12 Mar 2004 08:29:26 -0500

> connection might continue.

It does mean HTTP Error Status responses (any status > 299).

> request entity that it needs before transmitting any of the response.

Yes.

> to avoid deadlock by always reading request data when it arrives).

Yes.

> is it supported by HTTP/1.1 over a single connection?)

Sure - it's TCP, so it's actually more efficient to use it in both
directions at the same time (piggybacked acks).

> Does the connection have to be made non-persistent; i.e. closed afterwards?

Not if the client says it's HTTP/1.1.  If the client is 1.0, then you
have to assume a non-persistent connection by default.

If you have enough information to begin the response just from reading
the request headers, then you are free to send the response headers
right away; even the 1xx response is optional - you can skip right to
sending the final response.  (Note: the rules of the protocol don't
forbid this - what any given client will do with this is a different
question that can be answered only with testing).

> 100 (Continue) to produce a non-error response without using the

No, you can tell from the headers whether or not there is a request
body.  If there is a non-zero Content-Length, or a 'Transfer-Encoding:
chunked', or a Content-Type specifying a multipart encoding, then that
indicates the presence of a body.  If none of the above are there, then
there is no body and the next byte starts the next request.

> request entity is known to be very small).

Yes, to keep the connection open it must consume the request entity.

> on that connection.  Is that possible?

No, but even if it were it wouldn't save you anything - TCP is a stream;
there's no way to discard the bytes other than at the receiver (without
closing the connection, that is).

--

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 12 Mar 2004 19:07:10 +0000

> It does mean HTTP Error Status responses (any status > 299).

Thanks.  Why > 299?  To my mind, error status is > 399 (i.e. 4xx =
Client Error, 5xx = Server Error).

If redirections are Error Status responses for the purpose of low
level client behaviour in section 8.2.2, that is poorly worded and
should be clarified in the Errata.

If I am confused, surely some client implementations will have it
wrong too.  That's critical: the server _has_ to know whether the
client will abort sending the request body or not, to decide how to
continue with the connection.

> have to assume a non-persistent connection by default.

Assume persistence has already been determined appropriately, by
looking for Connection: close and/or Connection: keep-alive in the
request.

After transmitting the response entity and discovering that it didn't
need the request entity after all, and it wasn't an error response,
it's too late for the server to change the response headers: they are
transmitted already.  The server will not have put Connection: close
in the response header, unless that was requested by the client, or it
was HTTP/1.0 without keep-alive, or some other conditions applied.  So
the server will transmit the response...

And then it has to shut down the connection, right?  (All the while
reading data from the client to avoid the TCP RST problem).

> > question that can be answered only with testing).

Thanks.

> indicates the presence of a body.  If none of the above are there, the n

That's not what I mean.  What I'm saying is: if I _don't_ send 100
(Continue), I don't see a way to determine whether the client _chose_
to not transmit the request entity _despite_ the Content-Length or
Transfer-Encoding headers.  Therefore, after doing that, there's no way
for the server to read another request, and it must shut down the
connection.

(Btw, any Transfer-Encoding indicates a request body, and "chunked" is
required to be the last in the sequence of transfer-codings.  If
Content-Type specifies "multipart/byteranges", that indicates a body,
but I see nothing in RFC 2616 which indicates a body for any other
multipart encoding.  Right?)

But now on yet more diligent reading of RFC 2616, I have a question.
This whole mail revolves around the question so I'll make it stand
out:

    If the client sends Expect: 100-continue, and receives an HTTP/1.1
    (or later) response which is not preceded by 100 (Continue), is
    the client allowed to send more requests on that connection
    _without_ transmitting the request entity?

> closing the connection, that is).

Eh?  If you don't send 100 (Continue) then it does save you quite a
lot: the client might not transmit the request entity at all!

Also, if you stop reading at the receive, eventually the TCP window
fills and the transmitter stops, which reduces data transmitted
overall if not all of the request entity is read.  That may or may not
cause deadlock, depending on the transmitter's logic.  We can avoid
deadlock while reading the minimum at the server, by reading only when
the data is needed, writing response as we generate it, and if writing
blocks, then reading until writing is unblocked, buffering the read data.

This is the logic I have now.  Assume in every case that the
connection is persistent: the request doesn't have Connection: close,
and it's HTTP/1.1.

  1. If the server sends a non-error response, without 100 (Continue)
     and without observing any request entity prior to sending the
     response, then it must not try to read another request because of
     the ambiguity as to whether the client actually sent the request
     entity or not.  Is this correct?

  2. If the server sends an error response, without 100 (Continue) and
     without observing any request entity prior to sending the
     response, then it must not try to read another request for the
     same reason.  Is this correct?

  3. If the server sends a *non-error* response, after 100 (Continue),
     and before reading all of the request entity, it can assume the
     client will send all of the request entity or abort the connection.

     In this case if the server doesn't want to read all of the
     request entity, because it knows the entity is large and there
     is no point transferring the remaining data, then after it has
     completely sent the response entity it should signal connection
     shutdown (i.e. TCP FIN, shutdown(SHUT_WR)), and continue reading
     for a while to prevent the TCP RST problem.

     If the server does want to read all the request entity after it
     has started sending the respones, it can just do that and it
     should work.

  4. If the server sends an *error* response, after 100 (Continue),
     and before reading all of the request entity, it must assume the
     client MAY prematurely abort transmitting the request entity
     either by closing the connection or transmitting a zero length
     chunk.

     In this case if the server doesn't want to read all of the
     request entity, because it knows the entity is large and there
     is no point transferring the remaining data, then it doesn't have
     to shutdown the connection.  The client may terminate the request
     entity with a zero length chunk allowing the connection to
     continue.  However, the client might not, so if there's a lot of
     data still coming in, the server may choose to shutdown the
     connection, in the same way as for 3. above, a short time after
     sending the error response and failing to see the client
     prematurely terminate the request entity.

     If the server does want to read all the request entity after it
     has started sending the response, it cannot.  It has to _delay_
     sending any of the response until all of the request entity has
     been read.

  5. Because of the different required behaviours in 3. and 4., due to
     RFC 2616 section 8.2.2, the server and client must agree on what
     consitutes an "Error Status".

     Clearly 4xx and 5xx codes are; 1xx and 2xx are not.
     It isn't clear whether 3xx codes are errors.

     If there is disagreement either way, there are problems.  If the
     client receives a 3xx and treats that like a non-error, it will
     continue transmitting the request entity.  For the server to
     terminate that transmission, to save bandwidth and inform the
     client that it doesn't need all the response, it needs to
     shutdown the connection after the response is transmitted, in
     this case.

     If the client receives a 2xx and reacts in the same way as for an
     error status, it may prematurely terminate the request entity by
     sending a zero length chunk, or it may close the connection
     which, although always allowed perhaps due to user abort, is not
     expected for non-aborted non-error requests.  Therefore for the
     server to not cause premature request entity termination, it must
     _delay_ sending the response until it has read the whole request,
     in this case.

     So if there is any disagreement among implementations over what
     is an Error Status for the purpose of section 8.2.2, this creates
     a third category of responses that the server must treat
     differently from 3. and 4.  I see this category containing 3xx.

     If one of these responses is generated within the server, and the
     server wants to read all of the request entity eventually, then
     it must _delay_ sending any of the response until all of the
     request has been read.  However if the server does not want to
     read all of the request entity, it cannot assume the client is
     likely to prematurely terminate the request entity by closing or
     sending a zero length chunk.

     So the server's behaviour with this category of responses is
     different from its behaviour for responses which are consistently
     treated by implementations as errors or non-errors.  It has to
     use this third, more conservative strategy.

  6. If the server transmits a response, and in so doing discovers
     that it doesn't need any of the request entity, then it can avoid
     sending 100 (Continue).

     However, if it does not send 100 (Continue) is it true that it
     mustn't try to read another request on that connection?  I'm not
     sure.

     If the server discovers it doesn't need any of the request
     entity, but it would like the keep the connection persistent, it
     needs to use a different strategy for non-error and error
     responses:

         a. For non-error responses, if the request entity is known
            to be small from the Content-Length header or otherwise,
            it may be worth the server sending 100 (Continue) prior
            to the response anyway, reading the whole request entity
            and discarding it, and continuing to read another request.

         b. For non-error responses, if the request entity is known
            to be large, or the server doesn't wish to encourage
            an unknown size transmission over the network from the client,
            the server will avoid sending 100 (Continue), and instead
            shutdown the connection (carefully, avoiding the TCP RST
            problem) after sending the response.

         c. For error responses, the size of the request entity is not
            important.  The client will abort sending the request entity,
            and make an appropriate decision on whether to continue using
            the connection.

            100 (Continue) must be still be sent by the server if it
            wants to use the connection persistently after the error
            response.  Is this correct?  (I'm not sure).

This translates to the following logic in server code.  I would be
grateful to anyone who can offer corrections to or insight into this
logic.  It seems more complicated than I'd expected to need for
implementing lazy request entity reading for an application running on
top of the server.

    a. In all of the steps below, where it says transmit 100 (Continue)
       that may be avoided if some request entity has already
       been received.

    b. While transmitting the response, if at any time the
       transmission is blocked for writing, the server should try to
       read data from the client and save it for when the server
       application reads it, in order to resolve deadlock with less
       tolerant clients and large messages that fill TCP windows.

    c. As soon as the server application tries to read any of request
       entity, transmit 100 (Continue).

    d. If the application generates a *non-error* status, and might
       need to read some of the request entity but hasn't yet,
       transmit 100 (Continue) before the status.

    e. If the application generates a *non-error* status, and has
       already read or tried to read some of the request entity,
       transmit the status immediately.

    f. If the application generates a *non-error* status, and is
       already able to commit to not needing any of the request
       entity, but the size of the request entity is believed by the
       server to be sufficiently small that it's worth reading in
       order to maintain a persistent connection, and the connection
       is still eligible for persistence (neither side has transmitted
       Connection: close), transmit 100 (Continue) before the status.

    g. If the application generates a *non-error* status and is
       already able to commit to not needing any of the request
       entity, and the request entity is not believed by the server to
       be small enough to read in order to maintain a persistent
       connection, transmit the status without 100 (Continue).

    h. If the application has already generated a *non-error* status
       and is subsequently able to commit to not needing any more of
       the request entity than it has already read (which may be
       none), there is no need for the server to continue reading from
       the client except as noted for deadlock avoidance (b.), or if
       the server is going to read the whole request entity anyway (f.).

    i. If the application finishes handling a request and it generated
       a *non-error* status, and did not read all of the request
       entity, and the connection is still eligible for persistence
       (neither side has transmitted Connection: close), and the
       server believes that the amount of request entity that has not
       yet been received is small enough to be worth receiving in
       order to maintain a persistent connection, then it should read
       the remaining request entity and continue using the connection.

    j. If the application finishes handling a request and it generated
       a *non-error* status, and did not read all of the request
       entity, and the server believes that it is not worth receiving
       the remainder of the request entity, it should politely
       shutdown the connection, with a "lingering close" to avoid TCP
       RST problems.  Deadlock avoidance is required in conjunction
       with the lingering close as with (b.), by continuing to read
       incoming request data rather than letting the TCP receive window
       fill.

    k. If the application generates an *error* status, and might need
       to read some of the request entity but hasn't yet, *delay*
       transmitting the status until the application has completed
       handling this request, or can otherwise commit to not
       needing the request entity, or does actually read the
       entity and the *whole* entity has been received, or at least
       as much as the application will need of it.

    l. If the application generates an *error* status, and has
       already read or tried to read some of the request entity,
       *delay* transmitting the status until the *whole* entity
       has been received, or at least as much as the application
       will need of it.

    m. If the application generates an *error* status, and is already
       able to commit to not needing any of the request entity,
       transmit the status immediately.  The size of the request 
       entity is not important, because the client is expected to
       prematurely abort a long one.

    n. If the application has already generated an *error* status and
       is subsequently able to commit to not needing any more of the
       request entity than it has already read (which may be none),
       there is no need for the server to continue reading from the
       client except as noted for deadlock avoidance (b.).  However,
       if the connection is still eligible for persistence, the server
       should continue reading for a limited time, number of bytes or
       other policy determined by the server (see o.), speculating
       that the connection will be used for another request.

    o. If the application finishes handling a request and it generated
       an *error* status, and did not read all of the request
       entity, and the connection is still eligible for persistence
       (neither side has transmitted Connection: close), the server
       should continue to read and discard the request entity.  The
       client should prematurely terminate the request entity, although
       it might not.  Therefore the server should read and discard
       only for a limited time, number of bytes, or other policy
       determined by the server, and if the end of the request is not
       seen by then it should politely shutdown the connection, with
       a "lingering close", as in (j.).

    p. If the application generates a status which it's thought some
       clients may treat as an error and some as non-error for the
       purpose of RFC 2616 section 8.2.2, a conservative mix of the
       above rules is required.  3xx status codes might be in this
       category.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 14 Mar 2004 16:49:11 +0000

Joris Dobbelsteen wrote:
> Requests and responses are *always* fully completed. There is no way of
> canceling a request/response half-way, except by closing the connection.

Thank you.  I had managed to figure that out, more recently than my
mail on the subject :)

> Just ignore the 100 response. The protocol behaves exactly with or without
> it.

I can't ignore it: I'm writing a HTTP/1.1 server.  I must send the 100
response at the appropriate time, to prevent the client delay.  Also
it's a protocol requirement: section 8.2.3.  OTOH I don't want to send
it if I'm generating an error response.  Pretty much what 100 is for.

I delay sending the 100 response until the application (which sits on
top of the server) asks to read any of the request entity, which makes sense.

The complications arose from the question of what to do when the
server application doesn't read any of the request entity.
Conservatively the HTTP server just reads it anyway, but I was trying
to apply a close to optimal set of rules.  The obvious thing to do is
not send 100, so that the client might not sent the entity.

But I've looked a bit harder, and RFC 2616 section 8.2.3 states:

      - Upon receiving a request which includes an Expect request-header
        field with the "100-continue" expectation, an origin server MUST
        either respond with 100 (Continue) status and continue to read
        from the input stream, or respond with a final status code. The
        origin server MUST NOT wait for the request body before sending
        the 100 (Continue) response. If it responds with a final status
        code, it MAY close the transport connection or it MAY continue
        to read and discard the rest of the request.  It MUST NOT
        perform the requested method if it returns a final status code.

That final MUST NOT indicates that if my server application generates
a non-error response, i.e. a response which indicates it _did_ perform
the requested method, and the server application doesn't ever ask for
any of the request entity from the HTTP server, the HTTP server must
_still_ send 100.

Ok.

Furthermore, the HTTP server _must_ read _all_ of the request entity.
It cannot abort, because there is no way to know when the client
application has read the response.  It is possible to know when the
client's TCP stack has acknowledged all the response, but the client
_application_ might not see that data until it has sent all the request.

> I believe the above would impact most of your questions, as I believe you
> get meaning of this response incorrect. 

Yes I did.  I thought it meant the client could choose not to send the
response at all.  (Actually, for "Error Status" responses (section
8.2.2), the client _can_ choose not to send a response, as you
observe: the client can send a zero length chunk.  It can't do that
for non-error responses).  My understanding of 100 (Continue) is now
adequate.  Thank you :)

> *Remark* on the chunked transfer from clients to servers. I don't know
> whether implementations actually send a zero-sized chunk after getting a
> response. This behaviour might be implementation specific. I rather believe
> clients would complete the entire request.

If it's an Error Status they SHOULD send the zero-sized chunk or close
the connection.  Section 8.2.2 is quite prominent.  This has nothing
to do with 100 (Continue), by the way, it's orthogonal to it.

Consequently, if I'm generating an Error Status with a body, like a
404 with explanation page that *uses* the request entity to generate
that page, my server MUST NOT send the Error Status until reading
entire request entity.  If it did send the Error Status earlier, the
client is supposed to abort sending the request which the server needs
to construct the page which goes with that 404.

However, if I'm generating a 200 response where the page uses the
request entity, it appears I can transmit that before the request
entity is all read.

> When sending a entity body to the server and you receive a 3xx
> response, this means the request is redirected. The serer is
> unwilling/unable/... to handle the request.  The behaviour can be
> similar to the error responses (4xx and 5xx), because in this case
> the server is also unwilling/unable to handle the request.

You may be right, but I see no clear text which relates the meaning of
3xx to "Error Status" in section 8.2.2 - which is fairly important.

As I described in the example above, 200 can overlap the response and
request; 404 cannot.  Can 303?  It's allowed for a 303 response to
come with an explanatory page which _might_ be generated using data
from the client request entity.

I'll be conservative and assume clients are allowed to abort the
request entity on 3xx responses too (for 8.2.2).  *But* I have to be
conservative and assume I'm not allowed to transmit a 3xx without 100
when 100 is indicated (for 8.2.3): 303 being a valid response for a
POST method which *is* performed.

> After an error you should close the connection when:
> 1) You are not sure about the state (e.g. after 400 bad request).
> 2) Client is sending data and you don't want to waste bandwidth on it.

You can't fully close the connection, however, because of the TCP RST
problem.  And the client SHOULD abort the request with a zero length
chunk, but I haven't got as far as testing any clients for that yet.

> >After transmitting the response entity and discovering that it 
> >didn't need the request entity after all, and it wasn't an 
> >error response, it's too late for the server to change the 
> >response headers: they are transmitted already.  The server 
> >will not have put Connection: close in the response header, 
> >unless that was requested by the client, or it was HTTP/1.0 
> >without keep-alive, or some other conditions applied.  So the 
> >server will transmit the response...
> >
> >And then it has to shut down the connection, right?  (All the 
> >while reading data from the client to avoid the TCP RST problem).
> 
> A much better way to tranfer the data would be by using chunked transfers.
> Using trailers you can afterwards change the headers.

Do you trust any clients to honour trailers properly? :)

> The most obvious choice would be to close the connection and hope the user
> agent got the message correctly. Due to the fact that the user agent was
> unable to complete the request, this mgiht result in an error, despite
> receiving the entire response correctly. I don't have any experience with
> doing this.

Neither do I.  From RFC 2616, I think that a client _should_ display
an Error Status response correctly even if it is close-delimited, and
the (unidirectional) close reaches the client before the client has
finished transmitting.  It's clear that real web servers have to keep
reading at that point to make sure clients to eventually see the
error.  It's also clear that a client which doesn't keep monitoring
the stream for an error, i.e. tries to send an entire large request
first, won't look too impressive to its user: the server should always
be able to refuse a large upload, quickly.

> >> > If you have enough information to begin the response just from 
> >> > reading the request headers, then you are free to send the 
> >response 
> >> > headers right away; even the 1xx response is optional - 
> >you can skip 
> >> > right to sending the final response.  (Note: the rules of the 
> >> > protocol don't forbid this - what any given client will do 
> >with this 
> >> > is a different question that can be answered only with testing).
> >> 
> >> Yes.

No!  Section 8.2.3, about the origin server: "It MUST NOT perform the
requested method if it returns a final status code."  Here final
status code means without a preceding 100 (Continue), and with none of
the conditions for omitting 100 (such as already seeing some of the
request entity).

If it MUST NOT perform the requested method, that means it must be
sending an _error_ final status code, as a success code means that it
did perform the requested method.  Slightly round the houses logic I
know.

This includes at least some, maybe all, of 3xx.  I.e. for the purpose
of section 8.2.3, 3xx responses are in the same category as 2xx
responses.

> The request headers will ALWAYS be followed by the entity body. There is no
> way of cancelling transfering the entity body, other than to close the TCP
> connection.

Yes, I've got that now.  It is not possible to ask the client to abort
the request entity while accepting a non-error response, only an error
response.

> When analyzing the protocol, you can simply ignore the 100 reponse. With or
> without it, the results are exactly the same. It was rather developed to let
> a client wait a short while, so the server could return error when it didn't
> accepted the response for any reason, saving you from transmitting data.

My reading of 8.2.3 indicates a slight protocol implication: if the
server sees Expect: 100-continue, and sends a non-error response
(i.e. indicating it did perform the requested method) without a
preceding 100 Continue, and it sends the non-error status line
_before_ it receives any of the request entity, and it's a HTTP/1.1 or
greater request, then that server is not conforming with RFC 2616 8.2.3.

> >(Btw, any Transfer-Encoding indicates a request body, and 
> >"chunked" is required to be the last in the sequence of 
> >transfer-codings.  If Content-Type specifies 
> >"multipart/byteranges", that indicates a body, but I see 
> >nothing in RFC 2616 which indicates a body for any other 
> See the MIME specification, I believe...

RFC 2616 section 4.3:

   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers.

The existence of Content-Type without either of those two headers
would appear to not indicate a message-body.  Though section 4.4, part
4, offers multipart/byteranges without Content-Length and without
chunked, which logically implies Transfer-Encoding: identity, but
identity has been removed in the errata :)

Something is amiss...

> No, just ignore the 100 response when developing for HTTP/1.1. The server
> will always expect a entity body.

I'm writing the server ;)

> Same as above. Usually the connection will be closed to avoid receiving the
> unneeded entity body.

Yes, although you can't close the connection fully right away, because
the client will fail to see "in flight" response data.  Hence
lingering close etc.  And in theory, if the request is using chunked,
the client will use that zero length chunk instead of aborting the
connection...  (This makes chunked requests advantageous).  Then
again, how many clients transmit requests using chunked encoding?

> I don't know whether the request and response should be threated as an
> atomic piece. So an implementation might give an error, because it is unable
> to send the entire request.

It could be messy.  It'll need testing.

> I don't believe the transfer of data is aborted by a zero length chunk, when
> this is not the last one. More probably is the server closing the connection
> when it knows the client received all the data.

It's difficult to know when the client has received all the data so
that it's safe to close the connection...  A fair heuristic would be
wait for all ACKs for the data, then add a time limit for the client
application to read it from the client OS's TCP stack.  (If the
application doesn't read it soon enough, the request entity still
flowing to the server causes the server's closed socket to send RSTs
to the client OS, which then discards received data before the client
application can see it.)

> A 3xx (Redirection) indicates the server will not handle the response and
> will redirect it to another server. These is no point in this respect to
> transfer the data to the server and aborting the connection would seem the
> only logic one.

Sure, but what do actual client implementations do?  If the spec
_stated_ that 3xx responses are errors for the purpose of 8.2.2, fine.
But I don't see the language in the spec as indicating that at all
clearly.

In my mind, some 3xx responses are informative.  Certainly, 302, 303
and 307 are often used by servers wishing to direct the client around
the URL space of theserver, e.g. to append "/" when a URL corresponds
to a directory, or as an alternative form of content negotiation.

In particular, 303 is supposed to be used after a *successful* POST,
which wants to redirect the client to follow it with a GET.  That's
not an error at all.

So they're not really *errors*.

Also you said there's no point transferring data.  Servers certainly
transfer some data with 302/303/307 responses: a small page telling
older clients that there's a redirection, with a link.  That page can
depend on the request entity, especially if it's a 303 in response to
a POST.

Anyway I'll conservatively assumes clients treat 3xx like errors for the
purpose of 8.2.2 -- i.e. they may abort the request entity, so if the
server's application wants to read the request entity, it must delay
sending the status line until it's read all of the request.

At the same time, because 3xx aren't errors, I have to treat them like
2xx for the purpose of 8.2.3.

Fwiw, these are the rules my server now implements:

    - If the application asks for any of the request entity, send 100
      (if required by Expect: 100-continue or the PUT/POST rule).

    - If the application generates a 2xx status, and it hasn't sent
      100 yet, send it with the status (if required by....rule).  It
      is always sent, even if the application doesn't show any
      interest in the entity (because of 8.2.3).

    - If the application generates a 4xx/5xx status, and it hasn't
      asked for any of the request entity, send the error status
      without 100 and prevent the application from seeing any of the
      request entity.

    - If the application generates a 4xx/5xx and it does want the
      request entity, send 100 (if required by....rule), then read the
      entire request, and _then_ send the status line response
      (because of 8.2.2).

    - If the server application generates a 3xx, regardless of whether
      the application wants the request entity (because of 8.2.3) send
      100 (if required by....rule), then read the entire request,
      _then_ send the status line and response (because of 8.2.2, and the
      possibilities that clients might abort the request entity when
      they see a 3xx response).

The question of how to abort connections is another thorny one.  With
error responses, we want to close the connection after sending the
response and not reading the whole request but have to do so in a
careful way to be confident the client application sees the error.

With non-error 2xx responses (this is totally a guess based on how I
imagine some clients to be written), we might find that closing the
connection after sending the response and not yet having read the
whole request may lead clients to ignore the response, or to abort the
request.  So my conservative *guess* as to the right thing to do is to
send the status line as soon as possible (unlike for error responses),
but to refuse to send the close (for close-delimited) or zero chunk
(chunked) or final byte (content-length) until the all of the request
entity has been received.

That looks more complicated than necessary -- but is it?  Does anyone
have knowledge of how different well-known clients react to receiving a
whole response before all the request has been read?

Thanks,
-- Jamie

---

From: "Joris Dobbelsteen" <[email-masked]>
Date: Sun, 14 Mar 2004 15:54:53 +0100

Jamie,

This would be a short summary of my comment:
(Please correct me if I'm wrong anywhere)

Just ignore the 100 response. The protocol behaves exactly with or without
it.
The protocol is rather async, so the server can never guess the state of the
client for the data received from the client, it MUST always be absolutely
certain. A client is not required to wait for a 100 response, even when it
sends a Expect: 100-continue header. The server might just not sent a 100
response back.

When sending a entity body to the server and you receive a 3xx response,
this means the request is redirected. The serer is unwilling/unable/... to
handle the request.
The behaviour can be similar to the error responses (4xx and 5xx), because
in this case the server is also unwilling/unable to handle the request.

You should not close the connection when:
1) The state of the protocol is known (404 response)
2) You are willing to waste any more bandwidth on data from the client, you
will not handle.

There is some more inline...

Hopefully this clears up some questions.

- Joris

>-----Original Message-----
>From: [email-masked] 
>[mailto:[email-masked]] On Behalf Of Jamie Lokier
>Sent: Friday, 12 March 2004 20:07
>To: Scott Lawrence
>Cc: [email-masked]
>Subject: Re: Can the response entity be transmitted before all 
>the request entity has been read?
>
>
>Scott Lawrence wrote:
>> > it means HTTP Error Status codes, because of the implication that 
>> > the connection might continue.
>> 
>Thanks.  Why > 299?  To my mind, error status is > 399 (i.e. 
>4xx = Client Error, 5xx = Server Error).

The 3xx codes also indicate the server will not handle the request any
further, it is (probably) redirected to another server, who will handle the
request. There is no point in transfering any more data to a server
indicating not to handle the request.

>
>If redirections are Error Status responses for the purpose of 
>low level client behaviour in section 8.2.2, that is poorly 
>worded and should be clarified in the Errata.
>
>If I am confused, surely some client implementations will have 
>it wrong too.  That's critical: the server _has_ to know 
>whether the client will abort sending the request body or not, 
>to decide how to continue with the connection.

When you cancel a request, this can only be done by terminating the TCP
connection. HTTP doesn't provide any facility to cancel a request.

>Assume persistence has already been determined appropriately, 
>by looking for Connection: close and/or Connection: keep-alive 
>in the request.
>

>> > transmitting their request entities anyway.  Therefore there's no 
>> > way for the server to determine whether data received from the 
>> > client is the request entity or the next request, if it 
>wants to try 
>> > avoiding 100 (Continue) to produce a non-error response without 
>> > using the request entity.
>> 
>> indicates the presence of a body.  If none of the above are 
>there, the 
>> n there is no body and the next byte starts the next request.
>
>That's not what I mean.  What I'm saying is: if I _don't_ send 
>100 (Continue), I don't see a way to determine whether the 
>client _chose_ to not transmit the request entity _despite_ 
>the Content-Length or Transfer-Encoding headers.  Therefore, 
>after doing that, there's no way for the server to read 
>another request, and it must shut down the connection.

No, just ignore the 100 response when developing for HTTP/1.1. The server
will always expect a entity body.
The reasons are quite simple:
First of all, the client is allowed to start transfering data to the server
without waiting for the 100 response, regardless of the Expect header.
Second, there is NO way for a server (or a proxy) to distinguis a request
from an entity body. The entity body might look like a valid request, but is
in fact just data.
Cancelling an upload should always result in closing the connection.

>
>> > Or can it avoid reading the request entity completely, by not 
>> > sending 100 (Continue) and somehow determining when the 
>next request 
>> > arrives on that connection.  Is that possible?
>>
>> No, but even if it were it wouldn't save you anything - TCP is a 
>> stream; there's no way to discard the bytes other than at 
>the receiver 
>> (without closing the connection, that is).
>
>Also, if you stop reading at the receive, eventually the TCP 
>window fills and the transmitter stops, which reduces data 
>transmitted overall if not all of the request entity is read.  
>That may or may not cause deadlock, depending on the 
>transmitter's logic.  We can avoid deadlock while reading the 
>minimum at the server, by reading only when the data is 
>needed, writing response as we generate it, and if writing 
>connection is persistent: the request doesn't have Connection: 
>close, and it's HTTP/1.1.
>

No, the data will ALWAYS be sent or the connection will be closed. Requests
cannot be cancelled half-way. After the data is transferred, a new request
can be issued.

>     shutdown the connection after the response is transmitted, in

Indeed 3xx is non-error and proxies might cache them, especially 301
(permanent redirect). Still the server is unwilling/unable to fulfill the
request.

>     _delay_ sending the response until it has read the whole request,

I'm not sure about this. It seems quite logic to terminate sending after a
full request has been received. This might vary between implementation how
to handle such situations. I don't recall the spec defining any behaviour in
such cases.

>     differently from 3. and 4.  I see this category containing 3xx.

Generally a 3xx response can be handled very similar to a error response.
The server is unable/unwilling to fulfill the request.

>     mustn't try to read another request on that connection?  I'm not

The request is always complete. There is no way to cancel a request half-way
(except by closing the connection).

>            response.  Is this correct?  (I'm not sure).

Just ignore the 100 response. It does actually nothing, only indicate the
client should send data immediately and not wait for a possible error
closing the connection. The request/response behaviour is exactly the same
with or without the 100 response!
The 100 response was primarily designed to saves bandwidth. When an error is
discorved early (such as 404), this saves sending the entity header. In this
case the connection MUST be closed, because the server will expect the
entity body first and not a new request first.

>This translates to the following logic in server code.  I 
>would be grateful to anyone who can offer corrections to or 
>insight into this logic.  It seems more complicated than I'd 
>expected to need for implementing lazy request entity reading 
>for an application running on top of the server.
>

Indeed, remember, the client is not required to wait for a 100 response!
With or without the 100 reponse, the protocol behaves exactly the same.

---

From: Scott Lawrence <[email-masked]>
Date: Sun, 14 Mar 2004 19:10:07 -0500

> it if I'm generating an error response.  Pretty much what 100 is for.

100 is an attempt to optimize away some of the dumb things that happen
with > 299 responses.  It was an attempt to prevent the pipe from
getting filled with stuff the server doesn't want to read.   So, yes,
you had the right idea.

The qualifiers and exceptions all derive from a desire to be backward
compatible; since there were clients and servers out there using the 1.1
version number that didn't do this, we decided to leave in exceptions
for them. 

> _application_ might not see that data until it has sent all the request.

Right - no matter how long you wait, the client might see your failure
to read the body as an error, so you've got to read it, regardless of
whether or not you send a success status.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 15 Mar 2004 11:09:15 -0700 (MST)

Jamie,

	This thread contains too many arguable statements and RFC
interpretations for me to comment on each specific issue. In short,
the result of the response entity transmitted before all the request
entity has been read is unpredictable. For what it is worth, I would
like to try to change your development direction/tactic with a general
"Keep it Simple" response/statement below.

	If you do not control all clients, including forward and
reverse _proxies_, in front of your server, then I would advise that
you assume that some of your clients will not support some RFC 2616
MUSTs and will "violate" many RFC 2616 SHOULDs. Some of your clients
will be HTTP/1.0 clients. Some will not do what you expect them to do
when receiving unusual traffic. Some proxies have adaptation agents
attached to them (content modification and virus checking are some
examples). Such agents are even more vulnerable to HTTP surprises such
as "early" responses.

	You seem to be developing an HTTP-to-application proxy rather
than a pure origin server. In that case, the most reliable strategy in
the presence of questionable traffic is often to switch to a tunnel
behavior and terminate the connection as soon as the response is sent.
It may be a design mistake to give applications control knobs that
depend on proper HTTP/1.1 support in clients. Give your applications
just the basic knobs that minimize the amount of guess work on the
proxy. If an application requires fine control of 100-Continue or
status code sending time, the application will not work reliably on
top of HTTP. Do not do anything "unusual" or border-line, and do not
expose HTTP knobs that allow your applications to involve your proxy
in such "unusual" or border-line behavior.

	More specifically, do not expect clients to receive or support
100-Continue by default. Do not expect clients to be able to recover
from receiving responses prior to them sending complete requests. Do
not spend time deciding when to close a connection in the presence of
strange traffic: close as soon as you can, usually right after sending
the final response (erroneous or otherwise).

This subjective advice is based on testing actual HTTP
implementations. YMMV.

HTH,

Alex.

-- 
Protocol performance, functionality, and reliability testing.
Tools, services, and know-how.
http://www.measurement-factory.com/

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 15 Mar 2004 19:07:42 +0000

Alex Rousskov wrote:
> than a pure origin server.

It's not really any different to other well known origin servers.

It is structured like a server+CGI, or Apache+mod_perl.  I used the
term "server application" to indicate that there's a server component
and another component which communicates with the server in a simple
way: reading request entity, and writing response entity

> In that case, the most reliable strategy in the presence of
> questionable traffic is often to switch to a tunnel behavior and
> terminate the connection as soon as the response is sent.

I see that is the most common strategy :).  It seems a bit dubious,
though, because it's easy to imagine application code like this:

     write "Status: 200\n\n";
     write "Thank you for your submission.\n";

     while ($x = read (...)) { store ($x) }

     write "Great, all done.\n";

As you say, there are theoretical and practical requirements for
talking to real HTTP clients and proxies.  One of them may well be
that the response shouldn't be sent before the request is read.

I'd rather put the hard requirements, and every feature that will help
with robustness, in the _server_, rather than document it as a
requirement that applications have to follow.  It's the server's job
to keep the communication good as reliable as possible, insulating the
application.

> depend on proper HTTP/1.1 support in clients.

Exactly.  You see in the above example, _not_ giving the application a
control knob implies that the server reads the whole request before
sending any of the application's response, or refuses to let the
application read anything after any response has been sent.

That's contrary to most server implementations: they do give the
application control over when to read and write, which is the opposite
of what you're suggesting here.

> Give your applications just the basic knobs that minimize the amount
> of guess work on the proxy. If an application requires fine control
> of 100-Continue or status code sending time, the application will
> not work reliably on top of HTTP. Do not do anything "unusual" or
> border-line, and do not expose HTTP knobs that allow your
> applications to involve your proxy in such "unusual" or border-line
> behavior.

I agree.  I wish the application to do just the basic things people
expect:

    a function to set the status code + response headers
    ability to write the response entity
    ability to read the request entity
    no special requirement on when the request entity is read

> 100-Continue by default.

Ok.  I am so far following the advice in RFC 2616, which is to send
100-Continue if Expect: 100-continue is received and it's HTTP/1.1 or
greater, or if it's HTTP/1.1 (exactly that version) and it's POST or PUT.

Is that advice good?

Not sending 100-Continue is always safe, but may introduce delays.

> Do not expect clients to be able to recover from receiving responses
> prior to them sending complete requests.

That's a crucial question.  Should I either enforce that in the server,
by insisting on reading the whole request before allowing any text of
a non-error response to be sent (error can be sent immediately), or
document it as an application requirement: that the application must
do all its reading before it writes anything?

It seems that existing servers, e.g. Apache, thttpd and all the others
don't do either: they allow request to be read by the application when
it likes, the response to be sent before all the request is read if
the application likes, and don't document this as a problem.  It's for
application writers to be aware of it.

Yet, I want to document it if it's a real practical issue, because I
can see good uses for sending response before the request is all read:
a translation or file format server, which writes part of the
translation before all the text is uploaded, and an upload meter,
which writes some text as a large upload progresses, perhaps showing
thumbnails of the uploaded files as each one arrives too.

If these things are not possible, or if they have to be implemented in
other ways such as multiple frames so that uploads are separate
requests from the progressive display, or if they work with most
clients and there are a few known ones where it must be prevented,
then I accept that.

I'd simply like to know whether it's best to program the server to
enforce that, knowing it's a common/rare client weakness, or to not
enforce it but recommend it in the application interface
documentation, or to permit it if it actually works in practice.

> Do not spend time deciding when to close a connection in the
> presence of strange traffic: close as soon as you can, usually right
> after sending the final response (erroneous or otherwise).

My strategy is to copy Apache's well-tested "lingering close":
shutdown(fd,SHUT_WR) followed by reading everything for up to 30
seconds, or until 2 seconds passes with no incoming data, then the
full close().  Immediately fully closing the socket is known to cause
client problems for good reasons.

The other part of my strategy, based on what people have said here, is
to ensure I read the whole request if sending a non-error response,
but not if sending an error response - then I can just follow it with
lingering close.

> This subjective advice is based on testing actual HTTP

Your advice is very welcome!  Thank you!  I find it surprising that
response-during-request-upload isn't mentioned in any guidelines that
I've seen as either ok or a problem.  If there is knowledge about this
working or not working with common browser clients, I'd like to know
more.  Maybe I'm one of very few people to consider it :/

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 15 Mar 2004 16:36:21 -0700 (MST)

>      write "Great, all done.\n";

While it is easy to imagine such an application, tasking a proxy to
"rewrite" or "fix" application logic to fit HTTP restrictions seems
like a bad idea. IMO, upon receiving "Status: 200\n\n" and sending off
response headers, your proxy should become a "tunnel" and not try to
second-guess the application intent.

> I'd rather put the hard requirements, and every feature that will
> help with robustness, in the _server_, rather than document it as a
> to keep the communication good as reliable as possible, insulating
> the application.

IMO, the "Do No Harm" rule trumps the "Try to change the world to the
better" rule, especially for proxies (which is what you are
implementing in this context). If you can reliably convert garbage
into compliant output, do so. If your smart conversion algorithm
silently breaks a few innocent applications, then do no smart
conversion.

If you need a negative example, consider Apache 2.x problems with
smart content-length guessing algorithm that, AFAIK, still stalls a
few simple CGIs that work fine with Apache 1.x:
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23528

Specifically, if your server will be able to detect and reject
applications that write before reading, fine. If your server delays
any application output until it thinks there is no more input, your
feature is probably going to be a popular target for denial of service
attacks (for example) and you are probably going to deadlock
applications that write more than one buffer worth of data (another
example).

It is possible to have an HTTP-to-Applications API with enough logic
and controls that optimizations you mention are very appropriate and
safe. Is CGI such an interface? Do CGI specs document these things?

> application control over when to read and write, which is the
> opposite of what you're suggesting here.

Am I? A tunnel is exactly what gives the application unlimited ability
to read and write at any time, at will.

> I am so far following the advice in RFC 2616, which is to send
> 100-Continue if Expect: 100-continue is received and it's HTTP/1.1
> or greater, or if it's HTTP/1.1 (exactly that version) and it's POST
> or PUT.
>
> Is that advice good?

The former is not an advice, it is a MUST-level requirement :-)
so yes, you must do that.

The latter is a pro-active behavior intended to help RFC 2068 agents.
Unfortunately, it requires compliant RFC 2616 support for 100 Continue
in proxies. My bet that sending 100 Continue pro-actively will hurt in
more cases than it will help, but I have no data to prove that.

Moreover, there was a paper that formally proved that 100 Continue
leads to deadlocks in certain compliant environments, so we are
probably talking about a partially broken mechanism here anyway.

> That's a crucial question.  Should I either enforce that in the
> server, by insisting on reading the whole request before allowing
> any text of a non-error response to be sent (error can be sent
> immediately), or document it as an application requirement: that the
> application must do all its reading before it writes anything?

You cannot enforce this at the server without deadlocking or killing
the application or running out of server memory. Imagine an
application that writes more than you can buffer before it reads
anything.

> It seems that existing servers, e.g. Apache, thttpd and all the
> others don't do either: they allow request to be read by the
> application when it likes, the response to be sent before all the
> request is read if the application likes, and don't document this as
> a problem.  It's for application writers to be aware of it.

That's what I would do too, as far as code is concerned. Documenting
potential problems is always good, of course, especially if you can
give specific real-world examples.

> documentation, or to permit it if it actually works in practice.

Make your HTTP-to-application proxy as simple as possible. Warn of
possible problems if the tunnel interface is abused. Let applications
decide how they want to deal with those problems.

> full close().

Cool. I hope this well-tested algorithm is not what breaks CGIs in
Apache 2 :-). Also, FWIW, I recall half-close causing many problems
for Squid proxies for a while. It is probably fixed now.

Thanks,

Alex.

---

From: Jeffrey Mogul <[email-masked]>
Date: Mon, 15 Mar 2004 16:40:49 -0800

I believe Alex is referring to this paper:

	"Safe Composition of Web Communication Protocols for 
		Extensible Edge Services"
	Adam D. Bradley, Azer Bestavros, Assaf J. Kfoury (Boston University) 
	Proc. 7th Int' Workshop on Web Content Caching and Distribution (WCW)
	http://2002.iwcw.org/papers/18500001.pdf

The title is a bit of a misdirection; it's not really about edge
services.

     "The primary focus of this paper is on the applicability of formal
     methods to the verification of the correctness and
     interoperability of the HTTP protocol s revisions in all
     per-mutations of roles (client, proxy, server) and compositions
     thereof with respect to the 100 Continue feature.

Also, as I understand the paper, what they showed is that while
if everyone fully (and correctly) implemented the 100-Continue
specification of RFC2616, everything would be fine, we didn't
successfully design this feature to interoperate with RFC2068.
We also apparently said "MAY" instead of "MUST" in at least one
place, so partial (but legal) implementations of RFC2616 might
be a problem.

I suppose someone should take a look at the latest Internet-Draft
to see if it could be changed to avoid the last problem identified
in the paper.  However, even this might be too late to make
"Expect: 100-continue" a safe thing for clients to send, unless we
change the protocol version number.

-Jeff

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 16 Mar 2004 06:00:35 +0000

Thanks for your detailed response.

> second-guess the application intent.

Part of the mixed message here is that I'm simultaneously writing a
server which is intended to be well written, robust, protocol
compliant, persistent when possible, no deadlocks, and so forth; and
I'm also writing an application or two.  The applications are actually
the motivation for the server.

So the question of whether the above kind of application is worth
supporting is important, because I intend to try it out.

By the way, pure tunnelling leads to deadlock: the application can get
stuck writing if the client isn't reading the response until it
transmits all the request, and all the TCP windows fill up.  I don't
like that deadlock because it isn't necessary and it is practical to
eliminate it in the server.  It's messier to eliminate it in the
application, and anyway why do it once per application instead of once
in the server?

You're thinking: out of memory.  Actually no.  To avoid deadlock, this
is what I do: If writing would block, and there is data available to
read, read it.  The buffer holds _read_ data, and is therefore limited
by the maximum permitted request entity size.  The maximum is asserted
both for content-length and chunked requests.  The buffer is required
*somewhere*, either in the server or in the application itself, or in
backing storage for them, so there is no added resource consumption
from this technique when it's implemented properly.

> silently breaks a few innocent applications, then do no smart

:) Well, you'll be glad to know that for now, especially as I want to
try it out, the server allows the application to read and write as it
likes.

It's not entirely a tunnel: as said above, the server takes care of
deadlock avoidance for clients which aren't reading until they've
finished writing.  That simplifies the app code, without changing
anything unless that situation arises and the deadlock would really
occur.

Obviously the server is in change of chunking, (de)compression,
boundary checks against content-length etc.  I'll take that as understood.

> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23528

Yeah, that's a bug in Apache.  It's not at fault for being smart, it's
that it's code simply has a bug.  The idea is good as it allows some
HTTP/1.0 connections to be persistent when they otherwise wouldn't be.

> applications that write more than one buffer worth of data (another

Presently I am allowing the app to write before it's finished reading,
because I'd like to try out that capability.  I might add a
per-request option to buffer all the input before writing anything -
it's fits easily with the deadlock-avoidance buffering, and could be
useful to some applications.

If I did restrict writing until reading was complete, the DOS attack
would be no different than the DOS where people try to send maximum
size request entities, e.g. uploading lots of large files.  That's the
only effect on the buffering algorithm: it ends up buffering up to one
maximum size request entity per request, and the overall memory and
disk management for that can certainly be constrained.

Note that the response doesn't get buffered infinitely.  Response
generation is blocked while the request is being buffered up for
whatever reason.

It's not possible to omit the buffering *somewhere*: for clients which
send a whole request before reading the response, the entire request
has to be buffered or stored *somewhere*, either in the server or in
the application, to resolve the deadlock.

> safe. Is CGI such an interface? Do CGI specs document these things?

There is a CGI spec draft (1.2 at the moment), but it is not so well
written as HTTP/1.1.  Not to mention that quite a few of the CGI
meta-variables are implemented in various ways, and even the spec'd
ones miss out important info, so everyone adds a few more non-standard
variables.  (Like REQUEST_URI).

But that's beside the point, I'm not writing a CGI interface, I'm
writing and designing an HTTP-to-Application API with enough logic
etc. to do as you suggest.  At the same time, trying to keep it as
simple as possible - only the necessary controls.

> to read and write at any time, at will.

Exactly.  Too much control over the HTTP, like you said to avoid ;)
(Facetious comment; please ignore ;)

> The latter [100 Continue for PUT and POST] is a pro-active behavior
> intended to help RFC 2068 agents.  Unfortunately, it requires
> compliant RFC 2616 support for 100 Continue in proxies. My bet that
> sending 100 Continue pro-actively will hurt in more cases than it
> will help, but I have no data to prove that.

Hmm.  Maybe follow that SHOULD only if Via isn't defined?

If the client delay is small I'm not bothered.  Do you have any data
on how long those RFC 2068 agents will delay sending the request
entity?

> probably talking about a partially broken mechanism here anyway.

Hmm.  I'd like real data on what to do here.  If you can find the
paper or any other info, that would be very helpful.

I don't see any deadlock scenarios with the way I have implemented it.
Perhaps the deadlock occurs when it's coded in a different way (I've
been careful, most http implementors aren't half as cautious, judging
by the code I've read).  Or maybe I just have yet to see it.

I will have a peek at Apache's code to see what it does for the RFC
2068 clients. Squid won't give reliable answers: it's HTTP/1.1 support
is still in the development phase.

(I've looked at enough implementations to find loads of other quirks;
may as well see if anyone put in a comment about this).

  look..look..look

Hmm.  Apache 2 doesn't satisfy the MUST of 8.2.3.  It won't sent
100-continue if the request has Content-Length: 0.

Both Apache 1.3.29 and 2.0.48 (current versions) apply the rule that
Expect: 100-continue with HTTP-Version >= 1 causes 100-continue to be
sent.  Neither of them apply the rule for supporting RFC 2068 clients.

Apache 1.3.29 won't send it with error responses, according to a quick
skim of the code, but a couple of servers I poked at didn't behave
like maybe I misunderstood something, or those servers are configured
to be more complicated.

A quick try at www.microsoft.com :) reveals IIS/6.0 in their setup
sends 100-continue to a POST with Expect: 100-continue, despite the
404 response.  More checking: it sends 100-continue to a POST without
Expect: 100-continue, despite the 404 response.

phttpd-1.10.4, thttpd-2.25b and lighttpd-1.0.3 don't send it ever,
despite all of them claiming to offset some level of HTTP/1.1.  I
guess they predate RFC 2616.

> application that writes more than you can buffer before it reads

No.  The application can't do that: it will block when writing blocks.
The server's large buffer is the size of a maximum _request_ entity,
and that storage is unavoidable one way or another.  Everything else
is limited to an appropriate I/O block size.

> give specific real-world examples.

I will document it as well.  If there are clients which accept the
non-error response before all the request is read, that is certainly a
feature worth letting the app have access to.

The reason I asked all this is that if practically all real clients
fail with it (I haven't reached the stage of testing yet), then the
server may as well constrain the app, probably by complaining at it.

> decide how they want to deal with those problems.

Ok.  Now give me advice as application author: I'd like to know
whether it a common or rare client weakness, so whether I should
consider using that technique or not.

If there's only some well known, old clients which don't like it, then
a match on the User-Agent string which activates the request entity
buffer will be the right thing to do.  The server can include it in
the plethora of other User-Agent quirks it already works around.  (I'd
rather put such knowledge in the server which already has that buffer
for deadlock avoidance, than in N applications).

If, however, lots of clients don't like it, then it isn't worth using
the technique at all, and it would be better for the server to
complain when the app erroneously tries it -- even if it's just a
warning which can be disabled.

> Apache 2 :-).

The algorithm has been in Apache 1 for quite a while.

But maybe  Apache 2's version of it is to blame. :/

Apache 2's algorithm is different -- I think it is a coding error, not
intentional.  Apache 1 does what I describe above.  Apache 2 tries to
read with a 2 second timeout up to 30/2 times, and each read is 512
bytes max.  That means if the socket's receive buffer _already_ has
512*15 bytes in it, Apache 2 will terminate the lingering close
immediately, even if there is more data incoming from the client.
That seems very wrong: incoming data from the client after close is
what causes the transmitted data to be lost, which is why Apache 1
keeps trying until it sees a 2 second gap, which heuristically means
the client has stopped sending.

> Also, FWIW, I recall half-close causing many problems

I don't see how it could be a problem: server half-close with a
timeout (the 30 seconds timeout is still important) is invisible to
the client, except in terms of the timing.  The sequence of events the
client is able to observe is identical (unless the client is _so_
clever that is queries the socket to learn how much data has been
acknowledged by the server, but I'm sure Squid doesn't do that).

In effect, half-close with a 30 second timeout is equivalent, at the
socket interface level, to the network deciding to delay data flowing
from the client to the server for 30 seconds, so that the TCP RST
effect where response data disappears can't happen.  A real network
can cause a similar effect, so if half-close was causing Squid
problems, those problems would occasionally occur with real network
delays too.

Thanks,
-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 16 Mar 2004 06:40:51 +0000

> place, so partial (but legal) implementations of RFC2616 might

Thank you!  It is a most excellent paper.

The conclusion for someone writing an RFC2616 server is that "MAY send
a 100 [...] in response to an HTTP/1.1 PUT or POST request" must be
implemented as "MUST, after a timeout, ..." (or with no timeout), to
avoid certain deadlocks.

Other conclusions apply for someone writing a proxy or client.

Neither Apache 1 nor 2 actually do that, so I presume the theoretical
deadlock isn't a problem in practice.

Probably because the few clients which do wait for 100 (Continue), if
there are any at all, implement a reasonably short timeout before
sending the request entity anyway, even though that's prohibited by
RFC2068.

How wonderfully unclear.

- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 16 Mar 2004 09:42:19 -0700 (MST)

On Tue, 16 Mar 2004, Jamie Lokier wrote:

> > While it is easy to imagine such an application, tasking a proxy
> > to "rewrite" or "fix" application logic to fit HTTP restrictions
> > seems like a bad idea. IMO, upon receiving "Status: 200\n\n" and
> > sending off response headers, your proxy should become a "tunnel"
> > and not try to second-guess the application intent.
>
> By the way, pure tunnelling leads to deadlock: the application can
> get stuck writing if the client isn't reading the response until it
> transmits all the request, and all the TCP windows fill up.

Yes, but that's client- or application-caused deadlock, not a
server-caused deadlock. I am arguing that a general-purpose server
cannot resolve such deadlocks using a simple tunnel interface.
Applications acting unusually should bare the responsibility of
avoiding deadlocks, not the server.

> I don't like that deadlock because it isn't necessary and it is
> practical to eliminate it in the server.  It's messier to eliminate
> it in the application, and anyway why do it once per application
> instead of once in the server?

I do not know of a way to resolve the said deadlock in the server, in
a general way that will not break some valid applications.

> To avoid deadlock, this is what I do: If writing would block, and
> there is data available to read, read it.  The buffer holds _read_
> data, and is therefore limited by the maximum permitted request
> entity size.  The maximum is asserted both for content-length and
> chunked requests.  The buffer is required *somewhere*, either in the
> server or in the application itself, or in backing storage for them,
> so there is no added resource consumption from this technique when
> it's implemented properly.

Sorry, I do not follow. You seem to be limiting request bodies to
maximum buffer size, which seems like a bad idea to me (but I am not
sure).

Here are the facts: An application does not read until it writes a lot
of data. An application wants to write more than your server can
buffer. Your server can either pass those writes through or block
them. If your server passes those writes through, then your server
cannot break the application (the application may still block due to
client behavior, but that is not your problem). If your server blocks
those writes, the application author will be after you with bug
reports and tomatoes ("It worked fine with Foo server!").

Of course, if you develop a specialized server for specialized set of
applications, it is totally up to you how to handle early writes.

> generation is blocked while the request is being buffered up for

And if request body is larger than your server buffer? Temporary
buffering is like network delay, it cannot cause new deadlocks.
Buffering that depends on other buffer state may cause new deadlocks.

> the application, to resolve the deadlock.

For a baseline server algorithm, I would let broken clients and broken
applications to resolve their problems without server participation.
The server has to buffer request headers to set environmental
variables and such. After request headers are processed, the server
establishes a tunnel for the application to talk to the client and
just does chunk coding and/or connection termination management.

> > Imagine an application that writes more than you can buffer before
> > it reads anything.
>

With a simple tunneling server it will not block unless the client
does not read. Again, it becomes a client-application problem which is
what you want, as a server author, in a general case.

> is limited to an appropriate I/O block size.

I assume that request bodies should not be limited to your maximum
buffer size.

> server may as well constrain the app, probably by complaining at it.

And then you will find a customer who deploys your server in a
controlled environment where clients and applications need an
HTTP-compliant tunnel and your server breaks it...

> I'd like to know whether it a common or rare client weakness, so
> whether I should consider using that technique or not.

I do not know because we have not tested clients yet. If you have
insufficient information, I would suggest to allow possibly compliant
behavior by default. You can always implement complex
optimizations/restrictions later.

$0.02,

Alex.