Subject: WGLC p1: Tear-down

From: Ben Niven-Jenkins <[email-masked]>
Date: Mon, 29 Apr 2013 20:33:41 +0100

Section 6.6 of p1 states:

   A server that sends a close connection option MUST initiate a
   lingering close of the connection after it sends the response
   containing close.  The server MUST NOT process any further requests
   received on that connection.

   A client that receives a close connection option MUST cease sending
   requests on that connection and close the connection after reading
   the response message containing the close; if additional pipelined
   requests had been sent on the connection, the client SHOULD assume
   that they will not be processed by the server.

The last sentence can be interpreted one of two ways:
1) The client SHOULD assume the additional pipelined requests will NOT be processed by the server and therefore can happily re-try them knowing the server has not processed the previous ones.

2) The client SHOULD NOT assume the additional pipelined requests will be processed (which implies the client simply can not know whether the server has processed them or not).

As the client has no way of knowing whether the server may have processed them or not (e.g. the client may be talking to a proxy that has already relayed the pipelined requests to the origin and done so before the proxy was aware that it wanted to close the connection on this response) I would suggest rewording the last sentence quoted above:

OLD:
   the client SHOULD assume that they will not be processed by the server.
NEW:
   the client SHOULD NOT assume that they will be processed by the server.

Thanks
Ben

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 29 Apr 2013 21:37:44 +0200

>    the client SHOULD NOT assume that they will be processed by the server.

+1 that's perfectly true.

Willy

---

From: Zhong Yu <[email-masked]>
Date: Mon, 29 Apr 2013 16:49:36 -0500

On Mon, Apr 29, 2013 at 2:33 PM, Ben Niven-Jenkins
<[email-masked]>wrote:

> 1) The client SHOULD assume the additional pipelined requests will NOT be
> processed by the server and therefore can happily re-try them knowing the
> server has not processed the previous ones.
>
> 2) The client SHOULD NOT assume the additional pipelined requests will be
> processed (which implies the client simply can not know whether the server
> has processed them or not).
>
> As the client has no way of knowing whether the server may have processed
> them or not (e.g. the client may be talking to a proxy that has already
> relayed the pipelined requests to the origin and done so before the proxy
> was aware that it wanted to close the connection on this response) I would
> suggest rewording the last sentence quoted above:
>
> OLD:
Agreed; an origin server may also process pipelined requests concurrently,
so request#2 may have been processed when response#1 causes
Connection:close.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Tue, 30 Apr 2013 02:38:55 +0000

------ Original Message ------
From: "Zhong Yu" <[email-masked]>
><[email-masked]> wrote:
>Agreed; an origin server may also process pipelined requests 
>concurrently, so request#2 may have been processed when response#1 
>causes Connection:close.

Given that it's really hard for a client to predict the future, as to 
whether a pipelined request will be processed or not, or whether a 
previous resource will return Connection: close or not, and even to tell 
whether a request will be truly without side-effects, doesn't this mean 
it's just basically dangerous to pipeline full stop?

Do we need a way for a server to communicate which requests may be made 
with impunity multiple times, and which should only be made once?  e.g. 
safe to retry or not.  then only pipeline requests that are safe to 
retry according to the server (rather than according to some assumption 
or heuristic at the client, as such things are inevitably wrong on 
occasion).

Adrien

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 30 Apr 2013 12:48:10 +1000

On 30/04/2013, at 12:38 PM, Adrien W. de Croy <[email-masked]> wrote:

>> On Mon, Apr 29, 2013 at 2:33 PM, Ben Niven-Jenkins <[email-masked]> wrote:
>> Agreed; an origin server may also process pipelined requests concurrently, so request#2 may have been processed when response#1 causes Connection:close.
>  
> Given that it's really hard for a client to predict the future, as to whether a pipelined request will be processed or not, or whether a previous resource will return Connection: close or not, and even to tell whether a request will be truly without side-effects, doesn't this mean it's just basically dangerous to pipeline full stop?
>  
> Do we need a way for a server to communicate which requests may be made with impunity multiple times, and which should only be made once?  e.g. safe to retry or not.  then only pipeline requests that are safe to retry according to the server (rather than according to some assumption or heuristic at the client, as such things are inevitably wrong on occasion).

That's built into the method of the request...

> A user agent should not pipeline requests after a non-idempotent method until the final response status code for that method has been received, unless the user agent has a means to detect and recover from partial failure conditions involving the pipelined sequence.

--
Mark Nottingham   http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 30 Apr 2013 12:49:31 +1000

Now <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/477> (editorial).

Thanks,

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Tue, 30 Apr 2013 02:52:49 +0000

------ Original Message ------
From: "Mark Nottingham" <[email-masked]>
>>previous resource will return Connection: close or not, and even to 
>>tell whether a request will be truly without side-effects, doesn't 
>>this mean it's just basically dangerous to pipeline full stop?
>>
>>  Do we need a way for a server to communicate which requests may be 
>>made with impunity multiple times, and which should only be made once? 
>>e.g. safe to retry or not. then only pipeline requests that are safe 
>>to retry according to the server (rather than according to some 
>>assumption or heuristic at the client, as such things are inevitably 
>>wrong on occasion).
>
that's what I meant by assume.

UA authors might assume GET is idempotent.  It doesn't stop web 
developers from writing sites that have significant side-effects on GET. 
  Getting these people to indicate safety of retrying is another problem. 
  I guess this is one reason why pipelining isn't that widespread yet.  
Lots of problems with it.

Regards

Adrien

>
>>  A user agent should not pipeline requests after a non-idempotent 
>>method until the final response status code for that method has been 
>>received, unless the user agent has a means to detect and recover from 
>>partial failure conditions involving the pipelined sequence.
>
>--

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 30 Apr 2013 08:12:54 +0200

Hi Adrien,

On Tue, Apr 30, 2013 at 02:52:49AM +0000, Adrien W. de Croy wrote:
> UA authors might assume GET is idempotent.

UAs are the most well placed to know where they get the information they
send. I suspect that when the send a form using GET they don't trust
idempotence. However if a link has an embedded query-string, maybe they
consider the request idempotent as it's present in a link.

> It doesn't stop web 

We'll always get such things from clueless people, but that's also the
goal of the spec to insist on the risks of not respecting the standard.
If it's clearly written that GET/HEAD/PUT/DELETE are idempotent and that
browsers will consider this statement as true, then web developers will
have some guidance about the risk of doing stupid things.

> Getting these people to indicate safety of retrying is another problem. 

If they already use the wrong method and don't understand idempotence,
we can't expect them to advertise it correctly.

> Lots of problems with it.

No, it's really because many intermediaries and servers have had issues
with it, causing such requests to frequently stall or be dropped. It's
not always easy to get right, despite appearing obvious initially. I
recently managed to break it in haproxy without noticing until a user
reported some abnormal errors. Just to give you a rought idea, I believe
the issue was not caused by the request itself but by lack of space in
the response buffer when haproxy had to emit a redirect based on the
second request. Thus it forgot to wait for free space in the *response*
buffer to start to parse the *request* buffer... So it's easier to break
than to maintain in good shape.

Clearly, pipelining opens a new class of bugs, but there is no excuse
for not fixing them. If the spec provides some guidance on this, we'll
manage to slowly fix the web.

Regards,
Willy