Subject: RST_STREAM(OK) after an HTTP response

From: Jeff Pinner <[email-masked]>
Date: Wed, 24 Sep 2014 08:05:26 -0700

A server may respond to a request without completely reading the
message body. As an example, consider returning 403 Forbidden to a
large POST request.

In HTTP/2, the server would send a HEADERS frame containing the 403
response with the END_STREAM flag set, leaving the stream in the
half-closed (local) state.

My intuition is to follow up the response with a RST_STREAM (OK) in
order to close the stream entirely if I want to discard the body, but
also want to make sure that the client does not treat this as an error
and considers the response as valid.

The RST_STREAM text state that the frame is for "abnormal termination"
of a stream and that as a receiver it is to indicate rejection,
cancellation, or an error.

I propose that the text around RST_STREAM be clarified that from
something like "abnormal termination" to "abrupt termination" and that
this case be called out as an example in the HTTP mapping section to
make sure implementors treat the HTTP response as valid.

I opened issue https://github.com/http2/http2-spec/issues/618 on github.

---

From: Mike Bishop <[email-masked]>
Date: Wed, 24 Sep 2014 16:05:28 +0000

While I understand the scenario, it seems more appropriate for the client to send the RST_STREAM(OK) if it decides to abort sending the rest of the body.

-----Original Message-----
From: Jeff Pinner [mailto:[email-masked]] 
Sent: Wednesday, September 24, 2014 8:05 AM
To: HTTP Working Group
Subject: RST_STREAM(OK) after an HTTP response

A server may respond to a request without completely reading the message body. As an example, consider returning 403 Forbidden to a large POST request.

In HTTP/2, the server would send a HEADERS frame containing the 403 response with the END_STREAM flag set, leaving the stream in the half-closed (local) state.

My intuition is to follow up the response with a RST_STREAM (OK) in order to close the stream entirely if I want to discard the body, but also want to make sure that the client does not treat this as an error and considers the response as valid.

The RST_STREAM text state that the frame is for "abnormal termination"
of a stream and that as a receiver it is to indicate rejection, cancellation, or an error.

I propose that the text around RST_STREAM be clarified that from something like "abnormal termination" to "abrupt termination" and that this case be called out as an example in the HTTP mapping section to make sure implementors treat the HTTP response as valid.

---

From: Jeff Pinner <[email-masked]>
Date: Wed, 24 Sep 2014 09:11:39 -0700

> While I understand the scenario, it seems more appropriate for the client to send the RST_STREAM(OK) if it decides to abort sending the rest of the body.

I think that should be valid as well. Basically I think either should
be able to tear down the stream via RST_STREAM(OK) without the other
considering it an error, and that the text should be clarified to make
it explicit.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 24 Sep 2014 18:29:18 +0100

I’ve heard at least one other person bring up this scenario recently (forget who), FWIW. We should clarify, I think.

--
Mark Nottingham   http://www.mnot.net/

---

From: Michaela LaVan <[email-masked]>
Date: Wed, 24 Sep 2014 17:24:07 -0400

This use case has come up for us recently as well. I like Jeff's suggestion
both the client and server should be explicitly free to send a RST_STREAM
at any time without it necessarily indicating an error.

On Wed, Sep 24, 2014 at 1:29 PM, Mark Nottingham <[email-masked]> wrote:

> I’ve heard at least one other person bring up this scenario recently
> (forget who), FWIW. We should clarify, I think.
>
>
> >> While I understand the scenario, it seems more appropriate for the
> client to send the RST_STREAM(OK) if it decides to abort sending the rest
> of the body.
> >

---

From: Osama Mazahir <[email-masked]>
Date: Wed, 24 Sep 2014 23:28:24 +0000

Just to clarify, we can reword the text from “The RST_STREAM frame (type=0x3) allows for abnormal termination of a stream” to “The RST_STREAM frame (type=0x3) allows for immediate unconditional termination of a stream” (or “abrupt”).

But that the validity of the bodies in-flight (or buffered on either send/recv) is the same as the other reset codes.  That is, all bets are off and I expect most stacks to implement immediate stream termination by dropping any buffered data on the floor.  I do not want us to get into a situation where bodies/requests/responses are expected to be valid, be it partial or ‘complete’.

To me it appears we could just use CANCEL; the text already indicates that it used deliberately by an endpoint when it is no longer interested.  If we want to add a new code then please don’t confuse it by hinting at ‘success’ and instead use something along the lines of “deliberate”, “expected”, “terminate”, “close”

From: Michaela LaVan [mailto:[email-masked]]
Sent: Wednesday, September 24, 2014 2:24 PM
To: Mark Nottingham
Cc: Jeff Pinner; Mike Bishop; HTTP Working Group
Subject: Re: RST_STREAM(OK) after an HTTP response

This use case has come up for us recently as well. I like Jeff's suggestion both the client and server should be explicitly free to send a RST_STREAM at any time without it necessarily indicating an error.

On Wed, Sep 24, 2014 at 1:29 PM, Mark Nottingham <[email-masked]<mailto:[email-masked]>> wrote:
I’ve heard at least one other person bring up this scenario recently (forget who), FWIW. We should clarify, I think.

---

From: Jeff Pinner <[email-masked]>
Date: Wed, 24 Sep 2014 16:41:09 -0700

> But that the validity of the bodies in-flight (or buffered on either
> send/recv) is the same as the other reset codes.  That is, all bets are off
> and I expect most stacks to implement immediate stream termination by
> dropping any buffered data on the floor.  I do not want us to get into a
> situation where bodies/requests/responses are expected to be valid, be it
> partial or ‘complete’.

The point is that the received request be considered valid. Just as if
I sent a response to a 1.1 client and then closed the connection (even
if they were trying to send me data).

> To me it appears we could just use CANCEL; the text already indicates that
> it used deliberately by an endpoint when it is no longer interested.  If we
> want to add a new code then please don’t confuse it by hinting at ‘success’
> and instead use something along the lines of “deliberate”, “expected”,
> “terminate”, “close”

To me CANCEL implies I initiated a request and don't care about it
anymore. Here the request isn't being cancelled, it's being replied
to.

Furthermore, the server shouldn't have to wait for the client to "do
the right thing" and stop sending data and close the stream.

---

From: Jeff Pinner <[email-masked]>
Date: Wed, 24 Sep 2014 16:52:37 -0700

> if they were trying to send me data).

 * received response -- i need to proof read better :)

---

From: Martin Thomson <[email-masked]>
Date: Thu, 25 Sep 2014 00:17:22 -0700

> situation where bodies/requests/responses are expected to be valid, be it

There is another potential way this could work.  There are two ways a
client might consider a request "done", either the presence of
END_STREAM on the response side of the stream OR the transition of the
stream into the "closed" state.  This latter option that really causes
issues.

Then there is the concern above, where the RST_STREAM acts as an
interrupt and a trigger for the request being destroyed.  This seems
far more likely to me, partially because it's basically an essential
part of how you implement RST_STREAM handling anyway.

If people are generally OK with a clarification, either because their
implementation handles this gracefully, or because they are willing to
make the necessary changes, then I think we should make the change.
It seems like a valuable feature.  But I don't want to be saying
things about this if it is going to cause implementations considerable
pain.

Would the clarification Jeff requests break your implementation?  (And
are you unable or unwilling to make a change?)

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 26 Sep 2014 06:26:25 +1000

I'm good with the clarification.

But just a reminder that it should still be OK for a server to send an
early response and still consume the request body.

cheers

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Matthew Cox <[email-masked]>
Date: Thu, 25 Sep 2014 20:52:42 +0000

We’ve been discussing this internally, and we are unsure about what this actually means for the protocol.

Is there a pull request for this that we can review?

Thanks,

Matthew

From: Greg Wilkins [mailto:[email-masked]]
Sent: Thursday, September 25, 2014 1:26 PM
To: Martin Thomson
Cc: Osama Mazahir; Michaela LaVan; Mark Nottingham; Jeff Pinner; Mike Bishop; HTTP Working Group
Subject: Re: RST_STREAM(OK) after an HTTP response

I'm good with the clarification.
But just a reminder that it should still be OK for a server to send an early response and still consume the request body.
cheers

On 25 September 2014 17:17, Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

--
Greg Wilkins <[email-masked]<mailto:[email-masked]>>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 26 Sep 2014 07:53:01 +0100

> Is there a pull request for this that we can review?

Not yet, no.

The text would include a small tweak to RST_STREAM
(s/abnormal/abrupt/ as Jeff suggests, or something similar).

Then the HTTP mapping would note that a response should not be
considered failed if a RST_STREAM is received by a client in the
"half-closed (remote)" state, because the response would have been
completely received.

---

From: Matthew Cox <[email-masked]>
Date: Fri, 26 Sep 2014 17:09:05 +0000

So you are saying the request-side has failed, but the response-side hasn't?

This is what I would push back on.  Any buffered data at the time of receiving RST_STREAM should be allowed to be dropped on the floor.  There should be no guarantee that it will flow up the stack if the other side sends a RST_STREAM.

Even if I did allow the buffered data to stay, I'd have to fail the send side up the stack, and then the whole stack would have to know that receives are still valid even during a send failure.

To me the problem Jeff presented is an application layer problem.  In his example the termination of the request is based off of the receipt of a response, but since the application doesn't adhere to that, he wants to force it by resetting the stream.  What I think he should do is what most 1.1 stacks are doing now which is have a heuristic on how long you are willing to let the client keep sending data after you already told it that you weren't interested, and once it passes some threshold you reset it dropping everything on the floor.

Matthew

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Thursday, September 25, 2014 11:53 PM
To: Matthew Cox
Cc: Greg Wilkins; Osama Mazahir; Michaela LaVan; Mark Nottingham; Jeff Pinner; Mike Bishop; HTTP Working Group
Subject: Re: RST_STREAM(OK) after an HTTP response

Not yet, no.

The text would include a small tweak to RST_STREAM (s/abnormal/abrupt/ as Jeff suggests, or something similar).

Then the HTTP mapping would note that a response should not be considered failed if a RST_STREAM is received by a client in the "half-closed (remote)" state, because the response would have been completely received.

---

From: Jeff Pinner <[email-masked]>
Date: Fri, 26 Sep 2014 11:00:27 -0700

> This is what I would push back on.  Any buffered data at the time of receiving RST_STREAM should be allowed to be dropped on the floor.  There should be no guarantee that it will flow up the stack…

Why? This isn't TCP where you can receive a RST and have data buffered
in the receive buffer. By the time you receive the RST_STREAM you have
already read the END_STREAM flag. The data is received by your HTTP/2
layer in order.

> Even if I did allow the buffered data to stay, I'd have to fail the send side up the stack, and then the whole stack would have to know that receives are still valid even during a send failure.

Isn't this true with HTTP/1.1 -- if I send you a response before
you've finished sending request data your stack has to know that that
response is valid.

> To me the problem Jeff presented is an application layer problem.  In his example the termination of the request is based off of the receipt of a response, but since the application doesn't adhere…

A "time-wait" heuristic isn't very helpful for interop. A server could
have the valid time period be 1/2 RTT after it sends the reset frame
and you still have the same issue. The difference with HTTP/1.1 is the
server can signal it with a Connection: close header in the response
and tear down the connection. With HTTP/2, tearing down the connection
may be expensive if there are many streams over it, or impossible if
it's a proxied connection.

---

From: Osama Mazahir <[email-masked]>
Date: Fri, 26 Sep 2014 23:16:02 +0000

Doing this RST_STREAM_OK does not obviate heuristics.  The peer (client) could have already queued a large amount of DATA into TCP.  

>>The difference with HTTP/1.1 is the server can signal it with a Connection: close header in the response and tear down the connection.

If, after emitting the response, the server is no longer interested in the remainder of the request body then it can drain it (e.g. recv and dump) and do a graceful mutual HTTP/1.1 FIN exchange (or HTTP/2 END_STREAM exchange).  That is what servers have been doing.  If some threshold is exceeded (timeout, slowness, bytecount, etc) the server can take harsher reaction (HTTP/1.1 RST or HTTP/2 RST_STREAM).  When the body is large enough for it to matter, some clients will start with a 100-continue to avoid getting into the drain situation (others may send a canary request).  Client-servers applications that have intimate knowledge about each other and are using HTTP as a substrate could use chunked-encoding (over TLS) and 0-chunk-terminate (or really anything).

As an implementer of a general-purpose HTTP implementation, I understand the motivation.  But hacking RST_STREAM semantics is not good design.

What is desired here is to emit some signal that indicates: "hey...you don't need to bother sending me any more body because it's irrelevant".

This could be done using a bit flag ("TERMINAL") in the HEADERS frame (or make it a separate "TERMINAL" frame): A server MAY emit TERMINAL to indicate that it does not need the request body.  A TERMINAL flag MUST NOT be sent by a client.  A TERMINAL flag MUST NOT be sent in any state except half-closed-local. Upon receive, a client MAY prematurely END_STREAM the body it is sending.

A sophisticated client can use TERMINAL flag receipt to stop sending more DATA and instead END_STREAM to bring things to a graceful close.  Assuming the client is not already past the point-of-no-return.  An HTTP/1.1 to HTTP/2.0 intermediary can switch the parser mode on the HTTP/1.1 side.

It doesn’t even have to be a flag; it could be a new frame-type or an extension frame.  It's really an optimization for clients-servers that are not using any of the standard approaches.

-----Original Message-----
From: Jeff Pinner [mailto:[email-masked]] 
Sent: Friday, September 26, 2014 11:00 AM
To: Matthew Cox
Cc: Martin Thomson; Greg Wilkins; Osama Mazahir; Michaela LaVan; Mark Nottingham; Mike Bishop; HTTP Working Group
Subject: Re: RST_STREAM(OK) after an HTTP response

> This is what I would push back on.  Any buffered data at the time of receiving RST_STREAM should be allowed to be dropped on the floor.  There should be no guarantee that it will flow up the stack…

Why? This isn't TCP where you can receive a RST and have data buffered in the receive buffer. By the time you receive the RST_STREAM you have already read the END_STREAM flag. The data is received by your HTTP/2 layer in order.

> Even if I did allow the buffered data to stay, I'd have to fail the send side up the stack, and then the whole stack would have to know that receives are still valid even during a send failure.

Isn't this true with HTTP/1.1 -- if I send you a response before you've finished sending request data your stack has to know that that response is valid.

> To me the problem Jeff presented is an application layer problem.  In his example the termination of the request is based off of the receipt of a response, but since the application doesn't adhere…

A "time-wait" heuristic isn't very helpful for interop. A server could have the valid time period be 1/2 RTT after it sends the reset frame and you still have the same issue. The difference with HTTP/1.1 is the server can signal it with a Connection: close header in the response and tear down the connection. With HTTP/2, tearing down the connection may be expensive if there are many streams over it, or impossible if it's a proxied connection.

---

From: Martin Thomson <[email-masked]>
Date: Sat, 27 Sep 2014 11:24:39 +0100

> What is desired here is to emit some signal that indicates: "hey...you don't need to bother sending me any more body because it's irrelevant".

There is an alternative: recommend that clients reset their own
request streams once the response is received.

If the intent is to terminate the transmission of excess bytes of a
request, then this achieves that goal.  The disadvantage being that
there is less determinism from the server perspective.  The server
still has its flow control window, I guess.

I don't see any value in a new mechanism to solve this problem.

---

From: Jim Manico <[email-masked]>
Date: Sat, 27 Sep 2014 09:13:11 -0700

Isn't this helpful for "ajaxy" applications that only need to render 
part of the page based on UI dimensions? Only the client will reliably 
know the UI screen size in order to make that decision. So asking to 
terminating the body early via client request seems reasonable. Or even 
better, a request like "please only give me the second 1/2 of the body" 
also seem prudent for this workflow.

Is this the right use case in the first place?

- Jim

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 28 Sep 2014 13:28:17 +1000

> request streams once the response is received.

This would break existing applications.   It is perfectly legal HTTP to
send a response and then consume the request content.   Clients should not
reset the stream after the response is read as the server may still wish to
consume the body being sent.

Sure, in many/most use-cases it is entirely premature for a server to send
a response before consuming the body, but there are use-cases where it does
make sense... or at least where it is currently done and expected to keep
working.

The only party that knows if the request body is required or not is the
server.   It's perfectly able to send a reset now, and I think the only
real issue is that the spec says that resets are "abnormal termination".
Renaming that to an "immediate termination" and using the existing NO_ERROR
code would cover this case.

cheers