Subject: Re: #535: No 1xx Status Codes

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 15:39:23 +1000

On 27 Jun 2014, at 7:20 pm, Julian Reschke <[email-masked]> wrote:

> On 2014-06-27 09:56, Mark Nottingham wrote:
>> <https://github.com/http2/http2-spec/issues/535>
>> 
>> This seems like a re-opening of <https://github.com/http2/http2-spec/issues/264>. We discussed it a fair amount in the Seattle interim, and there was pretty strong support in the room for getting rid of 1xx status, especially since they're poorly supported in implementations, almost non-existant in APIs, and often don't survive hop-to-hop.
>> 
>> Julian, anything to add? I'm inclined to close this as a duplicate unless there's significant new information...
> 
> I still fail to see a compelling reason to remove them.
> 
> Why do we keep trailers, but not 1xx? I'd like to understand how we draw the line.

My .02 - trailers work in a way where they may not get used much, but some people still find them useful, and they don't cause significant issues. 

1xx, OTOH, has a track record of causing considerable havoc, and as has been pointed out many times, its semantics are better expressed in the framing layer.

That said, it's very much a judgement call. When we made that decision, we discussed it both in an interim and on the list:
  http://www.w3.org/mid/[email-masked]
... and there was considerable support for -- and no pushback against -- doing it. 

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 02 Jul 2014 07:58:31 +0200

> 1xx, OTOH, has a track record of causing considerable havoc, and as has been pointed out many times, its semantics are better expressed in the framing layer.

The status code *class* hasn't caused any havoc at all (unless I'm 
missing something), and expressing the semantics in the framing layer 
implies you need two different ways to express the semantics for 1.1 and 2.

> ...

While some people were busy getting six RFCs through IESG review :-)

I'll also cite:

> The feeling in the room in Seattle AIUI was that non-final status codes are a signalling mechanism that is no longer necessary in HTTP/2, since the framing layer can do a better job at these tasks. As such, we do not plan to accommodate non-final status codes (1xx) in HTTP/2.0, but instead will give advice about how they can be handled by intermediaries doing transitions between 1.x and 2.x (with one example above).

That advice isn't there. What can an intermediary do with a new 1xx 
status code other than dropping it?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 16:02:18 +1000

On 2 Jul 2014, at 3:58 pm, Julian Reschke <[email-masked]> wrote:

> That advice isn't there. What can an intermediary do with a new 1xx status code other than dropping it?

That was referring to the specific cases that are in use, not the class.

Can you compose a pull request outlining what you'd like to see, so we can get a sense of how intrusive it would be, and how likely it is that it'd be supported?

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Wed, 02 Jul 2014 08:25:16 +0200

> Can you compose a pull request outlining what you'd like to see, so we can get a sense of how intrusive it would be, and how likely it is that it'd be supported?

I can give it a try in the next few days. Before I start, can we do a 
sanity check on this example?

> A non-final response using a 1xx status code other than 100 or 101 is transmitted as a HEADERS frame, followed by zero or more CONTINUATION frames:
>
>   HTTP/1.1 103 BAR                 HEADERS
>   Extension-Field: bar       ==>     - END_STREAM
>                                      + END_HEADERS
>                                        :status = 103
>                                        extension-field = bar

So we'd just send multiple header frames except one.

If this makes sense, the other change would need to be applied to 8.1:

> An HTTP message (request or response) consists of:
>
>     1. one HEADERS frame (followed by zero or more CONTINUATION frames) containing the message headers (see [RFC7230], Section 3.2), and
>     2. zero or more DATA frames containing the message payload (see [RFC7230], Section 3.3), and
>     3. optionally, one HEADERS frame, followed by zero or more CONTINUATION frames containing the trailer-part, if present (see [RFC7230], Section 4.1.2).

to say:

"0. zero or more HEADERS frames (followed by zero or more CONTINUATION 
frames) containing the message headers of non-final (1xx) HTTP responses 
(see [RFC7230], Section 3.2 and 6.2), and"

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 16:30:21 +1000

On 2 Jul 2014, at 4:25 pm, Julian Reschke <[email-masked]> wrote:

> I can give it a try in the next few days. Before I start, can we do a sanity check on this example?
> 
> "0. zero or more HEADERS frames (followed by zero or more CONTINUATION frames) containing the message headers of non-final (1xx) HTTP responses (see [RFC7230], Section 3.2 and 6.2), and"

As long as that's contained to just responses, I think so.

An alternative resolution would be to mint a new, short consensus document that disallows minting new 1xx status codes -- due to the fact that they're not well-supported in implementations nor APIs, nor intermediaries (IIRC).

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Wed, 02 Jul 2014 08:40:51 +0200

> An alternative resolution would be to mint a new, short consensus document that disallows minting new 1xx status codes -- due to the fact that they're not well-supported in implementations nor APIs…

Do you have evidence of implementations that break on new 1xx codes?

I agree that APIs are a problem, but we have the same problem for 
trailers (which we keep), and anything that's new in the HTTP/2 world 
will need new APIs as well, so I'm not sure how that is a compelling 
argument.

Best regards, Julian

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Jul 2014 11:11:45 +0200

On 2 July 2014 08:30, Mark Nottingham <[email-masked]> wrote:

> they're not well-supported in implementations nor APIs, nor intermediaries
> (IIRC).

The Jetty server has long supported 100 continues and we believe that we
have good mechanism that does not depend on explicit API support.

Specifically we delay sending the 100 continues response until the Servlet
calls the Request.getInputStream() method (or any other method that
consumes the content).     A Servlet is then free to handle the request by
inspecting the URI and headers, and if it decides that it does not wish to
consume the input, then a 100 is never sent.  The Servlet can send an
arbitrary response, but the connection is always shutdown afterward in such
cases.     If the servlet does decide to consume the input, then the 100 is
sent when the input stream is obtained and the servlet typically blocks
until the content arrives (although we now have async IO in the API to
avoid this blocking if desired).

We have never had any problems reported with this mechanism, but before we
implemented it we had frequent complaints that it was not supported,
specially by SOAP and REST frameworks. These are from non-browser clients,
so I expect that they will wish to have similar semantics available in
http/2

We also have had some usage of 102 responses, which we implement by bending
the meaning of sendError in the servlet API.    I don't think that not
support 102 will cause widespread problems, but if we do support 100, then
102 support is probably good to at least ensure the mechanism is general
for all 1xx responses.

cheers

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 15 Jul 2014 18:13:05 +1000

On 2 Jul 2014, at 4:40 pm, Julian Reschke <[email-masked]> wrote:

> I agree that APIs are a problem, but we have the same problem for trailers (which we keep), and anything that's new in the HTTP/2 world will need new APIs as well, so I'm not sure how that is a compelling argument.

I think it's a decision that needs to be weighed on its merits; there is no single rule that we can apply to make all decisions (as much as I would enjoy automating the WG). 

My .02 -

Status codes have semantic "heft" in the protocol, and if an entire class of them is both poorly supported and has poor interest, it's not unreasonable to consider deprecating that class of status codes. If someone tries to use it in the future, they're going to be disappointed, because it isn't supported in software and APIs; that semantic won't be useful to express or expect (as was found with 102).

Persisting 1xx status codes has a real cost; besides the obvious cost of requiring HTTP/2 implementations to code for them, test them, and support them, there's also the cost of handling the interoperability problems when they don't work. 

OTOH the benefit of keeping them is quite nebulous; the two 1xx status codes originally defined ended up being a lot of trouble (100) and unused (101), and the only further one defined (102) turned out to be deprecated because of lack of support. 

So, my take is that making HTTP/2 more complex to satisfy use cases that haven't materialised in almost 20 years doesn't seem like a good tradeoff.

Re: Trailers - I don't think that we should use them as an exemplar for the features we put into HTTP/2; they barely made it in, and they are explicitly ignorable in HTTP/1. Personally, the argument you're making drives me to want to remove trailers, not add 1xx.

How do other folks feel about this? We can talk about it in Toronto, of course.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 15 Jul 2014 08:26:10 +0000

In message <[email-masked]>, Mark Nottingham writes:

>Re: Trailers - I don't think that we should use them as an exemplar for =
>the features we put into HTTP/2; they barely made it in, and they are =
>explicitly ignorable in HTTP/1. Personally, the argument you're making =
>drives me to want to remove trailers, not add 1xx.

The only active usecase for 1xx I have ever seen is 100-Continue, and that
functionality is already available using the windows in HTTP/2.0.

+1 for no 1xx.

At one point I surveyed trailer support and found that it was not support
at a level where one could even hope to spur further support.

Architecturally, trailers suffer from the exact same problem as CONTINUE,
pushing buffering costs from transmitters to receivers.

This is a bad idea because receivers suffer much higher traffic
densities than transmitters and they need to defend themselves
against DoS attacks.

Trailers, like CONTINUATION makes receiver-buffering available as a DoS tactic.

+1 for removing trailers.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Jul 2014 10:35:32 +0200

> Status codes have semantic "heft" in the protocol, and if an entire class of them is both poorly supported and has poor interest, it's not unreasonable to consider deprecating that class of status…

But we are not deprecating them, we are *removing* them.

> Persisting 1xx status codes has a real cost; besides the obvious cost of requiring HTTP/2 implementations to code for them, test them, and support them, there's also the cost of handling the intero…

HTTP/1.1<->2 converters need to code for them anyway: in one case 
they'll drop them (requires detecting them), in the other case they 
forward them.

HTTP stacks already deal with 1xx in some way (*), and not having them 
in 2.0 doesn't mean support in 1.1 can be dropped.

(*) And yes, there are bugs. What we need to do here is to have tests, 
raise bugs, etc. I already started with that.

> How do other folks feel about this? We can talk about it in Toronto, of course.

I agree that Trailers and 1xx are very similar. That being said, I'd 
rather get Trailer support fixed instead of them going away.

In general, while implementing HTTP/2, tons of code will need to be 
touched anyway; we should consider that an opportunity to also address 
1.1 related bugs.

Best regards, Julian

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 15 Jul 2014 18:58:47 +1000

Mark,

I'd like to see 100 semantic supported in h2, but it does not have to be
exactly by sending a :status:100.   I'm fine with dropping general 1xx
support.

I think there will be many real applications that just will not work over
h1-h2  proxies without 100 support.

I'm not a big fan of trailers and will ignore them in Jetty because there
is no way to plumb them into the servlet API.    However, I think there is
some value in leaving them (and mid stream meta data) in h2, as I think it
increases the chance that later extensions can be developed that will use
them and pass through intermediaries (but the frame should be called
METADATA rather than HEADERS), but this is very marginal.

regards

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 15 Jul 2014 09:18:49 +0000

In message <[email-masked]>, Greg Wilkins wri
tes:

>I'd like to see 100 semantic supported in h2, 

But isn't it already supported via the per-stream windows ?

It's true that the sender can fill the initial per stream window
before the receiver has a chance to decide if it wants the body
or not, but I've never seen the 100-Continue as a mechanism to
avoid sending a few unwanted tens of kilobyts, but rather to avoid
sending unwanted mega- and gigabytes ?

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 15 Jul 2014 11:29:25 +0200

> sending unwanted mega- and gigabytes ?

It's also used a lot by webservices and other clients relying on
connection pools to ensure that a connection is still alive before
sending a non-idempotent request over it. I don't see what benefit
it could provide in HTTP/2 however since we can abort an upload in
progress.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 15 Jul 2014 11:43:40 +0200

Hi Julian,

On Tue, Jul 15, 2014 at 10:35:32AM +0200, Julian Reschke wrote:
> >Status codes have semantic "heft" in the protocol, and if an entire class 
> >of them is both poorly supported and has poor interest, it's not 
> >unreasonable to consider deprecating that class of status codes. If 
> >someone tries to use it in the future, they're going to be disappointed, 
> >because it isn't supported in software and APIs; that semantic won't be 
> >useful to express or expect (as was found with 102).
> 
> >Persisting 1xx status codes has a real cost; besides the obvious cost of 
> >requiring HTTP/2 implementations to code for them, test them, and support 
> >them, there's also the cost of handling the interoperability problems when 
> >they don't work.
> 

I could be wrong, but the only possible issues I'm seeing with them removed
are :

  - H1->H2 gateways : have to respond 100-continue when getting an Expect
    header field, possibly to later break the connection if the server aborts
    the stream. I don't think it's a huge issue, it's similar to dealing with
    302 on POSTs in my opinion (not easy either but necessary anyway).

  - H2->H1 gateways : may benefit from sending Expect in outgoing POST requests
    or may have to do it over persistent connections, and to strip any 100
    response. This is a difficulty which already has to be dealt with in pure
    1.1.

I'm still seeing use cases for 101. If my H1->H2 gateway receives a WS
Upgrade request, I'm not sure how to pass it to local servers. It can be
worse for explicit proxies which will have to pass this to the rest of the
world. Maybe they'll have to pass them in 1.1, or maybe some will try to
emulate the CONNECT mode in H2, I don't know.

I remember you expressed some desire for 102, I don't know if that cannot
be done with some informational frames later. Or maybe this is the only
thing we'd need : send informational frames in either direction ?

> rather get Trailer support fixed instead of them going away.

Maybe one day it will be used to transport checksums, but I doubt it given
that I've yet to see a trailer somewhere :-/

> touched anyway; we should consider that an opportunity to also address 

I agree on this point. I'd formulate it differently though : opportunity
not to reproduce H1 mistakes :-)

Best regards,
Willy

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 16 Jul 2014 00:06:36 +1200

>     302 on POSTs in my opinion (not easy either but necessary anyway).

I have not checked scenarios, but I suspect that these gateways can emit
the 100-continue on a WINDOW_UPDATE from the server end of their h2
connection.

>     response. This is a difficulty which already has to be dealt with in pure

Indeed. This is probably a good recommendation for a SHOULD on emitting
Expect on the 1.1 connection, then emit a WINDOW_UPDATE down the h2
connection on recipt of the 100-continue or the Expect timeout.

If the above h1->h1 is correct this method will allow both types of
gateway to work in-sync across any number of hops of either type.

> that I've yet to see a trailer somewhere :-/

I'm told there are a few of our Squid clients using them already. But no
confirmation of that.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 15 Jul 2014 23:30:10 +1000

> sending unwanted mega- and gigabytes ?

The use case I've most often seen it used in is with REST/SOAP clients that
want to check that their credentials are correct or if they are talking to
the correct server before sending any payload.

Here is Amazon S3 talking about it:
http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTRedirect.html#RESTRedirect100Continue

In their use-case up to 64KB of data could be sent that then has to be
resent.  As such architectures often to many redirects, this could be an
issue.

Note also that if the client has started sending the request body, then it
is going to be harder for it to be stopped and redirected.  It may end up
serialising a lot more data and then having to do it again.

I think there is a tiny bit of a security issue as well, as sometimes a
request is redirected to a secure port, so 100 thus prevents the client
from sending data over an insecure connection.  (I'm not saying it is good
security, but I have definitely heard of it being relied on!)

Another aspect of this is that many REST/SOAP clients are not browsers, so
they will not be able to be upgraded easily.    So h1 to h2 proxies are
going to have to handle 100 continues correctly if these clients are to
work.    To well support the semantic, you need the application itself to
be able to inspect the request headers and then signal that it either needs
the request body or provides an alternate response.

So I really think this is a not uncommon http1 semantic that if we drop is
against our charter.

cheers

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Jul 2014 10:26:25 -0700

> +1 for no 1xx.

I'll note here that we've pushed a number of other things to
extensions.  Things that have use cases even.  1xx uses - or the
examples of these that we actually have - tend to be things that map
to HTTP/2 framing layer constructs.

With extensions, I think that we can provide for the potential uses
that 1xx codes were previously used for.  And I think that enabling
the functionality in this way is architecturally cleaner than
propagating the specific mechanism from HTTP/1.1.  1xx fits the
HTTP/1.1 architecture, but not HTTP/2.

> +1 for removing trailers.

It has always been marginal.  I've not seen a whole lot of support for
retaining them, but no one has really pushed hard.  Maybe you can try.