Subject: Re: h2 use of Upgrade

From: Martin Thomson <[email-masked]>
Date: Tue, 2 Sep 2014 15:18:48 -0700

On 31 August 2014 20:16, Roy T. Fielding <[email-masked]> wrote:
> Note that the SETTINGS frame is a sufficient response if the request
> was for OPTIONS.

Are you suggesting that we wouldn't provide an HTTP-level response
(other than the 101) to the OPTIONS request?

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 2 Sep 2014 16:02:12 -0700

> (other than the 101) to the OPTIONS request?

I am saying that the 101 (in HTTP/1.1 bits) is followed immediately by
a SETTINGS frame (in HTTP/2 bits), and that SETTINGS frame is a response
if the original request was for OPTIONS * (but not for something like
OPTIONS /).  OPTIONS * was designed for that purpose after the first
discussions of HTTPng.  SETTINGS is exactly what OPTIONS * is meant to
retrieve.  So, all we have to say is that an h2 response to an Upgrade
for an "OPTIONS * HTTP/1.1" request is the SETTINGS frame.

But that's just one alternative.  We could also simply tunnel the
HTTP/1.1 response back as a Headers+END_STREAM, which might be needed
for some unknown extensions if they already abuse OPTIONS *.

....Roy

---

From: Martin Thomson <[email-masked]>
Date: Tue, 2 Sep 2014 17:23:53 -0700

> for some unknown extensions if they already abuse OPTIONS *.

That was the thought.  Having special logic to detect OPTIONS requests
and then not expect a final response seems fraught.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Sep 2014 09:03:01 +0300

On 1 Sep 2014, at 6:16 am, Roy T. Fielding <[email-masked]> wrote:

> This comment is in reference to section 3.2 of
> 
> http://tools.ietf.org/id/draft-ietf-httpbis-http2-14.txt
> 
>> 3.2.  Starting HTTP/2 for "http" URIs
>> 
>>   A client that makes a request to an "http" URI without prior
>>   knowledge about support for HTTP/2 uses the HTTP Upgrade mechanism
>>   (Section 6.7 of [RFC7230]).  The client makes an HTTP/1.1 request
>>   that includes an Upgrade header field identifying HTTP/2 with the
>>   "h2c" token.  The HTTP/1.1 request MUST include exactly one
>>   HTTP2-Settings (Section 3.2.1) header field.
>> 
>>   For example:
>> 
>>     GET / HTTP/1.1
>>     Host: server.example.com
>>     Connection: Upgrade, HTTP2-Settings
>>     Upgrade: h2c
>>     HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>
> 
> The protocol-name is HTTP and the protocol-version is 2, so the correct
> upgrade protocol is HTTP/2 and the correct HTTP/1.1 response is
> 
>     GET / HTTP/1.1
>     Host: server.example.com
>     Connection: Upgrade, HTTP2-Settings
>     Upgrade: HTTP/2
>     HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>
> 
> That is, unless we want to change the name of this protocol to h2c
> (seems unlikely given the rest of the text).

This was always going to be a bit of a hack, because HTTP/1 and TLS each has their own name space to identify the protocol to be spoken. 

We chose to use the ALPN token because that’s the primary way to indicate breaking changes in HTTP/2, so future revisions (or variants) would presumably do the same thing. Given that the HTTP protocol name space is pretty static and small, that seemed like a reasonable thing to do.

Re-reading 7230 here, there isn’t a lot of semantic heft in the protocol-identifier. I think the most you can say is that we’re wilfully not reusing the existing “HTTP” protocol-name, and choosing not to use the (optional) protocol-version; instead, we’re choosing to align with ALPN, so that this revision (and future ones) don’t have to define both an ALPN token as well as a HTTP protocol-version.

Does doing so cause problems?

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 3 Sep 2014 14:08:55 -0700

> We chose to use the ALPN token because that’s the primary way to indicate breaking changes in HTTP/2, so future revisions (or variants) would presumably do the same thing. Given that the HTTP proto…

I don't see the point.  The protocol name is HTTP.  HTTP/1.1 is what you
are sending in this message, and HTTP/1.1 says that the protocol name is
HTTP.  It even uses HTTP/2.0 in the examples.

> Does doing so cause problems?

Yes, it contradicts RFC7230, which is a bad example to be setting.
Unless there is a strong reason to do so, and I have seen none here,
then the right choice is to follow the HTTP/1.1 design as intended.
Note also that

  Upgrade: TLS, HTTP/2

is already defined to mean: upgrade the connection to some version of
TLS and HTTP/2 at the same time.

....Roy

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Sep 2014 14:19:07 -0700

Roy--
Assuming that we went with upgrade as you specify, and we needed to specify
a different profile (e.g. HTTP2 but with lower default limits on the
various settings), how would that be negotiated?

-=R

On Wed, Sep 3, 2014 at 2:08 PM, Roy T. Fielding <[email-masked]> wrote:

> > This was always going to be a bit of a hack, because HTTP/1 and TLS each
> has their own name space to identify the protocol to be spoken.
> >
> > We chose to use the ALPN token because that’s the primary way to
> indicate breaking changes in HTTP/2, so future revisions (or variants)
> would presumably do the same thing. Given that the HTTP protocol name space
> is pretty static and small, that seemed like a reasonable thing to do.
>
> > Re-reading 7230 here, there isn’t a lot of semantic heft in the
> protocol-identifier. I think the most you can say is that we’re wilfully
> not reusing the existing “HTTP” protocol-name, and choosing not to use the
> (optional) protocol-version; instead, we’re choosing to align with ALPN, so
> that this revision (and future ones) don’t have to define both an ALPN
> token as well as a HTTP protocol-version.
> >

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 3 Sep 2014 15:28:19 -0700

On Sep 3, 2014, at 2:19 PM, Roberto Peon wrote:

> Roy--
> Assuming that we went with upgrade as you specify, and we needed to specify a different profile (e.g. HTTP2 but with lower default limits on the various settings), how would that be negotiated?

Wouldn't that be in the HTTP2-Settings field?

....Roy

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Sep 2014 15:37:50 -0700

Not in the ALPN-based connectivity case, at least, implying different
codepaths (and a required increase in verbosity near the start of the
communication) for upgrade vs ALPN-based negotiation.

Not specifying the ALPN token is also different in the upgrade case as the
ALPN token might indicate more information in the future about what
extensions are available, etc. which isn't something conveyed by settings
alone.
For anyone wishing to have equivalent functionality/latency for upgrade as
with ALPN or its future replacements, it would be good to offer the ALPN
token in there somewhere.
 -=R

On Wed, Sep 3, 2014 at 3:28 PM, Roy T. Fielding <[email-masked]> wrote:

> > Assuming that we went with upgrade as you specify, and we needed to
> specify a different profile (e.g. HTTP2 but with lower default limits on
> the various settings), how would that be negotiated?
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 3 Sep 2014 16:22:02 -0700

On Sep 3, 2014, at 3:37 PM, Roberto Peon wrote:

> Not in the ALPN-based connectivity case, at least, implying different codepaths (and a required increase in verbosity near the start of the communication) for upgrade vs ALPN-based negotiation.

That was already implied just by the use of Upgrade on an *existing*
connection.  The client is already telling the server what settings
it wants -- why does it need to repeat that in a painfully stupid
factorial combination of protocol name tokens?

....Roy

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Sep 2014 16:37:36 -0700

Well, it is no skin of my back if upgrade is less well supported than a TLS
based negotiation.

The difference extends beyond settings when one considers extensions.
-=R

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 3 Sep 2014 17:02:26 -0700

On Sep 3, 2014, at 4:37 PM, Roberto Peon wrote:

> Well, it is no skin of my back if upgrade is less well supported than a TLS based negotiation.

I think they are entirely different beasts, used in different contexts.

> The difference extends beyond settings when one considers extensions.

Hmm, I thought extensions are required to be negotiated with SETTINGS,
but it seems 5.5 doesn't mandate a specific method for negotiating.
That'll be fun.

....Roy

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 4 Sep 2014 08:19:42 +0200

> token in there somewhere.

The thing is that ALPN is a lower layer in the TLS stack, I mean it's
meta-data related to a stream connection that is offered by TLS to
transport HTTP/2 (or any other protocol). It could be seen as an
equivalent of the destination TCP port for TCP.

Currently we're using it in TLS to advertise the h2 name to indicate
to the server on the other side what application layer protocol we're
going to speak. In the case of HTTP/1 upgrade we have to advertise an
application layer protocol belonging to a different namespace.

Thus I'm seeing alternative possibilities :
  - either we consider that HTTP/2.0 designates exactly one application
    level protocol and that exactly one name+version must exist for each
    ALPN equivalent. That would more or less imply that the second digit
    continues to designate small protocol variations (eg: HTTP/2.1, etc).

  - or we consider that we're upgrading to an intermediate application
    layer that we could call "ALPN" and that we pass the protocol name
    from the ALPN namespace in a new header (sec-alpn-protocol-list or
    whatever we want to call it).

The second option doesn't seem stupid, as it would provide the closest
equivalent in clear over HTTP/1 to what is offered in secure mode over
TLS+ALPN and would make future upgrades easier. It could even make it
easier for intermediaries to be immediately compatible with future
versions.

Just my two cents,
Willy

---

From: Martin Thomson <[email-masked]>
Date: Thu, 4 Sep 2014 11:09:57 -0700

> Yes, it contradicts RFC7230, which is a bad example to be setting.

The contradiction seems to be dependent on interpretation, given the
actual usage of Upgrade, which is basically just websockets.

> Unless there is a strong reason to do so, and I have seen none here,

You obviously disagree, but I consider this to be a strong enough reason.

We did originally identify using the string "HTTP/2.0", but there were
concerns raised about the length of that string and something of a
lengthy discussion about the value of having distinct compartments in
the string that had semantics (or implied semantics). The length
concern is now less pressing since we know more about the bug in
question, but we did choose not to revert the change.

The net outcome of the discussion was that no one was able to identify
any value in having a regimented structure.  There was some discussion
about the virtues of structure in general and the structure defined
specifically in RFC 2817, but the conclusion there was that the value
structure provides was outweighed by the simplicity of an opaque
token.

---

From: Michael Sweet <[email-masked]>
Date: Thu, 04 Sep 2014 14:15:58 -0400

Martin,

> actual usage of Upgrade, which is basically just websockets.

and TLS for printing...

(IPP requires HTTP Upgrade to TLS support if you support TLS)

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 4 Sep 2014 13:05:09 -0700

> actual usage of Upgrade, which is basically just websockets.

*sigh*

I've had dozens of questions about Upgrade over the years, not including
my own use of it for internal projects like HTTPng and waka.  It is mostly
used in situations where gradual deployment of services is a necessity,
which might include anything from printers to battlefield sensor arrays
integrated by field laptops and relayed to AWACS.  Most of those uses
are fairly ephemeral in nature -- gradually removed over time.

So, when you say that actual usage if Upgrade is just websockets, I just
have to laugh (and cry a little) inside.  Websockets is a toy.  Juvenile
at best, actively harmful at worst.  It only uses Upgrade in order to
tunnel a raw socket over port 80, which is something the IETF used to
regard as a really bad idea (back when the IETF was mostly run by sysops).

Let's pretend for a second that that the SPDY team understands the entire
scope of SPDY usage.  That covers about 5% of HTTP usage, by my estimate,
just based on what people have told me they are doing over the past 20
years because they wanted to ask me a question not covered in the RFCs.
I suspect that only a small portion of people doing unusual things with
HTTP bothered to ask me a question.  I have been editing for 20 years
and only seem to understand maybe 60-70% of how HTTP is used in practice,
based on how much new I learn each year.  There is always a gap.

Ignorance of the scope of HTTP is not a design virtue.  The only hope that
we have of getting things right with a protocol as widely used as HTTP
(and as fundamentally important as HTTP) is that somebody will tell us
when we are wrong.  Please don't drive them away by making bad assumptions
about their problem space.  We don't own the problem space.

> structure provides was outweighed by the simplicity of an opaque

Yes, well, that's why we have last calls.  The value of a regimented
structure with orthogonal protocols provided as separate tokens is
that orthogonal protocols are supposed to change independently,
versioned over time, and separate-but-consistent tokens allow recipients
to easily distinguish between layered options (like TLS + HTTP/2) and
distinct choices (like HTTP/1 | 2).  That rationale will undoubtedly
become clearer after a few more iterations on the ALPN token set.

Nevertheless, the only problem here is that h2 is setting a bad example.
Any token is valid.  The ALPN token could even be used as the version.
If you want to stick with a bad example, then so be it.  It is no worse
than the painfully stupid requirements regarding TLS that have been
imposed by those who know nothing about the problem space.

Cheers,

Roy T. Fielding                     <http://roy.gbiv.com/>
Senior Principal Scientist, Adobe   <http://www.adobe.com/>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 4 Sep 2014 13:33:50 -0700

On 4 September 2014 13:05, Roy T. Fielding <[email-masked]> wrote:
> have to laugh (and cry a little) inside.

I apologize for trolling there.  I do forget myself sometimes.

You will admit that the registry is a little impoverished:
http://www.iana.org/assignments/http-upgrade-tokens/http-upgrade-tokens.xhtml#http-upgrade-tokens-1

Moreso considering the policy it operates on and its age.

---

From: Michael Sweet <[email-masked]>
Date: Thu, 04 Sep 2014 20:54:01 -0400

You could also use MIME media types - compared to the number of types in actual use, the MIME media type registry is a little impoverished.  Yet I don't think anyone would argue against using MIME media types in Internet protocols.

A lot of identifiers never get registered with IANA, even if all it takes is an email. Sometimes it is because developers don't understand the process or the importance of registering, and sometimes it is because their employers do not allow it (!).

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 05 Sep 2014 07:04:24 +0000

--------
In message <[email-masked]>, "Roy T. Fielding" w
rites:

>It is no worse

s/know nothing/cares/

It is certainly not because the WG quorum doesn't know about other parts
of the HTTP problem space, they just don't care about it.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.