Subject: alpn identifiers (not again!)

From: Stefan Eissing <[email-masked]>
Date: Mon, 8 Jun 2015 15:01:56 +0200

Probably everyone is tired of this already, and yes, there are tons of mails out there and several issues on github.

However. HTTP/2 is young, there are several draft specs building on top of it. The Alt-Svc spec is the prominent one stumbling over ALPN identifier use. Then there is tunnel-protocol. And more will come (QUIC anyone?). So I add my €0.02 in the hope of moving the mountain…

To have „h2c“ as an ALPN identifier that can never be used in ALPN and missing „h2“ in the Upgrade: header registry (why? enable http/2 on a https connection without ALPN?) shows that there are big holes here. It is unreasonable that any new transport layer protocol would need to write new alt-svc or tunnel-protocol specs or add names into the ALPN registry that will never be usable in TLS.

Will we have „h2q“ as Alt-Svc name for a DTLS secured QUIC transport of HTTP/2? Does not seem attractive to me.

And we *do* already have names and registry for names that define transport protocol stacks, the URL scheme. We have
- „http:“ as cleartext TCP HTTP/1 connection, with protocol negotiation over HTTP Upgrade
- „https:“ as TLS secured TCP HTTP/1 connection, with protocol negotiation over ALPN

With this in mind, a propsoed header like:
   Alt-Svc: h2c=":8000", h2=":443“
could be written as:
   Alt-Svc: h2=„http::8000", h2=„https:“
and in the future maybe:
   Alt-Svc: h2=„quics::1234“
without any confusion of ALPN identifiers and protocol stacks.

tl:dr

I think the distinction between „h2c“ and „h2" is wrong, confusing and limiting future development.

//Stefan

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782

---

From: Martin Thomson <[email-masked]>
Date: Mon, 8 Jun 2015 10:46:02 -0700

ALPN was invented to deal with a shortcoming of the URI scheme.
Specifically, because we needed to use either HTTP/1.1 or (originally)
SPDY to interact with https:// resources.

I see no problem with defining h2q, if that is the way we go.  And h2
isn't in the upgrade registry because we explicitly decided not to
support that option.

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 9 Jun 2015 11:44:57 +0200

I find ALPN identifiers highly useful and understand the distinction from URI schemes.

My argument is that ALPN identifier use has, because they are so nice, replaced URI scheme use where schemes are needed.

ALPN identifiers denote protocols. URI schemes denote protocol stacks. If we want something that denotes a named protocol over another named protocol stack, a simple concatenation. If we take ‚:‘ because it’s not part of a URI scheme (and assume that future ALPN identifier refrain from using it), we get Alt-Svc headers like

Alt-Svc: http:h2=„:8000", https:h2=":8001"

And if one really wants to keep it short, make "https:" the default if no ':' are present and have

Alt-Svc: http:h2=„:8000", h2=":8001"

> Am 08.06.2015 um 19:46 schrieb Martin Thomson <[email-masked]>:
> 

<green/>bytes GmbH

---

From: Martin Thomson <[email-masked]>
Date: Tue, 9 Jun 2015 10:05:31 -0700

On 9 June 2015 at 02:44, Stefan Eissing <[email-masked]> wrote:
> schemes are needed.

This is the part I'm not convinced of.  I think that it might be more
harmful in fact, but I'm not sure that I completely follow your logic.

> URI schemes denote protocol stacks.

This is something I disagree with, so maybe that's where we have a
disconnect.  URIs with a https scheme clearly do not describe a
protocol stack.

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 9 Jun 2015 19:39:36 +0200

The confusion is maybe in my head. I read an alt-svc with "h2c" as an advertisement that a client may connect via tcp, talk http in order to upgrade to h2. (or, outside the spec but implemented by many, as directly sending the PRIamble and switch to h2 that way.)

Similar "h2" in alt-svc seems to be the same with the addition of TLS in the stack and an additional way, ALPN, to upgrade.

Now, if I want to advertise that clients may connect via DTLS or maybe ssh to an endpoint that talks h2 (or h1 with upgrade), one would need a new ALPN identifier. Ok. But that id will never be meaningful in ALPN. That gives me the feeling that something is not right.

All that is specified will work. I am just looking for a way to do this better. 

> Am 09.06.2015 um 19:05 schrieb Martin Thomson <[email-masked]>:
>

---

From: Mike Bishop <[email-masked]>
Date: Tue, 9 Jun 2015 17:40:22 +0000

It boils down to whether versioning is a difference of protocol.  HTTPS denotes HTTP over SSL/TLS over TCP, which is a protocol stack.  But is that HTTP/1.1 or HTTP/2?  TLS 1.3 or SSL3?

Stefan is correct that those include built-in version negotiation mechanisms, so "HTTPS" is sufficiently precise to tell a client how to proceed.  Make a TLS connection and expect to negotiate some variety of HTTP over it.  You're also correct that it doesn't tell the client what version of HTTP (or TLS, for that matter) to expect on those endpoints.  But does the client actually need that information?  It knows what to do.

That said, *everything* in Alt-Svc has to ultimately carry the semantics of the original requesting protocol (HTTP in this case).  You're just telling the client how to reach an HTTP origin over some other combination of underlying protocols -- which may be TLS over TCP, or QUIC over UDP, or some SCTP mapping, or whatever else we dream up in 2024.  It may be immaterial to mention the HTTP semantic layer, since that ultimately has to come anyway.  However, those underlying layers can't currently be denoted by scheme, unless we define alternate schemes that http:// and https:// URIs can be mapped to.

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Tuesday, June 9, 2015 10:06 AM
To: Stefan Eissing
Cc: HTTP Working Group
Subject: Re: alpn identifiers (not again!)

> schemes are needed.

This is the part I'm not convinced of.  I think that it might be more harmful in fact, but I'm not sure that I completely follow your logic.

> URI schemes denote protocol stacks.

This is something I disagree with, so maybe that's where we have a disconnect.  URIs with a https scheme clearly do not describe a protocol stack.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 9 Jun 2015 12:02:00 -0700

> On Jun 9, 2015, at 10:40 AM, Mike Bishop <[email-masked]> wrote:
> 

URI schemes delineate naming authorities and their underlying namespaces.
The protocol stacks are a separate issue, defined largely by client config.
This is explained in RFC7230.

> Stefan is correct that those include built-in version negotiation mechanisms, so "HTTPS" is sufficiently precise to tell a client how to proceed.  Make a TLS connection and expect to negotiate some…

That's one way to configure a client, yes.

> That said, *everything* in Alt-Svc has to ultimately carry the semantics of the original requesting protocol (HTTP in this case).  You're just telling the client how to reach an HTTP origin over some other combination of underlying protocols -- which may be TLS over TCP, or QUIC over UDP, or some SCTP mapping, or whatever else we dream up in 2024.  It may be immaterial to mention the HTTP semantic layer, since that ultimately has to come anyway.  

Well, the alt service needs to be capable of supporting the semantics of *this* request,
which in most cases will be a GET or OPTIONS, and understand what to make of a request target.
All information protocols tend to support those semantics.  Presumably, the client will only
pick alt protocols that it would prefer over the one it is currently using, and thus has
some instruction on how to translate the old request into the new protocol(s).

> However, those underlying layers can't currently be denoted by scheme, unless we define alternate schemes that http:// and https:// URIs can be mapped to.

Alias schemes, yes.  Alias schemes are fine, but only if the client is instructed to
rewrite them as the more general scheme for presentation and links, since aliases in
general cause all sorts of damage to the Web.  Alt-Svc could then be replaced by a
Link rel=alternate, and not need to be limited to HTTP semantics.  But, that would have
the disadvantage of not being a hack specific to h2, so it is harder to parse and might
play havoc with folks who use Link for useful things.

Anyway, what you are basically saying is that ALPN identifiers are poorly suited for
this purpose, whereas a list of composed protocol name-ports like http.tcp.8080 or
h2c.ssh.tcp.22 would not have those limitations.  Whether or not we call those composed
names a scheme doesn't seem to matter, unless we use them as schemes.

....Roy

---

From: Martin Thomson <[email-masked]>
Date: Tue, 9 Jun 2015 15:08:51 -0700

On 9 June 2015 at 10:39, Stefan Eissing <[email-masked]> wrote:
> Now, if I want to advertise that clients may connect via DTLS or maybe ssh to an endpoint that talks h2 (or h1 with upgrade), one would need a new ALPN identifier. Ok. But that id will never be meaningful in ALPN.

Right.  Though it might be used in ALPN.  Say that HTTP/3 runs over
DTLS.  That does ALPN.

Or maybe there is a new protocol that decides to negotiate protocols
using exactly the same mechanism and identifiers as ALPN, just not in
TLS.  Is that ALPN?

> That gives me the feeling that something is not right.

Sure.  I understand.  Sometimes, to make a specification, you have to
break some abstractions.

---

From: Stefan Eissing <[email-masked]>
Date: Wed, 10 Jun 2015 11:40:37 +0200

> Am 10.06.2015 um 00:08 schrieb Martin Thomson <[email-masked]>:
> 

There is what I do not understand. If we ever make HTTP/2 work over DTLS, we would - in the current Alt-Svc spec - need a new ALPN identifier for that. So that a server can advertise this like:

  Alt-Svc: h2dtls=":443"

The server may offer ALPN with DTLS and what will the client send? "h2". Never will "h2dtls" be send as a protocol identifier over ALPN. Because "h2dtls" is not a single protocol but a stack.

  Alt-Svc: dtls.h2=":443", tls.h2=":443", ssh.h2=":1234", tcp.h2=":80"

If Alt-Svc would allow for a hierarchical notation, it would allow for a mix of a smaller set of identifiers that more readably state what they mean. (IMHO, etc.)

> Sure.  I understand.  Sometimes, to make a specification, you have to

Yes, but you need to make a case for it. Currently, I do not see that.

mod_h2 needs, to be fully compliant, to make checks that "h2" and "h2c" are used on the correct connections (have TLS or not) and some code paths are not supposed to work in some combinations. This ties the h2 code to exactly the two cases defined in the specification. Should *ever* someone use a secure connection protected by something other than TLS, these parts need to be rewritten. There is a price to breaking abstractions.

Cheers,

  Stefan

<green/>bytes GmbH

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 10 Jun 2015 11:34:49 +0000

On 9 June 2015, Stefan Eissing wrote:

> 
> The confusion is maybe in my head. I read an alt-svc with "h2c" as an
> advertisement that a client may connect via tcp, talk http in order to upgrade
> to h2. (or, outside the spec but implemented by many, as directly sending
> the PRIamble and switch to h2 that way.)
> 

I thought the alt-svc h2c behaviour was described in section 3.4 of the spec. The prior knowledge comes from alt-svc. The client MUST send the connection preface.

Lucas

---

From: Martin Thomson <[email-masked]>
Date: Wed, 10 Jun 2015 09:06:40 -0700

On 10 June 2015 at 02:40, Stefan Eissing <[email-masked]> wrote:
> The server may offer ALPN with DTLS and what will the client send? "h2".

It would send h2dtls: the version of HTTP/2 that runs over DTLS.  It
would be different from the one that runs over TLS in several crucial
ways, so it does need a new identifier (I mean, you could use other
information to disambiguate, but why do that when new identifiers are
cheap).

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 10 Jun 2015 21:58:30 +0000

new identifiers are far from cheap.

Any proxy that needs to understand them needs to be re-deployed.

I think since we're hell-bent on breaking layered networking basic 
tenets, we will run into these sorts of problems.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 10 Jun 2015 15:48:14 -0700

> new identifiers are far from cheap.

Cheaper than defining a whole new protocol.

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 10 Jun 2015 22:48:53 +0000

we didn't redesign TCP when we invented POP3 did we?

------ Original Message ------
From: "Martin Thomson" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "Stefan Eissing" <[email-masked]>; "HTTP Working 
Group" <[email-masked]>
Sent: 11/06/2015 10:48:14 a.m.
Subject: Re: alpn identifiers (not again!)

---

From: Martin Thomson <[email-masked]>
Date: Wed, 10 Jun 2015 16:11:17 -0700

> we didn't redesign TCP when we invented POP3 did we?

I'm not sure how that is relevant.

If we ported POP3 to SCTP, I'd imagine that would take quite a bit of
work.  Identifying the result as POP3+SCTP rather than POP3 might be a
good idea at that point.

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 11 Jun 2015 00:11:13 +0000

I would expect if we put POP3 over SCTP we shouldn't have to do any work 
to POP3 at all. Or SCTP

That's the whole point of having independent layers.

If we did have to, we got something wrong first by not making them 
independent.

Maybe POP3 and TCP is a bad example.

We don't change Ethernet to put IP over it, and we don't change IP to 
put TCP or UDP or GRE or the next protocol over it.

We shouldn't have to change TCP to put HTTP over it.  We didn't change 
HTTP to put it over TLS over TCP

------ Original Message ------
From: "Martin Thomson" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "Stefan Eissing" <[email-masked]>; "HTTP Working 
Group" <[email-masked]>
Sent: 11/06/2015 11:11:17 a.m.
Subject: Re: alpn identifiers (not again!)

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 11 Jun 2015 12:30:54 +1000

> HTTP to put it over TLS over TCP
​Not the protocol part of HTTP, but the identification part certainly
changed. (By minting the "https" URI scheme.)

I've written three drafts of this email, trying to keep it on-point, but
I'm having trouble working out exactly what problem is being discussed.
Here's my summary:

In places like HTTP Upgrade or Alt-Svc, I think the alternative service
types are:

* HTTP/1.x over TCP,
* HTTP2 over TCP, or
* TLS over TCP.

Here I use "TLS" to mean: open a TLS connection, if you both support ALPN
do that dance, otherwise it's old-school https

Since "TLS" isn't particularly informative (e.g if you care about the
differences between HTTP/1.1 and h2 transports), it would be nice to
further describe what gets carried within that TLS pipe. I.e. we're
identifying a protocol stack. So, you can enrich the options to:

* HTTP/1.x over TCP,
* HTTP2 over TCP,
* HTTP/1.x over TLS over TCP,
* HTTP2 over TLS over TCP.

Since the ALPN registry contains tokens for protocols carried within TLS,
and has a "shadow registry" of tokens that cannot be registered because
they apply to things that are never in TLS (like 'h2c'), we can write the
list as:

* <???>
* "h2c"
* "http/1.1"
* "h2"

Note that the tokens don't exactly define a "protocol stack", so much as:
the distinction between 'real' tokens and 'reserved indefinitely' shadow
tokens allows us to infer one extra layer below the identified protocol
(i.e. TLS, or not). As long as the stacks we want to identify only go as
deep as TLS/TCP, the stars remain aligned, and the registry exactly serves
our purposes.

Actually at this point I should be clear with what I'm saying: I'm only
considering protocol stacks looking downwards. Nothing stops the tokens
from identifying stacks that go *up* from TLS. For example, someone could
mint a token for "foobar-over-websockets-over-h2". Is that one of the
problems people are having? (That, by creating that token, proxies that are
happy with websockets-over-h2 have to learn a new magic word to know that
this also counts?)

Looking down again: if, in the future, we de-ossify the web and are able to
introduce a new lower-level protocol and use that to carry HTTP (is that
QUIC?), we'll have to add new tokens to the ALPN shadow registry (no big
deal?), but we'll also have to be sure to remember that "not TLS" in that
registry no longer necessarily just means "TCP".

I think that just means we're expanding the definition of the registry. I
don't know if that counts as a layering violation.

Have I missed anything?
Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Martin Thomson <[email-masked]>
Date: Wed, 10 Jun 2015 19:43:51 -0700

I think that you've summarized the situation well.  At least from my
perspective.

As for x-over-thing-we-already-know-about, that's a tough deal, and I
suspect that if we take this seriously, we need to consider whether
that gets a token or not.  That proxies will be affected is now a
consideration that needs to play into that.

Websockets was a good choice of example by the way.  Websocket
"sub-protocols", such as they are, aren't especially interesting
currently, and I wouldn't consider it worth adding new tokens for that
purpose.  But who can say if new uses of websockets might turn out to
be interesting.

On 10 June 2015 at 19:30, Matthew Kerwin <[email-masked]> wrote:
> Not the protocol part of HTTP, but the identification part certainly
> I've written three drafts of this email, trying to keep it on-point, but I'm
> having trouble working out exactly what problem is being discussed. Here's
> my summary:
>
>
> Here I use "TLS" to mean: open a TLS connection, if you both support ALPN do
> that dance, otherwise it's old-school https
>
> differences between HTTP/1.1 and h2 transports), it would be nice to further
> describe what gets carried within that TLS pipe. I.e. we're identifying a
> protocol stack. So, you can enrich the options to:
>
> * HTTP/1.x over TCP,
> * HTTP2 over TCP,
> and has a "shadow registry" of tokens that cannot be registered because they
> apply to things that are never in TLS (like 'h2c'), we can write the list
> as:
>
> * <???>
> * "h2c"
> * "http/1.1"
> * "h2"
>
> considering protocol stacks looking downwards. Nothing stops the tokens from
> identifying stacks that go *up* from TLS. For example, someone could mint a
> token for "foobar-over-websockets-over-h2". Is that one of the problems
> people are having? (That, by creating that token, proxies that are happy
> with websockets-over-h2 have to learn a new magic word to know that this
> also counts?)
>

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 12 Jun 2015 11:59:49 +1200

Some responses inline...

> changed. (By minting the "https" URI scheme.)

I've been thinking in the last few days that the problem may be that
there was never an "https" ALPN token minted to match it. At the time it
seemed okay since there was also never a HTTPS protocol name and nobody
hit problems. Now not so much.

This draft is possibly not the right place to do that minting, or maybe
it is.

> In places like HTTP Upgrade or Alt-Svc, I think the alternative service

* SPDY/.. over TLS

(adding SPDY to the mix since it has ALPN registered to make it a little
more obvious later where the odd-one-out is).

> * HTTP2 over TLS over TCP.

* HTTP/1 over SPDY/1 over TLS over TCP
* HTTP/1 over SPDY/2 over TLS over TCP
* HTTP/1 over SPDY/3 over TLS over TCP

> they apply to things that are never in TLS (like 'h2c'), we can write the

RFC 7301 did not specify non-TLS usage for these tokens. The initial
seeded registrations just got a reference to the protocol being used
over-TLS irregardless of whether it existed *only* over TLS or not.

The SPDY documents exlicitly forbid over-TCP usage. So we can make clean
assumptions there.

RFC 7230 defines two usages for over-TCP and over-TLS. Making RFC 7301
imply by omission that there is one ALPN token for both.

RFC 7540 defines the same, but with clear ALPN distinction between
over-TCP and over-TLS.

Leading to the list actually being this:

* "http/1.1"
* "h2c"
* "http/1.1"
* "h2"
* "spdy/1"
* "spdy/2"
* "spdy/3"

Spot the oddity.

And that is where a big part of the problem lies today. IMHO, what
happens in future tokens is secondary so long as it does not re-add this
problem.

Amos

---

From: Martin Thomson <[email-masked]>
Date: Thu, 11 Jun 2015 17:24:05 -0700

> hit problems. Now not so much.

RFC 7301 defines "http/1.1", which (I believe) refers to HTTP/1.1 over TLS.

> imply by omission that there is one ALPN token for both.

Or that it only applies to "x" over TLS.  My interpretation.

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 12 Jun 2015 11:42:59 +1000

> This draft is possibly not the right place to do that minting, or maybe
​Is that not what "http/1.1" is? Note that in my ​final list of tokens, I
left "HTTP/1.x over TCP" as "<???>" because, as I read it, there is no
token for that. That's the one we'd need to mint, if people want it, *not*
the HTTPS one.

​​

> over-TLS irregardless of whether it existed *only* over TLS or not.
Yes, the registry only identifies protocols negotiated/carried inside TLS.
Therefore everything in the registry is inside the TLS stack.​

> ​<...>​
​Actually I can't find any definitions of the "http/1.1" token except in
7301. Since the initial registry, as defined by RFC 7301, only contains
in-TLS protocols, the only implication is that the "http/1.1" token means
"HTTP/1.1 over TLS". "HTTP/1.1 over TCP" remains unspecified.

> Leading to the list actually being this:
​I write that list differently:

* <???>
* "h2c"
* "http/1.1"
* ..etc

> happens in future tokens is secondary so long as it does not re-add this
I think the problem is, of course, manyfold but boils down to two key
things:

1. The registry includes an entry for "HTTP/2 over TLS", which seems to
imply that any row that doesn't say "over X" means "over anything", instead
of "over TLS." (If the H2 entries were "HTTP/2" and "reserved" I think
there'd be less confusion.)

2. We're missing an equivalent 'reserved' token slot for "HTTP/1.1 over TCP"

That second point is easy to solve, and would allow us to move forward,
yeah? Assuming we actually need that token?

The confusion could be cleared up in a number of ways, which would reduce
the likelihood of future problems. (An informational RFC, a new column in
the registry table, maybe even just renaming the existing entries...)

Cheers