Subject: draft-kinnear-httpbis-http2-transport questions

From: "Martin Thomson" <[email-masked]>
Date: Wed, 20 Mar 2019 06:05:59 -0400

Lots to think about here.  Thanks for sharing.

I have a few fairly basic questions about the goals of the draft.

Why do you need to use both :protocol and a setting?  Isn't SETTINGS_ENABLE_CONNECT_PROTOCOL plus the new values for :protocol enough?

How does a :protocol of bytestream differ from a plan CONNECT?

What is the point of a :protocol of datagram?  Is the idea to use UDP?  The string "UDP" appears nowhere in the draft, so I've concluded that that isn't the case, except that using UDP would be the only way in which this would be different (and therefore marginally useful, modulo the obvious head-of-line stuff).

FWIW, the datagram thing might work, but DATA frames aren't really designed to work that way.  In order to use a datagram tunnel effectively you would have to change the complete stack for DATA to avoid coalescing of frames, buffering, splitting and all those sorts of things that are natural for something with an underlying assumption of streaming semantics.  It might be better to define a new frame type for this purpose.

If the point is to enable UDP, then there are a few features I might want to request.  For instance, the ability to setup a listener.  The ability to know what the IP and port of that listener are.  The ability to set rules about what remote addresses can send to that listener (other than a fixed, CONNECTed peer).  That might get closer to being useful for something like WebRTC.  I would want all of that for QUIC, for instance, where head-of-line is less of a concern.  But that gets dangerously close to TURN, so we'd have to have the NIH discussion in light of that.

Cheers,
Martin

---

From: Andy Green <[email-masked]>
Date: Wed, 20 Mar 2019 18:29:00 +0800

> Lots to think about here.  Thanks for sharing.

Thanks for discussing it on the list.

> How does a :protocol of bytestream differ from a plan CONNECT?

Putting datagram thing to one side, perhaps I missed it but it seems it 
doesn't buy anything compared to RFC8441:

https://datatracker.ietf.org/doc/rfc8441/?include_text=1

That already has the same idea of CONNECT-ing the stream to be 
different, non-http transport over stream DATA frames.  Although RFC8441 
is focused on transporting websockets, it defines an upgrade name 
registry so you can upgrade to something else (Section 9.2).

-Andy

---

From: Andy Green <[email-masked]>
Date: Fri, 22 Mar 2019 14:28:49 +0800

> Well, draft-kinnear-httpbis-http2-transport-01 seems do just that
> ( giving new upgrade token, as you suggest )

You're right, I scanned the introduction that was starting from CONNECT 
first principles and missed it mentions RFC8441 halfway through.

Still...

>> 5.  IANA Considerations
>>
>>    This specification registers two entries in the "HTTP Upgrade Tokens"
>>    registry that was established by [RFC7230].
> 
> What I missed ?

What it creates is a single registered upgrade type "bytestream" that is 
anonymous and has no description of what protocol or versioning or 
subframing is inside it.

 From ws perspective, it's the same as a ws protocol that doesn't 
provide subprotocol names... the vhost can only offer one of those and 
if you made that mistake, this doesn't have the escape hatch of 
subprotocol negotation that ws has.  If different vendors feel that 
different things should go in the anonymous transport, more than one 
vendor's products couldn't talk to the same vhost.

It could maybe get around it by registering something tied to a specific 
format, like "ethernet-encapsulation".  But if it wants to just be 
"bytestream" it should maybe think about ws-style subprotocol (which can 
include versioning in the name).

Compared to RFC8441's UPGRADE type, there is some low-hanging fruit 
because that deliberately encapsulated RFC6455 framing wholesale inside 
h2 DATA frames (including masking).  There is an opportunity to define 
an UPGRADE that uses h2 DATA framing directly, or a new h2 frame type 
that doesn't have the extra framing overhead and legacy ws framing stuff.

-Andy

---

From: Kari Hurtta <[email-masked]>
Date: Fri, 22 Mar 2019 07:52:21 +0200 (EET)

There is:

> 9.  IANA Considerations
> 
> 9.1.  A New HTTP/2 Setting
> 
>   This document registers an entry in the "HTTP/2 Settings" registry
>   that was established by Section 11.3 of [RFC7540].
>
>      Code: 0x8
>      Name: SETTINGS_ENABLE_CONNECT_PROTOCOL
>      Initial Value: 0
>      Specification: This document
>
> 9.2.  A New HTTP Upgrade Token
> 
>    This document registers an entry in the "HTTP Upgrade Tokens"
>       Value: websocket
>      Description: The Web Socket Protocol
>      Expected Version Tokens:
>      References: [RFC6455] [RFC8441]

https://tools.ietf.org/html/draft-kinnear-httpbis-http2-transport-01#section-3.1

> 3.1.  Initiating the Extended CONNECT Handshake
> 
>   Endpoints using this mechanism to establish byte stream or datagram
>   tunnels over HTTP/2 streams follow the CONNECT handshake procedure
>   defined in [RFC6455].  However, instead of supplying "websocket" for
>   the :protocol psuedo-header field to indicate a WebSocket connection,
>   they specify "bytestream" or "datagram" to indicate a byte stream or
>   datagram connection, respectively.

https://tools.ietf.org/html/draft-kinnear-httpbis-http2-transport-01#section-5

What I missed ?

/ Kari Hurtta

---

From: Kari Hurtta <[email-masked]>
Date: Fri, 22 Mar 2019 19:32:53 +0200 (EET)

Andy Green <[email-masked]>: (Fri Mar 22 08:28:49 2019)
> vendor's products couldn't talk to the same vhost.

It is not per vhost, it is per URL which includes also path.

Same vhost can provide different protocol on different path.

https://tools.ietf.org/html/rfc8441#section-4

|   o  On requests that contain the :protocol pseudo-header field, the
|      :scheme and :path pseudo-header fields of the target URI (see
|      Section 5) MUST also be included.

|   Endpoints using this mechanism to establish byte stream or datagram
|   tunnels over HTTP/2 streams follow the CONNECT handshake procedure
|   defined in [RFC6455].  However, instead of supplying "websocket" for
|   the :protocol psuedo-header field to indicate a WebSocket connection,
|   they specify "bytestream" or "datagram" to indicate a byte stream or
|   datagram connection, respectively.
|
|   The :scheme and :path psuedo-headers are required by [RFC6455].  The
|   scheme of the target URI MUST be set to "https" for both byte stream
|   and datagram tunnels.  The path is used in the same manner as for the
|   WebSocket protocol, and MAY be set to "/" (an empty path component)
|   if not desired for use.

You are saying that path is not enough to identify desired endpoint ?

Is subprotocols really used on Websocket ?

I suspect that path is changed instead.

/ Kari Hurtta

---

From: Andy Green <[email-masked]>
Date: Sat, 23 Mar 2019 05:24:14 +0800

>It is not per vhost, it is per URL which includes also path.

From a ws perspective the subprotocols buy you something different than path.

Ws also lets the upgrade use the path, since the path is there... and that's useful since the path may bind to url space on the vhost for purposes like basic auth requirement, which can also then apply to the http part of the ws upgrade cleanly.  In ws what's on the path is left wholly for a server + specific ws protocol to define or ignore.

Subprotocol is a negotiation...the client lists what he can handle and the server picks the best one he can handle, once at upgrade time, and informs the client in the response.  So you can deal with multiple versions in the field, rolling upgrades, and even eg, population of devices with different vendor protocols and quirks once.

Subprotocol is standardized outside of some arbitrary vendor url schema.  So packet inspection can know what is supposed to be in the upgraded stream.

Subprotocols allow the server to have one implementation which may be used by different vhosts, these may then be administratively enabled and disabled per-vhost at the server.

Clients can use the negotiation to support different server capabilities in one implementation cleanly.

There are other pressures on path to match applications and url space layout in the vhost, so it's not a free choice just for upgrades.

>You are saying that path is not enough to identify desired endpoint ?

No, you could put, eg, ?subprotocol=xxx on the path... it's not the same as ws style subprotocol negotiation but it will select a protocol.  You can put &quirks= &version= too.

However...

 - that draft does not define conventions for doing that

 - the draft specifies only two opaque, abstract transport :protocol names

Actually as it is if a vhost gets this request, he can't be sure of the client assumptions about what he'll read into the url and then no idea what's going to be coming from the opaque protocol name.  Anything that thought it might inspect the packet must learn everybody's random path conventions...

They can get around it by making the :protocol specify what it is in there, but that must go in the registry each time... it'd work for 'ethernet-encapsulation' or so though.

Or they can normatively tell how to specify the subprotocol another way.

>Is subprotocols really used on Websocket ?

Yes.

>I suspect that path is changed instead.

There are lots of ways to do it.  From what I have seen from many people using ws the first time, they are happy to ignore subprotocol and path and get on with their implementation.  But then they want to support more than one protocol on the same vhost.  Ws upgrades have no normative binding to server url space, they exist outside it unless the server implementation enforces something about path... they are normatively bound by subprotocol name.

-Andy

---

From: Eric Kinnear <[email-masked]>
Date: Fri, 22 Mar 2019 23:39:12 +0100

Hi Andy, Kari, 

Thanks for the comments and discussion here!

As Kari mentioned, this does specify two new :protocol names and is a fairly straightforward extension to RFC8441.

In terms of negotiation of subprotocols/next protocol there are a few options that come to mind: 

(1) Treat this like TCP and leave it up to the data running over this arbitrary bytestream to negotiate what it wants to do. Note that hostname/port/etc. are all just as available here as to TLS. That’s some of what’s currently in the document, but I suspect we’ll want more than that, for many of the reasons outlined in your emails. 
However, folks using existing protocols transported over H/2, which is one of the main use cases here, are probably okay with just (1). This also enables exactly the same API surface/mechanisms that are currently used for that protocol over TLS, except now it’s transparently using a single transport connection underneath. 

(2) The path is present and can be useful in designating different “endpoints” for different types of services. This is also in the document, and aims to look very similar to how WebSocket handles this in 8441. As Andy notes, this doesn’t quite provide a scenario where the client offers something and the server picks it. If you have multiple protocols and just need different “endpoints”, you can likely use this in combination with (1) above without too much trouble. 

(3) Incorporate a header that contains a list of next protocols and lets the responder choose. This can be very useful, but it’s not strongly defined in the current version of the document. I’d be open to adding it, and offhand it seems like it would look pretty much like WebSocket’s version in 8441.

(4) A different mechanism that would be helpful here? Ideally, it would be common to anyone and any protocol wanting to use the extended CONNECT handshake.

Operatively, the main point of the extended CONNECT part is to say “here’s this great thing for WebSocket, let’s generalize it to the base of what other :protocol values will need”.. 
In doing that, we also define a new :protocol that covers a number of use cases, but most certainly not everything that the extended CONNECT method could be useful for. 
Future :protocol values should be able to build on top of this, since it defines a minimal set (i.e. without WebSocket specific items) of functionality that is needed for generic use of that handshake. 

Thanks,
Eric

> On Mar 22, 2019, at 10:24 PM, Andy Green <[email-masked]> wrote:
> 
> 
> 
> On March 23, 2019 1:32:53 AM GMT+08:00, Kari Hurtta <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Andy Green <[email-masked]>
Date: Sat, 23 Mar 2019 09:17:43 +0800

On 23/03/2019 06:39, Eric Kinnear wrote:
> As Kari mentioned, this does specify two new :protocol names and is a 
> fairly straightforward extension to RFC8441.
> 
> (1) Treat this like TCP and leave it up to the data running over this 
> arbitrary bytestream to negotiate what it wants to do. Note that 
> hostname/port/etc. are all just as available here as to TLS. That’s some 
> of what’s currently in the document, but I suspect we’ll want more than 
> that, for many of the reasons outlined in your emails.
> However, folks using existing protocols transported over H/2, which is 
> one of the main use cases here, are probably okay with just (1). This 
> also enables exactly the same API surface/mechanisms that are currently 
> used for that protocol over TLS, except now it’s transparently using a 
> single transport connection underneath.

TLS is a good example... modern TLS does not take this completely opaque 
transport approach either... see SNI and ALPN / NPN.  They also found 
that it's necessary to have standardized OOB characterization of what it 
will carry.  In practice even h2 itself requires ALPN so the two sides 
can agree on "h2" as the protocol the opaque tunnel will carry.

Since that's only available at the network connection level but h2 can 
mux independent streams (upgraded to independent protocols...) inside 
it, I think you also have to take on replicating ALPN / ws subprotocol 
type metadata at that layer too, one way or another.

> (2) The path is present and can be useful in designating different 
> “endpoints” for different types of services. This is also in the 
> document, and aims to look very similar to how WebSocket handles this in 
> 8441. As Andy notes, this doesn’t quite provide a scenario where the 
> client offers something and the server picks it. If you have multiple 
> protocols and just need different “endpoints”, you can likely use this 
> in combination with (1) above without too much trouble.

You can rely on implied stuff like URL conventions as much as you want 
if you control both sides, and just need to do it once with no 
interoperability.  But for cases beyond that, you need to spell it out 
normatively how it should work.

Isn't it simpler to call this protocol metadata out into its own thing 
rather than overload the url path, in terms of defining it?  In some 
implementations the protocol handler will be intimately integrated with 
the whole URL space, but in others, it will not natively know what a URL 
is and definitely not know or care about at least the leftmost part of 
it already "dereferenced" by the server above it.

> (3) Incorporate a header that contains a list of next protocols and lets 
> the responder choose. This can be very useful, but it’s not strongly 
> defined in the current version of the document. I’d be open to adding 
> it, and offhand it seems like it would look pretty much like WebSocket’s 
> version in 8441.
> 
> (4) A different mechanism that would be helpful here? Ideally, it would 
> be common to anyone and any protocol wanting to use the extended CONNECT 
> handshake.

ws has the concept of a degenerate case where the protocol isn't 
given... you get a "default protocol" for that vhost whatever that means 
to the server (and it may mean something different to the client...).  I 
really recommend not copying that and enforcing roll-your-own protocol 
naming along the lines of "com.apple.myproduct.myprotocol.version".

> Operatively, the main point of the extended CONNECT part is to say 
> “here’s this great thing for WebSocket, let’s generalize it to the base 

I agree, but deployment situation is dismal on 8441.  And that has the 
advantage of ws semantics on the client side ultra-widely deployed.  So 
I also think as mentioned some examples selling why you would deploy it 
would be helpful in there.  For example, there's nothing saying why for 
your use-case, taking the hit to add 8441 encapsulated ws framing is out 
of the question.  Then you can just use 8441.

> of what other :protocol values will need”.
> In doing that, we also define a new :protocol that covers a number of 
> use cases, but most certainly not everything that the extended CONNECT 
> method could be useful for.
> Future :protocol values should be able to build on top of this, since it 
> defines a minimal set (i.e. without WebSocket specific items) of 
> functionality that is needed for generic use of that handshake.

Yes... I agitated a long while ago for this to directly be part of h2, 
at least something that was enough to carry ws in the base spec in a way 
compatible with the ws JS api.  so I agree it has been a missing 
piece...8441 largely ticked the box in terms of getting the 
(considerable) mux advantages but it has some deliberate inefficiencies 
and ws focus since it came to be from taking the temperature of what 
browser vendors were actually willing to discuss / implement not what 
was ideal.

-Andy

> Thanks,
> Eric
> 
> 
> 
>> On Mar 22, 2019, at 10:24 PM, Andy Green <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>>
>>
>> On March 23, 2019 1:32:53 AM GMT+08:00, Kari Hurtta 
>> <[email-masked] <mailto:[email-masked]>> 
>> wrote:
>>> Andy Green <[email-masked] <mailto:[email-masked]>>: (Fri Mar 22 
>>> 08:28:49 2019)
>>>>
>> Ws also lets the upgrade use the path, since the path is there... and 
>> that's useful since the path may bind to url space on the vhost for 
>> purposes like basic auth requirement, which can also then apply to the 
>> http part of the ws upgrade cleanly.  In ws what's on the path is left 
>> wholly for a server + specific ws protocol to define or ignore.
>>
>> Subprotocol is a negotiation...the client lists what he can handle and 
>> the server picks the best one he can handle, once at upgrade time, and 
>> informs the client in the response. So you can deal with multiple 
>> versions in the field, rolling upgrades, and even eg, population of 
>> devices with different vendor protocols and quirks once.
>>
>> Subprotocol is standardized outside of some arbitrary vendor url 
>> schema.  So packet inspection can know what is supposed to be in the 
>> upgraded stream.
>>
>> Subprotocols allow the server to have one implementation which may be 
>> used by different vhosts, these may then be administratively enabled 
>> and disabled per-vhost at the server.
>>
>> Clients can use the negotiation to support different server 
>> capabilities in one implementation cleanly.
>>
>> There are other pressures on path to match applications and url space 
>> layout in the vhost, so it's not a free choice just for upgrades.
>>
>> No, you could put, eg, ?subprotocol=xxx on the path... it's not the 
>> same as ws style subprotocol negotiation but it will select a 
>> protocol.  You can put &quirks= &version= too.
>>
>> However...
>>
>> Actually as it is if a vhost gets this request, he can't be sure of 
>> the client assumptions about what he'll read into the url and then no 
>> idea what's going to be coming from the opaque protocol name. 
>>  Anything that thought it might inspect the packet must learn 
>> everybody's random path conventions...
>>
>> They can get around it by making the :protocol specify what it is in 
>> there, but that must go in the registry each time... it'd work for 
>> 'ethernet-encapsulation' or so though.
>>
>> There are lots of ways to do it.  From what I have seen from many 
>> people using ws the first time, they are happy to ignore subprotocol 
>> and path and get on with their implementation.  But then they want to 
>> support more than one protocol on the same vhost.  Ws upgrades have no 
>> normative binding to server url space, they exist outside it unless 
>> the server implementation enforces something about path... they are 
>> normatively bound by subprotocol name.
>>
>> -Andy
>>
>>> / Kari Hurtta
>

---

From: Kari Hurtta <[email-masked]>
Date: Sat, 23 Mar 2019 09:24:34 +0200 (EET)

> Why do you need to use both :protocol and a setting?  Isn't SETTINGS_ENABLE_CONNECT_PROTOCOL 
> plus the new values for :protocol enough?

RFC 8441: Bootstrapping WebSockets with HTTP/2

SETTINGS_ENABLE_CONNECT_PROTOCOL SETTINGS seems do that partially:

https://tools.ietf.org/html/rfc8441#section-3

|   The new parameter name is SETTINGS_ENABLE_CONNECT_PROTOCOL.  The
|   value of the parameter MUST be 0 or 1.

|   Upon receipt of SETTINGS_ENABLE_CONNECT_PROTOCOL with a value of 1, a
|   client MAY use the Extended CONNECT as defined in this document when
*   creating new streams.  Receipt of this parameter by a server does not
*   have any impact.

|   Using a SETTINGS parameter to opt into an otherwise incompatible
|   protocol change is a use of "Extending HTTP/2" defined by Section 5.5
|   of [RFC7540].  Specifically, the addition a new pseudo-header field,
|   ":protocol", and the change in meaning of the :authority pseudo-
|   header field in Section 4 require opt-in negotiation. 

Specially it seems work on case

    :protocol=bytestream

when client is initializing this.

( :protocol=datagram is another thing )

https://tools.ietf.org/html/draft-kinnear-httpbis-http2-transport-01#section-2

|   As described in Section 5.5 of [RFC7540], SETTINGS parameters allow
|   endpoints to negotiate use of protocol extensions that would
|   otherwise generate protocol errors.  Use of the CONNECT method
|   extension defined in [RFC6455] requires the
|   SETTINGS_ENABLE_CONNECT_PROTOCOL parameter to be received by a client
|   prior to its use.

SETTINGS_ENABLE_CONNECT_PROTOCOL is on RFC 8441
it is NOT on RFC 6455: The WebSocket Protocol

|   After receiving both SETTINGS_ENABLE_CONNECT_PROTOCOL and
|   SETTINGS_ENABLE_BIDIRECTIONAL_CONNECT, either endpoint of an HTTP/2
|   connection can send a request in HEADERS frames to establish a byte
|   stream or datagram tunnel via the extended CONNECT method.
|   Similarly, either endpoint may be required to respond to an incoming
|   CONNECT request seeking to establish such a tunnel.

So I guess that server push on HTTP/2 is allowed to open
:protocol=bytestream  when 
  SETTINGS_ENABLE_BIDIRECTIONAL_CONNECT=1 is sent by client

> How does a :protocol of bytestream differ from a plan CONNECT?

Different semantic of :authority

https://tools.ietf.org/html/rfc7540#section-8.3

>   A proxy that supports CONNECT establishes a TCP connection [TCP] to
>   the server identified in the ":authority" pseudo-header field.  Once
>   this connection is successfully established, the proxy sends a
>   HEADERS frame containing a 2xx series status code to the client, as
>   defined in [RFC7231], Section 4.3.6.

So plain CONNECT is different.

|   o  On requests bearing the :protocol pseudo-header field, the
|      :authority pseudo-header field is interpreted according to
|      Section 8.1.2.3 of [RFC7540] instead of Section 8.3 of that
|      document.  In particular, the server MUST NOT create a tunnel to
|      the host indicated by the :authority as it would with a CONNECT
|      method request that was not modified by this extension.

So in summary :protocol=bytestream does not freate TCP connection
to the server identified in the ":authority" pseudo-header field. 

I think that this is the difference.

> What is the point of a :protocol of datagram?

Seems that :protocol=datagram implies that "packect boundaries" are 
preserved.  That seems to be the difference between :protocol=datagram
and :protocol=bytestream 

( :protocol=bytestream     is like SOCK_STREAM on socket()
  but :protocol=datagram  looks like SOCK_SEQPACKET on socket()
  -- not actually SOCK_DGRAM  which is "connectionless, unreliable messages". 

  So perhaps it should be :protocol=seqpacket instead ?
)

So that one HTTP/2 DATA frame is one "packet" and is preserved to 
application (so that frame boundaries are not removed by HTTP/2 stack).

Was that intended meaning of

 :protocol=datagram

and only difference between :protocol=bytestream and :protocol=datagram
(I suggest :protocol=seqpacket  ☺  )

https://tools.ietf.org/html/draft-kinnear-httpbis-http2-transport-01#section-4

|   If the application negotiated the "datagram" protocol, individual
|   DATA frames are considered complete messages on the stream.
|   Implementations SHOULD preserve these message boundaries when
|   delivering data to the application.  This prevents applications from
|   needing to insert another level of framing to delineate message
|   boundaries while transmitting datagram messages over HTTP/2 streams.
|   Additionally, if an application is forwarding messages received over
|   a "datagram" stream, the contents of each DATA frame should be sent
|   in individual datagrams where possible.

So looks like

    SETTINGS_ENABLE_CONNECT_PROTOCOL=1 from server

    :protocol=datagram  from client

    is not enough for negation of this.

Is now SETTINGS_ENABLE_BIDIRECTIONAL_CONNECT=1 from other end (from server
if client is using :protocol=datagram) that what guarantees that 
:protocol=datagram preserves message boundaries ?

This is not vey well spelled on SETTINGS_ENABLE_BIDIRECTIONAL_CONNECT
text.  And also name does not imply that.

Got I that correct?

/ Kari Hurtta

---

From: Eric Kinnear <[email-masked]>
Date: Fri, 22 Mar 2019 23:18:10 +0100

Hi Martin, 

Thanks for the read through and for the comments! 

You’ve managed to hit upon most of the open questions remaining for this approach, which I expect will be discussed in Prague in tsvwg and httpbis. 
My intent here is to have this document present (alongside its -00) to help discuss those questions, at which point we can rev the draft and start a more detailed discussion on the list with a more concrete proposal. 

> On Mar 20, 2019, at 3:05 AM, Martin Thomson <[email-masked]> wrote:
> 

Short answer: Yes, just doing a new :protocol value with the existing setting would be fine. There’s a big caveat around the “bidirectional” part, and I left the setting as covering both, when it probably should be only the second part. 

Long answer: 
So this brings me to some of the “where do we want this to go” questions. Right now, there are a few potentially competing goals, and I’ve left the setting to cover everything to ensure that any endpoint supporting this supports all of it. I expect that’s not what we really want long term, and that this can be split into two parts.  

A brief summary of those goals (this will be covered in a presentation in tsvwg this week): 

- Ability to establish transport of arbitrary bytes over a single stream on a multiplexed H/2 connection. At the API layer, we present this as if it were a TCP/TLS stream, with all the associated state machinery plumbed, but underneath we’re sharing a single H/2 connection. 

- Ability to share that H/2 connection with other streams, potentially also transporting arbitrary bytes, or maybe they’re performing standard H/2 requests/responses by sending HTTP messages. 

- Ability for the server to establish streams that transport arbitrary bytes (note that the client can reject these incoming requests for streams, just like any other H/2 stream). 
	This one is where it starts to get tricky, since we want everything to be bidirectional. This is the main reason to have a SETTING, since it’s a much larger leap than just adding a new :protocol value, and I’m not convinced that it’s currently allowed/even reasonable with CONNECT or any HTTP message. This is covered in -00 rather well by STREAM frames which are just defined as a new, bidirectional concept, but I’m not clear that -01 adequately solves this. (And ideas on the best way to accomplish this are most welcome!) 

In terms of use cases, we’re really looking at the intersection of a few things, and it may eventually be better to split this into two documents. 

1. Arbitrary transport of bytes (and potentially a framing layer that suffices for datagrams, see comments below) is useful for tunneling things over an H/2 connection while getting all the benefits of the shared underlying transport. 
	This could enable nice types of proxies, like for UDP, using existing CONNECT (not to the other endpoint), etc. 
	This is great to enable talking directly to the other endpoint if you wanted to bundle everything to that host together (of course, HoL blocking is still a thing)
	There are a several more that are written up across some of the HiNT (https://tools.ietf.org/html/draft-pardue-httpbis-http-network-tunnelling-01 <https://tools.ietf.org/html/draft-pardue-httpbis-http-network-tunnelling-01>) and HELIUM (https://tools.ietf.org/html/draft-schwartz-httpbis-helium-00 <https://tools.ietf.org/html/draft-schwartz-httpbis-helium-00>) drafts, which have lots of really great ideas and discussion.
	Having headers available in the HTTP message (as opposed to the -00 with STREAM frames) is useful for these types of use cases. 

	This starts to cover some of the “enhance CONNECT a little more and define ways to send the missing pieces over it/negotiate how to have both ends agree on what’s happening” strategy. 

2. Bidirectional establishment of arbitrary transport of bytes is another very useful concept
	I’d like to have both ends of a connection be able to establish a new stream that carries bytes — once you’re connected to someone, both sides should be able to initiate contact with the other. I use this today for making just a single link between devices and then having different services run on top of that shared link. 
	This is something that’s offered by QUIC Transport (i.e. not H/3) and there are lots of use cases for this - however we likely need to fall back to TLS/TCP for some percentage of connections. HTTP/3 falls back to HTTP/2 over TLS/TCP and is happy. QUIC Transport needs something to fall back to, and HTTP/2 Transport could be that thing.

For QUIC Transport falling back to TLS/TCP, some frame types aren’t necessary (ACK) since TLS/TCP provides reliability, etc. 
At that point, pretty much every frame maps to H/2, with the exception of being able to open new streams (and from either endpoint) without HTTP headers. I’m thinking that something along the lines of the -00 for this document works a bit more cleanly for this purpose, but that’s the main point of the discussion to be had. A QUIC STREAM frame would map pretty much directly to the STREAM frame in -00 and we can decide to use DATA for application data or keep it in the STREAM frame like QUIC. 

In terms of future direction, I’m thinking that this may need to be two documents, one for “how to run QUIC transport over TLS/TCP using H/2 framing / H/2” and one for “adding new :protocols to the extended CONNECT handshake”. The current document clubs beginnings of both together rather harshly and I’m not sure that’s helpful or useful. 

> How does a :protocol of bytestream differ from a plan CONNECT?

My take on this is that RFC8441 specifies a way to use CONNECT to establish a stream terminated by the other endpoint of the H/2 connection, whereas plain CONNECT establishes a new connection to (potentially) a different host entirely and then tunnels traffic through to that other host.
Because of that, you want many of the same things that WebSocket over H/2 needed, like ways to match up listening applications to these incoming streams.

> What is the point of a :protocol of datagram?  Is the idea to use UDP?  The string "UDP" appears nowhere in the draft, so I've concluded that that isn't the case, except that using UDP would be the…

I debated dropping “datagram” out as a potential :protocol value, but left it in for now, since one of the things that would be nice to have is the ability to encapsulate datagrams. 
The idea here was to support sending delineated messages, but if you really want to do that you should probably be adding some framing (or even WebSockets) if you’re coalescing DATA frames, etc. At that point, we really need some framing in here (which isn’t in -01) or to remove it entirely.

Just to make sure, by “use UDP” do you mean this would be H/2 over TLS encapsulating datagrams on a given stream, or using UDP/DTLS underneath and sending the H/2 frames over that?

> FWIW, the datagram thing might work, but DATA frames aren't really designed to work that way.  In order to use a datagram tunnel effectively you would have to change the complete stack for DATA to…

This is a really good point, since DATA frames aren’t guaranteed to get to the other side in the same shape that you tried to send them. If we’re going that far, then yes, additional frames may be easier to specify rather than changing the rules based upon the presence of an extension. 
Alternately, we go with yet-another-framing-layer and stick with that approach. 

> If the point is to enable UDP, then there are a few features I might want to request.  For instance, the ability to setup a listener.  The ability to know what the IP and port of that listener are.…

Would you communicate to the other endpoint about the listener or is that more of an API consideration? (All valid here, just curious)

Thanks again for the thoughtful review, and I’d love any ideas on how to get this where it needs to go! 

Thanks,
Eric

---

From: Eric Kinnear <[email-masked]>
Date: Sat, 23 Mar 2019 07:26:32 +0100

Hi Martin, 

> On Mar 20, 2019, at 3:05 AM, Martin Thomson <[email-masked] <mailto:[email-masked]>> wrote:
> 

Long answer: 

Thanks,
Eric