Subject: [QUIC] h2 mapping - extensions

From: Patrick McManus <[email-masked]>
Date: Thu, 3 Nov 2016 17:39:56 -0400

Hi Robbie, Mike, et al. Thanks for getting the ball rolling with
https://tools.ietf.org/html/draft-shade-quic-http2-mapping-00..

What is the thinking about rfc 7540 h2 extensions and the h2/quic mapping..
particularly the way it allows unspecified frame types which may or may not
require negotiation.

it seems some of these extensions may have stateful issues across streams,
ala hpack and some may not.

If the extension were only defined for h2-over-quic that would be for the
extension to sort out.. but it doesn't seem like we are limiting things to
quic vs tcp, or are we?

-Patrick

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 7 Nov 2016 11:26:25 -0800

On Thu, Nov 3, 2016 at 2:39 PM, Patrick McManus <[email-masked]>
wrote:

> What is the thinking about rfc 7540 h2 extensions and the h2/quic
> mapping.. particularly the way it allows unspecified frame types which may
> or may not require negotiation.
>

Not sure what you mean by not limiting things to QUIC vs. TCP -- do you
have a different mapping in mind?

My sense is that we'll need to
(i) define how current extensions will work with QUIC (eg, HPACK), and
(ii) require that future extensions be defined explicitly  for both TCP and
QUIC.

---

From: Patrick McManus <[email-masked]>
Date: Mon, 7 Nov 2016 14:34:41 -0500

On Mon, Nov 7, 2016 at 2:26 PM, Jana Iyengar <[email-masked]> wrote:

>> it seems some of these extensions may have stateful issues across
>> streams, ala hpack and some may not.
>>

I meant explicitly scoping h2 extensions so that they only apply to quic
and/or tcp.. and it does seem like that's what you mean :)

> (i) define how current extensions will work with QUIC (eg, HPACK), and

fwiw HPACK/QPACK is not an extension and you can't define it as one in quic
because it would need to be negotiated and that's going to break the 0RTT
goal of the protocol. I don't think that's a problem though as there is no
need to make it an extension :)

> (ii) require that future extensions be defined explicitly  for both TCP
> and QUIC.
>

So the mapping document will need to establish a new h2 extension registry?

-P

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 7 Nov 2016 14:44:03 -0800

On Mon, Nov 7, 2016 at 11:34 AM, Patrick McManus <[email-masked]>
wrote:

>>> If the extension were only defined for h2-over-quic that would be for
>>> the extension to sort out.. but it doesn't seem like we are limiting things
>>> to quic vs tcp, or are we?
>>>
>>

Gotcha. I think it makes sense to limit to QUIC and TCP at the moment since
there is a proposal to run HTTP over these, until such a time as there is a
need to define over another transport (let's call it Essy Transport
Protocol, you know, EssyTP). I suspect at that point the httpbis wg (hey,
that's you! :-)) can figure out how to define extensions over the other
transport.

> fwiw HPACK/QPACK is not an extension and you can't define it as one in
> quic because it would need to be negotiated and that's going to break the
> 0RTT goal of the protocol. I don't think that's a problem though as there
> is no need to make it an extension :)
>

Ah, sorry -- my grasp of which things are extensions and which ones not in
HTTP-land is weak. I think compression is likely to be one of those things
that is different in QUIC than in HTTP/2.

> So the mapping document will need to establish a new h2 extension registry?

Do you mean an IANA registry? It's probably safest to put these extensions
will be in different namespaces entirely, so yes, if I understand your
question correctly.

- jana

---

From: Mike Bishop <[email-masked]>
Date: Tue, 8 Nov 2016 02:23:12 +0000

Jana, h2 extensions are very broad – you can negotiate (method unspecified, but usually a mutual SETTING) any departure from or addition to the spec.  Additionally, clients MUST ignore unknown frame types, so anything that doesn’t change the core of the protocol (e.g. ORIGIN) can be sent speculatively, without knowing whether the peer will understand.

In the current mapping, you’re effectively running a full HTTP/2 framing layer and mux on Stream 3.  It’s referred to as “the HEADERS stream,” but it actually carries SETTINGS, PUSH_PROMISE, PRIORITY, and basically anything else.  The mapping document calls out things that are delegated to the QUIC layer (PING, GOAWAY, WINDOW_UPDATE) or other QUIC streams (DATA), but otherwise you could take arbitrary h2 extensions and drop it on Stream 3 without much change.  If we keep that model, that’s probably the default answer – an extension works unless it has some hard dependency on ordering w.r.t. message body.

That said, I’d prefer to break out of the double-mux model if we can, moving things to their own streams as much as possible to take advantage of getting out of head-of-line blocking.  I’m working on a draft, albeit too late for Seoul, and at least HEADERS/PUSH_PROMISE/DATA (and everyone’s favorite, CONTINUATION) can be dropped to other streams pretty easily.  We may still need a dedicated stream for SETTINGS, though, which means putting extensions there too still isn’t as much of a stretch – at least, for things that live on stream zero.  And on-stream extensions, too, perhaps.

From: QUIC [mailto:[email-masked]] On Behalf Of Jana Iyengar
Sent: Monday, November 7, 2016 2:44 PM
To: Patrick McManus <[email-masked]>
Cc: [email-masked]
Subject: Re: [QUIC] h2 mapping - extensions

On Mon, Nov 7, 2016 at 11:34 AM, Patrick McManus <[email-masked]<mailto:[email-masked]>> wrote:

On Mon, Nov 7, 2016 at 2:26 PM, Jana Iyengar <[email-masked]<mailto:[email-masked]>> wrote:
On Thu, Nov 3, 2016 at 2:39 PM, Patrick McManus <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robbie, Mike, et al. Thanks for getting the ball rolling with https://tools.ietf.org/html/draft-shade-quic-http2-mapping-00..
What is the thinking about rfc 7540 h2 extensions and the h2/quic mapping.. particularly the way it allows unspecified frame types which may or may not require negotiation.
it seems some of these extensions may have stateful issues across streams, ala hpack and some may not.
If the extension were only defined for h2-over-quic that would be for the extension to sort out.. but it doesn't seem like we are limiting things to quic vs tcp, or are we?

Not sure what you mean by not limiting things to QUIC vs. TCP -- do you have a different mapping in mind?

I meant explicitly scoping h2 extensions so that they only apply to quic and/or tcp.. and it does seem like that's what you mean :)

Gotcha. I think it makes sense to limit to QUIC and TCP at the moment since there is a proposal to run HTTP over these, until such a time as there is a need to define over another transport (let's call it Essy Transport Protocol, you know, EssyTP). I suspect at that point the httpbis wg (hey, that's you! :-)) can figure out how to define extensions over the other transport.

fwiw HPACK/QPACK is not an extension and you can't define it as one in quic because it would need to be negotiated and that's going to break the 0RTT goal of the protocol. I don't think that's a problem though as there is no need to make it an extension :)

Ah, sorry -- my grasp of which things are extensions and which ones not in HTTP-land is weak. I think compression is likely to be one of those things that is different in QUIC than in HTTP/2.

(ii) require that future extensions be defined explicitly  for both TCP and QUIC.

Do you mean an IANA registry? It's probably safest to put these extensions will be in different namespaces entirely, so yes, if I understand your question correctly.

- jana

-P

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 8 Nov 2016 09:57:08 +0100

> Am 08.11.2016 um 03:23 schrieb Mike Bishop <[email-masked]>:
> 

+1

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Nov 2016 13:38:10 -0500

On Mon, Nov 7, 2016 at 5:44 PM, Jana Iyengar <[email-masked]> wrote:

>
>
> Gotcha. I think it makes sense to limit to QUIC and TCP at the moment
> since there is a proposal to run HTTP over these, until such a time as
> there is a need to define over another transport (let's call it Essy
> Transport Protocol, you know, EssyTP). I suspect at that point the httpbis
> wg (hey, that's you! :-)) can figure out how to define extensions over the
> other transport.
>
>
This isn't quite what I mean - I have enough trouble with the 2 we've
already got :) As Mike mentioned, a 7540 extension can do almost anything
it likes and that might mean a baked in assumption about the ordering of
frames that is only valid on TCP. So I would sum the point of the issue to
be raised here as "h2-mapping should say 7540 based extensions only apply
to TCP and quic will maintain its own registry of extensions ala section 11
of 7540"

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 Nov 2016 10:16:44 +1100

On 10 November 2016 at 05:38, Patrick McManus <[email-masked]> wrote:
> "h2-mapping should say 7540 based extensions only apply to TCP and quic will
> maintain its own registry of extensions ala section 11 of 7540"

...or we insist on no assumption of absolute ordering between streams.
The only thing that I'm aware of that has this assumption currently is
HPACK.  I would like it if h2 extensions were usable here, most likely
will be.  But I'd be happy to simply say, no portability unless
explicitly noted.  If you want to talk registries (my goodness, are we
already doing that?) we could add a column to the registry that says
"good for" and list the protocols.

---

From: Cory Benfield <[email-masked]>
Date: Thu, 10 Nov 2016 08:21:37 +0000

> On 9 Nov 2016, at 23:16, Martin Thomson <[email-masked]> wrote:
> 

Don’t SETTINGS frames also imply an absolute ordering between stream 0 and every non-0 stream?

Cory

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 Nov 2016 20:49:03 +1100

> Don’t SETTINGS frames also imply an absolute ordering between stream 0 and every non-0 stream?

You are right.  I completely neglected that.  Use of SETTINGS to
initiate changes that affect streams requires a synchronization point.
Presumably we will have to solve that in the HTTP mapping document
somehow.  Hopefully we can have a generic solution.

---

From: Ian Swett <[email-masked]>
Date: Thu, 10 Nov 2016 05:04:12 -0500

If a single frame type always required in order delivery, but you didn't
want HOL blocking on everything, you could use a stream per frame type.

As such, any given HTTP/2 frame would need to specify if in-order delivery
is necessary, and if so, they should be sent on a single stream.
Otherwise, a stream per frame can be used.  That seems fairly generalizable
to me?

---

From: Cory Benfield <[email-masked]>
Date: Thu, 10 Nov 2016 10:09:45 +0000

> On 10 Nov 2016, at 10:04, Ian Swett <[email-masked]> wrote:
> 
> If a single frame type always required in order delivery, but you didn't want HOL blocking on everything, you could use a stream per frame type. 
> 
> As such, any given HTTP/2 frame would need to specify if in-order delivery is necessary, and if so, they should be sent on a single stream.  Otherwise, a stream per frame can be used.  That seems fairly generalizable to me?

I don’t think it resolves SETTINGS though. H2’s SETTINGS frame includes some connection-wide values that affect all other frames (e.g. SETTINGS_MAX_FRAME_SIZE), so with SETTINGS as written in 7540 it needs to be possible to sort all frames into “before ACK” and “after ACK” if that setting is changed: otherwise it’s very difficult to police the frame size.

A monotonic counter on SETTINGS could perhaps be used to this effect, but it’d require that all frames carry that counter on them and probably that frames from older settings regimes be discarded after a certain period of time. I’m not entirely sure that that’s a good plan, but it’s the first one that leaps out at me.

Cory

---

From: Ian Swett <[email-masked]>
Date: Thu, 10 Nov 2016 05:20:22 -0500

You're right, settings poses some unique challenges because of the
implication on the connection.  I don't think we want to add
settings-specific machinery to every other frame, as that would seem to
increase complexity greatly, though that epoch based approach would work.

It may be possible to constrain mid-connection settings frames in a way
that is not simple and practical.  ie: increasing the max number of streams
should always safe.  A similar approach may be possible with frame size?

On Thu, Nov 10, 2016 at 5:09 AM, Cory Benfield <[email-masked]> wrote:

> > As such, any given HTTP/2 frame would need to specify if in-order
> delivery is necessary, and if so, they should be sent on a single stream.
> I don’t think it resolves SETTINGS though. H2’s SETTINGS frame includes
> some connection-wide values that affect all other frames (e.g.
> SETTINGS_MAX_FRAME_SIZE), so with SETTINGS as written in 7540 it needs to
> be possible to sort all frames into “before ACK” and “after ACK” if that
> setting is changed: otherwise it’s very difficult to police the frame size.
>
> A monotonic counter on SETTINGS could perhaps be used to this effect, but
> it’d require that all frames carry that counter on them and probably that
> frames from older settings regimes be discarded after a certain period of
> time. I’m not entirely sure that that’s a good plan, but it’s the first one
> that leaps out at me.
>
> Cory
>
>

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 10 Nov 2016 11:25:18 +0100

> Am 10.11.2016 um 11:09 schrieb Cory Benfield <[email-masked]>:
> 
> 

Urgh. But if there is no notion of "before" and "after" between frames on different quic streams, as I understand it, the even this counter would not help. As it affects all frames? or?

This raises the question if the h2 framing layer can stay agnostic to the actual transport (I doubt that). And that opens discussions about how behaviour needs to change for these hop-by-hop properties of h2, doesn't it? We are not talking about h2 over a tcp-to-quic gateway, right?

-Stefan

---

From: Cory Benfield <[email-masked]>
Date: Thu, 10 Nov 2016 10:26:56 +0000

> On 10 Nov 2016, at 10:25, Stefan Eissing <[email-masked]> wrote:
> 
>> 

The counter helps only because you identify what SETTINGS regime applies to any given frame. This could also probably be done using some of QUICs packet-tracking and loss-recovery mechanisms (when there are no more frames under SETTINGS regime 3 undelivered we can now forget about SETTINGS regime 3), but I agree: this is pretty gross and I don’t think it’s a good idea. Just a strawman to kick around.

---

From: Mike Bishop <[email-masked]>
Date: Thu, 10 Nov 2016 19:28:58 +0000

It’s (unfortunately) fairly similar to what I’ve been kicking around with regard to loosening HPACK.  You ultimately have to have build some synchronization point to know when you can free memory safely, and that requires knowing when you’re done using the old state.  Or you have some level of optimism about discarding things and have a non-fatal error that forces a re-send on a new stream with the new state.

It seems like there are a couple paths we can go here:

  1.  HTTP/2 runs over a QUIC stream, modulo changes
  2.  Every request gets a stream; Frames related to request maintain order

     *   Requires framing within each stream
     *   Requires changes to HPACK

  1.  Every frame type gets a stream

     *   Cross-type dependencies (SETTINGS, HPACK) become problematic

  1.  Every frame gets a stream

     *   Neuters QUIC’s stream-based flow control, since you can no longer throttle individual requests
     *   Means arbitrarily-large HTTP/2 frames could be permitted without breaking multiplexing

The present draft is #1.  Combining #3 & #4 gives some flexibility, but then requires knowing on a per-frame type behavior which behavior is needed.  I currently lean toward #2, but I’m willing to be convinced otherwise.  Taking #2 to the extreme, you’re no longer carrying HTTP/2 or anything that pretends to be HTTP/2 – you’re defining a fresh way to carry HTTP using a new transport, and most extensions probably need to be redefined if they still apply.

From: QUIC [mailto:[email-masked]] On Behalf Of Cory Benfield
Sent: Thursday, November 10, 2016 2:27 AM
To: Stefan Eissing <[email-masked]>
Cc: Jana Iyengar <[email-masked]>; Ian Swett <[email-masked]>; [email-masked]; Martin Thomson <[email-masked]>; Patrick McManus <[email-masked]>
Subject: Re: [QUIC] h2 mapping - extensions

On 10 Nov 2016, at 10:25, Stefan Eissing <[email-masked]<mailto:[email-masked]>> wrote:

Am 10.11.2016 um 11:09 schrieb Cory Benfield <[email-masked]<mailto:[email-masked]>>:

On 10 Nov 2016, at 10:04, Ian Swett <[email-masked]<mailto:[email-masked]>> wrote:

---

From: Martin Thomson <[email-masked]>
Date: Fri, 11 Nov 2016 10:00:03 +1100

On 11 November 2016 at 06:28, Mike Bishop <[email-masked]> wrote:
> Requires framing within each stream
> Requires changes to HPACK
...
> I currently lean toward #2, but I’m willing to be convinced otherwise.  Taking
> #2 to the extreme, you’re no longer carrying HTTP/2 or anything that
> pretends to be HTTP/2 – you’re defining a fresh way to carry HTTP using a
> new transport, and most extensions probably need to be redefined if they
> still apply.

I also would prefer having everything for a single request on a
stream.  I think that HPACK (and SETTINGS) changes are the small piece
here in terms of absolute magnitude of changes.

That could only need small adjustments to get right. A few extra
pieces, such as a SETTINGS_ACK on every active stream might let you
use SETTINGS reliably, though it's probably a bit unwieldy (and you
would need some special accounting for both past and future streams to
make the feature complete).

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 11 Nov 2016 10:49:33 +0100

> Am 10.11.2016 um 20:28 schrieb Mike Bishop <[email-masked]>:
> [...]
> 	• HTTP/2 runs over a QUIC stream, modulo changes
> 	• Every request gets a stream; Frames related to request maintain order
> 		• Requires framing within each stream
> 		• Requires changes to HPACK
> 	• Every frame type gets a stream
> 		• Cross-type dependencies (SETTINGS, HPACK) become problematic
> 	• Every frame gets a stream
> 		• Neuters QUIC’s stream-based flow control, since you can no longer throttle individual requests
> 		• Means arbitrarily-large HTTP/2 frames could be permitted without breaking multiplexing
>  

I think #2 is the only way forward on this list that will end up with a HTTP transport that has comprehensible properties for the caller. (Well, apart from #1 maybe, but who wants that?)

For completeness sake, there is also #5:
- Every request is done in HTTP/1.1 format on a QUIC stream
- A special QUIC stream is used for signalling only

---

From: Mike Bishop <[email-masked]>
Date: Fri, 11 Nov 2016 18:10:46 +0000

Yes, #5 is basically draft-natarajan-http-over-sctp with a better multi-stream-based transport.  :-)  And while it's a valid option, there's been enough claimed benefit from header compression and a binary format that it's also not my preference.

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 11 Nov 2016 21:40:58 -0800

On Fri, Nov 11, 2016 at 1:49 AM, Stefan Eissing <
[email-masked]> wrote:

> > Am 10.11.2016 um 20:28 schrieb Mike Bishop <[email-masked]
> >:
> > [...]
> > The present draft is #1.  Combining #3 & #4 gives some flexibility, but
> then requires knowing on a per-frame type behavior which behavior is
> needed.  I currently lean toward #2, but I’m willing to be convinced
> otherwise.  Taking #2 to the extreme, you’re no longer carrying HTTP/2 or
> anything that pretends to be HTTP/2 – you’re defining a fresh way to carry
> HTTP using a new transport, and most extensions probably need to be
> redefined if they still apply.
>
> I think #2 is the only way forward on this list that will end up with a
> HTTP transport that has comprehensible properties for the caller. (Well,
> apart from #1 maybe, but who wants that?)
>

​That's basically what QUIC does today, right?​