Subject: [Masque] Thoughts on version conversion

From: Ben Schwartz <[email-masked]>
Date: Wed, 14 Apr 2021 14:59:49 -0400

For use of MASQUE on HTTP paths that use multiple versions of HTTP
(converted by intermediaries), I'd like to see if we can meet the following
requirements:

1. Unreliable datagrams are unreliable on H3 hops even if they have passed
through, or will pass through, H2 or HTTP/1.1.
2. H3<->HTTP/1.1 conversion is equivalent to chained conversion
(H3<->H2<->HTTP/1.1).
3. H3<->H2 conversion can be conducted without any special handling of DATA
frames.
4. "Legacy H2" (i.e. HTTP/2 without any new frame types or SETTINGS) is
also supported, using the same approach as for HTTP/1.1.
5. Intermediaries never have to buffer DATA frame contents (e.g. in order
to merge datagrams into a Body whose internal framing may not align with
the DATA frame boundaries).

I wonder if folks agree with these requirements.  I believe they are all
reasonably achievable.

--Ben Schwartz

---

From: David Schinazi <[email-masked]>
Date: Wed, 14 Apr 2021 12:09:10 -0700

Hi Ben,

- Requirements (1), (2) and (3) sounds good.
- Requirement (4) is a nice-to-have but not absolutely required, but I can
live with it.
- Requirement (5) is not possible given the other ones. If you want to
implement "Legacy H2" -> H3, then you need to buffer DATA frame contents to
parse out TLVs in order to extract the datagrams. This comes from the fact
that HTTP/(2|3) DATA frame boundaries don't carry semantics. Or am I
misunderstanding something?

David

On Wed, Apr 14, 2021 at 12:03 PM Ben Schwartz <bemasc=
[email-masked]> wrote:

> 3. H3<->H2 conversion can be conducted without any special handling of
> DATA frames.
> --
> Masque mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/masque
>

---

From: Ben Schwartz <[email-masked]>
Date: Wed, 14 Apr 2021 16:00:19 -0400

On Wed, Apr 14, 2021 at 3:09 PM David Schinazi <[email-masked]>
wrote:

> parse out TLVs in order to extract the datagrams.
>

Hmm.  I would argue that this conversion is most easily understood as
Legacy H2 -> HTTP/1.1 -> H2 -> H3.  No single step of that process "buffers
DATA frame contents" exactly.  Nonetheless, you're right.  Maybe there's a
clearer way to express this goal, which is really about the complexity of
merging datagrams into a mis-framed Body.

> This comes from the fact that HTTP/(2|3) DATA frame boundaries don't carry
> semantics.
>

This is really what I wanted to highlight.  Can we rely on H2 and H3
intermediaries to preserve DATA frame boundaries, if they also forward
datagrams?  There are some hints that they are supposed to (e.g. RFC 7540:
"Intermediaries SHOULD retain padding for DATA frames"), but it could be
clearer.  If we can't rely on this, then that may motivate defining a new
frame type and avoiding the use of DATA and datagrams on the same stream.

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 19 Apr 2021 12:06:55 +1000

On Thu, Apr 15, 2021, at 06:00, Ben Schwartz wrote:
> On Wed, Apr 14, 2021 at 3:09 PM David Schinazi <[email-masked]> wrote:
> > - Requirement (5) is not possible given the other ones. If you want to implement "Legacy H2" -> H3, then you need to buffer DATA frame contents to parse out TLVs in order to extract the datagrams. 
> 
> Legacy H2 -> HTTP/1.1 -> H2 -> H3.  No single step of that process 
> "buffers DATA frame contents" exactly.  Nonetheless, you're right.  
> Maybe there's a clearer way to express this goal, which is really about 
> the complexity of merging datagrams into a mis-framed Body.

Any buffering should not be possible unless you intend to lift things out into DATAGRAMs.  At that point, the buffering at an intermediary can be very minimal (up to 8 bytes, assuming I'm right, if we use QUIC-style varints).

> > This comes from the fact that HTTP/(2|3) DATA frame boundaries don't carry semantics.
> 
> datagrams?  There are some hints that they are supposed to (e.g. RFC 
> 7540: "Intermediaries SHOULD retain padding for DATA frames"), but it 
> could be clearer.  If we can't rely on this, then that may motivate 
> defining a new frame type and avoiding the use of DATA and datagrams on 
> the same stream.

We can't rely on DATA frame boundaries being preserved.  At a minimum, the maximum size of each is different depending on protocol (and the requirements that drive these maximums are different).

I think that the requirement you are looking to express is just that:  Any use of the underlying HTTP protocol does not depend on the position of units/chunks in the stream of data that is being sent.

That should allow any intermediary to choose the level at which it engages with the data being exchanged.  If it wants to reframe DATA, it can do so.  If it wants to send stuff as DATAGRAM frames in QUIC, then it needs to look at framing.

---

From: Ian Swett <[email-masked]>
Date: Mon, 19 Apr 2021 14:35:31 -0400

I agree with David and Martin that we definitely can't rely on DATA frame
boundaries being preserved.

One question this brought up for me is whether MASQUE can be transported
over 'Legacy HTTP/3' as well as 'Legacy HTTP/2'?  Presumably this question
should have a single answer?

On Sun, Apr 18, 2021 at 10:07 PM Martin Thomson <[email-masked]> wrote:

> Any buffering should not be possible unless you intend to lift things out
> into DATAGRAMs.  At that point, the buffering at an intermediary can be
> very minimal (up to 8 bytes, assuming I'm right, if we use QUIC-style
> varints).
>
> We can't rely on DATA frame boundaries being preserved.  At a minimum, the
> maximum size of each is different depending on protocol (and the
> requirements that drive these maximums are different).
>
> I think that the requirement you are looking to express is just that:  Any
> use of the underlying HTTP protocol does not depend on the position of
> units/chunks in the stream of data that is being sent.
>
> That should allow any intermediary to choose the level at which it engages
> with the data being exchanged.  If it wants to reframe DATA, it can do so.
> If it wants to send stuff as DATAGRAM frames in QUIC, then it needs to look
> at framing.
>
> --
> Masque mailing list
> [email-masked]

---

From: Ben Schwartz <[email-masked]>
Date: Mon, 19 Apr 2021 14:40:19 -0400

Assuming we've missed our chance to get DATAGRAM into H3 baseline, I think
we should support "Legacy HTTP/3" in the same fashion.

On Mon, Apr 19, 2021 at 2:36 PM Ian Swett <ianswett=
[email-masked]> wrote:

>> > > - Requirement (5) is not possible given the other ones. If you want
>> to implement "Legacy H2" -> H3, then you need to buffer DATA frame contents
>> to parse out TLVs in order to extract the datagrams.
>> >
>> We can't rely on DATA frame boundaries being preserved.  At a minimum,
>> the maximum size of each is different depending on protocol (and the
>> I think that the requirement you are looking to express is just that:
>> Any use of the underlying HTTP protocol does not depend on the position of
>> That should allow any intermediary to choose the level at which it
>> engages with the data being exchanged.  If it wants to reframe DATA, it can
>> do so.  If it wants to send stuff as DATAGRAM frames in QUIC, then it needs
>> to look at framing.
>>
>> --
>> Masque mailing list
>> [email-masked]

---

From: David Schinazi <[email-masked]>
Date: Mon, 19 Apr 2021 11:52:16 -0700

At the end of the day, since we've decided to support HTTP/1.1, we'll have
to create a TLV format for sending datagrams inside the data stream, no
matter what. All we need to say is that this format can be carried over the
data stream in more modern versions of HTTP and we're done. Additionally,
intermediaries that want to use the data stream on one hop and some more
modern framing on another hop will have to inspect the data stream. That's
all there is to it: this is pretty straightforward, there aren't many ways
to do this.

The one open question is whether we want to define the data stream framing
layer in the HTTP/3 Datagram document or in the CONNECT-UDP and CONNECT-IP
method documents.

David

On Mon, Apr 19, 2021 at 11:40 AM Ben Schwartz <bemasc=
[email-masked]> wrote:

>>> Any buffering should not be possible unless you intend to lift things
>>> out into DATAGRAMs.  At that point, the buffering at an intermediary can be

---

From: Alex Chernyakhovsky <[email-masked]>
Date: Mon, 19 Apr 2021 14:55:05 -0400

I think defining the data stream framing layer in both CONNECT-UDP and
CONNECT-IP would be redundant and confusing. I would like a single document
to define it, but it could be a separate document from the H3-DATAGRAM
specification. I believe someone (I forget who) expressed a preference it
be a separate document.

Sincerely,
-Alex

On Mon, Apr 19, 2021 at 2:52 PM David Schinazi <[email-masked]>
wrote:

>> Assuming we've missed our chance to get DATAGRAM into H3 baseline, I
>> think we should support "Legacy HTTP/3" in the same fashion.
>>
>>> I agree with David and Martin that we definitely can't rely on DATA
>>> frame boundaries being preserved.
>>>

---

From: Ian Swett <[email-masked]>
Date: Mon, 19 Apr 2021 15:17:57 -0400

On Mon, Apr 19, 2021 at 2:55 PM Alex Chernyakhovsky <[email-masked]>
wrote:

>> At the end of the day, since we've decided to support HTTP/1.1, we'll
>> have to create a TLV format for sending datagrams inside the data stream,
>> no matter what. All we need to say is that this format can be carried over
>> the data stream in more modern versions of HTTP and we're done.
>> Additionally, intermediaries that want to use the data stream on one hop
>> and some more modern framing on another hop will have to inspect the data
>> stream. That's all there is to it: this is pretty straightforward, there
>> aren't many ways to do this.
>>
>
Thanks, that's what I was hoping you'd say.

>
>> The one open question is whether we want to define the data stream
>> framing layer in the HTTP/3 Datagram document or in the CONNECT-UDP and
>> CONNECT-IP method documents.
>>
>
This seems like core functionality that belongs in the main doc to me,
since it's necessary for a proxy which speaks HTTP(/3?) Datagram, but may
not understand all applications which use it.

>>>> One question this brought up for me is whether MASQUE can be
>>>> transported over 'Legacy HTTP/3' as well as 'Legacy HTTP/2'?  Presumably
>>>> this question should have a single answer?
>>>>
>>>>> > > - Requirement (5) is not possible given the other ones. If you
>>>>> want to implement "Legacy H2" -> H3, then you need to buffer DATA frame
>>>>> contents to parse out TLVs in order to extract the datagrams.
>>>>> >

---

From: David Schinazi <[email-masked]>
Date: Mon, 19 Apr 2021 12:34:55 -0700

+1 to not further splitting documents. I think it was important for us to
split out HTTP/3 Datagrams from CONNECT-UDP so that CONNECT-IP and
WebTransport can use HTTP/3 Datagrams, but further splitting things sounds
overkill to me. There is non-negligeable per-draft process overhead that we
should keep in mind.

I'm leaning towards moving the data stream format to HTTP/3 Datagrams, but
not requiring that all applications that want datagrams use it. CONNECT-UDP
and CONNECT-IP will want to use it but maybe some others won't.

Also, we'll probably want to rename HTTP/3 Datagrams to HTTP Datagrams at
some point.

David

On Mon, Apr 19, 2021 at 12:18 PM Ian Swett <[email-masked]> wrote:

>>>>>> We can't rely on DATA frame boundaries being preserved.  At a
>>>>>> minimum, the maximum size of each is different depending on protocol (and
>>>>>> the requirements that drive these maximums are different).
>>>>>>
>>>>>> I think that the requirement you are looking to express is just
>>>>>> that:  Any use of the underlying HTTP protocol does not depend on the
>>>>>> position of units/chunks in the stream of data that is being sent.
>>>>>>

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 19 Apr 2021 20:35:14 +0100

> not understand all applications which use it.

In my view what's emerging is that via QUIC datagram, we're designing
"associated message contexts for HTTP" or whatever we might call it. This
feature has concrete signalling (TBD) and concrete formats (TBD).

The doc we have today is titled in a way that focuses on just one of those
concrete formats. There seems to be no suggestion about designing means to
carry QUIC datagrams in streams, just how to encode the L7 message as
stream data.

I'm with Ian that defining signalling and different message bearers in the
same document is a logically valid thing. I just think we might need to
re-orient the H3 dgram draft title and abstract to do so. Or spin out a
document that talks about abstract concepts.

Cheers
Lucas