Subject: Re: [Masque] ECN & Flow IDs

From: Martin Duke <[email-masked]>
Date: Sat, 3 Apr 2021 13:37:39 -0700

There's a problem with ECN (and anything else) that is both an extension
and implies a different datagram format: IIUC, you are likely to add 1RTT
of latency.

For example, with ECN, say there was just an "ecn" header field that mean
the first byte was the TOS byte. If the server doesn't understand the
extension, then datagrams that arrive before the response will be
incorrectly parsed. It is therefore unsafe to send packets before the
capability if fully negotiated.

With David's design, datagrams with ECT(0) are simply dropped by the proxy,
because it doesn't have a mapping for the flow ID. To avoid loss, the
client can simply send Not-ECT until it gets a response. This isn't a great
outcome, but far better than always having to withhold the first flight.

Am I misunderstanding how this would work in practice?

On Tue, Mar 30, 2021 at 7:31 PM Alex Chernyakhovsky <[email-masked]>
wrote:

> I don't think ECN is a good example for discussing datagram flow IDs. I
> have a github issue (
> https://github.com/ietf-wg-masque/draft-ietf-masque-h3-datagram/issues/35)
> out to remove the ECN example. (I much prefer passing the ECN information
> in the datagram payload, prefixed ahead of the packet as David describes).
>
> In addition to the issues that David describes, I think we should also
> consider whether the Datagram-Flow-Id header is a list separately from
> whether we allow a 1:many mapping between streams and datagram flow IDs. (I
> would like to make sure we do not have a many:many or many:1 mapping where
> multiple streams are responsible for multiple datagram flow IDs)
>
> Sincerely
> -Alex
>
> On Tue, Mar 30, 2021 at 8:36 PM David Schinazi <[email-masked]>
> wrote:
>
>> The trivial other approach to solving ECN would be to prefix the UDP
>> payload with a type byte that contains the two ECN bits and 6 unused bits.
>> That would definitely work, and therefore I don't think ECN is a good
>> example to use as a discussion starter for how flow IDs are managed.
>> I wrote that draft to test out the extensibility of the CONNECT-UDP
>> design, I'm not planning on moving it forward at this time.
>>
>> More conceptually, the main question is whether we allow one
>> client-initiated bidirectional stream to map to multiple datagram flow IDs.
>> The approach I've taken is to say yes: that allows us to reuse the flow
>> ID multiplexing logic to encode additional information in the flow ID.
>> You could build something isomorphic to this where you say no and have
>> one flow ID per stream, and then add a second varint right after the flow
>> ID.
>> Since datagrams have to fit in a QUIC packet, adding more varints eats
>> into available datagram MTU which makes me prefer the multiple flow IDs per
>> stream approach.
>> Both are pretty much equivalent in terms of implementation complexity:
>> either way you need a hash table mapping from a varint to what that varint
>> means.
>> There is more complexity in how you convey that mapping (right now this
>> is done using parameters on the Datagram-Flow-Id header), but I don't think
>> requiring one flow ID per stream solves any of that complexity, you'll
>> still need a way to convey extension information. The ECN example is so
>> simple that
>> it doesn't require sending much extension information, but if you look
>> further at enabling optional IP header compression in CONNECT-IP, then
>> you'll want
>> a way to associate a varint with which IP addresses you're compressing.
>>
>> I think the question we have to answer becomes: do we want MASQUE
>> protocols to be extensible? Our options are:
>> - disallow extensibility and slightly simplify the protocol
>> - allow extensibility via multiple flow IDs per stream, and deal with the
>> slight complexity
>> - allow extensibility with a single flow ID per stream, and deal with the
>> slight complexity
>> Personally, I feel strongly that we should allow extensibility.
>>
>> David
>>
>>
>> On Tue, Mar 30, 2021 at 3:40 PM Martin Duke <[email-masked]>
>> wrote:
>>
>>> Hello MASQUE,
>>>
>>> At IETF 110 there was a lot of good discussion challenging the
>>> foundations of the CONNECT-UDP framework, including the relationship
>>> between streams and Flow-IDs. While CONNECT-UDP happens to use flow IDs
>>> somewhat incidentally, the real action with the controversial
>>> multiple-flow-id-per-CONNECT happens in David's (unadopted) ECN draft:
>>> https://datatracker.ietf.org/doc/draft-schinazi-masque-connect-udp-ecn/
>>>
>>> Leading up to the interim, it would be great if one of the detractors of
>>> the flow-id mapping submitted their own approach to solve ECN. This would
>>> help to illuminate the tradeoffs. Speaking as an individual, I am also
>>> hoping to move the ECN work forward and having another design would help to
>>> do that.
>>>
>>> Thanks,
>>> Martin
>>> --
>>> Masque mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/masque
>>>
>> --
>> Masque mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/masque
>>
>

---

From: David Schinazi <[email-masked]>
Date: Sun, 4 Apr 2021 14:18:35 -0700

Martin, your understanding matches mine. I think using RFC3168-style ECN
(where the client's first flight isn't ECN-Capable) seems like a decent
solution here.

David

On Sat, Apr 3, 2021 at 1:37 PM Martin Duke <[email-masked]> wrote:

> With David's design, datagrams with ECT(0) are simply dropped by the
> proxy, because it doesn't have a mapping for the flow ID. To avoid loss,
> the client can simply send Not-ECT until it gets a response. This isn't a
> great outcome, but far better than always having to withhold the
> first flight.
>
>>> - allow extensibility via multiple flow IDs per stream, and deal with
>>> the slight complexity
>>> - allow extensibility with a single flow ID per stream, and deal with
>>> the slight complexity
>>>> Leading up to the interim, it would be great if one of the detractors
>>>> of the flow-id mapping submitted their own approach to solve ECN. This
>>>> would help to illuminate the tradeoffs. Speaking as an individual, I am
>>>> also hoping to move the ECN work forward and having another design would
>>>> help to do that.
>>>>
>>>> Thanks,
>>>> Martin
>>>> --
>>>> Masque mailing list
>>>> [email-masked]

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 7 Apr 2021 12:42:06 +0000

I wouldn’t call that a decent solution as there is work underway to make all packets ECN capable. This is especially important  since losing the first packet of a connection usually has a higher delay penalty than any other packet.

From: Masque <[email-masked]> on behalf of David Schinazi <[email-masked]>
Date: Sunday, 4. April 2021 at 23:19
To: Martin Duke <[email-masked]>
Cc: Alex Chernyakhovsky <[email-masked]>, MASQUE <[email-masked]>
Subject: Re: [Masque] ECN & Flow IDs

Martin, your understanding matches mine. I think using RFC3168-style ECN (where the client's first flight isn't ECN-Capable) seems like a decent solution here.

David

On Sat, Apr 3, 2021 at 1:37 PM Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:
There's a problem with ECN (and anything else) that is both an extension and implies a different datagram format: IIUC, you are likely to add 1RTT of latency.

For example, with ECN, say there was just an "ecn" header field that mean the first byte was the TOS byte. If the server doesn't understand the extension, then datagrams that arrive before the response will be incorrectly parsed. It is therefore unsafe to send packets before the capability if fully negotiated.

With David's design, datagrams with ECT(0) are simply dropped by the proxy, because it doesn't have a mapping for the flow ID. To avoid loss, the client can simply send Not-ECT until it gets a response. This isn't a great outcome, but far better than always having to withhold the first flight.

On Tue, Mar 30, 2021 at 7:31 PM Alex Chernyakhovsky <[email-masked]<mailto:[email-masked]>> wrote:
I don't think ECN is a good example for discussing datagram flow IDs. I have a github issue (https://github.com/ietf-wg-masque/draft-ietf-masque-h3-datagram/issues/35<https://protect2.fireeye.com/v1/url?k=408475ca-1f1f4cab-40843551-86d8a30ca42b-3047b2712c0c3067&q=1&e=090bd79d-94f2-4b16-9aa9-8dcf0b51791f&u=https%3A%2F%2Fgithub.com%2Fietf-wg-masque%2Fdraft-ietf-masque-h3-datagram%2Fissues%2F35>) out to remove the ECN example. (I much prefer passing the ECN information in the datagram payload, prefixed ahead of the packet as David describes).

In addition to the issues that David describes, I think we should also consider whether the Datagram-Flow-Id header is a list separately from whether we allow a 1:many mapping between streams and datagram flow IDs. (I would like to make sure we do not have a many:many or many:1 mapping where multiple streams are responsible for multiple datagram flow IDs)

Sincerely
-Alex

On Tue, Mar 30, 2021 at 8:36 PM David Schinazi <[email-masked]<mailto:[email-masked]>> wrote:
The trivial other approach to solving ECN would be to prefix the UDP payload with a type byte that contains the two ECN bits and 6 unused bits.
That would definitely work, and therefore I don't think ECN is a good example to use as a discussion starter for how flow IDs are managed.
I wrote that draft to test out the extensibility of the CONNECT-UDP design, I'm not planning on moving it forward at this time.

More conceptually, the main question is whether we allow one client-initiated bidirectional stream to map to multiple datagram flow IDs.
The approach I've taken is to say yes: that allows us to reuse the flow ID multiplexing logic to encode additional information in the flow ID.
You could build something isomorphic to this where you say no and have one flow ID per stream, and then add a second varint right after the flow ID.
Since datagrams have to fit in a QUIC packet, adding more varints eats into available datagram MTU which makes me prefer the multiple flow IDs per stream approach.
Both are pretty much equivalent in terms of implementation complexity: either way you need a hash table mapping from a varint to what that varint means.
There is more complexity in how you convey that mapping (right now this is done using parameters on the Datagram-Flow-Id header), but I don't think
requiring one flow ID per stream solves any of that complexity, you'll still need a way to convey extension information. The ECN example is so simple that
it doesn't require sending much extension information, but if you look further at enabling optional IP header compression in CONNECT-IP, then you'll want
a way to associate a varint with which IP addresses you're compressing.

I think the question we have to answer becomes: do we want MASQUE protocols to be extensible? Our options are:
- disallow extensibility and slightly simplify the protocol
- allow extensibility via multiple flow IDs per stream, and deal with the slight complexity
- allow extensibility with a single flow ID per stream, and deal with the slight complexity
Personally, I feel strongly that we should allow extensibility.

David

On Tue, Mar 30, 2021 at 3:40 PM Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:
Hello MASQUE,

At IETF 110 there was a lot of good discussion challenging the foundations of the CONNECT-UDP framework, including the relationship between streams and Flow-IDs. While CONNECT-UDP happens to use flow IDs somewhat incidentally, the real action with the controversial multiple-flow-id-per-CONNECT happens in David's (unadopted) ECN draft: https://datatracker.ietf.org/doc/draft-schinazi-masque-connect-udp-ecn/

Leading up to the interim, it would be great if one of the detractors of the flow-id mapping submitted their own approach to solve ECN. This would help to illuminate the tradeoffs. Speaking as an individual, I am also hoping to move the ECN work forward and having another design would help to do that.

Thanks,
Martin
--
Masque mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/masque
--
Masque mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/masque

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 7 Apr 2021 17:00:27 +0000

Hi David, hi all,

to also add to your question about multiple flow IDs. I guess we kind of agree that we don’t want flow ID for ECN information, as adding two bits is simple enough, however, then it is actually not fully clear to me why multiple flow IDs per connect request are needed. You briefly mentioned other extension below, however, not all, or only very few information, require per-packet information (as it was used for the ECN example). If you want to actually send multiple flows/connections to the same server you, can always send multiple connect requests. That’s slightly more overhead at connection establishment but not much, and I would say it’s architecturally more clean and therefore also simpler.

Mirja

From: Masque <[email-masked]> on behalf of David Schinazi <[email-masked]>
Date: Wednesday, 31. March 2021 at 02:36
To: Martin Duke <[email-masked]>
Cc: MASQUE <[email-masked]>
Subject: Re: [Masque] ECN & Flow IDs

David

Thanks,
Martin

---

From: Martin Duke <[email-masked]>
Date: Wed, 7 Apr 2021 10:06:37 -0700

Again, adding two bits to the datagram payload is not "simple enough" if
it's an extension.

If it's part of the CONNECT-UDP standard, then sure.

Otherwise, there is ambiguity in processing an H3 DATAGRAM frame sent by
the client before receiving the response from the server. The client can
send not-ECT or eat the 1RTT latency penalty.

With flow IDs, the proxy will simply drop the datagram, incurring a 1RTT
penalty only if it doesn't support the extension.

On Wed, Apr 7, 2021 at 10:00 AM Mirja Kuehlewind <
[email-masked]> wrote:

> Hi David, hi all,
>
>
>
> to also add to your question about multiple flow IDs. I guess we kind of
> agree that we don’t want flow ID for ECN information, as adding two bits is
> simple enough, however, then it is actually not fully clear to me why
> multiple flow IDs per connect request are needed. You briefly mentioned
> other extension below, however, not all, or only very few information,
> require per-packet information (as it was used for the ECN example). If you
> want to actually send multiple flows/connections to the same server you,
> can always send multiple connect requests. That’s slightly more overhead at
> connection establishment but not much, and I would say it’s architecturally
> more clean and therefore also simpler.
>
>
>
> Mirja
>
>
>
>
>
> *From: *Masque <[email-masked]> on behalf of David Schinazi <
> [email-masked]>
> *Date: *Wednesday, 31. March 2021 at 02:36
> *To: *Martin Duke <[email-masked]>
> *Cc: *MASQUE <[email-masked]>
> *Subject: *Re: [Masque] ECN & Flow IDs
>
>
>
> The approach I've taken is to say yes: that allows us to reuse the flow ID
> multiplexing logic to encode additional information in the flow ID.
>
> You could build something isomorphic to this where you say no and have one
> flow ID per stream, and then add a second varint right after the flow ID.
>
> There is more complexity in how you convey that mapping (right now this is
> done using parameters on the Datagram-Flow-Id header), but I don't think
>
> At IETF 110 there was a lot of good discussion challenging the foundations
> of the CONNECT-UDP framework, including the relationship between streams
> and Flow-IDs. While CONNECT-UDP happens to use flow IDs somewhat
> incidentally, the real action with the controversial

---

From: David Schinazi <[email-masked]>
Date: Wed, 7 Apr 2021 10:53:05 -0700

On Wed, Apr 7, 2021 at 10:00 AM Mirja Kuehlewind <
[email-masked]> wrote:

> require per-packet information (as it was used for the ECN example).
>

Imagine an extension to CONNECT-IP that allows compressing IP addresses.
That would change the datagram format.

If you want to actually send multiple flows/connections to the same server
> you, can always send multiple connect requests. That’s slightly more
> overhead at connection establishment but not much, and I would say it’s
> architecturally more clean and therefore also simpler.
>

That wouldn't work because two separate requests have (a) different
lifetimes and (b) can land on different HTTP backends.

David

Mirja

---

From: Alex Chernyakhovsky <[email-masked]>
Date: Wed, 7 Apr 2021 13:53:26 -0400

Hi Martin,

What's stopping the client from opportunistically sending the packet with
the extension in the same flight as requesting the extension? Then you'd
at-best get the expected behavior and at-worst fall back to the
1RTT penalty.

Sincerely,
-Alex

---

From: Martin Duke <[email-masked]>
Date: Wed, 7 Apr 2021 11:21:41 -0700

Alex,

Maybe I'm missing something obvious, but here's an example.

Let's say that the "ecn" extension means the first octet of the datagram
after the flow ID is the ToS byte.

1. The client sends a CONNECT-UDP request with the "ecn" extension
2. The client sends a datagram with the second byte reflecting the ToS.
3. The proxy does not understand the ecn extension and ignores it
4. The datagram arrives; the proxy sends a UDP packet with the ToS byte as
the first byte of payload.
5. The client receives the response without the "ecn" extension and stops
sending the ToS byte.

In #4 we're going to have some weird undefined behavior.

Compare this to David's draft

1. The client sends a CONNECT-UDP (flowID = 12) request with the "ecn"
extension (flowID = 16 for ECT(0))
2. The client sends a datagram with flowID = 16
3. The proxy does not understand the ecn extension and ignores it
4. The datagram arrives; the proxy drops flowID 16 because it doesn't know
what that is
5. The client receives the response without the "ecn" extension and stops
sending flowID 16.

packet losses aren't great, but are better than undefined behavior.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 8 Apr 2021 08:53:03 +0000

Hi Martin,

yes this is a problem that needs solving but I don’t think it’s unsolvable. It all depends on how exactly the negotiation works, e.g. the client could propose a flow ID with the connect request, however, if the proxy, does not accepted the proposed parameters it could select a different flow ID in its reply (and ignore all datagram with the old flow ID).

Mirja

From: Martin Duke <[email-masked]>
Date: Wednesday, 7. April 2021 at 20:22
To: Alex Chernyakhovsky <[email-masked]>
Cc: Mirja Kuehlewind <[email-masked]>, David Schinazi <[email-masked]>, MASQUE <[email-masked]>
Subject: Re: [Masque] ECN & Flow IDs

Alex,

Let's say that the "ecn" extension means the first octet of the datagram after the flow ID is the ToS byte.

1. The client sends a CONNECT-UDP request with the "ecn" extension
2. The client sends a datagram with the second byte reflecting the ToS.
3. The proxy does not understand the ecn extension and ignores it
4. The datagram arrives; the proxy sends a UDP packet with the ToS byte as the first byte of payload.
5. The client receives the response without the "ecn" extension and stops sending the ToS byte.

1. The client sends a CONNECT-UDP (flowID = 12) request with the "ecn" extension (flowID = 16 for ECT(0))
2. The client sends a datagram with flowID = 16
3. The proxy does not understand the ecn extension and ignores it
4. The datagram arrives; the proxy drops flowID 16 because it doesn't know what that is
5. The client receives the response without the "ecn" extension and stops sending flowID 16.

On Wed, Apr 7, 2021 at 10:53 AM Alex Chernyakhovsky <[email-masked]<mailto:[email-masked]>> wrote:
Hi Martin,

What's stopping the client from opportunistically sending the packet with the extension in the same flight as requesting the extension? Then you'd at-best get the expected behavior and at-worst fall back to the 1RTT penalty.

Sincerely,
-Alex

On Wed, Apr 7, 2021 at 1:07 PM Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:
Again, adding two bits to the datagram payload is not "simple enough" if it's an extension.

Otherwise, there is ambiguity in processing an H3 DATAGRAM frame sent by the client before receiving the response from the server. The client can send not-ECT or eat the 1RTT latency penalty.

With flow IDs, the proxy will simply drop the datagram, incurring a 1RTT penalty only if it doesn't support the extension.

On Wed, Apr 7, 2021 at 10:00 AM Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>> wrote:
Hi David, hi all,

Mirja

From: Masque <[email-masked]<mailto:[email-masked]>> on behalf of David Schinazi <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 31. March 2021 at 02:36
To: Martin Duke <[email-masked]<mailto:[email-masked]>>
Cc: MASQUE <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Masque] ECN & Flow IDs

David

Thanks,
Martin

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 8 Apr 2021 08:54:59 +0000

Hi David,

please see inline.

From: David Schinazi <[email-masked]>
Date: Wednesday, 7. April 2021 at 19:53
To: Mirja Kuehlewind <[email-masked]>
Cc: MASQUE <[email-masked]>
Subject: Re: [Masque] ECN & Flow IDs

to also add to your question about multiple flow IDs. I guess we kind of agree that we don’t want flow ID for ECN information, as adding two bits is simple enough, however, then it is actually not fully clear to me why multiple flow IDs per connect request are needed. You briefly mentioned other extension below, however, not all, or only very few information, require per-packet information (as it was used for the ECN example).

Imagine an extension to CONNECT-IP that allows compressing IP addresses. That would change the datagram format.

For compression I would usually think that things are negotiated on a per connect association basis and that you doesn’t necessarily need to provide any information on a per packet basis. However, if there are any per packet information needed, you can always add another extension specific field after the flow ID.

If you want to actually send multiple flows/connections to the same server you, can always send multiple connect requests. That’s slightly more overhead at connection establishment but not much, and I would say it’s architecturally more clean and therefore also simpler.

That wouldn't work because two separate requests have (a) different lifetimes and (b) can land on different HTTP backends.

If you have two different UDP “connections” without a proxy in between, this is the same, no? Maybe we are talking about different scenarios here?

Mirja

David

Mirja

David

Thanks,
Martin

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 8 Apr 2021 08:56:02 +0000

Hi David,

please see inline.

Mirja

David

Mirja

David

Thanks,
Martin

---

From: Alex Chernyakhovsky <[email-masked]>
Date: Thu, 8 Apr 2021 10:29:02 -0400

Hi Martin,

There's a few issues with your example, which is namely that step (4)
should not occur. The extension needs to be pre-declared, or else the proxy
does as you describe and we have undefined behavior. I instead was
imagining a header (or some other signalling mechanism) that says "we want
to use ecn extension" and when the proxy gets its first payload after that
header, it drops it because it doesn't know about the extension and replies
back that it doesn't support it. The client then has to send the 1RTT
value. You end up with the same behavior as David's draft.

of course, we can also fix this more explicitly in the CONNECT-UDP draft by
creating another stream-chunk type for UDP packet with additional data for
extensions in addition to the packet. Then all a proxy has to know is
there's data it doesn't support, and it just doesn't read and act upon it
when sending the packet payload.

Sincerely,
-Alex

On Wed, Apr 7, 2021 at 2:21 PM Martin Duke <[email-masked]> wrote:

>>>> to also add to your question about multiple flow IDs. I guess we kind
>>>> of agree that we don’t want flow ID for ECN information, as adding two bits
>>>> is simple enough, however, then it is actually not fully clear to me why

---

From: David Schinazi <[email-masked]>
Date: Thu, 8 Apr 2021 08:36:46 -0700

On Thu, Apr 8, 2021 at 1:56 AM Mirja Kuehlewind <
[email-masked]> wrote:

> Hi David,
>
>
>
> please see inline.
>
>
>
> *From: *David Schinazi <[email-masked]>
> *Date: *Wednesday, 7. April 2021 at 19:53
> *To: *Mirja Kuehlewind <[email-masked]>
> For compression I would usually think that things are negotiated on a per
> connect association basis and that you doesn’t necessarily need to provide
> any information on a per packet basis. However, if there are any per packet
> information needed, you can always add another extension specific field
> after the flow ID.
>
>
>
> If you have two different UDP “connections” without a proxy in between,
> this is the same, no? Maybe we are talking about different scenarios here?
>

If it's the same "connection" (let's say different ECN values for the same
UDP "connection", or different compression contexts for the same IP tunnel)
then you need to go through the same backend. That's why you need a way to
use multiple encodings in one "connection".

David

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 8 Apr 2021 19:38:44 +0000

See below.

Date: Thursday, 8. April 2021 at 17:37

On Thu, Apr 8, 2021 at 1:56 AM Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>> wrote:
Hi David,

please see inline.

From: David Schinazi <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 7. April 2021 at 19:53
To: Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>>
Cc: MASQUE <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Masque] ECN & Flow IDs

If it's the same "connection" (let's say different ECN values for the same UDP "connection", or different compression contexts for the same IP tunnel) then you need to go through the same backend. That's why you need a way to use multiple encodings in one "connection".

What I trying to say is that if it is the same connection, you should use the same flow ID. If additional per-packet information is needed an extension can add another field after the flow ID.

Mirja

David

Mirja

David

Mirja

David

Thanks,
Martin

---

From: Martin Duke <[email-masked]>
Date: Thu, 8 Apr 2021 13:28:49 -0700

Hi Alex,

I'm not sure what you mean by "sending the 1RTT value" but IIUC you're
proposing that the proxy drops all payloads if it doesn't understand the
extension? How would it know when DATAGRAMs transfer from the extension
format to the non-extension format?

Or are you saying this would be a connection-level capability negotiated by
SETTINGS or something?

On Thu, Apr 8, 2021 at 7:29 AM Alex Chernyakhovsky <[email-masked]>
wrote:

> of course, we can also fix this more explicitly in the CONNECT-UDP draft
> by creating another stream-chunk type for UDP packet with additional data
> for extensions in addition to the packet. Then all a proxy has to know is
>>> What's stopping the client from opportunistically sending the packet
>>> with the extension in the same flight as requesting the extension? Then
>>> you'd at-best get the expected behavior and at-worst fall back to the
>>> 1RTT penalty.
>>>
>>> Sincerely,
>>> -Alex
>>>
>>>> Otherwise, there is ambiguity in processing an H3 DATAGRAM frame sent
>>>> by the client before receiving the response from the server. The client can
>>>> With flow IDs, the proxy will simply drop the datagram, incurring a
>>>> 1RTT penalty only if it doesn't support the extension.
>>>>
>>>>> The approach I've taken is to say yes: that allows us to reuse the
>>>>> flow ID multiplexing logic to encode additional information in the flow ID.
>>>>>
>>>>> There is more complexity in how you convey that mapping (right now
>>>>> this is done using parameters on the Datagram-Flow-Id header), but I don't
>>>>> think
>>>>>

---

From: Alex Chernyakhovsky <[email-masked]>
Date: Thu, 8 Apr 2021 19:36:37 -0400

Hi Martin,

Yes, I was imagining that the presence if this extension is negotiated,
possibly by way of SETTINGS. Any

Sincerely,
-Alex

On Thu, Apr 8, 2021 at 4:29 PM Martin Duke <[email-masked]> wrote:

> Or are you saying this would be a connection-level capability negotiated
> by SETTINGS or something?
>
>>>>>> *From: *Masque <[email-masked]> on behalf of David Schinazi
>>>>>> <[email-masked]>
>>>>>> You could build something isomorphic to this where you say no and
>>>>>> have one flow ID per stream, and then add a second varint right after the
>>>>>> flow ID.
>>>>>>
>>>>>> Since datagrams have to fit in a QUIC packet, adding more
>>>>>> varints eats into available datagram MTU which makes me prefer the multiple
>>>>>> flow IDs per stream approach.
>>>>>>
>>>>>> Both are pretty much equivalent in terms of implementation
>>>>>> complexity: either way you need a hash table mapping from a varint to what
>>>>>> that varint means.
>>>>>>
>>>>>> requiring one flow ID per stream solves any of that
>>>>>> complexity, you'll still need a way to convey extension information. The
>>>>>> ECN example is so simple that
>>>>>>
>>>>>> it doesn't require sending much extension information, but if you
>>>>>> look further at enabling optional IP header compression in CONNECT-IP, then
>>>>>> you'll want
>>>>>>

---

From: Martin Duke <[email-masked]>
Date: Thu, 15 Apr 2021 09:02:38 -0700

Hi Alex,

Thanks for clarifying. IIUC one objective of David's design is to not
advertise MASQUE capabilities quite so explicitly, which is a drawback of a
SETTINGS-based mechanism.

On Thu, Apr 8, 2021, 16:36 Alex Chernyakhovsky <[email-masked]> wrote:

>>>> Let's say that the "ecn" extension means the first octet of the
>>>> datagram after the flow ID is the ToS byte.
>>>>
>>>> On Wed, Apr 7, 2021 at 10:53 AM Alex Chernyakhovsky <
>>>> [email-masked]> wrote:
>>>>
>>>>> Hi Martin,
>>>>>
>>>>>>> to also add to your question about multiple flow IDs. I guess we
>>>>>>> kind of agree that we don’t want flow ID for ECN information, as adding two
>>>>>>> bits is simple enough, however, then it is actually not fully clear to me
>>>>>>> why multiple flow IDs per connect request are needed. You briefly mentioned
>>>>>>> *From: *Masque <[email-masked]> on behalf of David
>>>>>>> Schinazi <[email-masked]>
>>>>>>> That would definitely work, and therefore I don't think ECN is a
>>>>>>> good example to use as a discussion starter for how flow IDs are managed.
>>>>>>>
>>>>>>> Leading up to the interim, it would be great if one of the
>>>>>>> detractors of the flow-id mapping submitted their own approach to solve
>>>>>>> ECN. This would help to illuminate the tradeoffs. Speaking as an
>>>>>>> individual, I am also hoping to move the ECN work forward and having
>>>>>>> another design would help to do that.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>> --
>>>>>>> Masque mailing list
>>>>>>> [email-masked]

---

From: David Schinazi <[email-masked]>
Date: Thu, 15 Apr 2021 09:44:41 -0700

You can't use SETTINGS for negotiating CONNECT-UDP extensions because
SETTINGS are per-hop and methods are end-to-end.

David

On Thu, Apr 15, 2021 at 9:03 AM Martin Duke <[email-masked]> wrote:

>>>> of course, we can also fix this more explicitly in the CONNECT-UDP
>>>> draft by creating another stream-chunk type for UDP packet with additional
>>>> data for extensions in addition to the packet. Then all a proxy has to know
>>>> is there's data it doesn't support, and it just doesn't read and act upon
>>>> it when sending the packet payload.
>>>>
>>>> Sincerely,
>>>> -Alex
>>>>
>>>>>>> Otherwise, there is ambiguity in processing an H3 DATAGRAM frame
>>>>>>> sent by the client before receiving the response from the server. The
>>>>>>> client can send not-ECT or eat the 1RTT latency penalty.
>>>>>>>
>>>>>>>> The trivial other approach to solving ECN would be to prefix the
>>>>>>>> UDP payload with a type byte that contains the two ECN bits and 6 unused
>>>>>>>> bits.
>>>>>>>>
>>>>>>>> On Tue, Mar 30, 2021 at 3:40 PM Martin Duke <
>>>>>>>> [email-masked]> wrote:
>>>>>>>>
>>>>>>>> Hello MASQUE,
>>>>>>>>
>>>>>>>>
>>>>>>>>