Subject: [Masque] An update con CONNECT-UDP

From: David Schinazi <[email-masked]>
Date: Sat, 12 Dec 2020 20:53:24 -0800

Hi MASQUE enthusiasts,

As discussed at IETF 109, I've made some changes to CONNECT-UDP.

Here are the highlights:

1) Draft structure / adoption inversion
- draft-schinazi-quic-h3-datagram has been renamed to
    draft-schinazi-masque-h3-datagram
- The Datagram-Flow-Id header has been moved from
    draft-ietf-masque-connect-udp to draft-schinazi-masque-h3-datagram

2) HTTP intermediaries
- There is now text in all drafts to properly handle intermediaries
- The technical solution is based on the H3_DATAGRAM SETTING

3) Extensibility
- The CONNECT-UDP stream format is now extensible
- Datagram-Flow-Id now allows multiple flow IDs for extensibility
- I wrote up an ECN extension to CONNECT-UDP to test out this extensibility

4) Misc
- Added text in order to close 16 issues

Here are the links to the drafts:
https://tools.ietf.org/html/draft-schinazi-masque-h3-datagram-01
https://tools.ietf.org/html/draft-ietf-masque-connect-udp-01
https://tools.ietf.org/html/draft-schinazi-masque-connect-udp-ecn-00

Let me know what you think!

Cheers,
David

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 16 Dec 2020 18:16:55 +0000

Hi David,

thanks for the updates. Most changes look good to me. As you may have seen, I left a few comments in the github issues. I think some might have been closed to premature, however, in any case, I think it would be helpful to also note in the issue when you close t how an issue was addressed in new draft version, rather than only saying that is was addressed, given that was in most cases not clear to me from the discussion within the issue.

I also reviewed the ECN document. This looks like a viable approach to me. I still think, while negotiation is needed anyway, that this should not be called an extension but part of the base spec that is implement by everybody who supports masque. The mechanism in the draft is not a lot of effort to implement and requiring it as part of the base protocol would help to makes sure all negotiation mechanisms specified to far are used, executed, and therefore also tested.

There would be an alternative design for ECN where you “just” provided ECN counter values when needed. This would be kind of similar to the feedback mechanism as used in AccECN. There are pros and cons here but it think it might be worth to have some discussion about possible design approach before moving ahead.

Mirja

From: Masque <[email-masked]> on behalf of David Schinazi <[email-masked]>
Date: Sunday, 13. December 2020 at 05:53
To: MASQUE <[email-masked]>
Subject: [Masque] An update con CONNECT-UDP

Hi MASQUE enthusiasts,

Here are the highlights:

2) HTTP intermediaries

3) Extensibility

4) Misc

Cheers,
David

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 16 Dec 2020 10:31:57 -0800

On Wed, Dec 16, 2020 at 10:17 AM Mirja Kuehlewind <mirja.kuehlewind=
[email-masked]> wrote:

> Hi David,
>
>
>
> thanks for the updates. Most changes look good to me. As you may have
> seen, I left a few comments in the github issues. I think some might have
> been closed to premature, however, in any case, I think it would be helpful
> to also note in the issue when you close t how an issue was addressed in
> new draft version, rather than only saying that is was addressed, given
> that was in most cases not clear to me from the discussion within the issue.
>
>
>
> I also reviewed the ECN document. This looks like a viable approach to me.
> I still think, while negotiation is needed anyway, that this should not be
> called an extension but part of the base spec that is implement by
> everybody who supports masque. The mechanism in the draft is not a lot of
> effort to implement and requiring it as part of the base protocol would
> help to makes sure all negotiation mechanisms specified to far are used,
> executed, and therefore also tested.
>

I would not be in favor of this. It's clearly separable and IMO the push to
add ECN to the core QUIC documents rather than as an extension was a source
of significant additional complexity that could have been factored out.

-Ekr

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 16 Dec 2020 20:53:53 +0000

In my personal opinion based on experience so far with implementing a
MASQUE prototype, CONNECT-UDP with a single flow has a lot of nice
similarity to CONNECT for TCP. This makes it conceptually straightforward
to port the code.

Furthermore, the extraction of Datagram-Flow-Id to the H3 DATAGRAM draft
means that the specification is now quite lean and focused.

I have nothing against ECN but I fear it would add the impression of
complication to CONNECT-UDP. That complication exists if a MASQUE server
chooses to exercise the ECN feature but since it's optional then I'd expect
several implementations to ignore it regardless of where the text lives.
For these reasons, I'm happy with separate documents.

Cheers
Lucas

---

From: David Schinazi <[email-masked]>
Date: Thu, 17 Dec 2020 18:46:09 +0100

Hi Mirja, I agree with EKR and Lucas here. Details inline.

> that was in most cases not clear to me from the discussion within the issue.

Sorry about that, I unfortunately didn't have the time to do that given
other work priorities.

> executed, and therefore also tested.

You cannot make ECN a mandatory part of CONNECT-UDP, for the same reason
that we didn't make it mandatory in QUIC: not all operating systems let you
read and write the ECN bits in the IP header. Additionally, some
implementers do not want to spend resources implementing a feature that is
currently unused on the Internet (see data from Apple proving that).

> There would be an alternative design for ECN where you “just” provided ECN
> counter values when needed. This would be kind of similar to the feedback
> mechanism as used in AccECN. There are pros and cons here but it think it
> might be worth to have some discussion about possible design approach
> before moving ahead.
>

I absolutely agree with this, and it's a great argument for making this an
extension - that way interested implementers can try out two separate ECN
extensions and compare real-world performance data. Furthermore, analyzing
these tradeoffs will take time, and that should delay the publication of
the ECN extension but it absolutely should not delay the publication of
CONNECT-UDP.

David

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 17 Dec 2020 17:56:16 +0000

Hi David,

regarding this point:

“You cannot make ECN a mandatory part of CONNECT-UDP, for the same reason that we didn't make it mandatory in QUIC: not all operating systems let you read and write the ECN bits in the IP header.“

You cannot make it mandatory to use and that’s why I said below we need negotiation in any case, however, in QUIC we explicitly decided to include it in the base spec in order to make it at least appear mandatory to implement. I think we should follow that example and logically consistent do the same here.

Mirja

From: David Schinazi <[email-masked]>
Date: Thursday, 17. December 2020 at 18:46
To: Mirja Kuehlewind <[email-masked]>
Cc: MASQUE <[email-masked]>
Subject: Re: [Masque] An update con CONNECT-UDP

On Wed, Dec 16, 2020 at 7:16 PM Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>> wrote:
Hi David,

Sorry about that, I unfortunately didn't have the time to do that given other work priorities.

You cannot make ECN a mandatory part of CONNECT-UDP, for the same reason that we didn't make it mandatory in QUIC: not all operating systems let you read and write the ECN bits in the IP header. Additionally, some implementers do not want to spend resources implementing a feature that is currently unused on the Internet (see data from Apple proving that).

I absolutely agree with this, and it's a great argument for making this an extension - that way interested implementers can try out two separate ECN extensions and compare real-world performance data. Furthermore, analyzing these tradeoffs will take time, and that should delay the publication of the ECN extension but it absolutely should not delay the publication of CONNECT-UDP.

David

---

From: David Schinazi <[email-masked]>
Date: Thu, 17 Dec 2020 19:04:56 +0100

I'm not sure what your point is, most QUIC stacks did not implement ECN.
What are you trying to accomplish?

David

On Thu, Dec 17, 2020 at 6:56 PM Mirja Kuehlewind <
[email-masked]> wrote:

> Hi David,
>
>
>
> regarding this point:
>
>
>
> “You cannot make ECN a mandatory part of CONNECT-UDP, for the same reason
> read and write the ECN bits in the IP header.“
>
>
>
> You cannot make it mandatory to use and that’s why I said below we need
> negotiation in any case, however, in QUIC we explicitly decided to include
> it in the base spec in order to make it at least appear mandatory to
> implement. I think we should follow that example and logically consistent
> do the same here.
>
>
>
> Mirja
>
>
>
> *From: *David Schinazi <[email-masked]>
> *Date: *Thursday, 17. December 2020 at 18:46
> *To: *Mirja Kuehlewind <[email-masked]>
> *Cc: *MASQUE <[email-masked]>
> *Subject: *Re: [Masque] An update con CONNECT-UDP
>
>
>

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 17 Dec 2020 20:10:07 +0000

I'm also wondering about the effective population that CONNECT-UDP ECN
support appeals to. CONNECT TCP doesn't support anything ECN, QUIC has ECN
support built in, so this works only for the subset of people tunneling
non-QUIC UDP protocols over MASQUE? Or am I missing something?

Cheers
Lucas

---

From: Tommy Pauly <[email-masked]>
Date: Sat, 19 Dec 2020 10:09:27 -0800

I think it’s slightly different from normal CONNECT, given that in normal CONNECT, there are separate TCP contexts on each side of the proxy that could theoretically react to ECN. With CONNECT-UDP proxying a QUIC connection, the signaling of ECN bits would need to be able to get back to the client to fully function. This is one of the many ways that we need to consider CONNECT-UDP as distinct from CONNECT. It’s also different in how nested congestion control interacts, how effective MTU works, etc.

That said, I think it is fine to have the mechanics of ECN for CONNECT-UDP be defined in a separate document—I totally expect that many use cases may not use it, particularly if they expect their primary benefit from ECN to be on the link between the client and the proxy, not between the proxy and the target. However, some deployments will care a lot about it.

I think what would be useful to have in the core CONNECT-UDP document is more discussion of the ramifications of using CONNECT-UDP, which include interactions with ECN, congestion control, nested ACKs, MTU impacts, etc, such that implementations are aware of what to consider. Those kinds of considerations seem doable in that document without holding it up, while the work on methods to account for those items can be defined as extensions that are applicable for various deployment scenarios.

Best,
Tommy

> On Dec 17, 2020, at 12:10 PM, Lucas Pardue <[email-masked]> wrote:
> 
> I'm also wondering about the effective population that CONNECT-UDP ECN support appeals to. CONNECT TCP doesn't support anything ECN, QUIC has ECN support built in, so this works only for the subset of people tunneling non-QUIC UDP protocols over MASQUE? Or am I missing something?
> 
> Cheers
> Lucas
> -- 
> Masque mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/masque

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 19 Dec 2020 18:53:20 +0000

On Sat, 19 Dec 2020, 18:09 Tommy Pauly, <[email-masked]> wrote:

> I think it’s slightly different from normal CONNECT, given that in normal
> CONNECT, there are separate TCP contexts on each side of the proxy that
> could theoretically react to ECN. With CONNECT-UDP proxying a QUIC
> connection, the signaling of ECN bits would need to be able to get back to
> the client to fully function. This is one of the many ways that we need to
> consider CONNECT-UDP as distinct from CONNECT. It’s also different in how
> nested congestion control interacts, how effective MTU works, etc.
>

Fair point. I might ask for evidence for the demand of ECN signalling of
some sorts for HTTP CONNECT.

> That said, I think it is fine to have the mechanics of ECN for CONNECT-UDP
> be defined in a separate document—I totally expect that many use cases may
> not use it, particularly if they expect their primary benefit from ECN to
> be on the link between the client and the proxy, not between the proxy and
> the target. However, some deployments will care a lot about it.
>
> I think what would be useful to have in the core CONNECT-UDP document is
> more discussion of the ramifications of using CONNECT-UDP, which include
> interactions with ECN, congestion control, nested ACKs, MTU impacts, etc,
> such that implementations are aware of what to consider. Those kinds of
> considerations seem doable in that document without holding it up, while
> the work on methods to account for those items can be defined as extensions
> that are applicable for various deployment scenarios.
>

I think including considerations in CONNECT-UDP and solutions as separate
extensions is a good idea.

Cheers
Lucas

---

From: David Schinazi <[email-masked]>
Date: Sun, 20 Dec 2020 10:29:36 +0100

+1, adding considerations to the core doc makes a lot of sense. I'll see
what I can come up with, but please send PRs, folks!

David

On Sat, Dec 19, 2020 at 7:53 PM Lucas Pardue <[email-masked]>
wrote:

>> That said, I think it is fine to have the mechanics of ECN for
>> CONNECT-UDP be defined in a separate document—I totally expect that many
>> use cases may not use it, particularly if they expect their primary benefit
>> from ECN to be on the link between the client and the proxy, not between
>> the proxy and the target. However, some deployments will care a lot about
>> it.
>>