Subject: Re: [Masque] HTTP Datagrams and MTU handling

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 3 Aug 2021 15:07:13 -0500

I'm not quite sure where this discussion ended up, but just to top-post in
case I'm just too far behind the curve ...

I'm seeing references to PMTUD and to ICMP PTB in this thread. For what
it's worth, in 2017 when the IETF was looking at advancing IPv6 to full
standard, one of the most contentious part of that conversation was about
whether ICMP PTB blackholing (and ICMP blackholing more generally) on the
Internet was too big a problem to advance a specification that relied on it
to full standard.

The Last Call discussion thread is "here", and the discussion was started
by Some Guy Named Lars Eggert.

https://mailarchive.ietf.org/arch/msg/tsv-area/mJOoovO90MjbF2gKQtHfov1c-nk/
I won't try to summarize every point, but
https://mailarchive.ietf.org/arch/msg/tsv-area/hVfA2klB7CnldJw4-tmCIBpAGl0/
(also from Lars) was pretty succinct.

"What Fernando said: I'm certainly not opposed to lifting this to Standard,
but it is painting an incorrect picture - PLPMTUD is de facto mandatory
these days, and has been for years."

That might be relevant to this discussion, I think?

Best,

Spencer

On Mon, Jul 26, 2021 at 2:35 PM Magnus Westerlund <magnus.westerlund=
[email-masked]> wrote:

> Hi Ben,
>
>
>
> My point is that if we can mimic the ICMP PTB mechanism that is
> sufficient. Yes, it will result in a few packets being dropped from the
> change until the ingress side of the tunnel gets the PTB message and can do
> something about it, either encapsulate if that is required to meet minimal
> MTU sizes, or alternatively from this point generate an ICMP or API
> indication to higher layer.
>
>
>
> But it prevents a MTU black hole and allows support IPv6 minimal MTU.
>
>
>
> Cheers
>
>
>
> Magnus Westerlund
>
>
>
> *From:* Ben Schwartz <bemasc=[email-masked]>
> *Sent:* den 26 juli 2021 17:27
> *To:* Magnus Westerlund <[email-masked]>
> *Cc:* [email-masked]; [email-masked]; [email-masked]
> *Subject:* Re: [Masque] HTTP Datagrams and MTU handling
>
>
>
>
>
>
>
> On Fri, Jul 23, 2021 at 4:32 AM Magnus Westerlund <magnus.westerlund=
> [email-masked]> wrote:
>
> On Tue, 2021-07-20 at 11:56 -0700, David Schinazi wrote:
>
>
>
>
>
> On Tue, Jul 20, 2021 at 1:52 AM Magnus Westerlund <
> [email-masked]> wrote:
>
> Hi David,
>
>
>
> How will the MASQUE Server know that it can't support CONNECT-IP if the
> restricted QUIC session is between HTTP intermediaries and not between
> either MASQUE endpoints?
>
>
>
>
>
> The MASQUE endpoints could perform PMTUD by sending ICMP echo requests
> through the tunnel.
>
>
>
> Yes, that is a possibility, but not something I like for the following
> reasons.
>
>
>
> 1. Forces a choice between awaiting the verification or provide an
> inconsistent behavior in regards to MTU if the tunnel drops oversized and
> then on later packets behave consistently.
>
>
>
> 2. In case the tunnel MTU changes during the sessions lifetime then we
> have no method other than continously run PMTUD over the HTTP datagrams to
> verify that the minimal size is still working.
>
>
>
> I would definitely prefer a mechanism where the HTTP implementation sent
> back a message indicating it dropped the HTTP datagram as oversize and
> provided the current MTU.
>
>
>
> I don't understand.  That would also result in problems 1 and 2.  If you
> want to avoid problems 1 and 2, you need an equivalent to "forward
> fragmentation" (like a "Datagram-Min-MTU" header).
>
>
>
> Alternatively, we can simply declare that the DATAGRAM path does not
> provide an MTU guarantee, and if you need such a guarantee, you have to use
> the DATAGRAM CAPSULE path.
>
>
>
>
>
>
> --
> Masque mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/masque
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 27 Aug 2021 08:11:22 +0000

Hi Spencer,

As I am back from vacation and catching up to email I am trying to understand
which aspect of the discussion you applying this comment to. 

I could interpret it to say, we don't need working creation of ICMP PTBs when a
MASQUE node drops a oversized packet that we can't forward without fragmentation
support or changing to reliable encapsulation. 

I hope you don't think you can apply that logic to dropping tunneled packets
that are 1280 or smaller size due to MTU issues. Becasue the PLPMTUD do assume
that IPv6 1280 packet size will work and thus also they will fail to handle the
situation if 1280 bytes are not supported.

My perspective here is that first of all we need a solution for ensuring that we
can get 1280 bytes packet forwarded over intermediary MASQUE nodes. I don't see
solutions that try to detect the situation and brings down the tunnel is
particullary good solution, as these limitations may exist both on tunnel
creation time as well as occurring during the session. 

I think sending ICMP PTBs would be good especially if that is minor extension of
the mechanism to handle how we ensure that <=1280 bytes IPv6 packets can be
sent.

I would appreciate some clarificaiton on the intention of this comment. 

/Magnus

On Tue, 2021-08-03 at 15:07 -0500, Spencer Dawkins at IETF wrote:
> I'm seeing references to PMTUD and to ICMP PTB in this thread. For what it's
> worth, in 2017 when the IETF was looking at advancing IPv6 to full standard,
> one of the most contentious part of that conversation was about whether ICMP
> PTB blackholing (and ICMP blackholing more generally) on the Internet was too
> big a problem to advance a specification that relied on it to full standard. 
> 
> The Last Call discussion thread is "here", and the discussion was started by
> Some Guy Named Lars Eggert. 
> 
> https://mailarchive.ietf.org/arch/msg/tsv-area/mJOoovO90MjbF2gKQtHfov1c-nk/ I
> won't try to summarize every point, but 
> On Mon, Jul 26, 2021 at 2:35 PM Magnus Westerlund <
> magnus.westerlund=[email-masked]> wrote:
> > Hi Ben,
> >  
> > My point is that if we can mimic the ICMP PTB mechanism that is sufficient.
> > Yes, it will result in a few packets being dropped from the change until the
> > ingress side of the tunnel gets the PTB message and can do something about
> > it, either encapsulate if that is required to meet minimal MTU sizes, or
> > alternatively from this point generate an ICMP or API indication to higher
> > layer.
> >  
> > From: Ben Schwartz <bemasc=[email-masked]> 
> > Sent: den 26 juli 2021 17:27
> > To: Magnus Westerlund <[email-masked]>
> > Cc: [email-masked]; [email-masked]; [email-masked]
> > Subject: Re: [Masque] HTTP Datagrams and MTU handling
> >  
> >  
> >  
> > On Fri, Jul 23, 2021 at 4:32 AM Magnus Westerlund <
> > magnus.westerlund=[email-masked]> wrote:
> > > > > How will the MASQUE Server know that it can't support CONNECT-IP if
> > > > > the restricted QUIC session is between HTTP intermediaries and not
> > > > > between either MASQUE endpoints?
> > > > >  
> > > > 
> > > >  
> > Alternatively, we can simply declare that the DATAGRAM path does not provide
> > an MTU guarantee, and if you need such a guarantee, you have to use the
> > DATAGRAM CAPSULE path.
> >  
> >  
> >  
> > -- 
> > Masque mailing list
> > [email-masked]

---

From: Ben Schwartz <[email-masked]>
Date: Fri, 27 Aug 2021 14:51:01 -0400

On Fri, Aug 27, 2021 at 4:11 AM Magnus Westerlund <
[email-masked]> wrote:
...

> situation if 1280 bytes are not supported.

PLPMTUD algorithms can be updated to avoid this assumption.  Of course,
there might be an install base of software that has this assumption baked
in.

...

> the mechanism to handle how we ensure that <=1280 bytes IPv6 packets can be

I don't think this would suffice.  If we need to support PLPMTUD that
assumes a 1280-byte minimum, presumably we also need to support
ICMPv6-based PMTUD that assumes a 1280-byte minimum.  A PTB message that
says "1280 is too big" won't be good enough.

I expect that the best solution is to direct such stiff endpoints to the
reliable transport (non-DATAGRAM), which has enormous MTU, and declare that
CONNECT-IP in DATAGRAM mode does not comply with the IP MTU rules.  If
that's not sufficient, I think we'll need a mechanism for "forward
reliability", with intermediaries upgrading oversize datagrams to the
reliable transport.

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 30 Aug 2021 09:49:56 +0000

On Fri, 2021-08-27 at 14:51 -0400, Ben Schwartz wrote:
> PLPMTUD algorithms can be updated to avoid this assumption.  Of course, there
> might be an install base of software that has this assumption baked in.
>  

I still think there are a number of IP usages that do assume certain base MTUs
will work because they are assumed to be below minimal and don't have any form
of PLPMTUD implemented.

> I don't think this would suffice.  If we need to support PLPMTUD that assumes
> a 1280-byte minimum, presumably we also need to support ICMPv6-based PMTUD
> that assumes a 1280-byte minimum.  A PTB message that says "1280 is too big"
> won't be good enough.
> 
> CONNECT-IP in DATAGRAM mode does not comply with the IP MTU rules.  If that's
> not sufficient, I think we'll need a mechanism for "forward reliability", with
> intermediaries upgrading oversize datagrams to the reliable transport.

Yes, we do have a reliable mode we can use. My argument is how the MASQUE
ingresses can determine when it is necessary to use this reliable mode for
packets to get them from MASQUE ingres to MASQUE egress. Simply dropping any
datagram that is oversized for the actual MTU will not work. There need to be
more functionality somewhere in the MASQUE system. If that is running ingress to
egress regular probing or a feedback mechanism to ingress that one dropped the
packet I don't care. 

Cheers

Magnus