Subject: Use cases for PMTUD and PLPMTUD (was: RE: 6MAN: Adoption call on draft-hinden-6man-rfc1981bis-01)

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 5 Feb 2016 18:13:50 +0000

Starting this under a new thread, IMHO if we want to promote RFC1981 to standard
we should understand its use cases as well as the use cases for RFC4821.

First, it is reasonable to expect that paths that begin and end within the same
well-managed administrative domain can be counted on to deliver the necessary
ICMPs. An example is my employer's corporate network. In that case, traditional
PMTUD per RFC1981(bis) can be applied alone w/o having to apply RFC4821.

On the other hand, paths that lead to Internet destinations cannot be counted
on to deliver the necessary ICMPs. In that case, RFC4821 provides a mitigation.

But, if we do not believe that there are paths for which traditional PMTUD
can still be used safely, then we should be working to deprecate RFC1981
instead of making it a standard.

Thanks - Fred
[email-masked]

---

From: Fernando Gont <[email-masked]>
Date: Fri, 5 Feb 2016 15:25:32 -0300

> instead of making it a standard.

Well, the thing here that you can do RFC1981-only, RFC4821-only, or
RFC1981/RFC4821 (should I say "dual stack"? :-) ).

With that in mind, one could as well have both RFC1981 and RFC4821 as
standards, I guess...

But yes, generally speaking, RFC1981-only is certaianly unreliable.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 5 Feb 2016 19:00:50 +0000

Hi Fernando,

> -----Original Message-----
> From: Fernando Gont [mailto:[email-masked]]
> Sent: Friday, February 05, 2016 10:26 AM
> To: Templin, Fred L; [email-masked]
> Cc: Bob Hinden; 6man WG; Fred Baker (fred)
> Subject: Re: Use cases for PMTUD and PLPMTUD (was: RE: 6MAN: Adoption call on draft-hinden-6man-rfc1981bis-01)
> 

Except for within well-managed administrative domains where RFC1981-only
is sufficient. They do exist; I am typing this message from within one right now.

Thanks - Fred

---

From: Bob Hinden <[email-masked]>
Date: Fri, 5 Feb 2016 11:04:31 -0800

Fred,

> On Feb 5, 2016, at 10:13 AM, Templin, Fred L <[email-masked]> wrote:
> 

Sorry for being argumentative, but this is not a requirement to move RFC1981 to Internet Standard.  It’s already at Draft Standard.  See RFC6410.

Instead continuing the debate over how RFC4821 should be cited in rfc1981bis, I am starting to think that a separate new document could discuss the relationship between the two approaches to learning the MTU, and make some recommendations on how they should be used together or separately.  Perhaps a BCP or update to Node Requirements.

I don’t think this topic needs to be part of advancing RFC1981.

Bob

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Feb 2016 14:16:13 -0500

Templin, Fred L <[email-masked]> wrote:
    > On the other hand, paths that lead to Internet destinations cannot be
    > counted on to deliver the necessary ICMPs. In that case, RFC4821
    > provides a mitigation.

    > But, if we do not believe that there are paths for which traditional
    > PMTUD can still be used safely, then we should be working to deprecate
    > RFC1981 instead of making it a standard.

I think that the mitigation is far more reliable than 1981.
So, I'd rather deprecate RFC1981.

{I noticed that 4821 is not on by default in most Linux distros/kernels,
and I actually want to know if there is some reason for it}

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 6 Feb 2016 08:52:26 +1300

> I don’t think this topic needs to be part of advancing RFC1981.

Yes, that seems right to me.

And in answer to other messages, I think it would be ludicrous to consider
obsoleting RFC 1981. It isn't broken. It is fair to note that in cases where
ICMP fails, PMTUD fails too.

   Brian

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 5 Feb 2016 20:00:18 +0000

> On Feb 5, 2016, at 11:04 AM, Bob Hinden <[email-masked]> wrote:
> 

That thought has crossed my mind as well. It is unusual to have a specification describing one approach comment on a different specification about a different approach.

If we were to write such a specification, what would it need to say?

I think the following would need to be present:

1) pointers to RFCs 1981bis and 4821
2) reports of experience (not just concern) with each
2a) environments in which each works well
2b) demonstrated issues with each
3) recommendations about deployment

2a for RFC 1981bis, would likely say that if ICMP PTB is not being blocked and a host is sending TCP segments that are too large, it quickly knocks them to a workable size. 2b for RFC 1981bis would likely say that if ICMP PTB is being blocked or spoofed, it doesn't behave well.

2a for RFC 4821 should likely say that it achieves a reasonable MSS. 2b might observe that it is not guaranteed to do so quickly.

As a recommendation, it seems like TCP should follow some kind of rule. There is an argument for sending segments in the initial burst of IW different sizes (including at least one derived from a 1280 byte MTU, one from a 1500 byte MTU, and one from a 9000 byte MTU, each less the sizes of any known headers) and either seeding the 4821 algorithm with the ack/sack response information, using the number found in a responding ICMP PTB, or, in the absence of an ICMP PTB, selecting the largest of those sizes that was acked or sacked. I'll bet Matt Mathis or one of his compadres would have a better approach.

We would also want to have an implementation report on the recommendations (at least one, ideally one used in each of several operational environments including virtualized data center, mobile wireless, wifi, broadband, and backbone).

---

From: Fernando Gont <[email-masked]>
Date: Fri, 5 Feb 2016 17:15:55 -0300

> is sufficient. They do exist; I am typing this message from within one right now.

I'd expect 6man to produce protocols that work everywhere, rather than
just on some subset scearios...

---

From: Fernando Gont <[email-masked]>
Date: Fri, 5 Feb 2016 17:18:28 -0300

> and I actually want to know if there is some reason for it}

Yes: convergence time.

-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Mark Andrews <[email-masked]>
Date: Sat, 06 Feb 2016 10:06:16 +1100

> just on some subset scearios...

RFC 1981 works for TCP unless you *deliberately* break it by dropping
PTB as TCP retries.

RFC 1981 does NOT work in general for UDP.  DNS/UDP is a perfect
example as it is the responding server that get the PTB and the DNS
client is trying multiple servers.  Often by the time the server
is tried again especially when a server is anycast the MTU knowledge
has been lost.

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Mark Smith <[email-masked]>
Date: Sat, 6 Feb 2016 10:26:51 +1100

> is tried again especially when a server is anycast the MTU knowledge

I think the main reason that fragments are dropped by some networks is
that they don't have any upper layer protocol header and port fields
in the payload.

One idea discussed a while ago (on this or the v6ops list) was to
shift fragmentation into the UDP layer, which would mean that every
UDP fragment would have the upper layer protocol header and port
numbers that some people want to look at.

At the time I thought UDP source port 0 could be used to indicate
there are UDP options in the packet, however I found that it actually
means no source port (e.g., protocols like DHCP and SNMP don't use
source ports because both client and server destination ports are well
known.)

I came across this draft by Joe last week, and I was thinking that
that might facilitate UDP layer fragmentation.

"Transport Options for UDP"
https://tools.ietf.org/html/draft-touch-tsvwg-udp-options-02

Fred has also started writing up a draft to shift fragmentation into
the GRE layer:

"GRE Tunnel Fragmentation"
https://tools.ietf.org/html/draft-templin-intarea-grefrag-02

Regards,
Mark.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 07 Feb 2016 11:15:17 -0500

Fernando Gont <[email-masked]> wrote:
    >> Templin, Fred L <[email-masked]> wrote: > On the other
    >> hand, paths that lead to Internet destinations cannot be > counted on
    >> to deliver the necessary ICMPs. In that case, RFC4821 > provides a
    >> mitigation.
    >>
    >> > PMTUD can still be used safely, then we should be working to
    >> deprecate > RFC1981 instead of making it a standard.
    >>
    >> I think that the mitigation is far more reliable than 1981.  So, I'd
    >> rather deprecate RFC1981.
    >>
    >> {I noticed that 4821 is not on by default in most Linux
    >> distros/kernels, and I actually want to know if there is some reason
    >> for it}

    > Yes: convergence time.

Yes, I've noticed that it takes a few re-transmissions on the first
connection to a site in the case where the MTU is limited and the ICMPs are
lost, but when the MTU is unblocked, there is no delay.

An implementation that wanted to avoid even the re-transmissions in the
blocked case could limit themselves to 1280 sized packets and send some 1500
byte probes have gotten through.

If 4821 is turned off, then the connection is mysteriously blocked, and
always fail.

So, your answer does not make sense to me.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 07 Feb 2016 11:19:24 -0500

Mark Andrews <[email-masked]> wrote:
    >> Except for within well-managed administrative domains where
    >> RFC1981-only > is sufficient. They do exist; I am typing this message
    >> from within one righ t now.
    >>

    > RFC 1981 works for TCP unless you *deliberately* break it by dropping

Yes, and we know that many networks do this.

{It appears that the Linux IPv6 XFRM/NetKey IPsec stack also fails to send
 PTBs when operating in one-arm router mode, but I haven't fully convinced
 myself that this is where the failure is yet}

    > client is trying multiple servers.  Often by the time the server is
    > tried again especially when a server is anycast the MTU knowledge has
    > been lost.

I didn't think PLPMTUD worked any better in this situation, or are I wrong here?

i.e: both are broken for UDP, and in the DNS/UDP case, the server never
     retransmits either... so absent using TCP, it will always just fail.

---

From: Mark Andrews <[email-masked]>
Date: Mon, 08 Feb 2016 11:48:18 +1100

> I didn't think PLPMTUD worked any better in this situation, or are I wrong he

Conceptually you do PLPMTUD by adjusting the EDNS UDP size advertised
in the query until you get a response.  This isn't strict PMTUD as
it also accounts for firewalls that are blocking fragments, firewalls
that still think DNS is limited to 512 byte UDP payloads.  Unfortunately
you only have about 3 seconds to do this all in as well as talking
to other servers.  The DNS doesn't have a way of signaling "fragment
at this size".

Named tracks successful answers/timeouts with different EDNS udp
sizes as well as the actual response sizes that have got through.

; [edns success/4096 timeout/1432 timeout/1232 timeout/512 timeout]

Too many timeouts with a particlar size and we stop offering a EDNS
UDP buffer of that size and fallback to the next break point.  These
values are set for 4in6/6in4 + a UDP header.  timeouts with a smaller
EDNS buffer size also count against larger sizes.

>      retransmits either... so absent using TCP, it will always just fail.

But the client re-queries leading to a re-transmission.
 
> --=-=-=
> Content-Type: application/pgp-signature; name="signature.asc"
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> 
> iQEVAwUBVrduiYCLcPvd0N1lAQJ7RQf/bInuCsHmVH6AONDSavt/Ut1/xo0NdnRp
> 1vtfg2LwHG5cUJGNpHOdnlg0OdhIJS9KVYN6xP0FQqbfTq4E6zRpoBkw4kMFQFWK
> cYOe8MaXx4FJFWF6MWcugglhWRkFQeSaQyYcn0Odhmeoc7w2/ehlsTfuFNGCnpH9
> 2bJS2Og80JWMDy7Q95IL6JrsKE1/VDDLpJJkQTfqPboZ1BG5Vn6IlDUT+lEMsZly
> By2CSNBglAqVtU5fUPYCa6yCx92I2g8MYePqkjclN1dAlELhlOUllsrJ8F3kCuFs
> 4eT9yAOtU4W7tP/Ka3bBdJTRYnS31NXozgdIlPsnhZ7mXdrks1tQcg==
> =kitT
> -----END PGP SIGNATURE-----
> --=-=-=--

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 8 Feb 2016 16:05:38 +0000

Hi Mark,

> https://tools.ietf.org/html/draft-templin-intarea-grefrag-02

Yes, also "Fragmentation Option for Generic UDP Encapsulation":

https://datatracker.ietf.org/doc/draft-herbert-gue-fragmentation/

Both are essentially the same concept - hike the fragmentation header
up to the encapsulation layer instead of the IP layer. The idea originally
came from RFC2764.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 8 Feb 2016 16:30:58 +0000

Hi Fred,

> From: Fred Baker (fred) [mailto:[email-masked]]
> Sent: Friday, February 05, 2016 12:00 PM
> To: Bob Hinden
> Cc: Templin, Fred L; Ole Trøan; IPv6 List
> > Instead continuing the debate over how RFC4821 should be cited in rfc1981bis, I am starting to think that a separate new document
> could discuss the relationship between the two approaches to learning the MTU, and make some recommendations on how they
> should be used together or separately.  Perhaps a BCP or update to Node Requirements.
> >
> That thought has crossed my mind as well. It is unusual to have a specification describing one approach comment on a different
> specification about a different approach.
>
> 2a for RFC 1981bis, would likely say that if ICMP PTB is not being blocked and a host is sending TCP segments that are too large, it
> quickly knocks them to a workable size. 2b for RFC 1981bis would likely say that if ICMP PTB is being blocked or spoofed, it doesn't
> behave well.
> 

Another point that needs to be considered is interactions with multipath. With
RFC1981(bis), the path MTU will eventually converge to the minimum MTU of
all paths that contribute to the multipath. But, with RFC4821, each path in the
multipath would need to be tested independently, i.e., per-flow and not just
per-destination path MTU probing.

Implementations might approach this by starting the RFC4821 path MTU
probing process from scratch for every new TCP connection and not cache
the MTU discovered by previous connections. It would work, but could
result in lots of probing.

Thanks - Fred

> As a recommendation, it seems like TCP should follow some kind of rule. There is an argument for sending segments in the initial burst
> of IW different sizes (including at least one derived from a 1280 byte MTU, one from a 1500 byte MTU, and one from a 9000 byte MTU,
> each less the sizes of any known headers) and either seeding the 4821 algorithm with the ack/sack response information, using the
> number found in a responding ICMP PTB, or, in the absence of an ICMP PTB, selecting the largest of those sizes that was acked or
> sacked. I'll bet Matt Mathis or one of his compadres would have a better approach.
> 
> We would also want to have an implementation report on the recommendations (at least one, ideally one used in each of several
> operational environments including virtualized data center, mobile wireless, wifi, broadband, and backbone).
>

---

From: Michael Richardson <[email-masked]>
Date: Mon, 08 Feb 2016 13:21:08 -0500

Mark Andrews <[email-masked]> wrote:
    >> I didn't think PLPMTUD worked any better in this situation, or are I
    >> wrong he re?

    > in the query until you get a response.  This isn't strict PMTUD as it
    > also accounts for firewalls that are blocking fragments, firewalls that
    > still think DNS is limited to 512 byte UDP payloads.  Unfortunately you
    > only have about 3 seconds to do this all in as well as talking to other
    > servers.  The DNS doesn't have a way of signaling "fragment at this
    > size".

I'd forgotten about doing it this way.
I now recall you explaining it many times before :-)

It would seem that ICMP based PMTUD is useless for DNS too, and that we
should do away with 1981?

    > Named tracks successful answers/timeouts with different EDNS udp sizes
    > as well as the actual response sizes that have got through.

    > Too many timeouts with a particlar size and we stop offering a EDNS UDP
    > buffer of that size and fallback to the next break point.  These values
    > are set for 4in6/6in4 + a UDP header.  timeouts with a smaller EDNS
    > buffer size also count against larger sizes.

    >> i.e: both are broken for UDP, and in the DNS/UDP case, the server
    >> never retransmits either... so absent using TCP, it will always just
    >> fail.

    >> --=-=-= Content-Type: application/pgp-signature; name="signature.asc"
    >>
    >> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux)
    >>
    >> 4eT9yAOtU4W7tP/Ka3bBdJTRYnS31NXozgdIlPsnhZ7mXdrks1tQcg== =kitT
    >> -----END PGP SIGNATURE----- --=-=-=--
    > --
    > Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia

---

From: Mark Andrews <[email-masked]>
Date: Tue, 09 Feb 2016 10:33:07 +1100

In message <[email-masked]>, Michael Richardson writes:
> Content-Type: text/plain

There is a difference between trying to avoid something and doing
away with something.  Named turns off PMTUD for the sockets it opens
because even with TCP there isn't the recovery time available to
deal with lost PTBs.

That said RFC 1981 works well when the PTB get through for TCP and
if most of the DNS/UDP traffic was fragmented it would work reasonable
well there too however DNS/UDP packets are variable sizes and you
have 1 in N packets being fragmented and the MTU knowledge times
out.  N is dropping and more sites sign their zones.

Firewall developers really don't know how to handle DNS.  They do
lots of really stupid things like dropping EDNS != 0 packets for
no sane reason, dropping EDNS requests with a EDNS option or EDNS
flag set.  EDNS specified safe default behaviours for all these
event but firewall developers seem to think they know better.

Ahhhh this EDNS flag bit is set, the world may end, lets drop the
packet and break the protocol instead of reading the protocol spec
and seeing that the default is for the server to ignore the bit.

Similarly dropping ICMP PTB and fragments.  The world will not end
if you let through a "bad" ICMP PTB or a IP fragment.

Mark

> iQEVAwUBVrjckYCLcPvd0N1lAQI/1wf/f42aJtsGxKjkEwNFSedh2jJCEs7H9hhW
> onkwzKj+d+D9zhWwJyiqrZ2C/jLaxz+N6KowvCO8/+xNrbo0yDca5RLfcqWyEMkz
> hb3zwQiDoBdoTFiZ/R+leTWclTx2h9lMzqyCB4cL69PNgmQBFy5NxwSdGE79AUiP
> uQJhbfxxjkKXNkddgAj3z6q8UFuikZQSeYfrAUyoqF+6L/ZlqDIvHLZnw8w1YA5p
> 2krSML8S5XdA0YBcOuH3CAXvLG9tYQiLkRso3UG4OPqD9JgydSUJM/4ef+7cIOOq
> LSl9YKXN2r2E9rcCCye0RL5LPyYo+deL1ZieYOAqaUuKgxW9e7Ozaw==
> =Nax3

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 19:30:07 -0300

> lost, but when the MTU is unblocked, there is no delay.

If you employ RFC4821 as icmp-less PMTUD, it doesn't matter whether the
ICMPs get blocked or not. If you implement RFC4821 for blackhole
detection, then I agree with you.

> So, your answer does not make sense to me.

"My" answer is actually what I recall from discussing this with Linux
kernel developers 10 years ago -- I guess that they were referring to
"RFC4821 as icmp-less PMTUD".

That said, if the toggle in Linux is to use RFC4821 as icmp-less PMTUD,
then my (relayed) response would apply. OTOH, if the toggle is simply to
employ RFC4821 for blackhole detection then I'd agree with you.

FWIW, I'd expect RFC4821 fo blackhole detection to be "on by default",
and, if anything, to be able to turn on RFC4821 as icmp-less PMTUD (but
this one *not* "on by default", for the reasons already stated).

Thanks!

Best regards,