Subject: Re: PMTUD and MTU < 1280

From: RJ Atkinson <[email-masked]>
Date: Wed, 20 Jul 2011 17:35:31 -0400

Earlier, Brian Carpenter wrote:
> It's always been my understanding that an interface sending IPv6 packets
> MUST implement some (unspecified) form of framentation and reassembly
> *below layer 3* if the link MTU is less than 1280. In other words a
> PTB for a packet of length 1280 is an unrecoverable violation.

I also understand that the IPv6 specifications say that 
the minimum IPv6 MTU for any link type is 1280 bytes.

I am not sure the specs insist that an IPv6 implementation 
must treat an ICMPv6 Packet-Too-Big for less than 1280 bytes 
as "unrecoverable". (I haven't re-read the IPv6 specs recently.)

I understand that some deployed IPv6 implementations will
tolerate and support Packet-Too-Big messages with a link MTU
at least as small as 576 -- following the Postel Principle
of "generous in what one accepts" and being practical about
several radio link technologies that do not support a
1280 byte IP MTU.  Such implementations are more resilient
than the IPv6 specifications require; this is helpful.

> For example, if you're tunneling IPv6 over an IPv4 network whose PMTU
> (to the other end of the tunnel) is, to take a random example, 576,
> the tunnel end points could use IPv4 fragmentation and reassembly
> to provide a 1280 MTU for the IPv6 traffic.

One could do that.  Over low-bandwidth radio links the overhead of doing
that would effectively make it preferable to deploy an IPv6::IPv4 translator
or some other protocol translation scheme (and continue with IPv4 forever, 
at least on that link) than to encapsulate IPv6 in anything more than 
just the radio link layer just to meet the 1280 byte requirement.

As I recall, I noted this problem to the IPv6 WG back in the middle 1990s.[1]
Sadly. that prediction seems to have come true.  So I understand that today, 
over some deployed radio link technologies, several IPv6 implementations use 
the raw link encapsulation and fragment IPv6 into chunks at that raw link
MTU size (even though that is below 1280 bytes).

> Of course this doesn't cover the NAT64 case; so what? NAT breaks many things.
> We just have to rely on the real world, where the layer 2 link MTU is pretty
> much always 1500 today.

I agree that the IEEE standard Ethernet link MTU is very widely deployed
today.  Many, possibly most, Ethernet switches also support a link MTU
somewhat over 9K bytes.  

I'm told that SONET links most commonly use the old FDDI link MTU (roughly 4500 bytes).  
ATM links, which seem to be going away, most commonly support an IP MTU near
9180 bytes (which value seems to have driven the Jumbo Ethernet implementations).  
However, there are still a number of link types, mostly radio link types 
(as near as I can tell), that have a link MTU well below 1280 bytes.  As 
near as I can tell, IPv6 would have been much better off selecting 512 bytes
as the minimum IPv6 MTU size.[1]

It is very unfortunate that IPv6 specifies this very large minimum MTU size
(i.e. 1280 bytes), because it is impeding the ability of various organisations 
to deploy IPv6 on a number of existing links that already support IPv4.

This is just the kind of obstacle to deployment that IPv6 does not need.
Specifying the value 1280 is not solving a real technical problem, IMHO.  
A lower value would facilitate faster, easier, simpler, and more widespread 
deployment and use of IPv6, IMHO.

IPv4 can run over nearly any link without protocol changes or requiring
special case handling.  Sadly, IPv6 cannot make the same claim.  One hopes 
IPv6 implementers will be tolerant of IPv6 MTUs below 1280 bytes, because
they do exist in the deployed world and aren't going away anytime soon.

Yours,

Ran Atkinson

[1] <ftp://ftp.ietf.org/ietf-online-proceedings/94dec/area.and.wg.reports/ipng/ipngwg/ipngwg-minutes-94dec.txt>

---

From: james woodyatt <[email-masked]>
Date: Wed, 20 Jul 2011 14:44:23 -0700

On Jul 20, 2011, at 14:35 , RJ Atkinson wrote:

> One hopes IPv6 implementers will be tolerant of IPv6 MTUs below 1280 bytes, because they do exist in the deployed world and aren't going away anytime soon.

Those hopes are not well placed.

I am aware of at least one packet filter implementation-- which might be refined in the future, one hopes, to conform fully with RFC 6092-- that treats this behavior as a resource exhaustion attack and drops flows accordingly.

--
james woodyatt <[email-masked]>
member of technical staff, core os networking

---

From: Philip Homburg <[email-masked]>
Date: Thu, 21 Jul 2011 08:30:51 +0200

>as "unrecoverable". (I haven't re-read the IPv6 specs recently.)

Some services, like big DNS server cannot afford to do PMTU. It requires them
to store the original DNS reply just in case an ICMP comes back.

Without that, the host will see that its first request fails and has to retry.
Over time, this will happen again and again because big servers cannot
maintain all PMTU state forever.

>to deploy IPv6 on a number of existing links that already support IPv4.

The root cause of this are routers that don't do fragmentation. Any lower
minimum MTU would force any host that cannot maintain PMTU state to fragment
in small sizes, which is bad for the 99% of the world that has MTUs bigger
than 1280.

I'd say that for any link that is mostly reliable, adding a fragmentation
layer should not cost more than 16 bits. There are even the payload length,
flow label and traffic class fields ready to be abused if need be. So
it can we without any increase in length.

>they do exist in the deployed world and aren't going away anytime soon.

I don't think that silently doing something that goes against the standard
is a good thing.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 21 Jul 2011 17:12:48 +1000

> to store the original DNS reply just in case an ICMP comes back.

This was originally noted in draft-ietf-ipngwg-bsd-frag-00 (January
1998) and became IPV6_USE_MIN_MTU (RFC 3542, May 2003).  Now if we
can just get it into POSIX ....

And, yes, nameservers do set this option.  FreeBSD's implementation
however is partially broken.  See:
http://www.freebsd.org/cgi/query-pr.cgi?pr=158307.

> maintain all PMTU state forever.

It takes multiple requests usually and yes the state does get lost
within minutes on busy nameservers.

What should have been specified is that hosts need to fragment
packets into roughly equal sizes fragments.  This reduces the
probabilty of having to deal with multiple PTB packets.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 22 Jul 2011 19:32:50 -0700

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> RJ Atkinson
> Sent: Wednesday, July 20, 2011 2:36 PM
> To: [email-masked]
> Subject: Re: PMTUD and MTU < 1280
> 

IPv6 specs have always said that ICMPv6 packet-too-big for
less than the minimum MTU should cause the IPv6 host to send 
packets (of that minimum MTU size) with an additional fragmentation 
header.  This existed in the first IPv6 RFC, RFC1883 (when the IPv6 
minimum MTU was 576):

   In response to an IPv6 packet that is sent to an IPv4 destination
   (i.e., a packet that undergoes translation from IPv6 to IPv4), the
   originating IPv6 node may receive an ICMP Packet Too Big message
   reporting a Next-Hop MTU less than 576.  In that case, the IPv6 node
   is not required to reduce the size of subsequent packets to less than
   576, but must include a Fragment header in those packets so that the
   IPv6-to-IPv4 translating router can obtain a suitable Identification
   value to use in resulting IPv4 fragments.  Note that this means the
   payload may have to be reduced to 528 octets (576 minus 40 for the
   IPv6 header and 8 for the Fragment header), and smaller still if
   additional extension headers are used.

and in the current IPv6 RFC, RFC2460:

   In response to an IPv6 packet that is sent to an IPv4 destination
   (i.e., a packet that undergoes translation from IPv6 to IPv4), the
   originating IPv6 node may receive an ICMP Packet Too Big message
   reporting a Next-Hop MTU less than 1280.  In that case, the IPv6 node
   is not required to reduce the size of subsequent packets to less than
   1280, but must include a Fragment header in those packets so that the
   IPv6-to-IPv4 translating router can obtain a suitable Identification
   value to use in resulting IPv4 fragments.  Note that this means the
   payload may have to be reduced to 1232 octets (1280 minus 40 for the
   IPv6 header and 8 for the Fragment header), and smaller still if
   additional extension headers are used.

-d

> [1] <ftp://ftp.ietf.org/ietf-online-
> proceedings/94dec/area.and.wg.reports/ipng/ipngwg/ipngwg-minutes-
> 94dec.txt>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 22 Jul 2011 19:34:21 -0700

> james woodyatt
> Sent: Wednesday, July 20, 2011 2:44 PM
> To: RJ Atkinson
> Cc: [email-masked]
> > One hopes IPv6 implementers will be tolerant of IPv6 MTUs below 1280
> bytes, because they do exist in the deployed world and aren't going
> away anytime soon.
> 
> I am aware of at least one packet filter implementation-- which might
> be refined in the future, one hopes, to conform fully with RFC 6092--
> that treats this behavior as a resource exhaustion attack and drops
> flows accordingly.

Its behavior violates the last paragraph of Section 5 of RFC2460.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 25 Jul 2011 09:36:23 -0400

> From: Rémi Després [mailto:[email-masked]]
> Sent: Monday, July 25, 2011 8:38 AM
> To: Dan Wing
> Cc: 'james woodyatt'; 'RJ Atkinson'; [email-masked]
> Le 23 juil. 2011 à 04:34, Dan Wing a écrit :
> 
> Violation _only in case_ of "an IPv6 packet that is sent to an IPv4
> destination".

But how does one determine an IPv6 packet is, or isn't, going 
to an IPv4 destination?  I don't think it's possible to determine
if there is an IPv6/IPv4 translator on the path.

-d

> If the destination is IPv6, a PMTU below 1280 remains therefore a
> network failure.
> This authorizes a simple IPv6 host to refuse packets beyond 1280 octets
> and to have no support of packet-reassembly.
> 
> Right?
> 
> Regards,
> RD
> 
> 
> 
> >
> > -d
> >
> >>
> >> --

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 25 Jul 2011 14:50:19 -0400

> From: Rémi Després [mailto:[email-masked]]
> Sent: Monday, July 25, 2011 1:43 PM
> To: Dan Wing
> Dan,
> 1.
> The point I wanted to check is just, slightly reformulated):
> "May a simple IPv6 host have no support of packet-reassembly, and
> simply accept packets up to 1280 octets."

The earlier part of this thread was talking about sending; you're
now bringing up receiving.

IMO, if the packet came from IPv4, and that IPv4 network had a small
MTU (e.g., 576) causing fragmentation, then such an IPv6 receiver
will be unable to receive the packet.

> In my understanding, the answer should be yes.
> - This doesn't depend on whether sources know or not whether their
> destinations are IPv6 or IPv4 only.
> - If the destination happens to be IPv6, current RFC's don't permit
> intermediate nodes to refuse 1280 packets as being too big.
> 
> 2.
> How sources can be sure to have e2e transparency in IPv6 is a different
> question, but IMHO an important one.
> 
> For instance, if a destination address is obtained from the DNS in a
> AAAA, with no A for the same URL  and without any well-known prefix
> indicating that there is an embedded-IPv4-address, I hope the source
> can be guaranteed that e2e transparency won't be broken?

I don't think so.  DNS64 comes to mind.

> I won't have time personally to contribute much on this, but the
> subject would usefully be clarified, IMHO.

The RFCs are pretty clear, IMO.  Implementers don't want to read
them all the way.

-d

> Regards,
> RD
> 
> 
>  Le 25 juil. 2011 à 15:36, Dan Wing a écrit :
> 
> >>>>
> >>>> ...
> >>>
> >>>> Administrative Requests:
> https://www.ietf.org/mailman/listinfo/ipv6

---

From: Rémi Després <[email-masked]>
Date: Mon, 25 Jul 2011 19:42:59 +0200

Dan, 

1.
The point I wanted to check is just, slightly reformulated):
"May a simple IPv6 host have no support of packet-reassembly, and simply accept packets up to 1280 octets."

In my understanding, the answer should be yes.
- This doesn't depend on whether sources know or not whether their destinations are IPv6 or IPv4 only. 
- If the destination happens to be IPv6, current RFC's don't permit intermediate nodes to refuse 1280 packets as being too big.

2. 
How sources can be sure to have e2e transparency in IPv6 is a different question, but IMHO an important one.

For instance, if a destination address is obtained from the DNS in a AAAA, with no A for the same URL  and without any well-known prefix indicating that there is an embedded-IPv4-address, I hope the source can be guaranteed that e2e transparency won't be broken? 

I won't have time personally to contribute much on this, but the subject would usefully be clarified, IMHO.

Regards,
RD

---

From: Mark Andrews <[email-masked]>
Date: Tue, 26 Jul 2011 12:57:51 +1000

In message <071801cc4afb$b3ead4a0$1bc07de0$@com>, "Dan Wing" writes:
> > From: R=E9mi Despr=E9s [mailto:[email-masked]]
> > =
> > =
> > =
> > =

And there is missing functionality in the API's.

IPV6_USE_MIN_MTU doesn't force a fragment header to be added to
packets less than 1280 octets for example and I'm not aware of any
other API hooks which will do this.  As a result a node still has
to do some parts of path mtu discovery even if it doesn't want to
and it is undesirable for it to do so.

RFC 3542

   This specification defines a mechanism to avoid path MTU discovery by
   sending at the minimum IPv6 MTU [RFC-2460].  If the packet is larger
   than the minimum MTU and this feature has been enabled the IP layer
   will fragment to the minimum MTU.  To control the policy about path
   MTU discovery, applications can use the IPV6_USE_MIN_MTU socket
   option.

A application should be able to send a IPv6/UDP packet, without resorting
to using raw sockets, and not have a PTB be generated as a result.

Mark

> -d
> 
> 
> > Regards,
> > RD
> > =
> 
> > =
> 
> >  Le 25 juil. 2011 =E0 15:36, Dan Wing a =E9crit :
> > =
> 
> > >>>>
> > >>>> ...
> > >>>

---

From: Rémi Després <[email-masked]>
Date: Mon, 25 Jul 2011 23:47:21 +0200

> now bringing up receiving.

Yes.

The point made is about the difference between IPv6-to-IPv4 and IPv6-to-IPv6 PMTU's (an IPv6 PTB with MTU < 1280 remaining excluded, AFAIK, if both source and destination are IPv6.) 

> will be unable to receive the packet.

The point is only about IPv6 to IPv6.

> I don't think so.  DNS64 comes to mind.

In my understanding, a host that requests both for A's and AAAA's, and receives no A, knows it talks to an IPv6-only host, with or without DNS64.
There may be other ways to know it, e.g. for some IPv6-only sensors talking only with IPv6-capable dedicated servers. 

In any case, I have no problem with leaving this subject, as many others may be found more urgent.

Thanks,
RD