Subject: Feedback on draft-generic-6man-tunfrag-07

From: Fernando Gont <[email-masked]>
Date: Fri, 08 Feb 2013 07:16:25 -0300

Hi, Fred,

Sorry for the delay in providing comments (as promised). Here they are:

Section 2:

>    The current "Internet cell size" is effectively 1500 bytes, i.e., the
>    minimum MTU configured by the vast majority of links in the Internet.
>    IPv6 constrains this even further by specifying a minimum link MTU of
>    1280 bytes [RFC2460].  However, due to operational issues with Path
>    MTU Discovery (PMTUD) [RFC1981] these sizes can often only be
>    accommodated when links with smaller link-layer segment sizes are
>    configured to perform link adaptation.

We could probably add some references regarding PMTUD issues in IPv6
(some folks think/thought/expected them to be gone with v6... but they
are not) -- found them later... could you move them to this para?

>    Unfortunately, link adaptation can present a significant burden to
>    the link endpoints, i.e., especially when the link supports high data
>    rates and/or is located nearer the "middle" of the network instead of
>    nearer the "edge".  An alternative therefore is to ask the
>    originating IPv6 node to perform fragmentation on the packets it
>    sends, in which case reassembly would be performed by the final
>    destination.

This is okay, but.. in the previous para, you started with the premise
that you have to rely on link-adaptation as a result of PMTUD failures,
so.... if ICMPv6 PTB are going to be blocked, how an you expect the
sources to fragment their packets? (i.e., receive the corresponding signal)

Section 3:

> 
>    This document does not propose to change these requirements, but
>    notes that link adaptation can be burdensome for some links to the
>    point that it would be highly desirable to push the fragmentation and
>    reassembly responsibility to the IPv6 communication endpoints.  In
>    order to accommodate this, when the router at the link ingress
>    performs link adaptation on a packet it should also send an ICMPv6
>    PTB message back to the original source (subject to rate limiting)
>    with a Next-Hop MTU less than 1280 and with Code field set to 1
>    [RFC4443].  (Note that these PTB messages are advisory in nature and
>    do not necessarily indicate packet loss.)

Some implementations validate ICMP error messages based on
connection-progress (please see RFC 5927)-- in v6 this is as good as it
can get, since you might not even receive the eg TCP header in the
ICMPv6 payload. Since you'd be both sending an error *and* performing
link adaptation, the connection would progress, and hence such
implementations would drop the ICMPv6 message. -- workaround: keep track
of whether the other endpoint reduces the size of its packets. If it
doesn't, try "drop & send ICMPv6 PTB". (and yes, if packets stop
flowing, you should assume there's a PMTUD blackhole)

Section 3:

>    "In response to an IPv6 packet that is sent to a destination located
>    beyond an IPv6 link that must perform link adaptation, the
>    originating IPv6 node may receive an ICMP Packet Too Big message
>    reporting a Next-Hop MTU less than 1280 and with Code=1.  In that
>    case, the IPv6 node is not required to reduce the size of subsequent
>    packets to less than 1500 bytes, but must perform IPv6 fragmentation
>    on those packets by breaking the packet into N roughly equal-length
>    pieces (where N is minimized and the length of each piece is smaller
>    than the Next-Hop MTU).  These fragments will be reassembled by the
>    destination."

I guess you meant "1280" rather than 1500?

That aside: the text is a bit confusing. what's should be the maximum
size of each fragment? The one reported by the ICMPv6 PTB code 1?

Section 4:
>    Regardless of whether there is a link in the path that performs link
>    adaptation, when an originating IPv6 node receives a PTB message
>    reporting a Next-Hop MTU value between 1280 and 1500 bytes, the node
>    need not reduce the size of the packets it sends.  The node may
>    instead invoke fragmentation for packets between 1281 and 1500 bytes
>    (again, by splitting the packet into N roughly equal-length pieces)
>    before submitting each fragment for link-layer framing.  These
>    fragments will be reassembled by the final destination

Why don't you want the packet size to be reduced in this case? That
aside, if you're fragmenting the packets, you're reducing their size....
so I don't get why the text says "need not reduce..."

Section 4:
>    A more interesting situation arises when PTB messages are lost on the
>    return path to the originating IPv6 node.  Since the node has no way
>    of discerning which paths may exhibit this condition, it may be
>    better served to assume the worst case for all paths and take
>    precautionary measures to avoid silent packet loss.  For example, an
>    originating IPv6 node that wishes to ensure that packets between 1281
>    and 1500 bytes will reach the final destination can use "proactive
>    fragmentation" to fragment each packet (again, by splitting the
>    packet into N roughly equal-length pieces). 

Fragmentation is not a panacea. Please check the recent discussion on
v6ops regarding IPv6 fragment filtering, and
<http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-thesis.pdf>.

> 6.  IANA Considerations
> 
>    There are no IANA considerations for this document.

Of the top of my head: Aren't you specifying a new Code value for ICMPv6
PTB? -- Then it looks like you do have actions for IANA.

Important item: At the end of the day, you're somehow saying that the
IPv6 minimum MTU is no longer 1280 -- but not explicitly. So this should
be made explicit (yes, yes.. "controversial"... but I like to call a
lemon a lemon :-) ). -- FWIW, some folks have already noted that the
hassle of having to do link adaptation, and v6-enabled links failing to
to that (I have always been for references on this topic, though).

Thanks!

Best regards,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 8 Feb 2013 10:16:57 -0800

Hi Fernando,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Fernando Gont
> Sent: Friday, February 08, 2013 2:16 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Feedback on draft-generic-6man-tunfrag-07
> 
> Hi, Fred,
> 

Thank you for taking the time to read and comment. Please see below

> are not) -- found them later... could you move them to this para?

Yes, absolutely. The studies by Matthew Luckie and his colleagues would
fit in here, as would the reference you cited below.

> that you have to rely on link-adaptation as a result of PMTUD failures,

Link adaptation is not necessarily as a result of PMTUD failures.
We already know that, for IPv6, link adaptation is a MUST if the
link MTU is smaller than 1280. What I am saying is that some links
may wish to set an even larger link adaptation threshold (say 1500)
under the presumption that PMTUD *might* fail.  

> sources to fragment their packets? (i.e., receive the corresponding

The ICMPv6 PTB messages I am proposing would be advisory in nature
and not an indication of packet loss. So, delivery of the PTB's to
the source is best-effort, and loss of the messages would not result
in a black hole. It is just that the link would need to continue to
work hard if the messages either don't get delivered to the source
or are ignored by the source. 

> ICMPv6 payload.

I'm curious as to what you mean by this. ICMPv6 PTB messages are
mandated to include "As much of (the) invoking packet as possible".
Has this not proven to be the case in operational practice? Or, are
you saying that the TCP header is sometimes not available in-the-clear,
e.g., if IP security is applied?

> Since you'd be both sending an error *and* performing
> implementations would drop the ICMPv6 message.

The system will still work if the source drops the ICMPv6 PTB
message - it's just that the system would work *better* if the
source would heed the message as specified in this document.

> -- workaround: keep track

It would be too much to ask for the link adaptation endpoint to
keep track of all of the sources and observe their behavior.

> I guess you meant "1280" rather than 1500?

No, I actually meant 1500 bytes. This is the minimum-sized reassembly
buffer that the destination is required to configure per RFC2460. So,
this is the maximum size for which the source can expect the destination
to perform reassembly unless it has some way of knowing that the
destination is capable of reassembling more.

> size of each fragment? The one reported by the ICMPv6 PTB code 1?

Each piece must be smaller than the Next-Hop MTU as reported by the
ICMPv6 PTB message, and the source must also minimize the number of
pieces it sends.  

> so I don't get why the text says "need not reduce..."

Again, this is leveraging the fact that all IPv6 nodes are
required to reassemble 1500 bytes at a minimum. It is up to
the source whether it would be better served to reduce the
size of its packets to something smaller than 1500 bytes or
to perform fragmentation and let the destination perform
reassembly. But, I get your point about confusing wording
so here is a proposed re-write:

  "...when an originating IPv6 node receives a PTB message
   reporting a Next-Hop MTU value between 1280 and 1500 bytes, the node
   can reduce the size of the packets it sends as specified in RFC1981.
   Alternatively, the node can invoke fragmentation for packets that are
   larger than the Next-Hop MTU value but no larger than 1500 bytes, since
   IPv6 destinations are required to reassemble at least that much [RFC2460]."   

> <http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-

That is worrisome. I don't understand that, if an IPv6 destination
is required to reassemble at least 1500 bytes, that middleboxes in
the network would prevent it from doing so. This seems to be even
more broken than the failure of the network to deliver ICMPv6 PTB
messages. If these core features don't work as advertised, can we
really ever expect to do anything better than 1280? 

> lemon a lemon :-) ).

What I would like to say is that the IPv6 minimum MTU size
should be bounded by the IPv6 minimum reassembly size (which is
1500 bytes) instead of bounded by some curious constant value
such as 1280. Packets larger than 1500 bytes are also possible
when the IPv6 source uses RFC4821, but these packets would be
delivered without fragmentation if they are delivered at all.

> -- FWIW, some folks have already noted that the

I think 6lowpan (RFC4944) might be one example?

Thanks - Fred
[email-masked]

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Fernando Gont <[email-masked]>
Date: Fri, 08 Feb 2013 18:19:56 -0300

> under the presumption that PMTUD *might* fail.  

That wasn't what I got in my first read of the document (at least).
WOuld it be possible to clarify that?

> e.g., if IP security is applied?

I'm saying that, at least theoretically speaking, such header might not
be there. Even if a node complies with
draft-ietf-6man-oversized-header-chain (and e.g. puts lots of extension
headers and then the tcp header in the first frag), when an error is
elicited, the tcp header might need to be left out.

That said, some implementations check for progress on the connection
before honoring ICMPv6 PTB -- which means such implementations would
drop the ICMPv6 PTB your proposing.

> source would heed the message as specified in this document.

Agreed. Just wanted to point this one out, since I don't know whether
you had considered it. -- it might be worth a para.

> keep track of all of the sources and observe their behavior.

Agreed.

> ICMPv6 PTB message, and the source must also minimize the number of

But then you *are* reducing the size of your packets (the text says "is
not required..."). Or am I missing something?

May be you mean "is not required to reduce the assumed path mtu"?

>> so I don't get why the text says "need not reduce..."
[...]
>    IPv6 destinations are required to reassemble at least that much [RFC2460]."   

Probably saying "..can reduce the assumed path mtu" (RFC1918-terminology
IIRC) might make it even clearer?

> That is worrisome.

Yes. :-)

> I don't understand that, if an IPv6 destination

I guess when it comes to packet size, something along the lines of <
1400 would generally work (i.e., the usual internet cell size, minus
some bytes for possible tunnels).

But it looks like as long as you need ICMPv6 PTB and/or fragmentation,
you're in trouble.

> such as 1280.

I wasn't objecting at all -- for instance, at least at first thought,
what you say makes sense to me.

> Packets larger than 1500 bytes are also possible

What if you need to send, say, a 5000 byte UDP datagram?

> I think 6lowpan (RFC4944) might be one example?

To be honest, I haven't picked at 6lowpan, yet. Do they implement an
adaptation layer, or they simply say "sorry, our minimum MTU is
something smaller than 1280"?

Cheers,

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 8 Feb 2013 14:41:44 -0800

Hi Fernando,

> From: Fernando Gont [mailto:[email-masked]]
> Sent: Friday, February 08, 2013 1:20 PM
> To: Templin, Fred L
> Cc: [email-masked]
> Subject: Re: Feedback on draft-generic-6man-tunfrag-07
> 

Sure. I will write up a clarification in a new draft and post it
along with the diff's.

> elicited, the tcp header might need to be left out.

Thanks - this clarifies.

> drop the ICMPv6 PTB your proposing.

It is OK if some implementations drop this "new kind" of PTB;
correct operation will still result. It would be better however
if those implementations were updated to honor the messages, and
this can be done in an incremental deployment fashion.
 
> you had considered it. -- it might be worth a para.

Yes; I will add a paragraph that captures your above points.

> > keep track of all of the sources and observe their behavior.

OK.

> May be you mean "is not required to reduce the assumed path mtu"?

Yes - I think your statement is more accurate. I will fix it.

> IIRC) might make it even clearer?

OK.
 
> >> <http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-

Ok, but what can be done to fix it?

> some bytes for possible tunnels).

I don't think even that is safe. We are entering an age where we
will more and more see multiple levels of nested tunnels within
tunnels where the innermost nesting levels will see progressively
smaller MTUs. For example, RFC2473 tunnels invoke fragmentation
when the tunnel nesting level becomes too deep to fit within the
path MTU. Unfortunately, that can only be detected if PMTUD works.

> But it looks like as long as you need ICMPv6 PTB and/or fragmentation,

Then, links that perform link adaptation will have no way of
offloading fragmentation and reassembly to the end systems.
With working IPv6 frag/reas, we should be able to do better
than that.

> what you say makes sense to me.

OK, thanks.

> What if you need to send, say, a 5000 byte UDP datagram?

The sender needs to know that the receiver can reassemble at least
as much as 5000. It should then use IPv6 fragmentation to break the
packet up into fragments no larger than 1280 and send each fragment
to the receiver (another reason why IPv6 frag/reas *must* be fixed).
 
> something smaller than 1280"?

Yes; RFC4944 specifies a link adaptation method for IPv6 over
6lowpan.

Thanks - Fred