Subject: RE: Meta-issues: On the deprecation of the fragmentation function

From: RJ Atkinson <[email-masked]>
Date: Tue, 9 Jul 2013 11:02:16 -0400

All,

I think everyone agrees that intermediate fragmentation
(e.g. by routers having high-speed links in the middle of the path) 
can be problematic,and that is why IPv6 does NOT require that 
(i.e. RFC-2460 says that the "DF" bit is invisible but always 
set for IPv6).

That noted, I agree very much with Fernando Gont's
comments on 5th July 2013 that IPv6 fragmentation in
end systems ought NOT be deprecated -- and I find his
reasoning compelling.

(SEPARATELY)

Not all deployed links are built out of wired/wireless
Ethernet or SONET.  The assumption that everything is either
Ethernet or SONET (or perhaps WDM) is a bit of a "First
World" assumption and puts lesser-developed countries/regions
at a significant disadvantage for Internet access.  

Frankly it is embarrassing for IETF participants to make 
such an assumption (i.e. that all links are Ethernet, 
SONET, WDM, or better).

While it is fine to optimise so that Ethernet or SONET links 
have great performance, for example by eliminating the
requirement for backbone routers to fragment packets,
we ought to retain the ancient  "IP OVER EVERYTHING"  mantra 
that has served the Internet (and ALL regions of the world) 
so well for so many years now.

I know of a number of link technologies that simply can't 
support the existing 1280 byte minimum IPv6 MTU.  Links 
with the lower MTU overwhelmingly are relatively low speed 
wireless links (everything in the gamut from HF radio 
to SATCOM).  My concerns about this have been expressed in
IPv6 WG meetings dating back to the early 1990s, as IETF
meeting minutes show.  All the data available to me indicates 
that these lower-speed (and smaller MTU) links are NOT going away.

For HF radio links, link transmission speed is usually 
measured in Kbps, with some deployed IPv4/HF links today
running as low as 2.4 Kbps.  VHF and UHF line-of-sight
links tend to be much faster than HF, but are still well 
below the slowest modern Ethernet link (i.e. 10 Mbps).

The IPv6 concept of adding a special link-specific
fragmentation/reassembly protocol layer has never been
practical.  By definition, these links have severe bandwidth
limitations already.  I've seen multiple instances where
specialised routers that are connected to these LOW speed
links insert IPv6 Fragmentation Headers and perform
intermediate fragmentation to get IPv6 packets through
such links.  In these special cases, there is no performance
issue because these wireless links have such low data rates.

A result of deprecating the IPv6 Fragmentation Header will be
that such links will be running IPv4 *forever*.  This is
the opposite of encouraging IPv6 transition -- it is preventing
IPv6 transition.  It also probably requires the ongoing deployment 
of IPv6::IPv4 protocol translation gateways -- and also
probably inhibits Internet deployment outside of "1st World"
countries.

Sigh.

Deprecating IPv6 Fragmentation Header and related end-system
support is going to harm users and inhibit transition to IPv6.

PLEASE, PLEASE, let us NOT go there.

Sincerely Yours,

Ran Atkinson

---

From: Randy Bush <[email-masked]>
Date: Tue, 09 Jul 2013 06:24:35 -1000

> at a significant disadvantage for Internet access.  

my small experience is that there are no technologies unique
to the Nth world.  we all buy and use the same crap from the
carriers, vendors, ...

---

From: Brian Jones <[email-masked]>
Date: Tue, 9 Jul 2013 11:16:20 -0400

+1 - All of RJ's comments.

There has to be a solution that not only allows for but encourages IPv6
deployment, not prevents it.
--
Brian

On Tue, Jul 9, 2013 at 11:02 AM, RJ Atkinson <[email-masked]> wrote:

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 9 Jul 2013 18:38:05 +0200

On Jul 9, 2013, at 17:02, RJ Atkinson <[email-masked]> wrote:

> practical.  

For 6LoWPAN, we did (we had to).  And it works well enough.

But for one problem: adaptation layer fragmentation is *transparent*, so there is no way for an application to do the equivalent of PMTUD to avoid/minimize adaptation layer fragmentation.

A feeble attempt at addressing this:

http://tools.ietf.org/html/draft-bormann-intarea-alfi-03

Grüße, Carsten

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 9 Jul 2013 13:28:02 -0400

>> fragmentation/reassembly protocol layer has never been

My comment (above) was in the context of the kinds of
low-bandwidth links with smaller MTU sizes that I was
discussing, of course. :-)

> For 6LoWPAN, we did (we had to).  And it works well enough.

I am happy to accept your word that it works well enough
for 6LoWPAN (i.e. IEEE 802.15.4).  Of course, it helps
that IEEE 802.15.4 links support up to 250 Kbps, which is 
visibly higher capacity than a number of RF links that
are deployed with IPv4 today.

However, I also have experience indicating that link-specific
fragmentation/reassembly does NOT work as well as simply
inserting an IPv6 Fragmentation Header -- at least for 
some other low-speed, smaller MTU, RF link layers that
work adequately with IPv4 today.  

Various obvious things, such as header compression, 
already are commonly used on such low-bandwidth RF links.

> But for one problem: adaptation layer fragmentation is
> *transparent*, so there is no way for an application
> to do the equivalent of PMTUD to avoid/minimize adaptation
> layer fragmentation.

Good point.

Yours,

Ran

---

From: Doug Barton <[email-masked]>
Date: Tue, 09 Jul 2013 10:31:35 -0700

I stated it a while back, but now that folks seem to be coming around I 
thought it might be worthwhile to restate that I agree that deprecating 
fragmentation is a bad idea. My part of this elephant is that we need 
fragmentation/PMTUD/Window Scaling to work reliably as we look toward 
future networks that are faster, and require larger MTU to be more 
efficient.

Doug

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 9 Jul 2013 18:12:38 +0000

Doug,

It might be interesting to revisit what we mean by deprecating IPv6 fragmentation....

It means that the IETF will not approve any new protocols that rely upon IPv6 fragmentation. Nothing more, nothing less.

Old protocols will continue to emit IPv6 fragments. In order to achieve backwards compatibility, new IPv6 implementations will MUST continue to support reassembly of incoming fragments. New IPv6 implementations MAY even support transmission of IPv6 fragments, if they want to support legacy applications that rely on the transmission of IPv6 fragments.

Network operators will make up their own minds as to whether they will forward IPv6 fragments. They do this today. The draft offers no guidance, one way or the other.

Brian Carpenter recommended some clarifying text regarding what the recommendation to deprecate actually means. You will see this in an updated version of the draft in the next day or so.

                                            Ron

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Doug Barton
> Sent: Tuesday, July 09, 2013 1:32 PM
> To: [email-masked]
> Subject: Re: Meta-issues: On the deprecation of the fragmentation
> function
>

---

From: Doug Barton <[email-masked]>
Date: Tue, 09 Jul 2013 11:41:25 -0700

> It means that the IETF will not approve any new protocols that rely upon IPv6 fragmentation. Nothing more, nothing less.

Thanks for clarifying. FWIW, I understand what is being proposed, and I 
still think it's a bad idea.

Doug

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 9 Jul 2013 19:11:42 +0000

Hi,

Going back to our discussions from last week, here is a draft that
captures the proposed uses of fragmentation for tunneling and other
purposes. It is called: "Fragmentation Revisited", and it offers an
approach to limited use of fragmentation in a way that can begin to
remove all barriers to supporting larger MTUs in the Internet, i.e.,
even with broken PMTUD, even with broken fragmentation/reassembly,
and even with tunnels.

Think about that for a second. While we currently have burned-in
numbers (576, 1280, 1500) that will never go away, the adoption of
these techniques will ensure that there will never again need to
be another burned-in number even as the Internet naturally grows
to support larger packet sizes. We will then be able to say that
the MTU of the Internet is "unlimited" or rather limited only by
the constituent links, whatever they happen to be. Comments on
the list would be welcome.

Thanks - Fred
[email-masked]
 
---

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, July 09, 2013 11:50 AM
To: [email-masked]
Subject: I-D Action: draft-generic-6man-tunfrag-08.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Fragmentation Revisited
	Author(s)       : Fred L. Templin
	Filename        : draft-generic-6man-tunfrag-08.txt
	Pages           : 8
	Date            : 2013-07-09

Abstract:
   IP fragmentation has long been subject for scrutiny since the
   publication of "Fragmentation Considered Harmful" in 1987.  This work
   cast fragmentation in a negative light that has persisted to the
   present day.  However, the tone of the work failed to honor two
   principles of creative thinking: never say "always" and never say
   "never".  This document discusses uses for fragmentation that apply
   both to the present day and moving forward into the future.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-generic-6man-tunfrag

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-generic-6man-tunfrag-08

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-generic-6man-tunfrag-08

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 10 Jul 2013 08:50:57 +1200

On 10/07/2013 05:28, RJ Atkinson wrote:
...
>> to do the equivalent of PMTUD to avoid/minimize adaptation

And you can't get round it. The decision to raise the minimum link
MTU from 576 to 1280 was not a change in principle; it was just
raising the level at which transparent adaptation layers were
presumed to be acceptable. The big change in principle was the
abolition of on-path fragmentation.

   Brian

---

From: "Manfredi, Albert E" <[email-masked]>
Date: Tue, 9 Jul 2013 21:21:00 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian
> E Carpenter

> abolition of on-path fragmentation.

I'm not sure I follow this.

First is, the change of MTU was not one of 576 to 1280. It was one of 64 to 1280. That 576 figure is the smallest maximum RECONSTITUTED packet size that IPv4 is supposed to support. Individual fragments need not be larger than 64 bytes. If that same principle had been applied to IPv6, maybe we wouldn't be having these discussions at all.

The other point is, I don't see how transparent adaptation layers are an issue at all? I don't think anyone is saying that it's impossible to transmit IPv6 over ATM cells? Or are they?

Bert

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 9 Jul 2013 23:53:24 +0200

On Jul 9, 2013, at 22:50, Brian E Carpenter <[email-masked]> wrote:

> And you can't get round it.

(I don't know if you mean that the way I read it, but:)
Yes, I'm trying to do ALFI because I think we do need a way to get around this lack of information.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 10 Jul 2013 00:08:05 +0200

> First is, the change of MTU was not one of 576 to 1280.

I think he was talking about changing the minMTU of Steve Deering's Simple IP from 576 first into the mistaken 1500 and then into 1280 in the process of turning it into IPv6.
http://tools.ietf.org/html/draft-deering-sip-00
(BTW, is that process documented anywhere? RFC1710 doesn't seem to mention the MTU.)

> The other point is, I don't see how transparent adaptation layers are an issue at all? I don't think anyone is saying that it's impossible to transmit IPv6 over ATM cells? Or are they?

The problem comes up where the transparency is a leaky abstraction (http://en.wikipedia.org/wiki/Leaky_abstraction).

For ATM, the abstraction was not very leaky as the performance impact of sending an MTU-size packet was negligible (at least for the more recent cases of ATM).

For the constrained networks Ran and I are interested in, the performance impact of adaptation layer fragmentation can be significant, and application layer protocols that get to choose their packet sizes can benefit quite a bit if they know this fragmentation is taking place and what packet sizes are the thresholds.

Please read the short ALFI draft for more background.
http://tools.ietf.org/html/draft-bormann-intarea-alfi-03
http://www.ietf.org/proceedings/85/slides/slides-85-intarea-1.pdf

Grüße, Carsten

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 9 Jul 2013 22:27:05 +0000

Hi,

> Carsten Bormann
> Sent: Tuesday, July 09, 2013 3:08 PM
> To: Manfredi, Albert E
> Cc: [email-masked] List
> I think he was talking about changing the minMTU of Steve Deering's
> Simple IP from 576 first into the mistaken 1500 and then into 1280 in
> the process of turning it into IPv6.
> > The other point is, I don't see how transparent adaptation layers are
> an issue at all? I don't think anyone is saying that it's impossible to
> transmit IPv6 over ATM cells? Or are they?
> 
> The problem comes up where the transparency is a leaky abstraction
> (http://en.wikipedia.org/wiki/Leaky_abstraction).
> 
> For ATM, the abstraction was not very leaky as the performance impact
> of sending an MTU-size packet was negligible (at least for the more
> recent cases of ATM).
> 
> For the constrained networks Ran and I are interested in, the
> performance impact of adaptation layer fragmentation can be
> significant, and application layer protocols that get to choose their
> packet sizes can benefit quite a bit if they know this fragmentation is
> taking place and what packet sizes are the thresholds.

I already said in the other thread that aviation and tactical
military links are examples of slow RF links where even as much
as 1280 is asking a lot.

About size issues, the first IPv6 spec (RFC1883) had a 576 minMTU,
and that was changed to 1280 during publication of RFC2460 based
on Steve Deering's Nov. 1997 proposal on the IPng list. Other
sizes of interest:

  68 - real IPv4 minMTU
 576 - de-facto IPv4 minMTU (the one everyone assumes)
 576 - real IPv4 minMRU
 576 - old IPv6 minMTU
1280 - current IPv6 minMTU
1500 - current IPv6 minMRU
1500 - de-facto IPv4 minMRU (the one everyone assumes)
1500 - legacy Ethernet MTU
2048 - proposed new IPv6 minMRU

Fred

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 10 Jul 2013 00:32:58 +0000

Doug,

Let's see if we can find some common ground. 

Assume that the IETF is considering a new protocol that doesn't run over TCP. In order to deal with MTU issues, the new protocol must do one of the following:

a) implement PLMTUD or PMTUD
b) restrict itself to sending PDUs so small that when they are encapsulated in an IPv6 header, the resulting packet will not exceed 1280 bytes
c) rely on IPv6 fragmentation

Is there ever a reason why c) is better than a) or b). For that matter, is c) ever an acceptable solution?

                                                  Ron

> From: Doug Barton [mailto:[email-masked]]
> Sent: Tuesday, July 09, 2013 2:41 PM
> To: Ronald Bonica
> Cc: [email-masked]
> > It means that the IETF will not approve any new protocols that rely
> upon IPv6 fragmentation. Nothing more, nothing less.
>

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 9 Jul 2013 20:42:22 -0400

On 09  Jul 2013, at 16:50 , Brian E Carpenter wrote:
> MTU from 576 to 1280 was not a change in principle;

For IPv6, the specification of any minimum MTU might well 
have been the fundamental mistake here.

Specifying a minimum MTU above 576 definitely was a mistake 
in practical terms of IPv6 deployability -- and some of us 
objected to the change at the time (giving examples of 
links that would be broken -- which have come true since).

As you know already, IPv4 simply does not have any
Minimum Link MTU.  So it was probably over-specification 
for IPv6 to specify any minimum link MTU.

For IPv4, RFC-791 says 576 actually was the size of buffer space
that hosts had to have available for fragment reassembly.  

RFC-1122 agrees with RFC-791 and gives it the name EMTU_R 
  ("Effective MTU to receive").  

RFC-1122 further defines EMTU_S ("Effective MTU to send")
and says: 
  "... the IP layer SHOULD use EMTU_S <= 576 whenever
  the destination address is not a a connected network,
  and otherwise use the connected network's MTU."

A bit later on, RFC-1122 also says:
   "Since nearly all networks in the Internet currently
   support an MTU of 576 or greater, we strongly recommend
   the use of 576 for datagrams sent to non-local networks."

Again, this doesn't make 576 a minimum link MTU for IPv4.

> The big change in principle was the abolition of on-path fragmentation.

Well, at the moment, on-path fragmentation is not actually prohibited,
as near as several implementers (including me) can tell, although it 
absolutely is never required to be supported by routers.  

Everyone agrees that on-path fragmentation by routers is generally 
undesirable.  

Everyone also agrees that a router, for example a backbone high-speed router, 
must not be forced to fragment anything -- for practical performance reasons 
that are widely understood.  

However, the IPv6 specs don't prohibit a router from choosing to fragment 
a packet in the way described in my earlier note in those circumstances.  
Such fragmentation is generally undetectable at present, except to the 
router that performs the fragmentation in order to transmit the IPv6 packet 
over a link having a smaller MTU.  

This difference between "not required" and "prohibited" is critical 
for IPv6 deployment over low-rate links having a smaller MTU size, 
as I outlined in my previous note within this thread.

If a goal of this WG is to encourage deployment of IPv6 and transition
from IPv4 to IPv6, then it would be best not to change the current status
of fragmentation in IPv6 -- and it would be helpful to change the minimum
Link MTU to a value lower than 1280 (or deprecate the Minimum Link MTU
specification entirely, although that seems quite unlikely here now).

I'd really like to see IPv6 get universally deployed.  These particular
changes are significant barriers to that happening and create perverse
incentives either to stay with IPv6 forever or to build & deploy
IPv6::IPv4 translational gateways (ugh).

Cheers,

Ran

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 14:58:44 +0000

Hi Ron,

> Ronald Bonica
> Sent: Tuesday, July 09, 2013 5:33 PM
> To: Doug Barton
> Cc: [email-masked]
> Subject: RE: Meta-issues: On the deprecation of the fragmentation
> function
> 
> Doug,
> 
> Assume that the IETF is considering a new protocol that doesn't run
> over TCP. In order to deal with MTU issues, the new protocol must do
> one of the following:
> 
> b) restrict itself to sending PDUs so small that when they are
> encapsulated in an IPv6 header, the resulting packet will not exceed
> 1280 bytes

Fragmentation at a tunnel ingress router is unavoidable. Proof:

  - a tunnel configures a 1280 MTU
  - When its packets are encapsulated they emerge as (1280 + HLEN)
    (the length of the encapsulating headers)
  - the tunnel crosses a 1280 link somewhere in the path to the egress
  - the packet is dropped with a PTB signal sent back
  - the ingress now has two choices: 1) start fragmenting, 2) quit.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 15:23:57 +0000

Hi Ran,

Cutting a bunch of stuff that basically repeats what I

> absolutely is never required to be supported by routers.

Routers that configure RFC2473 tunnels require fragmentation
in order to meet the IPv6 minMTU. This is not fragmentation
of the original packet before encapsulation but rather
fragmentation of the tunneled packet after encapsulation.
 
> Everyone agrees that on-path fragmentation by routers is generally

Again, never say "always" and never say "never". Undesirable
or not, fragmentation at tunnel routers is unavoidable in
some circumstances.

> that are widely understood.

As long as there is an IPv6 minMTU, tunnels will need to support
some form of fragmentation. And, because PMTUD doesn't work very
well, there needs to be an IPv6 minMTU below which the host is
guaranteed that its packets will be delivered. 

> as I outlined in my previous note within this thread.

You seem to have not seen the previous thread where I already
showed the need to accommodate low-end links (6lowpan, aviation,
tactical military, etc.).

> specification entirely, although that seems quite unlikely here now).

I don't think changing the minMTU to something smaller than
1280 now is necessary or even desirable. And, since PMTUD
doesn't work very well, there needs to be *some* minMTU
below which the host can be absolutely assured that its
packets will get through.

> IPv6::IPv4 translational gateways (ugh).

I want IPv6 too. But, I want it to go forward with no
limits on migration to larger packets. In other words,
no more magic numbers except 2^16 for IPv4 and 2^32
for IPv6.

Thanks - Fred

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 10 Jul 2013 15:49:50 +0000

Fred,

There are alternatives....

Probably, the best alternative is for the tunnel ingress router to tunnel ingress router to discover the PMTU to the egress. When the tunnel ingress router receives a packet that is so large that it cannot be forwarded through the tunnel, it discards the packet and sends an ICMP PTB to the packet's originator. The packet's originator then modifies its sending behavior based upon its new estimate of the PMTU associated with the destination.

So, for the purposes of MTU management, the tunnel is just another link.

                                                     Ron

> From: Templin, Fred L [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2013 10:59 AM
> To: Ronald Bonica; Doug Barton
> Cc: [email-masked]
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Ronald Bonica
> > Is there ever a reason why c) is better than a) or b). For that
> > matter, is c) ever an acceptable solution?
> 
> > > > It might be interesting to revisit what we mean by deprecating
> > > > IPv6
> > > fragmentation....
> > > >
> > > Thanks for clarifying. FWIW, I understand what is being proposed,
> > > and
> > I

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 15:59:29 +0000

Hi Ron,

> From: Ronald Bonica [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2013 8:50 AM
> To: Templin, Fred L; Doug Barton
> Cc: [email-masked]
> Probably, the best alternative is for the tunnel ingress router to
> tunnel ingress router to discover the PMTU to the egress. When the
> tunnel ingress router receives a packet that is so large that it cannot
> be forwarded through the tunnel, it discards the packet and sends an
> ICMP PTB to the packet's originator. The packet's originator then
> modifies its sending behavior based upon its new estimate of the PMTU
> associated with the destination.

Sure, the tunnel ingress can probe the path to the egress; such a
probing method is already covered by SEAL. But, if the path MTU will
not accommodate a packet that after encapsulation is as large as
(1280 + HLEN) there is no alternative for the ingress other than to
start fragmenting since the ingress is not allowed to send a PTB
message reporting a size smaller than 1280. 

> So, for the purposes of MTU management, the tunnel is just another

True, but I want that link to have an unbounded MTU. In other
words, encapsulate and send everything regardless of its size
even if a little bit of fragmentation is necessary (but try to
tune out the fragmentation if possible).

You should really have a look at my new draft:

Thanks - Fred

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 10 Jul 2013 17:11:51 +0000

> probing method is already covered by SEAL.

Most GRE implementations do this, too.

 But, if the path MTU will
> message reporting a size smaller than 1280.

I understand that you want to solve for the use-case in which a tunnel interior link has MTU < (1280 + HLEN). But before solving for that use-case, we need to do a cost/benefit analysis.

We understand the cost of solving for this use-case. The task of reassembly is moved to the egress router. So, we need to make sure that the egress router is large enough to handle the task of reassembly and we need to make sure that its resources cannot be monopolized by a DoS attack. We also have to maintain our fragmentation capability.

Note that some of the cost is absorbed by the owner of the egress router. However, a portion of the cost is absorbed by the entire community, as they deal with the operation complexity associated with fragmentation.

Now let's try to understand the benefit. Is there an installed base of IPv6-capable links with MTU < (1280 + HLEN) that carry traffic between tunnel endpoints? Is there a reason why someone might want to design a network this way?

If we were to solve for this use-case, who would be the beneficiary? Possibly the party deploying the MTU-challenged link? Or, asked another way, would cost be assigned to the beneficiary?

                                                              Ron

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 17:23:35 +0000

Hi Ron,

> Sent: Wednesday, July 10, 2013 10:12 AM
> I understand that you want to solve for the use-case in which a tunnel
> interior link has MTU < (1280 + HLEN).

Yes; for example, a 1280 MTU tunnel crossing another 1280 MTU
tunnel.

> But before solving for that use-
> case, we need to do a cost/benefit analysis.
> 
> We understand the cost of solving for this use-case. The task of
> reassembly is moved to the egress router. So, we need to make sure that
> the egress router is large enough to handle the task of reassembly and
> we need to make sure that its resources cannot be monopolized by a DoS
> attack. We also have to maintain our fragmentation capability.

I understand that. But a couple of points:

1) the egress will never be asked to reassemble more than (1500 + HLEN)
2) the ingress will feel the pain too and will be motivated to tune out
   the fragmentation
 
> Note that some of the cost is absorbed by the owner of the egress
> router. However, a portion of the cost is absorbed by the entire
> community, as they deal with the operation complexity associated with
> fragmentation.

The community can help to get rid of fragmentation. All they have
to do is configure a larger MTU on links that connect routers to
other routers. The MTU should be (1500 + HLEN) or larger.

> Now let's try to understand the benefit. Is there an installed base of
> IPv6-capable links with MTU < (1280 + HLEN) that carry traffic between
> tunnel endpoints? Is there a reason why someone might want to design a
> network this way?

Tunnels within tunnels, as one example. They are a common use case
in operational practice today.

> If we were to solve for this use-case, who would be the beneficiary?

The entire Internet would benefit, since all barriers to larger
MTUs would be removed.

> Possibly the party deploying the MTU-challenged link? Or, asked another
> way, would cost be assigned to the beneficiary?

Not sure I understood that. Everyone benefits, but the tunnel
endpoints doing the fragmentation have to do what is necessary
to maintain the tunnel in any event.

Thanks - Fred

---

From: james woodyatt <[email-masked]>
Date: Wed, 10 Jul 2013 11:22:49 -0700

On Jul 10, 2013, at 08:49 , Ronald Bonica <[email-masked]> wrote:
> [...]
> [...]

ICMPv6 packet too big errors are unreliable on the real-world Internet.

I hate to sound like a broken record, but I will: I look forward to reviewing a proposal to update to Generic Packet Tunneling in IPv6 [RFC 2473] for implementing tunnel path MTU discovery at the encapsulation layer [c.f. RFC 4821].

--
james woodyatt <[email-masked]>
core os networking

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 10 Jul 2013 19:21:30 +0000

Hi James,

If ICMPv6 PTB messages are unreliable, fragmentation breaks just as badly as PMTUD.

At the risk of going off-topic, please take a look at draft-bonica-intarea-gre-mtu-02.

                                                 Ron

> james woodyatt
> Sent: Wednesday, July 10, 2013 2:23 PM
> To: [email-masked]
> I hate to sound like a broken record, but I will: I look forward to
> reviewing a proposal to update to Generic Packet Tunneling in IPv6 [RFC
> 2473] for implementing tunnel path MTU discovery at the encapsulation
> layer [c.f. RFC 4821].
> 
> 
> --

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 19:39:27 +0000

Hi James,

> Sent: Wednesday, July 10, 2013 11:23 AM
> To: [email-masked]

Wrap your RFC2473 tunnel in SEAL and you are good to go
(see draft-templin-intarea-seal). A quick read of Section
4 of my new draft should be all you need to understand
what I'm talking about:

Note however that all the tunnel needs is a limited application
of RFC4821 to ensure that the *required* packet sizes are
accommodated. For larger packets, assume that the hosts sending
large packets are using RFC4821 themselves (so the tunnel doesn't
have to). All the tunnel needs to do is "take care of the smalls,
and let the bigs take care of themselves".

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 19:55:46 +0000

Hi Ron,

> Sent: Wednesday, July 10, 2013 12:22 PM
> To: james woodyatt; [email-masked]

I'm trying to understand that, but maybe you are talking about
IP fragmentation? SEAL fragmentation is a different thing, and
can be used in place of IP fragmentation.

Thanks - Fred

> Of

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jul 2013 20:07:31 +0000

Hi,

> A quick read of Section

Here is the text I'm talking about so I won't have to keep
repeating myself:

Thanks - Fred

---

4.  IPv6 Tunnels

   IPv6 tunnels are used for many purposes, including transition,
   security, mobility, routing control, etc.  While it is assumed that
   transition mechanisms will eventually give way to native IPv6, it is
   clear that the use of tunnels for other purposes will continue and
   even expand.  A long term strategy for dealing with tunnel MTUs is
   therefore required.

   Tunnels may cross links (perhaps even other tunnels) that configue
   only the IPv6 minMTU of 1280 bytes while the tunnel ingress must be
   able to send packets that are at least 1280 bytes in length so that
   the IPv6 minMTU is extended to the source.  However, these tunneled
   packets become (1280 + HLEN) bytes on the wire (where HLEN is the
   length of the encapsulating headers), meaning that they would be
   vulerable to loss at a link within the tunnel that configures a
   smaller MTU.  Therefore, the only way to satisfy the IPv6 minMTU is
   through network layer fragmentation and reassembly between the tunnel
   ingress and egress, where the ingress fragments its tunneled packets
   that are larger than (1280 - HLEN) bytes.

   Unfortunately, fragmentation and reassembly are a pain point for in-
   the-network routers - especially for those that are nearer the core
   of the network.  It is therefore highly desirable for the tunnel
   ingress to discover whether this fragmentation and reassembly can be
   avoided.  This can only be done by allowing the ingress to probe the
   path to the egress by sending whole 1500 byte probe packets to
   discover whether the probes can be delivered to the egress without
   fragmentation.  These 1500 byte probes appear as (1500 + HLEN) bytes
   on the wire, therefore the path must support an MTU of at least this
   size in order for the probe to succeed.

   The tunnel fragmentation and reassembly strategy is therefore as
   follows:

   1.  When the tunnel ingress receives a packet that is no larger than
       (1280-HLEN) bytes, it encapsulates the packet and sends it to the
       egress without fragmentation.  The egress will receive the packet
       since it is small enough to fit within the IPv6 minMTU of 1280
       bytes.

   2.  When the tunnel egress receives a packet that is larger than 1500
       bytes, it encapsulates the packet and sends it to the egress
       without fragmentation.  If the packet is lost in the network due
       to a size restriction, the ingress may or may not reeceive a PTB
       message which it can then forward to the original soruce.
       Whether or not a PTB message is received, however, it is the
       responsibility of the original source to ensure that its packets
       larger than 1500 bytes are making it to the final destination by
       using a path probing technique such as specified by [RFC4821].

   3.  When the tunnel ingress receives a packet larger than (1280 -
       HLEN) but no larger than 1500 bytes, and it is not yet known
       whether packets of this size can reach the egress without
       fragmentation, the ingress encapsulates the packet and uses
       network layer fragmentation to fragment it into two pieces that
       are each signifiicantly smaller than (1280 - HLEN) bytes.  At the
       same time, the tunnel ingress sends an unfragmented 1500 byte
       probe packet toward the egress (subject to rate limiting) which
       will appear as (1500 + HLEN) bytes on the wire.  If the egress
       receives the probe, it informs the ingress that the probe
       succeeded.  If the probe succeeds, the ingress can suspend the
       fragmentation process and send packets between (1280-HLEN) and
       1500 bytes without using fragmentation.  This probing process
       exactly parallels [RFC4821].

   In this method, the tunnel egress must configure a slightly larger
   MRU than the minMRU specified for IPv6 in order to accommodate the
   HLEN bytes of tunnel encapsulation during reassembly. 2KB is
   recommended as the minMRU for this reason.

   These procedures give way to the ability for the tunnel ingress to
   configure an unlimited MTU (theoretical limit is 2^16 bytes for IPv4
   and 2^32 bytes for IPv6).  They will therefore naturally lead to the
   Internet migrating to larger packet sizes with no dependence on
   traditional path MTU discovery.  Operators will also soon discover
   that configuring larger MTUs on links between routers (e.g., 2KB or
   larger) will dampen the fragmentation and reassembly requirements
   until fragmentation and reassembly usage is gradually tuned out of
   the network.

   These procedures are not supported by the existing IPv6 fragmentation
   procedures, however they are exactly those specified in the
   Subnetwork Encapsulation and Adaptation Layer (SEAL)
   [I-D.templin-intarea-seal].  Widespread adoption of SEAL will
   therefore naturally lead to an Internet which no longer places MTU
   restrictions on tunnels and therefore supports natural migration to
   unbounded packet sizes.

---

---

From: james woodyatt <[email-masked]>
Date: Wed, 10 Jul 2013 14:29:10 -0700

> At the risk of going off-topic, please take a look at draft-bonica-intarea-gre-mtu-02.

I did. I won't comment about that here, because I too don't want to drift off topic.

Extending my remarks: Mr. Templin is right to point out that tunnels over networks where ICMPv6 Packet Too Big errors are not reliably delivered from the egress to the ingress, and also where such errors are not delivered from destination hosts back to the egress, have no choice but to assume the PMTU from the ingress to the egress is 1280 octets, and they must fragment packets that would exceed that length when encapsulated in an IPv6 header chain. If they cannot fragment, than paths through the tunnel do not meet the minimum MTU requirement.

I agree with him that SEAL addresses this problem, by using a different mechanism to fragment that the RFC 2460 extension header we are proposing to deprecate, but RFC 2473 doesn't do that, and there is a procedural issue I am trying to surface here: if we're going to deprecate the fragmentation header, while at the same time recognizing that the Internet is unable to deliver ICMPv6 Packet Too Big errors reliably, then we are blowing a big hole in the standards by not promoting SEAL or something like it as a STANDARD replacement for GRE and RFC 2473 encapsulation.

All I'm saying is that if we're going to start deprecating fragment headers, then let's make sure we are deprecating all of the things that rely on them, and providing a standard replacement for each of them in turn.

[Insert here shopped version of classic Hyperbole And A Half Cartoon, saying "DEPRECATE ALL THE THINGS!"]

---

From: Ronald Bonica <[email-masked]>
Date: Thu, 11 Jul 2013 13:50:07 +0000

Yes, I am talking about IPv6 fragmentation.

                     Ron

> Sent: Wednesday, July 10, 2013 3:56 PM
> To: Ronald Bonica; james woodyatt; [email-masked]
> I'm trying to understand that, but maybe you are talking about IP
> fragmentation? SEAL fragmentation is a different thing, and can be used
> in place of IP fragmentation.
> 
> Thanks - Fred
> > > modifies its sending behavior based upon its new estimate of the
> > > PMTU associated with the destination.
> > > > [...]
> > >
> > > 2473] for implementing tunnel path MTU discovery at the
> > > encapsulation layer [c.f. RFC 4821].
> > >
> > >
> > > --
> > > IETF IPv6 working group mailing list [email-masked] Administrative
> > > Requests: https://www.ietf.org/mailman/listinfo/ipv6

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 11 Jul 2013 16:41:47 +0000

Hi Ron,

> Sent: Thursday, July 11, 2013 6:50 AM
> To: Templin, Fred L; james woodyatt; [email-masked]

OK, but if you want to deprecate that you need to replace it
with something else like SEAL.

Thanks - Fred

> > > > reviewing a proposal to update to Generic Packet Tunneling in
> IPv6
> > > [RFC
> > > > -----------------------------------------------------------------
> --
> > -
> > > > -----------------------------------------------------------------
> --
> > -
> > > >
> > >
> > >
> > >

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 Jul 2013 08:34:57 +1200

> with something else like SEAL.

If you mean that 1280 must be provided everywhere regardless of
nested tunnels, yes. (Setting aside the NAT64 mess.)

   Brian