Subject: [RRG] LISP PMTU & fragmentation problems

From: Robin Whittle <[email-masked]>
Date: Sat, 08 Mar 2008 02:40:53 +1100

Short version:    lisp-06's new material on Path MTU limits includes
                  some text on how to resolve the problems if they
                  are deemed to be bad enough to need a solution
                  within LISP.  I can't understand this text in any
                  way which makes practical sense.

                  Also, it would be good to publish the research
                  which indicates that 1500 byte MTU limits are
                  relatively rare.

                  If the problems of PMTU are deemed not worth
                  solving within LISP, then LISP would be deployed
                  on the assumption that all transit links would
                  be capable of some much higher than 1500 byte
                  PMTU.

                  This would tend to constrain the locations of ITR
                  and ETR functions to be at or near border routers,
                  in order that they have unfettered access to jumboframe
                  capable links to the core of the Internet.

                  This would seem to be a major restriction on the
                  ability of operators to place ITRs and ETRs wherever
                  they like.

                  Likewise, it would seem to reduce the number of
                  devices which could do ITR or ETR functions and
                  thereby lead to bottlenecks and to these devices
                  needing to be large and expensive.

Hi Dino and Team,

Below is the text from draft-farinacci-lisp-06 which concerns path
MTU limits, packets being too long due to encapsulation overhead,
fragmentation etc.

=======

5. Tunnelling Details

   ...

   Since additional tunnel headers are prepended, the packet becomes
   larger and in theory can exceed the MTU of any link traversed from
   the ITR to the ETR.  It is recommended, in IPv4 that packets do not
   get fragmented as they are encapsulated by the ITR.  Instead, the
   packet is dropped and an ICMP Too Big message is returned to the
   source.

   Based on informal surveys of large ISP traffic patterns, it appears
   that most transit paths can accommodate a path MTU of at least 4470
   bytes.  The exceptions, in terms of data rate, number of hosts
   affected, or any other metric are expected to be vanishingly small.

   To address MTU concerns, mainly raised on the RRG mailing list, the
   LISP deployment process will include collecting data during its pilot
   phase to either verify or refute the assumption about minimum
   available MTU.  If the assumption proves true and transit networks
   with links limited to 1500 byte MTUs are corner cases, it would seem
   more cost-effective to either upgrade or modify the equipment in
   those transit networks to support larger MTUs or to use existing
   mechanisms for accommodating packets that are too large.

   For this reason, there is currently no plan for LISP to add an
   additional, complex mechanism for implementing fragmentation and
   reassembly in the face of limited-MTU transit links.  If analysis
   during LISP pilot deployment reveals that the assumption of
   essentially ubiquitous, 4470+ byte transit path MTUs, is incorrect,
   then LISP can be modified prior to protocol standardization to add
   support for one of the proposed fragmentation and reassembly schemes.
   Note that one simple scheme is detailed in Section 5.4.

5.4. Dealing with Large Encapsulated Packets

   In the event that the MTU issues mentioned above prove to be more
   serious than expected, this section proposes a simple and stateless
   mechanism to deal with large packets.  The mechanism is described as
   follows:

   1.  Define an architectural constant S for the maximum size of a
       packet, in bytes, an ITR would receive from a source inside of
       its site.

   2.  Define L to be the maximum size, in bytes, a packet of size S
       would be after the ITR prepends the LISP header, UDP header, and
       outer network layer header of size H.

   3.  Calculate: S + H = L.

   When an ITR receives a packet of size greater than L on a site-facing
   interface and that packet needs to be encapsulated, it resolves the
   MTU issue by first splitting the original packet into 2 equal-sized
   fragments.  A LISP header is then pre-pended to each fragment.  This
   will ensure that the new, encapsulated packets are of size (S/2 + H),
   which is always below the effective tunnel MTU.

   When an ETR receives encapsulated fragments, it treats them as two
   individually encapsulated packets.  It strips the LISP headers then
   forwards each packet to the destination host of the destination site.
   The two fragments are reassembled at the destination host into the
   single IP datagram that was originated by the source host.

   This behavior is performed by the ITR when the source host originates
   a packet when the DF field of the IP header is set to 0.  When the DF
   field of the IP header is set to 1, or the packet is an IPv6 packet
   originated by the source host, the ITR will drop the packet when the
   size is greater than L, and sends an ICMP Too Big message to the
   source with a value of S, where S is (L - H).

   This specification recommends that L be defined as 1500.

=======

I will deal first with the section 5.4 material.  I will assume a header
size of 36 bytes for IPv4.  This is 20 bytes for the IPv4 header, 8 for
the UDP header and 8 for the LISP Locator Reach Bits and nonce.  In fact,
you need to allow more than this, depending on whether you need 4 bytes
for up to 31 Locator Reach Bits, 8 bytes for 62 etc.  Similarly, for IPv6
I assume a 56 byte overhead.

Jumboframes can be up to 16110 bytes, according to a cavalcade of MTU
values compiled by Joe St Sauver:

  http://darkwing.uoregon.edu/~joe/jumbo-clean-gear.html

According to:

  http://www.cisco.com/warp/public/121/mtu_atm.html

the figure of 4470 bytes was chosen to match FDDI and HSSI.

My understanding of what you wrote in 5.4 is:

IPv4 with Don't Fragment set to 0:
----------------------------------

> This specification recommends that L be defined as 1500.

> 2.  Define L to be the maximum size, in bytes, a packet of size S
>     would be after the ITR prepends the LISP header, UDP header, and
>     outer network layer header of size H.
>
> 3.  Calculate: S + H = L.

L = 1500
H =   36
S = 1464

> 1.  Define an architectural constant S for the maximum size of a
>     packet, in bytes, an ITR would receive from a source inside of
>     its site.

S is 1464 bytes.  But an ITR could receive a packet of any length from a
source inside its site.  So this sentence makes no proper sense to me.

>  When an ITR receives a packet of size greater than L on a site-facing
>  interface and that packet needs to be encapsulated, it resolves the
>  MTU issue by first splitting the original packet into 2 equal-sized
>  fragments.  A LISP header is then pre-pended to each fragment.

I guess you meant 'S' (1464) rather than 'L' (1500).

>  This
>  will ensure that the new, encapsulated packets are of size (S/2 + H),
>  which is always below the effective tunnel MTU.

This last sentence would only be true (assuming PMTU to the ETR of 1500
bytes, whereas it could be somewhat lower than this) if the ITR had an
admission limit of 2908 bytes. I am ignoring Ethernet headers and maybe
some other details here, so these figures may not be precise.

 Traffic packet:

    2908 bytes = 20 bytes IPv4 header + 2888 bytes payload.

 Fragments into two packets:

    1464 bytes =     20 bytes IPv4 header
                 + 1444 bytes payload

 Add LISP outer header:

    1500 bytes =     36 bytes LISP header
                 +   20 bytes IPv4 header
                 + 1444 bytes payload

The problem is that this paragraph:

>  When an ITR receives a packet of size greater than L on a site-facing
>  interface and that packet needs to be encapsulated, it resolves the
>  MTU issue by first splitting the original packet into 2 equal-sized
>  fragments.  A LISP header is then pre-pended to each fragment.

doesn't specify an upper limit to the packet which will be handled in this
way.

Since you only allow two fragments, I think there needs to be some upper
limit on the admitted packet size.

However, all this assumes that the ITR has a 1500 byte PMTU to the ETR.
In many cases, the PMTU will be a lot higher.  So the above algorithm does
not allow the ITR to send longer packets without fragmentation.

I think any solution needs to enable larger than ~1500 byte traffic
packets to be encapsulated and sent without fragmentation, assuming the
PMTU to the ETR allows this.  This will be complex, since the ITR needs to
discover this PMTU, and can't do this instantly.  Fred Templin and I have
approaches to this:

  http://www.firstpr.com.au/ip/ivip/pmtud-frag/
  http://tools.ietf.org/html/draft-templin-seal

I want to update my proposal somewhat, based on some of Fred's ideas.

IPv6 or IPv4 with DF = 1
------------------------

>  ...  the ITR will drop the packet when the size is greater than L, and
>  sends an ICMP Too Big message to the source with a value of S, where S
>  is (L - H).

This makes no sense to me.  It would make more sense if this occurred when
the packet length was greater than S (1464 bytes for IPv4 or 1444 for IPv6).

But you still have the problem that the system makes no use of longer
PMTUs, which will frequently exist, to the ETR.

I think there is no way around the thorny task of the ITR having to
discover the PMTU to the ETR, and while it is doing that, there remains
the problem of what to do with packets which would need to be fragmented
in order to be delivered if the PMTU was (as must be assumed) only 1500 bytes.

Research and upgrading transit networks . . .

>  Based on informal surveys of large ISP traffic patterns, it appears
>  that most transit paths can accommodate a path MTU of at least 4470
>  bytes.

It would be good for this research to be made public.

> The exceptions, in terms of data rate, number of hosts affected, or any
> other metric are expected to be vanishingly small.

Again, this needs to be justified with arguments and observations.  By the
time a map-encap scheme could be implemented, we can expect gigabit
Ethernet to be more widely deployed.  Also, if there is some forewarning
that the new map-encap scheme doesn't work well with PMTU of 1500 or so
bytes, then this might be an incentive for people to upgrade their
systems.  However, it would also be an incentive not to use the map-encap
scheme.

>  To address MTU concerns, mainly raised on the RRG mailing list, the
>  LISP deployment process will include collecting data during its pilot
>  phase to either verify or refute the assumption about minimum
>  available MTU.

I don't think a pilot scheme would be very useful for this, since it would
only involve a small number of ASes.  A better approach would be a
thorough survey of all links between ASes.

Still, even if you can prove that all links have a PMTU of 4470 or
whatever, your plan of ignoring PMTU and fragmentation problems still
leaves the ITR with the task of handling traffic packets which would be
short enough if sent directly on the transit link (meaning that some hosts
will do this, and can do it fine for non-EID destinations), but will be
too long after the addition of the LISP header.

The ITR really needs to solve this problem, since if a packet is dropped
en-route to the ETR due to some PMTU limit, the sending host will not hear
about it.  I wrote a long piece on the list last year about why it is
impractical for the ITR to handle ICMP PTB messages coming back from the
tunnel so as to generate valid PTB messages to the sending host from which
the traffic packet originated.

You could, perhaps, assume some fixed PMTU of 4470 or whatever, but then
in the future, when some or many transit links support 64k byte packets or
the like, you are still going to be stuck limiting traffic packets to 4470
minus the length of the LISP header.

>  If the assumption proves true and transit networks
>  with links limited to 1500 byte MTUs are corner cases, it would seem
>  more cost-effective to either upgrade or modify the equipment in
>  those transit networks to support larger MTUs or to use existing
>  mechanisms for accommodating packets that are too large.

OK - who is going to pay for truckloads of new routers or line cards?

This is before LISP is introduced, so there no-one is benefiting yet.

More likely, LISP would be resisted or rejected due to its incompatibility
with a small but significant subset of transit links.

I still think your preferred approach involves a heavy cost of
inflexibility and bottlenecks at border routers - because you force all
ITRs and ETRs to be on (in effect) gigabit links to the outside world.

   - Robin              http://www.firstpr.com.au/ip/ivip/

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 8 Mar 2008 20:46:40 -0800

>                  way which makes practical sense.

Well, I'm sorry about that. The text is written in such simple and  
precise terms, I am surprised you wouldn't understand it.

>                  which indicates that 1500 byte MTU limits are

There wasn't research. It was a survey. I asked 10 people, and all 10  
made this statement. So there isn't much to publish.

>                  be capable of some much higher than 1500 byte

That is correct.

>                  capable links to the core of the Internet.

Right, or you do fragmentation. We did have an out.

>                  ability of operators to place ITRs and ETRs wherever

For Loc/ID purposes (there are several other reasons to use LISP than  
what is intended by this venue), we want to strongly suggest that xTRs  
be placed on CE (CPE) routers. We think that is the best balance of  
tradeoffs.

>                  needing to be large and expensive.

No, not bottlenecks, easier deployability.

You make it sound that by adding LISP to a router, it will get  
overloaded. You capacity design a router to deal with the input rate  
and density of the box and the amount of work you have to do. ITRs  
won't attract more traffic when they are at the CE. They get traffic  
based on who wants to send data to external destinations.

> source inside its site.  So this sentence makes no proper sense to me.

It means that if the packet from the source is >= 1464, the packet  
will be fragmented.

> I guess you meant 'S' (1464) rather than 'L' (1500).

The total packet size after the ITR is finished with it is L.

> not allow the ITR to send longer packets without fragmentation.

That is correct. But maybe a source site that talks to a destination  
site of the same ISP that advertises support for larger MTUs, then the  
ITR be configured with an L value of 4470 or 9182 perhaps.

> IPv6 or IPv4 with DF = 1

You want the ITR to tell the host to send a size so when the outer IP  
header, UDP header, and  LISP header are prepended, the size of the  
packet is L.

Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 9 Mar 2008 17:39:06 -0700

Dino,

I have to agree that the new LISP text reads like a
"slacker's guide" to MTU handling. First, it only does
the splitting for IPv4 packets that had DF=0 from the
original host, and there really aren't very many of those
in-the-wild these days. Second, for IPv6 and IPv4 with DF=1,
the original source will be told a degenerate MTU that would
go against the principle of least surprise ("I expected 1500,
but only got 1464") plus it *always* results in packet loss
until MTU discovery has converged. Finally, there is no
provision for discovering MTUs larger than 1500 even though
the core may soon transition to all-GigE.

You have said that you believe the core is comprised almost
completely of links that can do quite a bit more than 1500.
Why not trust your judgement and just run SEAL; that way, you
get to enjoy the larger MTUs and will have no fragmentation
unless a degenerate link shows up on the path - in which case
SEAL will detect and correct it.

Thanks - Fred
[email-masked] 

>-----Original Message-----
>From: Dino Farinacci [mailto:[email-masked]] 
>Sent: Saturday, March 08, 2008 8:47 PM
>To: Robin Whittle
>Cc: Routing Research Group
>Subject: [RRG] Re: LISP PMTU & fragmentation problems
>

---

From: Robin Whittle <[email-masked]>
Date: Mon, 10 Mar 2008 14:30:41 +1100

Hi Dino,

> precise terms, I am surprised you wouldn't understand it.

I think I understood it OK, though it would be better if you put the
1500 byte recommendation at the beginning, and broke it into two
sections: IPv4 DF=0 first and IPv6 or IPv4 DF=1 second.

What I understood from the ID seemed so at odds with a desirable
outcome that I assumed you were mistaken in some way, writing text
which conveyed something different from what you really intended.

> made this statement. So there isn't much to publish.

Since the PMTU/fragmentation problem needs to be tackled by any
mature map-encap scheme, and since your intention is to ignore the
problem if you think some high proportion of links in the DFZ
("transit paths" was the term you used in section 5.) support
jumboframes, I think it would help folks evaluating LISP if you had
more substantial research.

For instance, somehow get a bunch of border routers in a bunch of
ISPs and AS end-user networks and find out what the PMTU is from
each such router to all the rest.  Then, somehow characterise how
representative the sample is, so that the results can be
extrapolated in some meaningful way to all existing ISP and AS
end-user networks.

Your plan to wait until you have a LISP pilot running doesn't seem
as good as doing proper research now.  Waiting until a pilot
deployment delays the gathering of the important information, and
would be for only a small number of participating networks.

Some other things I am keen to know are:

1 - How far do "jumboframe" links go into these ISP and AS
    end-user networks.  (Therefore how deep in these networks -
    meaning close to and ISPs' customers' networks - can you place
    ITRs and ETRs.  This strongly affects how you spread the load,
    the capacity these devices must have etc.)

     (Later in your response, you surprised me by stating
      you recommend ITR and ETR functions be in CE routers.
      I had assumed you meant ISP border routers.)

2 - If hosts are expecting some longer MTU, 4470 or whatever, and
    the LISP ITR adds a header which makes the packet exceed a
    4470 limit en-route to the ETR, without any additional
    mechanisms (your preferred solution), then how would the sending
    host know the packet had been dropped?  The PTB will go to the
    ITR (in Ivip, it goes to the sending host, but contains
    information different to the packet sent by the host) - and it
    is not practical to have the ITR figuring out which sending host
    to generate a corresponding PTB for.  See:

      http://www.ietf.org/mail-archive/web/ram/current/msg01766.html

3 - How stable would the PMTU be between any two routers?  Just
    because right now the packets go through gigabit Ethernet links
    which support some longish MTU, how do we know that when there
    is some outage en-route, that BGP won't cause the packets to go
    via some other links, one or more of which has a 1500 MTU?

      (I recall discussion of how 100Mbps Ethernet is still widely
       used at peering exchanges, and that you can't mix gigabit and
       100Mbps Ethernet without having them all run with the 1500
       byte MTU which is typical of 100Mbps Ethernet.)

4 - How could anyone adopt LISP with the "do nothing" approach
    to PMTU and fragmentation, when even a handful of links in the
    DFZ are not capable of handling jumboframes?  They couldn't
    ensure their packets are not sent on those links.

5 - How reluctant will networks be to adopt LISP if they can only
    put their ITRs and ETRs in locations which will always have
    jumboframe-compatible links to their border routers?

In version 06 of your ID, you wrote:

   Based on informal surveys of large ISP traffic patterns, it
   appears that most transit paths can accommodate a path MTU of
   at least 4470 bytes.  The exceptions, in terms of data rate,
   number of hosts affected, or any other metric are expected to be
   vanishingly small.

By "transit link" I assume you mean from any one border router of
any one ISP (or AS end-user network) to border routers of all other
such networks.

Therefore, I assumed you were intending to have the ITR and ETR
functions at these border routers.  But in your response, you wrote
something very different: CE routers.

>>             be capable of some much higher than 1500 byte PMTU.
> 
> That is correct.

OK.

> Right, or you do fragmentation. We did have an out.

This isn't clear, nor do I recall anything like this in the ID.

What does the fragmentation?

The ITR could fragment 4.5k packets into four or so ~1.4k packets,
but this involves four packets, which is really pushing your luck in
terms of efficiency and robustness.  I would contemplate doing that
for a few initial packets while the ITR was probing PMTU to the ETR,
which should take a few seconds - but LISP has no such robust PMTUD
mechanism, so you would have to keep fragmenting all such traffic
packets into four.  I don't think anyone would regard that as
sustainable.

Should ITRs which are configured in some way that they "know" their
one or more upstream links is not jumboframe compatible drop the
>~1500 byte packets and send a PTB?   That's fine, but then you have
to have a configuration item in the ITR, and it could get complex if
the ITR has one jumboframe compatible upstream link you want to use
fully, and one which is not.

There is no PMTU Discovery process in your proposal, as far as I recall.

So in the "do nothing" option, you have most ITRs knowing they can
send jumboframes, and a few as noted above knowing they can't.

Now what does a network do regarding placement of ETRs?  You can't
place an ETR on a non-jumboframe link, because the majority of LISP
ITRs don't have a way of knowing that ETR can't receive jumboframes.

What would happen is that the ITR would send a jumboframe and get
back a PTB.  Then it would have to either:

1 - Forget about it.  Unacceptable packet loss.

2 - Keep a copy of the packet and fragment it into some number of
    pieces, sending them to the ETR, and hopefully having them all
    arrive, since you don't have a way of checking for the
    successful arrival of all packets.   Inefficient with
    significantly higher packet loss rates.  Also, the ITR has
    to keep a lot of state, including whole traffic packets, and
    hope that it does get a PTB message, meaning the PTB packet
    is firstly generated, secondly not filtered out and thirdly not
    dropped.

3 - Somehow recognise which of the recently tunnelled packets the
    PTB is for, look up the sending host and, with a copy of
    the first part of the original packet, send a PTB to the
    sending host so it tries again with a smaller packet.  But
    see my above-linked RAM list message from July last year
    about why this is extremely onerous to do in a secure way.

> be placed on CE (CPE) routers. We think that is the best balance of

What other reasons would LISP be implemented for other than solving
the routing scaling problem?

I don't recall this suggestion for ITRs and ETRs being at Customer
Edge routers being mentioned in a LISP draft, but I haven't
necessarily kept up with every change.

My understanding of CE routers is shown by CEx in the figure
below, connecting to Provider Edge routers.

   ----------------        -----------------------    -->  Other
  |  End-user net- |      |       ISP-A           |  /     border
  |  work 1        |      |                       | /      and
  |                |     PE1                     BR1---->  transit
  |                |      |                       | \      routers
  |               CE1--- PE2                      |  \     in the
  |                |      |                       |  |     DFZ.
  |                |       -----------------------   |
  |                |                                 |
  |                |                                 |
  |                |       -----------------------   |
  |                |      |       ISP-B           |  /
  |               CE2----PE3                      | /
  |                |      |                      BR2----> As above.
  |                |      |                       |
  |                |     PE4                      |
   ----------------       |                      BR3---->
                          |                       | \
                           -----------------------   \
                                                      -->

Alternatively, there could be just a single Customer Edge router CE1
in this example, with its second link going to PE3.

I think this means that all ISPs which adopt LISP would need to be
entirely jumboframe compatible, assuming your preferred "do nothing"
scenario is the case.

If my understanding is correct, locating the ITR and ETR functions
in the CE routers also requires all links to the PE routers be
jumboframe compatible.

This may make sense at the "top-end of town", but I think the future
map-encap system needs to be extremely broadly applicable, to
end-users of all types and sizes, including especially businesses
with a DSL link and a cable modem link, for instance.  These common
types of data link will not be jumboframe compatible in the
foreseeable future.

> No, not bottlenecks, easier deployability.

My comment was based on the notion that the ITR and ETR functions
would be located a the Border Routers of ISPs, because I figured
that was the most jumboframe compatible location in many ISPs.

If you can place the ITRs and ETRs within or near the CE or PE
routers, then this greatly reduces bottlenecks and makes the system
more deployable.  But then you need to extend gigabit Ethernet to
all those places.

> overloaded. You capacity design a router to deal with the input rate and
> density of the box and the amount of work you have to do. ITRs won't
> attract more traffic when they are at the CE. They get traffic based on
> who wants to send data to external destinations.

Yes, there is no more traffic.  However, the ITR functionality in
LISP is complex and so the router has more work to do than before.
Likewise ETR functionality is an additional burden.

I am planning Ivip so that the whole thing could be deployed without
changing a single router.  All ITR and ETR functions could be done
in ordinary servers - and likewise the fast-push system.  In many
cases it would be better if the ITR and ETR functions were
integrated into routers, but it would not be necessary to deploy the
system successfully.

LISP ITR and ETR functionality is a lot more complex than with Ivip,
and if you are assuming that the LISP ITR and ETR work will always
be done by upgraded routers, I think this would be a major barrier
to many networks adopting your system.

> It means that if the packet from the source is >= 1464, the packet will
> be fragmented.

This is what I understood from later in your text, but this does not
match the sentence:

since an ITR could receive packets from inside the site of any
length which fits within the MTU of the link it arrives on.  If you
added something like "and which would be sent without
fragmentation." Then this would make more sense to me.

> The total packet size after the ITR is finished with it is L.

Yes, but your text refers to "S" as the length of the packet as it
is received, before it is encapsulated.  I think your text could be
modified to "greater than S" to make it more in accordance with your
intended meaning.

Working backwards:

> the ITR be configured with an L value of 4470 or 9182 perhaps.

This wasn't mentioned in the ID - only "1500 bytes".

How does anything advertise MTUs, or larger MTUs?  There is nothing
in LISP headers or map reply messages concerning MTUs.  BGP doesn't
carry MTU information.  LISP has no reliable Path MTU Discovery
mechanism.  The closest you get is the ITR receiving a PTB message
if the encapsulated packet is too big for some link en-route to the
ETR.  But that is not robust.  Fred Templin and I believe that the
only way to do robust PMTUD is with explicit probe packets from the
ITR to the ETR, with nonce-protected acknowledgements of proper
reception.

I am confused about the "same ISP" above.  Perhaps you mean the
destination end-user network has its ETR connected to an ISP which
provides it with 4470 or 9182 byte MTUs to the DFZ.

How could the ITR in the first site know this?  How could it build
up a set of information about PMTUs to various ETRs, so it could
reject packets which were too long (after encapsulation) for each
PMTU, when the sending host sends that packet, depending on which
link the encapsulated packet was to be sent out on, without any
robust mechanism for PMTU discovery?

> header, UDP header, and  LISP header are prepended, the size of the

Yes, that is what I meant.  Your text would make more sense to me if
it was something like:

  the ITR will drop the packet when the size is greater than S ...

---

From: Dino Farinacci <[email-masked]>
Date: Sun, 9 Mar 2008 18:29:22 -0700

> "slacker's guide" to MTU handling. First, it only does

Call it what you wish, but we are trying to be practical and simple at  
the same time. You know we really want to deploy this and not just  
create paper with all kinds of protocol machinery just for the sake of  
solving problems. Some problems don't need to be solved. Some problems  
are not real problems.

> original host, and there really aren't very many of those

So if you say DF=0 is non existent, then we say that small MTU links  
are non-existent as well.

> but only got 1464") plus it *always* results in packet loss

Well, sorry it's the real effective MTU, 1464 that is. And why is this  
surprising?

> the core may soon transition to all-GigE.

The MTU discovery will be contained inside of the site. That is a lot  
better than all the way to the destination host.

> SEAL will detect and correct it.

Too much machinery, it is not needed and we want to be stateless. Do  
you not agree that our proposal is even simpler than SEAL?

Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Mar 2008 06:58:39 -0700

Hi Dino, 

>Sent: Sunday, March 09, 2008 6:29 PM
>To: Templin, Fred L
>Cc: Robin Whittle; Routing Research Group
>Subject: Re: [RRG] Re: LISP PMTU & fragmentation problems
>

>are not real problems.

That's not at all what SEAL is about, if that is what you
mean. SEAL actively fixes real-world problems when there
are *definitely* small MTU links, and is available as hot
standby when there *may* be small MTU links. Think of it
as an uninterruptible power supply; it is ordinarily
dormant, but kicks in when needed and keeps the system
up and running until the small MTU links are identified
and replaced.

>are non-existent as well.

I didn't say DF=0 was non-existent; only that it is rare.
Two words that don't fly in this context are "always" and
"never".

>surprising?

With SEAL, the effective MTU is 1500 and the hosts never have
to deal with PTBs. No loss due to MTU restrictions; no wasted
resources for delivering PTBs, plus the 1500 MSS negotiated
by the TCPs never has to be revised.

The other aspect of sending the PTB with MTU=1464 is that,
if the source was another tunnel endpoint (e.g., a security
gateway) there is no guarantee that it will be able to
translate the PTB for transmission back to the original
source. RFC2923 also identifies other uncertainties about
classical path MTU discovery.

>better than all the way to the destination host.

So, you seem to be arguing against large MTUs in the interdomain
context? RFC4821 supports that scenario, but won't work if your
routers won't let it.

IMHO, we want to evolve to a "GigE-jumboframe-everywhere"
Internet, and setting a fixed and too-small MTU at the
routers will not allow that to happen.

>you not agree that our proposal is even simpler than SEAL?

In the interdomain, I believe it will be not unreasonable
to find 1500 MTU links but completely unreasonable to find
anything appreciably smaller. As such, SEAL will never split
a packet into more than 2 pieces the same as what the LISP
text says. The only difference is that SEAL will split DF=1
packets the same as for DF=0 packets. The final destination
will reassemble the latter, while the ETR will reassemble
the former (up to a modest limit of 2KB). But, the ETR
already has to support reassembly for IPv4 and IPv6 anyway.

AFAICT, the choice is to tell the original source a too-small
MTU once and for always (which may not always work), or to
allow a little bit of robust segmentation and reassembly
between the ITR and ETR. All things in moderation...

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Mar 2008 07:02:07 -0700

Robin,

>Fred Templin and I believe that the

Slight clarification; SEAL supports both explicit *and*
implicit probes. In SEAL, the data packets themselves
serve as implicit probes without exposing them to loss
under normal circumstances. (Plus, the SEAL nonce is
based on the 32-bit Identification field.)

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Mar 2008 07:35:27 -0700

Dino, 

Just one more word about practical and simple, I have already
built and tested an alpha version of SEAL in Linux that does
the ID extension (code was submitted on the MANET wg list).
The next version will pick up the MTU handling.

Thanks - Fred

---

From: Scott Brim <[email-masked]>
Date: Mon, 10 Mar 2008 10:12:32 -0400

Do you believe every encaps technique should carry fragmentation info in 
its own headers, or should there be a more general approach?  If the 
latter, where in the IETF/IRTF should MTU problems be solved?

---

From: Scott Brim <[email-masked]>
Date: Sun, 09 Mar 2008 21:18:04 -0400

Why is this a RRG subject anyway?  If LISP (or anything else in RRG) has 
an MTU problem, then all encapsulations do, and you want a general approach.

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 11 Mar 2008 06:14:49 -0700

>its own headers, or should there be a more general approach?

I believe the general approach does entail a little bit of
fragmentation info - 4 extra bytes, to be specific.

>If the latter, where in the IETF/IRTF should MTU problems be solved?

Looks like it is going to be discussed in the INTAREA.

Thanks - Fred

---

From: Randall Atkinson <[email-masked]>
Date: Tue, 11 Mar 2008 06:41:28 -0700

% Why is this a RRG subject anyway?  If LISP (or anything else in RRG)
% has an MTU problem, then all encapsulations do, and you want
% a general approach.

As I understand it, not all RRG proposals involve encapsulation.

I agree that most (all ?) forms of encapsulation have issues
when a full-MTU packet arrives and needs to be encapsulated.

Those proposals that do involve encapsulation might want to
at least discuss the issues related with the use of encapsulation
(example: MTU discovery, potential issue of fragmentation) and
explain why those are managable or acceptable in the context
of that particular proposal.

Cheers,

Ran
[email-masked]