Subject: [RRG] Critique of Sprite-MTU: PMTUD, fragmentation etc.

From: Robin Whittle <[email-masked]>
Date: Tue, 30 Oct 2007 20:08:01 +1100

I have a revised critique of Fred Templin's proposal:

  http://tools.ietf.org/html/draft-templin-inetmtu-05

at my IPTM (ITR Probes Tunnel MTU) page:

  http://www.firstpr.com.au/ip/ivip/pmtud-frag/#sprite_critique

The three elements of the critique apply to the use of Sprite-MTU in
an ITR-ETR system (at least for IPv4).  The third element may be
applicable to the many other uses which Sprite-MTU is intended to
help with.

In brief (please read the full critique before responding), they are:

  1 - The ITR has to set up state even when a single short
      (too short to be a problem for MTU limits) packet is
      tunneled to an ETR.

  2 - The ITR and ETR have to continually engage in Sprite-MTU
      communications as long as packets are tunneled to the ETR -
      and this should arguably start with the first packet.

Both these place a huge load on the ITR, for just a single packet -
which doesn't even need any MTU management.  My proposal avoids such
overheads, by being triggered only by longer packets.

  3 - The initial response of the ITR to a longer packet is to
      send back a Packet To Big message to the sending host and
      to tunnel the packet.  The sending host will then resend
      the packet (and probably subsequent packets) in multiple
      small fragments - but the original packet may arrive at
      the destination host and be acknowledged.  This contradictory
      response is likely to cause trouble for sending hosts,
      including those implementing RFC 4821, as well as being a
      source of extra packets, confusion etc.

This tunneling, MTU, fragmentation, PMTUD stuff is really challenging.

So far, Fred's proposal and my own IPTM (ITR Probes Tunnel MTU) are
the only two which are applicable to the ITR-ETR schemes we are
discussing on this list.

I would really appreciate some critical assessment of IPTM, and
likewise some confirmation or otherwise of my critique of Sprite-MTU.

  - Robin

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 30 Oct 2007 08:44:32 -0700

Robin,

I will address the points from the critique on your
webpage below: 

> 1. The ITR needs to create state for every "tunnel" - and
> a "tunnel" is created by a single packet being sent to a
> single ETR.  This packet could be very short, as might all
> subsequent packets - such as for VoIP, DNS or other protocols,
> and so would never actually need any intervention on the part
> of the ITR regarding PMTU, PMTUD or fragmentation.  Creation
> of this state - and the consequent need to manage it and
> eventually delete it - seems to be an onerous and unnecessary
> burden on ITRs for just one traffic packet.  Creation of this
> state, and management of it, would probably involve (in a
> traditional router) considerable effort by the router's
> central CPU, rather than straightforward effort by the FIB
> processors/hardware.

There may be wording in the draft that led you to conclude
that there are absolutes, and if so I would be happy to fix
it in the next draft version. But, there is nothing wrong
with an implementation that initially operates in "legacy mode"
with minimal soft state and then shifts over to sprite-mtu mode
when it becomes evident that a tunnel is seeing non-negligible
use. Moreover, soft state is required for any approach that
would aspire to larger MTUs than the minimums cited for the
legacy mode of operation. Finally, the degree of effort for
managing soft state depends somewhat on the location of the
ITR; ITRs that are located closer to the edge of the network
might have less trouble managing the soft state than those
located nearer to the core.  
  
> 2. Continuation of mandatory sprite communications for probing
> PMTU seems to be an onerous and unnecessary burden on ITRs
> which handle only one packet, or only a few, or only packets
> which are so short as to not require any special management of
> PMTU, fragmentation etc.  Sprite communications are supposed to
> continue as long as the traffic is flowing in the tunnel, and
> one interpretation of that is that the sprite communications
> should begin with the first traffic packet.  So a single packet
> would trigger the generation of state in the ITR and the ETR,
> with multiple packets flowing back and forth, until the ITR
> decided that there was no need to continue - some time after
> the one and only traffic packet arrived.

The draft doesn't currently say this, but there is no need
for the continuous sprite exchanges once there is evidence
that the path MTU is larger than the minimums (1280+ENCAPS
for IPv6/*/IP and 576 for IPv4/*/IP), since the purpose for
the continuous sprite exchanges is to mitigate fragmentation
related misassociations at high data rates. This can be
addressed in the next draft version.  

> 3. Under certain common circumstances, in particular the
> first traffic packet which is longer than a certain length,
> the ITR would send back an ICMP Packet Too Big (PTB) message
> to the sending host, even though it sends the resultant
> encapsulated packet.  This would mean (for IPv4) that the
> first attempt by the sending host to send a packet longer
> than about (576 minus encapsulation overhead) bytes would 
> result in the sending host reducing its estimation of the
> Path MTU to the destination host - to an unrealistically
> low value.  Meanwhile, the ITR is required to send the packet,
> with DF=1, to the ETR, where it may well arrive and be
> forwarded to the destination host.  So the sending host has
> contradictory signals from the outside world.  It is likely
> to get the PTB message first, and so send the same packet as
> multiple fragments.  Meanwhile, the receiving host responds 
> to the first packet and sends a response (actually, not all
> packets are part of protocols which would generate a response),
> which may arrive before, during or after the sending host
> emits the fragmented form of its initial packet.  There is
> great potential for confusion at the sending host, especially
> with RFC4821

It seems reasonable to expect that the initial packets over
a tunnel (e.g., TCP SYN, TCP ACK, etc) will be small enough
that the TNE would not need to generate a PTB back to the
original source. (Isolated large packets would typically be
sent with DF=0 for IPv4, or as 1280byte or smaller fragments
for IPv6.) Moreover, RFC4821 implementations will take care
to not expose packets of an unprobed size to loss due to an
MTU restriction such that for those implementations the most
likely packets to trigger a PTB from the TFE are the probe
packets themselves.

There is evidence in RFC4821 that the sending host would
not be confused by the contradictory information of both
a probe confirmation and a PTB response for the same probe
packet. In Section  2:

  "The isolated loss of a probe packet (with or without an
   ICMP Packet Too Big message) is treated as an indication
   of an MTU limit"

and:

  "Packetization Layer PMTUD (PLPMTUD) introduces some
   flexibility in the implementation of classical Path
   MTU Discovery.  It can be configured to perform just
   ICMP black hole recovery to increase the robustness of
   classical Path MTU Discovery, or at the other extreme,
   all ICMP processing can be disabled and PLPMTUD can
   completely replace classical Path MTU Discovery."

and:

  "In the limiting case, all ICMP PTB messages might be
   unconditionally ignored, and PLPMTUD can be used as
   the sole method to discover the Path MTU."

Also, in Section 7.6.2:

  "If an ICMP PTB message is received matching the probe
   packet, then search_high and eff_pmtu MAY be set from
   the MTU value indicated in the message.  Note that the
   ICMP message may be received either before or after the
   protocol loss indication."

Fred
[email-masked] 

> -----Original Message-----
> From: Robin Whittle [mailto:[email-masked]] 
> Sent: Tuesday, October 30, 2007 2:08 AM
> To: Routing Research Group list; Templin, Fred L
> Subject: Critique of Sprite-MTU: PMTUD, fragmentation etc.
>

---

From: Robin Whittle <[email-masked]>
Date: Wed, 31 Oct 2007 05:12:20 +1100

Hi Fred,

Thanks very much for your response.  I linked to it from my critique
page.  I will think about this some more and respond on the list in
the next day or two.  My 5AM thoughts are:

A future version of the Sprite-MTU draft is likely to resolve my
first two concerns.

I still think sending a Packet To Big message while sending the
packet onwards is likely to cause trouble - and that my lightweight
approach of fragmenting larger outer packets (for reassembly in the
ETR, before decapsulation), at first, even if DF=1 is set in the
original packet, has significant advantages, at least in an ITR-ETR
setting.  I think being lightweight, transparent and non-confusing
(such as not sending back PTB messages with spuriously low MTU
values), while getting the packet to the ETR even in the face of
lower than maximum MTU limits, are really important goals.

Also, my IPTM proposal with "outer source address = sending host's
address" is capable of supporting Traceroute, with a moderate
upgrade of the Traceroute client software.

I would appreciate some comment on my idea of setting some baseline
of PMTU for IPv4 which is much higher than 576, with the following
arrangement:

1 - I assume that (however defined) the "core" of the Net supports
    MTUs well in excess of 1000, and in in excess of some carefully
    chosen value well above 1000, and so probably more than double
    576.  I wrote "1280" but perhaps the value needs to be somewhat
    below this.

2 - Which "core" routers have a lower basic MTU value than 1500?
    Probably none - I guess.  If there are a few, lets put them on
    notice they should lift their game by the time an ITR-ETR scheme
    is deployed.

3 - State in the ITR-ETR scheme that all ITR and ETR functions
    should be located such that they have at least a "1280"
    or whatever value MTU to the "core" of the Net.

Can anyone think of regions of the Net where some value pretty close
to 1500 would not be a good choice?  How many levels of tunneling,
and of what kinds, are likely to be encountered?  Are there in
practice MPLS tunnels which limit MTU below 1500?

You can't necessarily make such assumptions for IPv4 in your
Sprite-MTU system, because you intend to use it in many settings
other than from an ITR to an ETR.  However, perhaps some agreement
could be reached about the final ITR-ETR scheme requiring ITRs and
ETRs to have a higher MTU (1280 or maybe >1400?) to the "core" of
the Net.  Then Sprite-MTU could have a special IPv4 threshold much
higher than 576 bytes when operating in an ITR-ETR setting.

  Cheers

    - Robin

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 30 Oct 2007 13:25:43 -0700

Robin,

The 576 comes from the minimum EMTU_R required of all IPv4
nodes. Unless the sprite-mtu TNE has some other a-priori 
knowledge of the TFE's capabilities, it can assume no more
than this.

That said, if the sprite-mtu TNE *does* have a-priori info
(e.g., that the TFE can accept non-initial */IPv4 fragments
and can reassemble up to 1500 bytes or so) it can use a
larger initial value in deciding whether/not to send a PTB.
Maybe it would help if the document said this explicitly.

Fred

> Sent: Tuesday, October 30, 2007 11:12 AM
> To: Routing Research Group list
> Cc: Templin, Fred L
> Subject: Re: Critique of Sprite-MTU: PMTUD, fragmentation etc.
> 
> Hi Fred,
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 20 Nov 2007 09:43:44 -0800

FYI, a new version of the sprite-mtu proposal is available:

http://www.ietf.org/internet-drafts/draft-templin-inetmtu-06.txt

This version addresses the list comments and also clarifies
the use of temporary private addresses for soft state
synchronization.

Fred

> From: Templin, Fred L 
> Sent: Tuesday, October 30, 2007 1:26 PM
> To: Robin Whittle; Routing Research Group list
> Subject: [RRG] RE: Critique of Sprite-MTU: PMTUD, fragmentation etc.
> 
> Robin,
>