Subject: transmech MTU comments

From: Fred Templin <[email-masked]>
Date: Wed, 12 Nov 2003 14:43:28 -0800 (PST)

Erik,
 
My understanding of RFC 3168 is that when the tunnel interface
forwards an IPv6 packet to an IPv4 interface with an ECT(0) or ECT(1)
codepoint in the traffic class field, it MAY set the codepoint to CE if
congestion is experienced. I interpret the MAY to mean that the other
option is to drop the packet. If the packet is to be dropped, should it
be dropped silently? Also, could a link restriction be considered as
congestion? If so, does drop silent also entail NOT sending an
ICMPv6 "packet too big" message back to the source?
 
Thanks - Fred
[email-masked]

---

From: Fred Templin <[email-masked]>
Date: Wed, 12 Nov 2003 14:52:57 -0800 (PST)

Along with this, what if we were to set the tunnel MTU to a
value that is slightly larger than the size of the underlying
IPv4 interface (minus 20bytes for IPv4 header)? In the dynamic
case, the test could be:
 
  - packet has ECT(0), ECT(1) codepoint - forward if smaller than
    link MTU, setting to CE if necessary; dropping if too large
 
  - packet does not have ECT(0), ECT(1) - check the IPv4 path
    MTU, forward if not larger, else send ICMPv6 "packet too big"
 
Can something like this go as a text change suggestion for MECH?
 
Thanks - Fred
[email-masked]

Fred Templin <[email-masked]> wrote:
Erik,

Thanks - Fred
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Thu, 13 Nov 2003 07:15:14 +0200 (EET)

> ICMPv6 "packet too big" message back to the source?

Where in the spec do you read that the MAY?   What I basically read is 
9.1.1, first paragraph, a couple of the last sentences.  But that applies 
to the decapsulating router only.

RFC3168 doesn't seem to define how to drop the packet, by the way, so I 
assume by "dropping" they mean a silent discard.

I'm not sure whether it makes sense to delve into ECN details in the 
spec..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Fred Templin <[email-masked]>
Date: Thu, 13 Nov 2003 01:33:03 -0800 (PST)

Pekka,
 
> 9.1.1, first paragraph, a couple of the last sentences. 
 
Check section 5, second to last paragraph.
 
> But that applies to the decapsulating router only.
 
I don't think so. The processing entity inside the encapsulator
accepts a packet from the overlying IPv6 interface and enqueues
it to an underlying IPv6-in-IPv4 interface, so I would call that entity
a "router"

> assume by "dropping" they mean a silent discard.
 
OK.

> I'm not sure whether it makes sense to delve into ECN details in the 
 
Well, it seems it should be documented somewhere.
What do you suggest?
 
Fred
[email-masked]

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 13 Nov 2003 17:32:22 +0100 (CET)

> ICMPv6 "packet too big" message back to the source?

My understanding is that larger than MTU is not congestion, that
packets would not be dropped as an alternative to setting CE, and
that congestion for tunnels isn't any different than congestion in general.

What tunnel adds is the question of what the encasulator puts in the ECT
bits in the outer header (where copying them from the inner header
makes sense in many cases) and whether the decapulator does anything with
the received ECT bits in the outer header (and I think that 3168 says
that unless there are security concerns, copying them from the outer
to the inner header makes sense).

Thus I think the text in the mech document about ECN are sufficient.

  Erik

---

From: Fred Templin <[email-masked]>
Date: Thu, 13 Nov 2003 17:50:51 -0800 (PST)

Erik,

> to the inner header makes sense).

My only concern in this case is what happens at the decapsulator
if an ECT/CE codepoint is set in the inner header but the outer
header contains Not-ECT. Which one do we trust, in that case,
and should the behavior be documented?
 
> Thus I think the text in the mech document about ECN are sufficient.

Agreed on other fronts except the above.

Fred
[email-masked]

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 14 Nov 2003 03:12:48 +0100 (CET)

> and should the behavior be documented?

Section 9.1 in RFC 3168 already documents this.

  Erik

---

From: Fred Templin <[email-masked]>
Date: Fri, 14 Nov 2003 10:29:21 -0800

>Section 9.1 in RFC 3168 already documents this.

Then, can we have a more-specific reference in the document to provide
better guidance to implementors, e.g., ([RFC3168], section 9.1)?

Thanks - Fred
[email-masked]

---

From: Erik Nordmark <[email-masked]>
Date: Mon, 17 Nov 2003 15:44:52 +0100 (CET)

> better guidance to implementors, e.g., ([RFC3168], section 9.1)?

Will do.

  Erik

---

From: Fred Templin <[email-masked]>
Date: Wed, 19 Nov 2003 17:33:05 -0800

Erik,

I would like to discuss some aspects of the current text in [MECH], 
section 3:

>3.2.1.  Static Tunnel MTU
>
>   A node using a static tunnel MTU MUST limit the size of the IPv6
>   packets it tunnels to 1280 bytes i.e., treat the tunnel interface as
>   having a fixed interface MTU of 1280 bytes.  An implementation MAY
>   have a configuration knob which can be used to set a larger value of
>   the tunnel MTU than 1280 bytes, but if so the default MUST be 1280
>   bytes.  A larger fixed MTU should not be configured unless it has
>   been administratively ensured that the decapsulator can reassemble
>   packets of that size.  Care should be taken when manually configuring
>   large tunnel MTUs to only do so when the MTU of the IPv4 path to the
>   tunnel endpoint is large to avoid causing excessive fragmentation.
>
>   When using the static tunnel MTU the Don't Fragment bit MUST NOT be
>   set in the encapsulating IPv4 header.  As a result the encapsulator
>   should not receive any ICMPv4 "packet too big" message as a result of
>   the packets it has encapsulated.
>

The latter paragraph implies that either all links on the path will
be at least as large as the static MTU, or that nodes with constricting
links will use IPv4 fragmentation to split the packet into pieces small
enough to traverse the constricting link. The former case will not be
true in general, because certain bandwidth constrained links will choose
smaller-than-1280-byte MTUs for their IPv4 interfaces if BCP 48, 50,
and 71 recommendations are followed. Also, we cannot be assured that
all forwarding nodes will correctly implement IPv4 fragmentation.
So, we have a very real possibility for black holes here.

>3.2.2.  Dynamic Tunnel MTU
>
>   The dynamic MTU determination is OPTIONAL.  However, if it is
>   implemented, it SHOULD have the behavior described in this document.
>
>   The fragmentation inside the tunnel can be reduced to a minimum by
>   having the encapsulator track the IPv4 Path MTU across the tunnel,
>   using the IPv4 Path MTU Discovery Protocol [RFC1191] and recording
>   the resulting path MTU.  The IPv6 layer in the encapsulator can then
>   view a tunnel as a link layer with an MTU equal to the IPv4 path MTU,
>   minus the size of the encapsulating IPv4 header.
>
>   Note that this does not eliminate IPv4 fragmentation in the case when
>   the IPv4 path MTU would result in an IPv6 MTU less than 1280 bytes.
>   (Any link layer used by IPv6 has to have an MTU of at least 1280
>   bytes [RFC2460].)  In this case the IPv6 layer has to "see" a link
>   layer with an MTU of 1280 bytes and the encapsulator has to use IPv4
>   fragmentation in order to forward the 1280 byte IPv6 packets.
>

But, shouldn't the encapsulator send a "packet too big" to the source
in this case even if the MTU it reports is less than 1280 bytes? In 
response,
the source should then include a fragment header in the packets it sends
as a signal to the encapsulator that IPv4 fragmentation is permissible
(see RFC 2460, section 5). More discussion on this below:

>   The encapsulator SHOULD employ the following algorithm to determine
>   when to forward an IPv6 packet that is larger than the tunnel's path
>   MTU using IPv4 fragmentation, and when to return an IPv6 ICMP "packet
>   too big" message per [RFC1981]:
>
>           if (IPv4 path MTU - 20) is less than 1280
>                   if packet is larger than 1280 bytes
>                           Send IPv6 ICMP "packet too big" with MTU = 1280.
>                           Drop packet.
>                   else
>                           Encapsulate but do not set the Don't Fragment
>                           flag in the IPv4 header.  The resulting IPv4
>                           packet might be fragmented by the IPv4 layer on
>                           the encapsulator or by some router along
>                           the IPv4 path.
>                   endif
>

I believe the above "else" case should be re-worded as follows:

        else
                if packet does not contain a fragment header
                        Send IPv6 ICMP "packet too big" with MTU
                        = (IPv4 path MTU - 20). Drop packet.
                else
                        Encapsulate and fragment the packet using IPv4
                        fragmentation with a maximum fragment size
                        of (IPv4 path MTU - 20). The lower 16 bits of
                        the Identification field in the fragment header
                        is used as the Identification field for each IPv4
                        fragment header, and the Don't Fragment field
                        is not set.
                endif
        endif

First, about sending the "packet too big" with an MTU size less
than 1280, this seems to me to be consistent with the expectation
specified in RFC 2460, section 5. This is what an "IPv6-to-IPv4
translator" is supposed to do, and from the perspective of the
original IPv6 host it makes no difference whether the node
that sends the packet too big is a translator or an IPv6-in-IPv4
tunnel endpoint.

As to fragmenting the packet in the enapsulator instead of
just sending it with the DF bit not set, the encapsulator has no
way of knowing whether there are forwarding nodes in the IPv4
path with broken, non-existent, or slow-path IPv4 fragmentation
implementations and so the only safe option is for the tunnel
encapsulator itself to do the fragmentation.

As to the setting of the fragment ID field, my suggested text
above reflects my best understanding of the normative ref's, but
I believe we have the following problem. What if the original IPv6
source wanted to do host-based IPv6 fragmentation (e.g., for large
UDP packets) even though the IPv6 path MTU was less than 1280
bytes?

The source would send a series of N IPv6 fragments, each of which
would have the same value in  the fragment ID field. But then, the
tunnel encapsulator would use IPv4 fragmentation to split each of the
N IPv6 fragments into M IPv4 fragments again using the *same*
fragment ID value! We would then have a collision in the decapsulator's
IPv4 reassembly buffer, since there would be no way of knowing to
which one of the N IPv6 fragments a particular IPv4 fragment belonged!

So, either my understanding of the normative references is wrong,
or the normative references themselves are wrong. Can you help?

Fred
[email-masked]

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 20 Nov 2003 06:31:44 -0800 (PST)

> and 71 recommendations are followed. 

Agreed that there might be IPv4 fragmentation in this case.

> Also, we cannot be assured that

Huh? If we can't assume that IPv4 fragmentation works we might as well
assume that IP routing doesn't work and we should be using the postal
service to have this discussion instead of email.

> (see RFC 2460, section 5). More discussion on this below:

IPv4 fragmentation is always permissible.
The text you are referring to in RFC 2460 is to enable SIIT (and NAT-PT?)
to operate over IPv4 paths with a path MTU less than 1280 bytes without
having to make up IPv4 fragment IDs on the fly.
RFC 2460 says "undergoes translation" as the motivation.
Thus this has nothing to do with encapsulation.

  Erik

---

From: Fred Templin <[email-masked]>
Date: Thu, 20 Nov 2003 08:05:36 -0800 (PST)

Erik,
 
I appreciate the humor, but this is a critical design point that needs
to be resolved:
 
Should we have as a goal the ability to support L2 bridges, switches,
hubs, etc. that join media with dissimilar MTUs, but do not support
IPv4 fragmentation and do not send "packet too big" ICMPs?
 
We answered "NO" to this 15 years ago when we allowed the
approach now specified in RFC 1191 to go forward, and that
decision took a viable design alternative off the table which had
a profound effect on the shaping of the industry.
 
We're not going to fix this overnight by anything we do here, but
allowing for generality (i.e., not expecting any IPv4 fragmentation
support along the forwarding path) would be an important step
toward restoring a level playing field. Let's not blow it again,
given new opportunities to get things right.
 
Fred Templin
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Thu, 20 Nov 2003 19:06:58 +0200 (EET)

Hi,

> given new opportunities to get things right.

Fred, I appreciate your concern about creating a robust and optimal
Path MTU Discovery mechanism, but IMHO this document -- which we're
moving towards DS -- just simply doesn't seem to be the right time and
place to do so.

There is little we can do to fix IPv4 at this point; IPv6 already
doesn't include fragmentation support along the path, and might be
enhanceable, in future, to support new methods.  We'll need to live
with what we have.

---

From: Fred Templin <[email-masked]>
Date: Thu, 20 Nov 2003 10:37:47 -0800

Your message was a cop-out, Pekka. Please address the question:

  Do we, or do we not want to allow for the possibility of
  future L2 bridges, switches, hubs etc. that connect media
  with dissimilar MTUs but that don't get involved with
  L3 functions like IPv4 fragmentation and sending
  "packet too big" ICMPs?

Let's look at a real-world example of where our failure to
support such L2 devices is currently hurting us. Look at the
interface MTU for your 802.11 card under your favorite host
OS (e.g., Microsoft, Linux, etc.) and you will see that it says
1500 Bytes. Next, go to the IEEE specs and you will see that
802.11 frames can support a body of 2312 bytes.

So, why can't we set the MTU of our 802.11 interfaces to
2312 and take immediate advantage of the 35% gain in
efficiency? Because there might be an 802.11/Ethernet
L2 bridge somewhere on the path and IPv4 path MTU
discovery would break!

The current IPv4 path MTU discovery scheme is limiting
potential growth and precluding legitimate solution alternatives.
While we can't fix it all in one fell swoop, we can start making
informed decisions (e.g., non-assumption of IPv4 fragmentation,
support for packetization-layer path MTU discovery that does
not require ICMPs, etc.) that will eventually bring about healing
and new growth opportunities.

Fred
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Thu, 20 Nov 2003 20:39:06 +0200 (EET)

>   Your message was a cop-out, Pekka. Please address the question:

Ok..

>   L3 functions like IPv4 fragmentation and sending

No, I don't believe this is an immediate goal.
 
> and new growth opportunities.

I agree that this may be an interesting direction to explore, but 
there does not yet seem to be a general agreement about whether the 
IETF will be moving to this direction or not.  (Not sure if people 
even agree about a problem statement?)

When such mechanisms have not been even fully? specified(?) for IPv4,
it would seem to be premature to do it for IPv6-over-IPv4, especially
considering the stage where we're at now.

(I don't have full knowledge of the situation where this stands at the 
moment, but this is my impression..)

This seems to be close to a pre-engineering/research question at the
moment;  not something I think is suitable for inserting into a
specification moving towards Draft Standard.

---

From: Fred Templin <[email-masked]>
Date: Thu, 20 Nov 2003 10:57:26 -0800

>No, I don't believe this is an immediate goal.

"Immediate goal" was not the question; "allow for the
future possibility of" was the question. I would have a
hard time believing any one of us could honestly
answer "no" to the latter.

Fred
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Thu, 20 Nov 2003 21:02:18 +0200 (EET)

> answer "no" to the latter.

Ok, trying to clarify what I said.

No, I don't believe it's an immediate goal to provide future
possibility of [...].

That doesn't mean we may want to start building that future
extensibility in a couple of years (for example), but providing that
future extensibility does not seem a requirement *at the moment*.

There's just no way to provide future extensibility possibility for
every possible future, while staying robust and simple for the
mechanisms and methods that exist today.  Future extensibility is a
tradeoff just like any other..

---

From: bill <[email-masked]>
Date: Thu, 20 Nov 2003 11:19:45 -0800 (PST)

> No, I don't believe this is an immediate goal.

	Hum... lost knowledge.  anyone here still remember the
	MTU mismatches btwn 802.3, 802.5 and FDDI?  there were
	any number of L2 bridges that did the negotiation to
	LCD-MTU w/o mucking w/ L3 functions... (to the extent
	possible)

> specification moving towards Draft Standard.

	see above.  why is a problem solved more than a decade ago
	now a research topic?

> Pekka Savola                 "You each name yourselves king, yet the

--bill

---

From: Pekka Savola <[email-masked]>
Date: Thu, 20 Nov 2003 21:26:22 +0200 (EET)

> 	LCD-MTU w/o mucking w/ L3 functions... (to the extent

Sure, but ...
 
> 	see above.  why is a problem solved more than a decade ago

As far as I understand, the proposition was to solve the problem at
the IP layer, not L2 what was the case (AFAIK) a decade ago.

I have nothing against L2 solutions as long as they're transparent to
IP, and that's certainly not a research topic :-).

---

From: bill <[email-masked]>
Date: Thu, 20 Nov 2003 11:57:56 -0800 (PST)

> IP, and that's certainly not a research topic :-).

	read Freds note carefully.  "...might be an 802.11/Ethernet L2 bridge..."
	which clearly calls out a L2 device.  The kicker is that pesky
	802.3 spec that calls for a 1500 mtu. darn that pesky ethernet
	replacement spec that everyone trys to use as a default for transit
	infrastructure... 

--bill