Subject: [mobile-ip] RH and path MTU discovery

From: Erik Nordmark <[email-masked]>
Date: Thu, 31 Jan 2002 10:31:03 +0100 (CET)

Question for the list:

I was thinking about "RH works for MIPv6" - have implementors
tested the interaction of MIPv6/RH with Path MTU discovery
and figured out how to make sure it works?

The mipv6 draft doesn't say anything about this but it seems like
there are similar path MTU issues RH insertion by the IP stack
as there are with IPsec tunnel insertion by the IP stack
thus this is something that we presumably need to make sure is
well documented.

The issues are:
TCP when initiating a connection needs to know which MSS to use.
The MSS will be different if the IP layer, e.g. based on a binding cache
entry, inserts things in the packet. This is independent whether what is
added is a RH, a HOA, or an extra IP header.
It seems like some advise would be useful to point out this issue.

If a TCP connection starts out without using RH and later route optimization
has been setup (causing RH to be added by the IP layer) then a packet
that previously fit in the MTU will no longer fit. Thus the IP layer needs
to be able to notify the TCP layer that it is adding N bytes to the packets
so that TCP can adjust its understanding of the effective path MTU.

Finally, during the connection there could be changes in the routing path
to the MN causing routers to send back ICMP packet too big messages.
Such packets would need to be recognized by TCP as effecting the MTU
for the connection even though the destination field in the
"packet in error" in the ICMP packet is the CoA instead of the HoA.
An explicit example:
	TCP generates a packet with
		src = CN
		dst = HoA
	(and TCP already knows that IP will add 24 bytes due to the RH so
	it doesn't send packets that are too big)

	This causes IP to send
		src = CN
		dst = CoA
		routing header with seg-left = 1, address = HoA

	If the packet is too big somewhere along the path an ICMP
	error will be sent back with
		src = some router
		dst = CN
		ICMP packet too big
			mtu = 1300
		packet in error
			src = CN
			dst = CoA
			routing header with seg-left = 1, address = HoA

	The actual processing of this can be split between IP and TCP but
	the effect must be the same as TCP having received a too big
	packet reporting a smaller MTU and where the dst was the HoA i.e.
			src = some router
		dst = CN
		ICMP packet too big
			mtu = 1276  <=== NOTE
		packet in error
			src = CN
			dst = HoA   <=== NOTE

	One possible way to implement this is to have the IP layer convert
	the ICMP error before passing it to TCP. Another possible way
	would be to have TCP be able to adjust the reported MTU
	and extract the destination address from the RH.

Do implementations already solve this? Has it been widely tested?

RFC 2473 describes the solution to this in the more general case of
IP-in-IPv6 tunneling, thus that might be useful as background reading.

  Erik

---

From: Francis Dupont <[email-masked]>
Date: Thu, 31 Jan 2002 11:55:44 +0100

=> note this is obviously an IPv6 WG item (:-).

=> basically there are two kind of issues with RH and path MTU discovery,
one is about path MTU discovery real implementation (here) and
the second is about stuff insertion (after).
All path MTU discovery implementations I know (including mine)
don't cache the MTU per path as they should do but per (final)
destination. In the real world this doesn't harm. The second
feature is the MTU of the interface to the first intermediate
destination, in general this is checked very late but not too late,
i.e. if the guessed MTU is too large the error is detected when the
first packet has to be sent, fixed and a retry occurs (to optimize
(to be politically correct :-) the path MTU guessing routine has to
know RHs).

   The issues are:

=> this is the path MTU guessing routine, with RFC 2461 words the MTU
is cached in the destination cache and initialized if needed to the
MTU of the outgoing interface (in RFC 2461 the destination cache is
per interface but in general the interface is pointed by the neighbor
cache entry, itself pointed by the destination cache aka routing table).

=> this kind of stuff should be cached per PCB (i.e. per connection).
There are two ways:
 - the active way: when the BCE is created, all PCBs connected to the
   MN destination are notified they have to recompute the MTU/MSS
   (the routine already exists for ICMP packet too big arrival)
 - the reactive way: when the stuff is added the packet becomes too
   large, an error is returned and the sending routine recomputes
   the MTU/MSS and retries.
In fact both are used because of unconnected PCBs, i.e. the reactive
way is considered as the not optimized but always working way.

=> I agree but if an ICMP packet too big is received the TCP layer has
to adjust too.

=> exactly, this is the main source of MTU/MSS adjustments.

=> the packet in error has the RH too so the problem is only on the
code which peels headers in order to do the proper action (this is
easier than reception code because headers are dropped). Of course
this works only if the packet in error was not too truncated.
(PS: if headers are replaced by hard state this is still possible
but can become a bit hairy).

   An explicit example:
   
   	This causes IP to send
   		src = CN
   		dst = CoA

=> with this extended example you can see that header peeling is needed
because the next-header field with TCP is the RH one.

   	One possible way to implement this is to have the IP layer convert
   	the ICMP error before passing it to TCP.

=> usually the IP layer gives the essential informations to TCP,
not the packet and manage by yourself (:-). These informations
are addresses (HoA!), ports, the type of problem (too big).
In my implementation I don't give the MTU because the MTU/MSS
(re)guessing routine will find it at a know place.
There is an implementation choice: to take into account or not
the extra stuff in the cached MTU (I prefer "not" because the
extra stuff is not in general per destination when the MTU is).

        Another possible way
   
=> this should be in the IP layer because the code can be shared.
   
   Do implementations already solve this?

=> yes of course!

   Has it been widely tested?
   
=> each time FTP is used in a mobility demo (FTP has a nice "hash" option
which shows progress. BTW FTP shows the impact of optimizations for
interactive (control) or bulk transfer (data): the interactive restart
after the handoff "hole" is far more aggressive, this is a good
argument for TCP algorithm adjustments for mobility).

=> RFC 2473 is about something a bit different (near purely transit
stuff where this whole message is about end node stuff).

Regards

[email-masked]

PS: in conclusion, this is not easy but within the reach of a skilled
programmer (so get a good implementation and learn/copy :-).
PPS: my advantage here is mainly the time zone (Erik lives in France
and sent the message when you are sleeping :-).

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 31 Jan 2002 16:31:31 +0100 (CET)

> destination.

I thought this was what Solaris did as well, until I looked.
Yes, I know of know implementations which associate the path mtu with
the actual source routed path.
But when there is a source route and the ICMP error arrives the Solaris
implementation just looks at the destination field in the IP header
in error. In fact it would need to see if there are "unconsumed" elements
in the source route/routing header and find the last one in order to
do this for the final destination.
This might be a Solaris specific bug - haven't looked at other pmtu
enabled OSs.

But that doesn't cause problems as long as TCP is told of the right
MTU to use.

> way is considered as the not optimized but always working way.

So existing MIPv6 implementations do this?

> => I agree but if an ICMP packet too big is received the TCP layer has

So existing MIPv6 implementations do this?

> (re)guessing routine will find it at a know place.

If so you need to have the ICMP handling code in IP always find the final
destination. I hope only Solaris has the bug I mentioned above.

> argument for TCP algorithm adjustments for mobility).

Was there explicit tests where during such a connection a router in the path
have its MTU lowered?

Finally, how did the implementors figure this out and share the need for it
amongst themselves given that the specification has zero mention of it?

   Erik

---

From: "James Kempf" <[email-masked]>
Date: Thu, 31 Jan 2002 08:51:56 -0800

Erik,

How is this different from the case where a router fails on the
original path and the route changes so that the original
path MTU is no longer valid?

            jak

---

From: Francis Dupont <[email-masked]>
Date: Thu, 31 Jan 2002 19:02:49 +0100

=> my code is too old for the "existing" word but one has existed
and I believe this can be copied or done again.

=> yes, this was tested with native IPv6 and IPv6 over IPv4 paths.
But I agree we should verify if this is explicitely tested in test suites
(I have some contacts with all testers at the last ETSI IPv6 plugtest,
I can forward the message).

=> should I answer?

Regards