Subject: RE: New draft: "Link Adaptation for IPv6-in-IPv4 Tunnels"

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 19 Aug 2005 08:16:34 -0700

Mark,

Understand your point about setting TCP MSS, but regardless of
whether the tunnel occurs on the same node as one of the TCPs
or somewhere along the path the key point is whether/not the
implementation takes the SHOULD and configures the largest
possible MTU during tunnel setup time. This might require some
pre-probing of SEGSIZE, e.g., when sending IPv6 ND messages
during tunnel setup.

Alternatively, the tunnel could optimistically set a larger MTU
and return locally-generated ICMPv6 "packet too big" messages
in case it gets ICMPv4 "fragmentation needed" messages back
from the network. But, this method is susceptible to path
MTU-related black-holes when the network fails to deliver
accurate ICMPv4s.

The best implementations might do some kind of fusion of the
two approaches, e.g., optimistically set a larger initial MTU
and actively probe the tunnel segment size in the early going
while watching for ICMPv4's. So, the degree of sophistication
here is really up to the implementation rather than being
mandated by the spec.

Thanks - Fred
[email-masked]

> -----Original Message-----
> From: Mark Smith 
> [mailto:[email-masked]] 
> Sent: Friday, August 19, 2005 12:38 AM
> To: Templin, Fred L
> Cc: [email-masked]
> Subject: Re: New draft: "Link Adaptation for IPv6-in-IPv4 Tunnels"
> 
> Hi Fred,
> 
> On Thu, 18 Aug 2005 10:20:01 -0700
> "Templin, Fred L" <[email-masked]> wrote:
> 
> > Mark,
> > 
> > Thanks for the comments. The paragraph you cited needs to be read in
> > conjunction with the one that immediately follows, however, i.e.:
> > 
> >    Since LinkMTU values larger than 1280 bytes may result 
> in [ICMPv6]
> >    "packet too big" messages due to temporary segmentation 
> restrictions
> >    (see: section 3.3), ULPs SHOULD employ a probing 
> strategy that begins
> >    with a smaller payload size (on the order of 1KB) and 
> probes upward
> >    [PMTUD].  (Note that this may not be possible for some ULPs.)
> > 
> > This says that ULPs (like TCP) can do their own probing if they want
> > to discover larger MTUs beyond what the initial tunnel interface MTU
> > allows independent of the tunnel segment size probing. BTW, 
> this is true
> > also for the case of no tunnel interfaces in the path, 
> e.g., a TCP could
> > start out on a path with a small MTU and re-routed to an 
> all-GIGE path.
> > 
> 
> Is this based on the assumption that the tunnel is a link within the
> path, rather than possibly at one end of it ?
> 
> The reason I ask is that I was looking at this issue in the context of
> one end of the TCP connection being on the same node as the tunnel end
> point, which happens to be how I'm currently running IPv6 - this PC is
> both a tunnel end point and the node where my TCP connections 
> originate.
> 
> For example, say Node A is both the tunnel end point and one 
> of the TCP
> connection end points. Node A starts the 3 way TCP hand shake 
> with remote
> node B, which is sitting behind, rather than on, the remote tunnel
> end-point. In that case, Node A would set it's announced MSS value to
> something derived from the initial tunnel MTU i.e. 1280 
> bytes, or a MSS
> value of 1220. The other end might set MSS to 1440, because it is
> attached to a 1500 byte MTU ethernet.
> 
> Even though there might be a larger possible path MTU between 
> Node A and
> B, including the tunnel within that path, e.g., 1500 - 20 (IPv4) - 40
> (IPv6) or a path MTU of 1460, Node B would never attempt to or cause a
> probe for anything larger, because it would never send a 
> segment to Node
> A that was larger than Node A's initial MSS value. 
> 
> I'm not sure if TCP adapts to a larger local interface MTU if 
> it changes
> during the connection. If it does, Node A could send segments larger
> than the initial MTU (minus IPv6 and TCP hdrs) to Node B if the local
> MTU increases, up to the maximum segment size that Node B 
> announced. If
> TCP doesn't behave that way, then Node A would also never 
> send a larger
> than it's original MTU derived segment to Node B, also never 
> causing the
> tunnel MTU to increase. That would mean that in both directions of the
> path, the MTU would never be increased beyond the initial tunnel MTU
> value.
> 
> My thoughts were that setting the initial tunnel MTU to a 
> value derived
> from todays common Internet path MTU, rather than IPv6's minimum
> required MTU value, might overcome this problem in this scenario where
> the TCP and tunnel end points are on the same node, without 
> it being too
> detrimental to any other cases, such as when the tunnel is 
> within the path,
> rather than at one of the ends of it.
> 
> HTH,
> Mark.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 23 Aug 2005 09:10:18 -0700

Mark,

Thanks for the follow-up; see responses inline below: 

> Sent: Monday, August 22, 2005 6:20 PM
> To: Templin, Fred L
> Cc: [email-masked]
> On Mon, 22 Aug 2005 09:19:00 -0700
> > Another implementation option which wasn't mentioned is to
> > designate a tunnel pseudo-interface that acts as a "lid" on
> > the can-of-worms consisting of all active tunnels configured
> > on the endpoint. The lid would present a consistent MTU
> > (e.g., 9180 bytes) to upper layers, e.g., for setting the
> > TCP MSS, but there may be instances of locally-generated
> > ICMPv6 "packet too big" messages coming from tunnels inside
> > the can.
> 
> I might get you to clarify this a bit further, so I'm sure I'm
> commenting in the right context. Here is what I understand you're
> describing :
> 
> (1) (Probably) Single "parent" tunnel interface on the 
> tunnel-end-point
> host
> 
> (2) the parent tunnel interface would be used by the host as the
> TCP end point, with the MSS value derived from the interface MTU.
> Normally, the source IPv6 address of the TCP end point would 
> be one of the
> addresses assigned to this parent tunnel interface.
> 
> (3) child tunnels are used to manage actual IPv4 tunnel MTUs, and
> performs fragmentation of the large, assuming the other end also sets
> MSS to be around 9000 bytes, IPv6 segments into IPv4 path MTU 
> sized IPv4
> packets.

Yes to all of the above.
 
> (4) child tunnels would have to all be from and to the same tunnel
> end-point, similar to how ATM virtual channels are all part 
> of the same
> virtual path. I think this would be a requirement because the IPv6
> addressing used for the TCP connections would be derived from 
> the parent
> pseudo-tunnel interface.
> 
> (5) as a consequence of (4), multiple pseudo tunnel interfaces would
> have to exist for different sets of tunnel end points.

Not necessarily on points (4) and (5); I was thinking of having
all of the "worms" in a single "can".
 
> Alternatively, I could see how the pseudo tunnel interface 
> could be used
> to as the IPv6 MTU / MSS value, yet the source IPv6 address 
> used for TCP
> connections is the IPv6 address assigned to the child tunnel 
> being used.
> However, I'd think that would mean modifications to how IPv6 would, by
> default, pick IPv6 source addresses, as I'd think it would usually
> select an address from the outgoing interface that the route table has
> indicated is to be used.

I was thinking multiple IPv6 addresses per node and the addresses
serving more as application endpoint identifiers rather than
node identifiers. I haven't looked to see how well this view
is supported under existing OS's.
 
> If my understanding is correct, then I'd think this parent,
> psuedo-tunnel interface method could suffer from the "fragmentation
> considered harmful" issue, i.e., at the IPv6 TCP connection level, the
> unit of transmission isn't the unit of control.

Independent of whether the pseudo-interface method is used, route
changes can certainly occur after a tunnel's IPv4 segment size has
been determined which could result in ICMPv4 "fragmentation needed"
messages that indicate the loss of a segment smaller than the
TCP unit of transmission. The originating tunnel endpoint can
then either re-segment and retransmit the IPv6 packet or return
an ICMPv6 "packet too big" message back to the original source.
But, that would entail caching original IPv6 packets at the
encapsulator for up to the tunnel's RTT which could be
difficult for large bandwidth*delay tunnels.

Another alternative is for the encapsulator to probe the IPv4
segment size periodically in a proactive "black hole detection"
sort of posture. The frequency for probing could be based on
the arrival of ICMPv4s, which may/may not be accurate (see below).  
 
> I could be completely confused though :-)
> 
> > Again, this seems outside the scope of the
> > specification - do you concur?
> >
> 
> I'd agree that how the tunnels are representated within the 
> OS is mostly
> an implementation detail, although I have had an experience with an
> IPsec implementation that didn't represent tunnels as virtual
> interfaces, which caused a number of issues (routing protocol
> association with the tunnel, applying traffic filtering to traffic
> before it entered the tunnel - a lot of functions on a router are
> commonly oriented towards applying the functions to an specified
> interface). To get around this problem, the vendor 
> recommended solution
> was to create GRE tunnels and then use IPsec transport mode to the GRE
> traffic, which is a bit of a kludge. It probably would have 
> been better
> to have at least a recommendation in the IPsec RFCs as to how to
> represent the IPsec tunnels within an implementation, similar 
> to the way
> the IPv6 neighbour discovery RFC describes a conceptual host
> representation.
> 
> My original thoughts were that the default, initial tunnel MTU was a
> protocol constant, making it in the scope of the RFC, and that setting
> it initially to 1280 bytes would mean that all 
> implementations would use
> 1280 bytes as the default value. Unfortunately it ends up creating a
> constraint on the maximum PMTU value possible, at least when 
> the TCP end
> point is on the same node as the tunnel end-point. 
> 
> Your suggestion in the text to set it higher overcomes that 
> limitation,
> I was making the suggestion to set it higher to try to 
> optimise for the
> common case without requiring any further tunnel configuration.
> 
> Commenting on this from your earlier email :
> 
> When you say "accurate", are you refering to routers sending incorrect
> link MTU values when they issue an ICMPv4 Fragmentation 
> Required message
> ? If you are, I personally haven't come across that problem, at least
> that I'm aware of. Is it common ? The only common problem I'm aware of
> in this area is misconfigured firewalls blocking incoming these ICMP
> Fragmentation Required messages, which breaks PMTUD.

Consider an adversary that can wiretap a path and examine tunneled
segments as they fly by. There is nothing to stop such adversaries
from sending gratuitous ICMPv4 "fragmentation needed" messages with
wrong link MTU values back to the originating tunnel endpoint, i.e.,
even though the tunneled segments are actually making it through to
the other side. I personally know of no real-life examples of such
adversaries, but it is easy to postulate their existence.

Fred