Subject: [Int-area] draft-ietf-intarea-tunnels comments

From: Toerless Eckert <[email-masked]>
Date: Sat, 15 Aug 2015 02:24:25 -0700

Nice start. Some comments:

1. diagnostics

It would be lovely to have a section about diagnostics,
eg: fragmentation/reassembly counters, status of PMTUD
and the like and explain at least which of the relevant
counters are derived/mandated from tunnel-type independent
RFCs (eg: for interfaces). If i am not mistaken,
PMTUD status and reassembly counters and the like
would more likely be subject to each tunnel type spec
(including any potential MIB spec). Anything the draft
can do to help kick tunnel solutions to provide the
best diagnostics currently defined via RFCs - and ideally
pointing out gaps.

2. rfc4459

You make it sound as if rfc4459 is misleading. How would you
like to see in the RFC repository the relationship ?
"rfc4459 updated by..." ?

3. Readability

The style is somewhat terse and hard to read and terminology is inconsistent.

For example:, section 4: "The requirements of an IP tunnel" ...
Eg: This section defines requirements for the operations of tunnels
derived from standards for the tunneled protocol and tunnel
exchange message protocol. It only discusses cases where these
are either IPv4 or IPv6, but the rules may? also apply to other
protocols.

You first define tunnel MTU as TMTU, but use TunMTU in the 4.1 pseudocode.
You also use couple other abbrevitations in the 4.1 pseudocode that
are either not defined on inconsistent with 2.2 Terminology or redefined
multiple times. Eg: "egressRMTU is the receive MTU", and a few lines later 
"egress reassembly MTU (egressRMTU)", TTP abbreviation not in 2.2...  Please fix.

Likewise consistencies in the psudocode, example:

      and send each fragment into the tunnel ingress
      send TTP into the tunnel "interface" (the ingress)

And then in the ingres you say "encapsulate TTP as received and emit",
which is confusing when considering the case that the ingres was
given TTP fragments from the first step.

4. Fragementation

The fragmentation code is confusing because it looks like two parts in the forwarding
plane where fragmentation could happen, and i am sure in reality products
do not implement it that way. And then you have to figure out that
at last it's not possible to do fragmentation twice because its the same
condition. Which looks like even worse ral-world code path. And doubtfully
better architectural pseudo-code..

Then your text also mentions "address resolution" without referring to it,
but if there is any implication, i think its for example per-egress TMTU
and egresRMTU considerations.

Let me know if this pseudocode makes more sense . thats
at least hwat i figured out is what you intended to say. If not than
i couldn't parse your text/code correctly:

      # In a tunnel with multiple egres, each egres may have a
      # separate RMTU and different TMTU
      (egresRMTU,egresTMTU)=resolve(TTP destination)

      # linkMTU is an architectural IP link property. It could be derived
      # the maximimum egresRMTU across all egres. Administratively setting
      # linkMTU to a value smaller than any possible egresRMTU allows
      # tunnels not having to know (or assume) egresRMTU.

      if ( TTP > linkMTU || TTP > egresRMTU) && (TTP can not be fragmented))
         drop TTP and send ICMP "too big" to TTP source
         return
      endif

      maxfragment = min(egressRMTU,egressTMTU)
      if( TTP > maxfragment)
         split TTP into fragments of egresTMTU size
         encapsulate each TTP fragment and emit as TLP (Tunnel Link Packet)
      else
         encapsulate TTP and emit as TLP (Tunnel Link Packet)
      endif

Next:

Why does the doc not talk about fragmenting the TLP (Tunnel Link Packet) ?
Are there any hard rules prohibiting that ? Would be nice to include
a reference to those rules if possible if there are any for IP.

IMHO, fragmenting the encapsulated packet can make perfect sense. You
do it at the non-IP layer anyhow, eg: within (ml)PPP. And if you happen to
have a tunnel encap that didn't (like PPP) added its own fragmentation
(within the encap mechanism), then it can make perfect sense to fragment
the IP TLP. AFAIK, DF just implies that you can not fragment before
encap. So If you have a device specifically designed for HW tunneling
and good at fragmentation, and you have traffic that wants to be nice
to the network and sets DF to avoid fragmentation on poor normal routers,
then your tunnel devices best choice in the absence of a self-fragmenting
encap mechanism is to fragment the TLPs - and reassemble those on the
other side.

But i am sure i am missing a decade of politics on this option ;-))

5.  IP multicast

In your history section you may also want to note:

Doing tunneling right has been a learning curve for the Internet
that has not fully settled yet which is the reason for this document.

The first attempt to use tunnels in a large scale to transit IP
multicast across the (non-IP-Multicast-enabled) Internet did lead
in 1988 to a collapse of many paths these tunnels where running
over due to a misunderstanding of how to correctly do tunneling
(see section 4. paragraph 2 of RFC1075). Redoing the tunneling did
then allow to use tunnels to proliferate that new IP multicast
technology much better [ER94], but it took two more decades to
revisit the tunneling problem for this use-case to be better
deploable, which lead to RFC7450 (AMT).

Aka, also add AMT to your list of tunnel protocols please: 

  o Automatic Multicast Tunneling (AMT) [RFC7450] - Tunnels IPv4/IPv6
    multicast via p2mp into IPv4/IPv6 UDP to pass through NAT/FW
    and enable IP multicast across the Internet without need for
    configuration of the remote tunnel endpoint.

Wrt to 4.10.: 
   
There is no working PMTUD for IP multicast packets due to scale.
Attempts to solve that problem are just a waste of effort IMHO.

IPv6 has good enough MTU requirements:  The gain of going
beyond 1280 application packet size packets for IP multicast is highly
questionable (aka: + tunnel header < 1500). Then again, IMHO all
(including unicast) uses of > 1280 for anything but tunneling are just
indications of really bad systems. I've only come to see larger MTU be 
requested by really expensive "High Performance Supercomputer" that had
an 8086 front-end-CPU for network processing - and therefore could
only get any reasonable throughput with close to 64k size packets.
Braindead.

Ok, sorry i hate jumbo packets, nothing personal.

In any case: I'd write that Tunnels should for IPv6 multicast
packets simply reject packets larger than their MTU, and that
IPv6 multicast applicastions are expected to follow RFC3542, section 11.1,
which means they never send > 1280.

Not sure what other aspects for multicast you thought to be of
interest. Let me know, i surely will have opinins about them ;-)

Cheers
    Toerless

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Aug 2015 11:43:16 -0700

Hi, Toerless,

Thank you for the detailed comments.

I'll roll most of these into the next version, but had a few points
below that it might be useful to discuss.

Joe

> It would be lovely to have a section about diagnostics,

I'll add that, thanks.

> "rfc4459 updated by..." ?

Ideally, yes.

> The style is somewhat terse and hard to read and terminology is inconsistent.

I'll check the specific issues you raised; if you can point out any
others either on-list or off-list, that'd be useful.

> do not implement it that way.

I am not as sure.

Both routers and hosts make fragmentation decisions in the forwarding
plane (for hosts this is where the source IP address is determined) and
then also at the tunnel device driver.

> And then you have to figure out that
> condition. Which looks like even worse real-world code path. And doubtfully

Yes, it might be the case that the forwarding plane sends a packet to a
device driver that decides it cannot process the message.

> and egresRMTU considerations.

I'll expand that section accordingly, but as you note it might be useful
to give an example of the pseudocode for the less common case of
multiple egresses as well.

I would expect that to be useful as a separate example because I don't
think it's a common case.

>       # In a tunnel with multiple egress, each egress may have a

The resolution would return the necessary properties of the tunnel,
which needs to also include the Tunnel Link Packet header.

While they are determined by the TTP destination, I hesitate to refer to
them as "egressNAME" because they could depend on the tunnel ingress
selected as well.

So I would instead use:

	(TLPheader, RMTU, TMTU) = resolve(TTP destination)

I'll revise the pseudocode to make it more clear.

> Why does the doc not talk about fragmenting the TLP (Tunnel Link Packet) ?

It talks about both fragmenting the TTP (the first pseudocode block) and
the TLP. The latter is described as a process in the second pseudocode
block but should be stated more clearly, i.e., as if we create the
larger TLP and source-fragment it.

...
>     configuration of the remote tunnel endpoint.

Will do, thanks.

> which means they never send > 1280.

Good point, will add as well.

----