Subject: [Int-area] Questions to draft-intarea-tunnels-03

From: Linda Dunbar <[email-masked]>
Date: Thu, 6 Oct 2016 20:34:56 +0000

Joe and Mark,

You said "Because tunnels are links, they are subject to the same issues as any link, e.g. MTU.."
The MTU issue exist between any two points in a network. The MTU issue for tunnel is more like the MTU issue between any two points in the network (i.e. traverse many links, and can change over time), less like the MTU for a single link because the single link MTU is only determined by the fixed TWO endpoints of the link, whereas the MTU of a tunnel (and a path) is impacted by many links in between, i.e. more dynamic and can change depending which path the packet traverse through.

In the section 1 of your draft, you stated:  ".. tunnels emulate a link".  To me, "tunnel" hide the inner addresses, but the packets can traverse many links.  So Tunnels hardly emulate a link.

My two cents.

Linda

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Oct 2016 14:10:05 -0700

Hi, Linda,

On 10/6/2016 1:34 PM, Linda Dunbar wrote:
>
> Joe and Mark,
>
>  
>
>  
>
> You said “Because tunnels are links, they are subject to the same
> issues as any link, e.g. MTU..”
>
> The MTU issue exist between any two points in a network.
>

A link MTU is different from a path MTU, as explained in the document.
The MTU issues of a link are different than those of a path, as described.

> The MTU issue for tunnel is more like the MTU issue between any two
> points in the network (i.e. traverse many links, and can change over
> time), less like the MTU for a single link because the single link MTU
> is only determined by the fixed TWO endpoints of the link, whereas the
> MTU of a tunnel (and a path) is impacted by many links in between,
> i.e. more dynamic and can change depending which path the packet
> traverse through.
>

Both a link and tunnel may have multiple hops not visible to IP, each of
which may have a different MTU; this is no different than an IP path
MTU. The difference is that the link/tunnel hops are not visible to IP.

>  
>
> In the section 1 of your draft, you stated:  “.. tunnels emulate a
> link”.  To me, “tunnel” hide the inner addresses, but the packets can
> traverse many links.  So Tunnels hardly emulate a link.
>

There are many link layer technologies which use multiple link-layer
hops and relaying, including Ethernet, ATM, SONET, etc.

Joe

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 6 Oct 2016 23:36:38 +0000

Joe,

I strongly agree with what you said "A link MTU is different from a path MTU".

A Tunnel MTU is more like "path MTU" because

-           the Tunnel MTU is the minimum of all the links' MTU that the "tunnel" traverse,

-          the fragmented segments may be out of order when arriving at the egress node of the tunnel, whereas over a link, the fragmented frames always arrive at egress port in the same order (therefore the reassembly is much easier over a link).

Another question, in Section 5.2: why you specifically discuss the "egress  MTU" ?  A tunnel MTU is determined by the minimum MTU of all the links' MTU  that the tunnel traverse.
The HSrc and HDst would have considered all the links' MTU when they choose the frame size. So the tunnel egress MTU would be big enough for the original  packet size between HSrc & HDst.

A tunnel MTU is a critical issue, not because of Tunnel being like a link", but

-          because the ingress adds the outer header encapsulation which may make the total  frame size chosen by the original source to exceed the MTU of  some intermediate links,

-          because the fragmented segments may be out of order when arriving at the egress node of the tunnel, whereas over a link the fragmented frames always arrive at egress port in the same order (therefore the reassembly is much easier over the link).

A couple of minor typo:

Section 3.6.1: "the term "inner")..", is the ")" extra?

Section 4.2 first sentence: ".... It messages might be ...", do you mean "... its messages might be..."

Linda

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, October 06, 2016 4:10 PM
To: Linda Dunbar <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: Questions to draft-intarea-tunnels-03

Hi, Linda,

A link MTU is different from a path MTU, as explained in the document. The MTU issues of a link are different than those of a path, as described.

The MTU issue for tunnel is more like the MTU issue between any two points in the network (i.e. traverse many links, and can change over time), less like the MTU for a single link because the single link MTU is only determined by the fixed TWO endpoints of the link, whereas the MTU of a tunnel (and a path) is impacted by many links in between, i.e. more dynamic and can change depending which path the packet traverse through.

Both a link and tunnel may have multiple hops not visible to IP, each of which may have a different MTU; this is no different than an IP path MTU. The difference is that the link/tunnel hops are not visible to IP.

There are many link layer technologies which use multiple link-layer hops and relaying, including Ethernet, ATM, SONET, etc.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Oct 2016 21:48:23 -0700

Hi, Linda,

On 10/6/2016 4:36 PM, Linda Dunbar wrote:
>
> Joe,
>
>  
>
> I strongly agree with what you said “A link MTU is different from a
> path MTU”.
>
>  
>
> A Tunnel MTU is more like “path MTU” because
>
> -           the Tunnel MTU is the minimum of all the links’ MTU that
> the “tunnel” traverse,
>

A link MTU is the largest IP packet that can traverse the link (i.e.,
without needing to be fragmented to go over the link).

A tunnel MTU, like a link MTU, is the largest IP packet that can
traverse the tunnel or link layer without needing to be fragmented
FIRST, which is one of two things, whichever is larger:
    1. - the min of the MTUs of the links along the tunnel or the
link-layer hops along the path of the link protocol, minus the
tunnel/link encapsulation overhead
    2. - the reassembly size of the tunnel/link egress, minus the
tunnel/link encapsulation overhead

Note that (1) would never be larger than (2), so (2) is sufficient for
both a tunnel and a link protocol.

An IP path MTU is determined only by (1).

This is one way in which a tunnel is more like a link than an IP path.

One other way is that an IP packet hopcount/ttl is never decremented
when traversing a link or a tunnel *because* it does not encounter an IP
router along the way, and only IP routers perform this decrement, i.e.,
an IP packet with TTL=0 should be able to traverse both a link and a
tunnel, but would not traverse an IP path.

> -          the fragmented segments may be out of order when arriving
> at the egress node of the tunnel, whereas over a link, the fragmented
> frames always arrive at egress port in the same order (therefore the
> reassembly is much easier over a link). 
>

A link is not guaranteed to avoid reordering or loss. A particular link
protocol may have one or both of these properties, but it is not a
requirement to be a link.

>  
>
> Another question, in Section 5.2: why you specifically discuss the
> “egress  MTU” ? 
>
See above.

> A tunnel MTU is determined by the minimum MTU of all the links’ MTU
>  that the tunnel traverse.
>
This is incorrect, as noted above (and described in the document). The
egress MTU determines the largest IP packet (or fragment) that can
traverse either a link or a tunnel.

> The HSrc and HDst would have considered all the links’ MTU when they
> choose the frame size. So the tunnel egress MTU would be big enough
> for the original  packet size between HSrc & HDst.
>

Consider a link such as ATM: the ATM cell size is not relevant, only the
ATM AAL5 reassembly payload size. That determines the MTU because an IP
packet that large can traverse an ATM link (with multiple ATM link hops)
without needing to be fragmented before traversal.

The same is true for an IP tunnel. The hops of the tunnel are not
relevant *to the packet arriving at the ingress*. The only thing that is
relevant is the largest payload the tunnel egress can reassemble. The
hops within the tunnel determine how the ingress fragments, not whether
a packet arriving at the ingress must be dropped.

>  
>
> A tunnel MTU is a critical issue, not because of Tunnel being like a
> link”, but
>
> -          because the ingress adds the outer header encapsulation
> which may make the total  frame size chosen by the original source to
> exceed the MTU of  some intermediate links,
>

The overhead of these encapsulations is relevant only at the egress
reassembly; the effect on the individual fragments is not relevant. IP
doesn't care about the 5-byte overheads for ATM cells.

> -          because the fragmented segments may be out of order when
> arriving at the egress node of the tunnel, whereas over a link the
> fragmented frames always arrive at egress port in the same order
> (therefore the reassembly is much easier over the link). 
>
As noted above, in-order delivery is ensured only for some link
technologies/protocols.

>  
>
>  A couple of minor typo:
>
>  
>
> Section 3.6.1: “the term "inner")..”, is the “)” extra?
>
Yes - thanks for pointing that out. It'll be fixed in the next rev.

>  
>
> Section 4.2 first sentence: “…. It messages might be …”, do you mean
> “… its messages might be…”
>

I'll double-check tomorrow, but I'm fairly sure I meant to just drop the
"it" part. Again, it'll be fixed in the next rev.

Joe

>  
>
> Linda
>
>  
>
>  
>
> *From:*Joe Touch [mailto:[email-masked]]
> *Sent:* Thursday, October 06, 2016 4:10 PM
> *To:* Linda Dunbar <[email-masked]>; [email-masked]
> *Cc:* [email-masked]
> *Subject:* Re: Questions to draft-intarea-tunnels-03
>
>  
>
> Hi, Linda,
>
>  
>
>     The MTU issue for tunnel is more like the MTU issue between any
>     two points in the network (i.e. traverse many links, and can
>     change over time), less like the MTU for a single link because the
>     single link MTU is only determined by the fixed TWO endpoints of
>     the link, whereas the MTU of a tunnel (and a path) is impacted by
>     many links in between, i.e. more dynamic and can change depending
>     which path the packet traverse through.
>
>
> Both a link and tunnel may have multiple hops not visible to IP, each
> of which may have a different MTU; this is no different than an IP
> path MTU. The difference is that the link/tunnel hops are not visible
> to IP.
>
>
>      
>
>     link”.  To me, “tunnel” hide the inner addresses, but the packets
>     can traverse many links.  So Tunnels hardly emulate a link.
>
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 14:43:19 +0000

Hi, Joe and Mark are right - a tunnel has the same characteristics as a link. Just
as you would never admit a 9KB packet into a 1500B link without fragmentation,
a tunnel ingress must never admit a packet into the tunnel if it is larger than the
egress reassembly buffer size.

Joe's analogy of ATM AAL5 is also exactly correct. And, as Joe says, the cell
size does not matter *as long as the cell size can traverse the tunnel without
loss due to a size restriction*. That is why the nominal cell size should be no
larger than 1280B (minimum IPv6 link MTU) unless there is assurance that
larger cell sizes can be used.

Thanks - Fred

From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
Sent: Thursday, October 06, 2016 9:48 PM
To: Linda Dunbar <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: [Int-area] Questions to draft-intarea-tunnels-03

Hi, Linda,

A link MTU is the largest IP packet that can traverse the link (i.e., without needing to be fragmented to go over the link).

A tunnel MTU, like a link MTU, is the largest IP packet that can traverse the tunnel or link layer without needing to be fragmented FIRST, which is one of two things, whichever is larger:
    1. - the min of the MTUs of the links along the tunnel or the link-layer hops along the path of the link protocol, minus the tunnel/link encapsulation overhead
    2. - the reassembly size of the tunnel/link egress, minus the tunnel/link encapsulation overhead

Note that (1) would never be larger than (2), so (2) is sufficient for both a tunnel and a link protocol.

One other way is that an IP packet hopcount/ttl is never decremented when traversing a link or a tunnel *because* it does not encounter an IP router along the way, and only IP routers perform this decrement, i.e., an IP packet with TTL=0 should be able to traverse both a link and a tunnel, but would not traverse an IP path.

A link is not guaranteed to avoid reordering or loss. A particular link protocol may have one or both of these properties, but it is not a requirement to be a link.

Another question, in Section 5.2: why you specifically discuss the "egress  MTU" ?
See above.

A tunnel MTU is determined by the minimum MTU of all the links' MTU  that the tunnel traverse.
This is incorrect, as noted above (and described in the document). The egress MTU determines the largest IP packet (or fragment) that can traverse either a link or a tunnel.

Consider a link such as ATM: the ATM cell size is not relevant, only the ATM AAL5 reassembly payload size. That determines the MTU because an IP packet that large can traverse an ATM link (with multiple ATM link hops) without needing to be fragmented before traversal.

The same is true for an IP tunnel. The hops of the tunnel are not relevant *to the packet arriving at the ingress*. The only thing that is relevant is the largest payload the tunnel egress can reassemble. The hops within the tunnel determine how the ingress fragments, not whether a packet arriving at the ingress must be dropped.

The overhead of these encapsulations is relevant only at the egress reassembly; the effect on the individual fragments is not relevant. IP doesn't care about the 5-byte overheads for ATM cells.

-          because the fragmented segments may be out of order when arriving at the egress node of the tunnel, whereas over a link the fragmented frames always arrive at egress port in the same order (therefore the reassembly is much easier over the link).
As noted above, in-order delivery is ensured only for some link technologies/protocols.

 A couple of minor typo:

I'll double-check tomorrow, but I'm fairly sure I meant to just drop the "it" part. Again, it'll be fixed in the next rev.

Joe

Linda

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, October 06, 2016 4:10 PM
To: Linda Dunbar <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: Questions to draft-intarea-tunnels-03

Hi, Linda,

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 08:00:49 -0700

Hi, Fred,

On 10/7/2016 7:43 AM, Templin, Fred L wrote:
>
> Joe’s analogy of ATM AAL5 is also exactly correct. And, as Joe says,
> the cell
>
The link (or tunnel) *payload* size needs to be at least 1280B unless
larger sizes can be used.

And (I know you know, but for others) that's why there needs to be
ingress/egress fragmentation and reassembly to ensure that when the
payload is smaller, the edge-to-edge transit payload size meets these
minimums.

All of this is in the doc already; if it's not clear from the doc, I
would welcome suggestions on what to add.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 15:06:56 +0000

Hi Joe,

?  The link (or tunnel) *payload* size needs to be at least 1280B unless larger sizes can be used.

I was only referring to the (fragmentation) *cell* size as being no larger than 1280B
unless there is assurance of larger sizes. It goes without saying that the *payload*
size needs to be at least 1280B unless larger sizes can be used, but since you have
said it you have spoken correctly.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Friday, October 07, 2016 8:01 AM
To: Templin, Fred L <[email-masked]>; Linda Dunbar <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: [Int-area] Questions to draft-intarea-tunnels-03

Hi, Fred,

The link (or tunnel) *payload* size needs to be at least 1280B unless larger sizes can be used.

And (I know you know, but for others) that's why there needs to be ingress/egress fragmentation and reassembly to ensure that when the payload is smaller, the edge-to-edge transit payload size meets these minimums.

All of this is in the doc already; if it's not clear from the doc, I would welcome suggestions on what to add.

Joe

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 7 Oct 2016 15:15:48 +0000

Joe,

You said "The egress MTU determines the largest IP packet (or fragment) that can traverse either a link or a tunnel."

If a Tunnel traverse some segments of very old layer 2 network with MTU = 1500, but the Tunnel Egress node is a modern node with MTU = 2000 bytes.
I would think the Tunnel MTU is 1500, not the 2000 at the Tunnel egress node, correct?

Linda

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, October 06, 2016 11:48 PM
To: Linda Dunbar <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: Questions to draft-intarea-tunnels-03

Hi, Linda,

 A couple of minor typo:

Joe

Linda

Hi, Linda,

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 08:32:33 -0700

> On Oct 7, 2016, at 8:15 AM, Linda Dunbar <[email-masked]> wrote:
> 
> Joe,
>  
> You said “The egress MTU determines the largest IP packet (or fragment) that can traverse either a link or a tunnel.”
>  

If the ingress fragments and egress reassembles, the MTU of the tunnel(or link) is 2000 - encaps overhead. The MTU of the  hops inside the tunnel/link determine how the Ingress needs to fragment, not ultimately what can traverse the tunnel/link.

I.e. The difference is between the MTU *of* the tunnel/link and the MTU *inside* the tunnel/link.

Joe

> I strongly agree with what you said “A link MTU is different from a path MTU”.
>  
> -         the Tunnel MTU is the minimum of all the links’ MTU that the “tunnel” traverse,
> 
> Another question, in Section 5.2: why you specifically discuss the “egress  MTU” ? 
> See above. 
> 
> 
> A tunnel MTU is determined by the minimum MTU of all the links’ MTU  that the tunnel traverse.
> The HSrc and HDst would have considered all the links’ MTU when they choose the frame size. So the tunnel egress MTU would be big enough for the original  packet size between HSrc & HDst.
> 
> A tunnel MTU is a critical issue, not because of Tunnel being like a link”, but
> Section 4.2 first sentence: “…. It messages might be …”, do you mean “… its messages might be…”
> 
> You said “Because tunnels are links, they are subject to the same issues as any link, e.g. MTU..”
> In the section 1 of your draft, you stated:  “.. tunnels emulate a link”.  To me, “tunnel” hide the inner addresses, but the packets can traverse many links.  So Tunnels hardly emulate a link.
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 09:07:53 -0700

re-sending cut down to try to get past the intarea size filters (seriously?)

Joe

On 10/7/2016 9:05 AM, Joe Touch wrote:
>
> Linda,
>
>
> On 10/7/2016 8:50 AM, Linda Dunbar wrote:
>>
>> Joe,
>>
>>  
>>
>> Most hosts use MTU path discovery to choose the smallest MTU before
>> sending the data frame.
>>
> Yes, with the caveat that PMTUD doesn't always work (due to ICMP
> blocking) and PLMTUD is supported only in a few protocols.
>
>> If the Tunnel Ingress only uses the Tunnel egress MTU as the size to
>> determine if the frames need to be fragmented,
>>
>
> It does not. It uses the MTU of the path inside the tunnel to decide
> how to fragment.
>
> Again, this is already in the doc. See the pseudocode in Section 4.2.
>
>> then it will need immediate nodes to do further fragmentation. What
>> if the immediate nodes can’t do fragmentation/reassembly?
>>
>
> (this is irrelevant because it doesn't happen this way).
>
>>  
>>
>>  Here is the design guideline for end hosts. Why can’t Tunnel Ingress
>> follow this principle?
>>
>
> Because a tunnel is one "segment" as far as the transiting IP is
> concerned.
>
> You are confusing how a tunnel appears to the transiting packet to how
> the tunnel insides appear to the ingress.
>
> Joe
>
>>  To utilize a path in the most efficient manner possible, hosts must
>> find the /path MTU/; this is the smallest MTU of any link in the path
>> to the distant end. For example, for two hosts communicating across
>> three routed links with independent MTUs of 1500, 800, and 1200
>> bytes, the smallest (800 bytes) must be assumed by each end host to
>> avoid fragmentation.
>>
>> path_mtu.png
>>
>>  
>>
>>  
>>
>>  
>>
>>

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 7 Oct 2016 16:23:27 +0000

Joe,

I agree this sentence of you said “. It uses the MTU of the path inside the tunnel to decide how to fragment.”

Your above statement contradicts with the  following sentence in the draft:
“The egress MTU determines the largest IP packet (or fragment) that can traverse either a link or a tunnel.”

An overlay “ tunnel” described in your draft appears as one “link” from “transiting IP” (or IP routing) perspective, but it is not from MTU perspective.

Linda

From: Joe Touch [mailto:[email-masked]]
Sent: Friday, October 07, 2016 11:08 AM
To: Linda Dunbar <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: Questions to draft-intarea-tunnels-03

Joe

Linda,

Most hosts use MTU path discovery to choose the smallest MTU before sending the data frame.
Yes, with the caveat that PMTUD doesn't always work (due to ICMP blocking) and PLMTUD is supported only in a few protocols.

If the Tunnel Ingress only uses the Tunnel egress MTU as the size to determine if the frames need to be fragmented,

It does not. It uses the MTU of the path inside the tunnel to decide how to fragment.

then it will need immediate nodes to do further fragmentation. What if the immediate nodes can’t do fragmentation/reassembly?

 Here is the design guideline for end hosts. Why can’t Tunnel Ingress follow this principle?

Because a tunnel is one "segment" as far as the transiting IP is concerned.

You are confusing how a tunnel appears to the transiting packet to how the tunnel insides appear to the ingress.

Joe

 To utilize a path in the most efficient manner possible, hosts must find the path MTU; this is the smallest MTU of any link in the path to the distant end. For example, for two hosts communicating across three routed links with independent MTUs of 1500, 800, and 1200 bytes, the smallest (800 bytes) must be assumed by each end host to avoid fragmentation.

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 10:15:40 -0700

On 10/7/2016 9:23 AM, Linda Dunbar wrote:
>
> Joe,
>
>  
>
> I agree this sentence of you said “. It uses the MTU of the path
> inside the tunnel to decide how to fragment.”
>

There are two separate things:

- the largest packet that can traverse a tunnel as transit (i.e., that
can use the tunnel as a link)
    - this is determined by the largest payload the egress can reassemble

- the largest packet that an ingress can send to the egress
    - this is determined by the min of the MTU of the hops within the link

E.g, let's see what happens when IPv4 goes over an IPv6 tunnel.

An IPv6 tunnel uses IPv6 hops (1280B packets) and can reassemble up to
1500B at the egress (as per RFC2460).

However, that means that the *payload* across the hops of the tunnel is
1280-40 = 1240, and the largest packet that can traverse the tunnel is
1500-40 = 1460.

If an IPv4 packet with DF=1 shows up at the ingress and it is smaller
than 1240, it is sent inside one IPv6 packet to the egress.

If an IPv4 packet with DF=1 shows up at the ingress and is between 1241
and 1460, it is encapsulated as a single IPv6 packet that is then source
fragmented at the ingress into a 1240B first fragment and a second
fragment of whatever remains. Those fragments are reassembled at the
egress and continue.

If an IPv4 packet with DF=1 shows up at the ingress and is larger than
1460, it needs to be dropped by the forwarding plane as exceeding the
MTU *of the tunnel*.

This is how the tunnel has an MTU of 1460 and how the ingress uses a
path MTU to the egress of 1240.

>  
>
>
> An overlay “ tunnel” described in your draft appears as one “link”
> from “transiting IP” (or IP routing) perspective, but it is not from
> MTU perspective.
>
It is, as per above.

Joe

> *Sent:* Friday, October 07, 2016 11:08 AM
> *To:* Linda Dunbar <[email-masked]>
> *Cc:* [email-masked]; [email-masked]
>         If the Tunnel Ingress only uses the Tunnel egress MTU as the
>         size to determine if the frames need to be fragmented,
>
>
>         then it will need immediate nodes to do further fragmentation.
>         What if the immediate nodes can’t do fragmentation/reassembly?
>
>
>     You are confusing how a tunnel appears to the transiting packet to
>     how the tunnel insides appear to the ingress.
>
>     Joe
>
>
>          To utilize a path in the most efficient manner possible,
>         hosts must find the /path MTU/; this is the smallest MTU of
>         any link in the path to the distant end. For example, for two
>         hosts communicating across three routed links with independent
>         MTUs of 1500, 800, and 1200 bytes, the smallest (800 bytes)
>         must be assumed by each end host to avoid fragmentation.
>
>         path_mtu.png
>
>          
>
>          
>
>          
>
>          
>
>  
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 17:17:25 +0000

The draft is correct as-written. It is exactly like ATM, where the reassembly buffer size
determines the MTU. Where it is slightly different than ATM is that we know that the
*minimum* cell size is 1280 (for IPv6), but if there is administrative assurance that a
larger cell size could be used then it is OK to use it.

Thanks - Fred

From: Int-area [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Friday, October 07, 2016 9:23 AM
To: Joe Touch <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [Int-area] Questions to draft-intarea-tunnels-03

Joe,

---

From: t.petch <[email-masked]>
Date: Fri, 7 Oct 2016 18:15:20 +0100

----- Original Message -----
From: "Joe Touch" <[email-masked]>
To: "Templin, Fred L" <[email-masked]>; "Linda Dunbar"
<[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, October 07, 2016 4:00 PM

> > size does not matter *as long as the cell size can traverse the
tunnel
> > without

Joe

I find paragraphs such as

"   Fragmentation is critical tunnels that support TTP packets for
   protocols with minimum MTU requirements, while operating over tunnel
   paths using protocols with minimum MTU requirements. Depending on the
   amount of space used by encapsulation, these two minimums will
   ultimately interfere, and the TTP will need to be fragmented to both
   support a TTP minimum MTU while traversing tunnels with their own TLP
   minimum MTUs."

rather hard to follow (I am a native English speaker with some years of
experience with IPv4)

Tom Petch
> Joe

------------------------------------------------------------------------
--------

> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 17:45:18 +0000

Hi Tom,

> rather hard to follow (I am a native English speaker with some years of

I agree that the text could be improved by simplifying some sentences and
using a few less acronyms. But, the architectural description is correct as it
stands.

Thanks - Fred
[email-masked]

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 15:48:23 +0000

Hi Joe,

Ø  If the ingress fragments and egress reassembles, the MTU of the tunnel(or link) is 2000 - encaps overhead.

Unless I am misunderstanding Linda’s comment and your response, the fact that the
egress attaches to a 2000B link has no bearing on the MTU size the egress can configure.

As per your doc, it is the reassembly buffer size (and not the sizes of any links the
tunnel is configured over) that determines the tunnel MTU. So, in this example, the
egress could just as well configure a 9KB reassembly buffer size and hence the
tunnel would have a 9KB minus ENCAPS MTU.

Thanks - Fred

Sent: Friday, October 07, 2016 8:33 AM

On Oct 7, 2016, at 8:15 AM, Linda Dunbar <[email-masked]<mailto:[email-masked]>> wrote:
Joe,

Joe

Linda

To: Linda Dunbar <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Hi, Linda,

 A couple of minor typo:

Joe

Linda

Hi, Linda,

Joe

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 7 Oct 2016 15:50:10 +0000

Joe,

Most hosts use MTU path discovery to choose the smallest MTU before sending the data frame. If the Tunnel Ingress only uses the Tunnel egress MTU as the size to determine if the frames need to be fragmented, then it will need immediate nodes to do further fragmentation. What if the immediate nodes can’t do fragmentation/reassembly?

Linda

Sent: Friday, October 07, 2016 10:33 AM

Joe

Linda

Hi, Linda,

 A couple of minor typo:

Joe

Linda

Hi, Linda,

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 09:01:26 -0700

On 10/7/2016 8:48 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  If the ingress fragments and egress reassembles, the MTU of the
> tunnel(or link) is 2000 - encaps overhead.
>
>  
>

You are right; I was reading Linda's note as indicating a 2000B reassembly.

As you note, the MTUs of the hops of the tunnel have no bearing on the
tunnel MTU *if* the egress does reassembly. The egress reassembly
payload determines the tunnel's MTU.

FYI, all, this is already all in the doc.

Joe
> *From:*Int-area [mailto:[email-masked]] *On Behalf Of *Joe
> Touch
> *Sent:* Friday, October 07, 2016 8:33 AM
> *Subject:* Re: [Int-area] Questions to draft-intarea-tunnels-03
>
>  
>
>  
>
>
>     You said “The egress MTU determines the largest IP packet (or
>     fragment) that can traverse either a link or a tunnel.”
>
>      
>
>     If a Tunnel traverse some segments of very old layer 2 network
>     with MTU = 1500, but the Tunnel Egress node is a modern node with
>     MTU = 2000 bytes.
>
> If the ingress fragments and egress reassembles, the MTU of the
> tunnel(or link) is 2000 - encaps overhead. The MTU of the  hops inside
> the tunnel/link determine how the Ingress needs to fragment, not
> ultimately what can traverse the tunnel/link.
>
>  
>
>     *Sent:* Thursday, October 06, 2016 11:48 PM
>     *To:* Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; [email-masked]
>     <mailto:[email-masked]>
>     *Cc:* [email-masked] <mailto:[email-masked]>
>     A link MTU is the largest IP packet that can traverse the link
>     (i.e., without needing to be fragmented to go over the link).
>
>     One other way is that an IP packet hopcount/ttl is never
>     decremented when traversing a link or a tunnel *because* it does
>     not encounter an IP router along the way, and only IP routers
>     perform this decrement, i.e., an IP packet with TTL=0 should be
>     able to traverse both a link and a tunnel, but would not traverse
>     an IP path.
>
>
>
>         -          the fragmented segments may be out of order when
>         arriving at the egress node of the tunnel, whereas over a
>         link, the fragmented frames always arrive at egress port in
>         the same order (therefore the reassembly is much easier over a
>         link). 
>
>
>     A link is not guaranteed to avoid reordering or loss. A particular
>     link protocol may have one or both of these properties, but it is
>     not a requirement to be a link.
>
>
>
>          
>
>     This is incorrect, as noted above (and described in the document).
>     The egress MTU determines the largest IP packet (or fragment) that
>     can traverse either a link or a tunnel.
>
>
>
>         The HSrc and HDst would have considered all the links’ MTU
>         when they choose the frame size. So the tunnel egress MTU
>         would be big enough for the original  packet size between HSrc
>         & HDst.
>
>
>     Consider a link such as ATM: the ATM cell size is not relevant,
>     only the ATM AAL5 reassembly payload size. That determines the MTU
>     because an IP packet that large can traverse an ATM link (with
>     multiple ATM link hops) without needing to be fragmented before
>     traversal.
>
>     relevant *to the packet arriving at the ingress*. The only thing
>     that is relevant is the largest payload the tunnel egress can
>     reassemble. The hops within the tunnel determine how the ingress
>     fragments, not whether a packet arriving at the ingress must be
>     dropped.
>
>
>
>          
>
>         -          because the ingress adds the outer header
>         encapsulation which may make the total  frame size chosen by
>         the original source to exceed the MTU of  some intermediate
>         links,
>
>
>     The overhead of these encapsulations is relevant only at the
>     egress reassembly; the effect on the individual fragments is not
>     relevant. IP doesn't care about the 5-byte overheads for ATM cells.
>
>
>
>         -          because the fragmented segments may be out of order
>         when arriving at the egress node of the tunnel, whereas over a
>         link the fragmented frames always arrive at egress port in the
>         same order (therefore the reassembly is much easier over the
>         link). 
>
>     I'll double-check tomorrow, but I'm fairly sure I meant to just
>     drop the "it" part. Again, it'll be fixed in the next rev.
>
>     Joe
>
>
>
>          
>
>         Linda
>
>          
>
>          
>
>         <mailto:[email-masked]>; [email-masked]
>         <mailto:[email-masked]>
>         *Cc:* [email-masked] <mailto:[email-masked]>
>         A link MTU is different from a path MTU, as explained in the
>         document. The MTU issues of a link are different than those of
>         a path, as described.
>
>
>
>
>
>             The MTU issue for tunnel is more like the MTU issue
>             between any two points in the network (i.e. traverse many
>             links, and can change over time), less like the MTU for a
>             single link because the single link MTU is only determined
>             by the fixed TWO endpoints of the link, whereas the MTU of
>             a tunnel (and a path) is impacted by many links in
>             between, i.e. more dynamic and can change depending which
>             path the packet traverse through.
>
>
>         Both a link and tunnel may have multiple hops not visible to
>         IP, each of which may have a different MTU; this is no
>         different than an IP path MTU. The difference is that the
>         link/tunnel hops are not visible to IP.
>
>
>
>
>              
>
>             In the section 1 of your draft, you stated:  “.. tunnels
>             emulate a link”.  To me, “tunnel” hide the inner
>             addresses, but the packets can traverse many links.  So
>             Tunnels hardly emulate a link.
>
>
>         There are many link layer technologies which use multiple
>         link-layer hops and relaying, including Ethernet, ATM, SONET, etc.
>
>         Joe
>
>
>
>      
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 09:05:34 -0700

Linda,

> determine if the frames need to be fragmented,

It does not. It uses the MTU of the path inside the tunnel to decide how
to fragment.

> then it will need immediate nodes to do further fragmentation. What if
> the immediate nodes can’t do fragmentation/reassembly?
>

Joe

> three routed links with independent MTUs of 1500, 800, and 1200 bytes,
> the smallest (800 bytes) must be assumed by each end host to avoid
> fragmentation.
>
> path_mtu.png
>
>  
>
>  
>
>  
>
> Linda
>
>  
>
> *Sent:* Friday, October 07, 2016 10:33 AM

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 7 Oct 2016 19:56:29 +0000

Hi, I just submitted a new version of the AERO spec that includes a section on
packet sizing issues that relates directly to this discussion.

Please review Section 3.12 of the AERO document and compare it to what
appears in intarea-tunnels. The two documents should be in agreement.

Thanks - Fred

---
A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : Asymmetric Extended Route Optimization (AERO)
        Author          : Fred L. Templin
	Filename        : draft-templin-aerolink-72.txt
	Pages           : 62
	Date            : 2016-10-07

Abstract:
   This document specifies the operation of IP over tunnel virtual links
   using Asymmetric Extended Route Optimization (AERO).  Nodes attached
   to AERO links can exchange packets via trusted intermediate routers
   that provide forwarding services to reach off-link destinations and
   redirection services for route optimization.  AERO provides an IPv6
   link-local address format that supports operation of the IPv6
   Neighbor Discovery (ND) protocol and links IPv6 ND to IP forwarding.
   Admission control and address/prefix provisioning are supported by
   the Dynamic Host Configuration Protocol for IPv6 (DHCPv6), while
   mobility management and route optimization are naturally supported
   through dynamic neighbor cache updates.  Although DHCPv6 and IPv6 ND
   messaging are used in the control plane, both IPv4 and IPv6 are
   supported in the data plane.  AERO is a widely-applicable tunneling
   solution especially well suited to mobile Virtual Private Networks
   (VPNs) and other applications as described in this document.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-templin-aerolink/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-templin-aerolink-72

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-templin-aerolink-72

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 16:24:51 -0700

Hi, Tom,

Thanks for pointing that out. I'll try to revise to make that easier.

If you have any other specific examples, please let me know (either
on-list or directly).

Thanks,

Joe

---

From: "Christian Huitema" <[email-masked]>
Date: Fri, 7 Oct 2016 21:57:35 -0700

On Friday, October 7, 2016 8:48 AM, Fred Templin wrote:
> ...

Fred, technically, you are correct. But let's go back to the argument developed by Kent and Mogul, "Fragmentation considered harmful." The core of the argument is that if the network loses a fragment, TCP has to retransmit the whole packet. It is always inefficient; if the stars are aligned just wrong, it can be extremely inefficient. In contrast, if there is no fragmentation, the unit of transmission is the unit of control, and TCP performs efficiently.

If each link on a path has a fixed MTU, MTU discovery will ensure that packets are not fragmented. If links have a variable MTU, you will want MTU discovery to discover the MTU "of the moment," so as to ensure TCP efficiency. Tunnels are links with variable MTU. A change in route could cause the tunnel MTU to drop from 9K to 1.5K. 

The two classic strategies for tunnel MTU is to ensure a lowest common denominator, or to use a reassembly buffer and go for some high MTU size. Neither is very satisfactory. The lowest common denominator approach foregoes the advantage of large MTU when they are actually available. The reassembly approach relies on intermediate fragmentation, with the corresponding retransmission inefficiency. 

My preferred solution would have the tunnel endpoints perform MTU discovery. They could track the tunnel MTU as it evolves over time, send ICMP "too big" messages when packets are too long, and effectively enable end-to-end MTU discovery.

-- Christian Huitema

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Oct 2016 23:21:50 -0700

Christian,

> On Oct 7, 2016, at 9:57 PM, Christian Huitema <[email-masked]> wrote:
> 

It is always less efficient than not fragmenting, but the loss issue matters only when the probability of losing an entire packet is much lower then the probability of losing one or more fragments. If congestion losses are correlated (drop one fragment means drop the rest of the packet) or non-congestion losses are spread out (so the likelihood of losing only any fragment of a packet the same as losing the packet if it were whole) then it doesn't matter. 

> 
> If each link on a path has a fixed MTU, MTU discovery will ensure that packets are not fragmented.
 If/when it works. PMTUD depends on ICMPs that are often blocked. Only PLMUTD will succeed in those cases. 

> If links have a variable MTU, you will want MTU discovery to discover the MTU "of the moment," so as to ensure TCP efficiency.

Yes, and any variation in the IP path can cause that change in path MTU.

> Tunnels are links with variable MTU. A change in route could cause the tunnel MTU to drop from 9K to 1.5K. 

Not necessarily.  A tunnel with static end points would not have a variable MTU because the MTU of the tunnel (not inside the tunnel, which is not visible to the transiting packet) is determined by.the semblance limit, not the path inside the tunnel.

> 
> The two classic strategies for tunnel MTU is to ensure a lowest common denominator, or to use a reassembly buffer and go for some high MTU size.

You are assuming it is critical to avoid fragmentation inside the tunnel. There are links that almost never avoid this (ATM). It's not critical unless the losses are dominant inside the tunnel and are impacted by the fragmentation (e.g., fragment losses result in higher packet losses than there would have been with unfragmented packets, as described above).

> Neither is very satisfactory. The lowest common denominator approach foregoes the advantage of large MTU when they are actually available. The reassembly approach relies on intermediate fragmentation, with the corresponding retransmission inefficiency. 

Many links live with this inefficiency. It's provably impossible to avoid this for IP over IP because of minimum MTU requirements.

> 
> My preferred solution would have the tunnel endpoints perform MTU discovery.

Sure - to guide the ingress source fragmentation (or not) of encapsulated packets.

> They could track the tunnel MTU as it evolves over time, send ICMP "too big" messages when packets are too long, and effectively enable end-to-end MTU discovery.

They should perform PLMTUD with the egress to figure out bout the atom MTU inside the tunnel and the egress reassembly limit - only the latter determines the tunnel MTU.

That tunnel MTU should be used by the node where the tunnel attaches to decide what to do - whether to fragment, whether to send an ICMP too-big (which will likely be dropped) or whatever. It is never the job of the tunnel to send these messages, though. The tunnel ingress is directly equivalent to a network interface and its MTU value.

All of this is already in the draft, FWIW.

Joe

---

From: t.petch <[email-masked]>
Date: Sun, 9 Oct 2016 11:05:15 +0100

----- Original Message -----
From: "Joe Touch" <[email-masked]>
Sent: Saturday, October 08, 2016 12:24 AM

Joe

How about

'   o  Tunnel internal MTU (TIMTU): the largest message that a tunnel
      egress can emit into a tunnel without requiring further
      fragmentation to reach the tunnel egress.' ?

You may have to read it twice to see my confusion.

More generally, I think that Fred's point is well made, that the
terminology, particularly the abbreviations thereof - e.g. LMTU, TIMTU,
TMTU, PMTU, ERMTU, RMTU - make the I-D harder to follow.

Tom Petch

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Oct 2016 16:26:35 +0000

Hi Christian,

> -----Original Message-----
> From: Christian Huitema [mailto:[email-masked]]
> Sent: Friday, October 07, 2016 9:58 PM
> To: Templin, Fred L <[email-masked]>; 'Joe Touch' <[email-masked]>; 'Linda Dunbar' <[email-masked]>
> Subject: RE: [Int-area] Questions to draft-intarea-tunnels-03
> 
> Fred, technically, you are correct. But let's go back to the argument developed by Kent and Mogul, "Fragmentation considered
> harmful." The core of the argument is that if the network loses a fragment, TCP has to retransmit the whole packet. It is always
> inefficient; if the stars are aligned just wrong, it can be extremely inefficient. In contrast, if there is no fragmentation, the unit of
> transmission is the unit of control, and TCP performs efficiently.

Fragmentation considered harmful was written in 1987, and many of the arguments
made there have been overcome by modern technology. You are pointing out
perhaps the only argument that still applies in the modern era, but even so packet
loss is far less prevalent along most paths now than it used to be "back in the day".

> If each link on a path has a fixed MTU, MTU discovery will ensure that packets are not fragmented. If links have a variable MTU, you
> will want MTU discovery to discover the MTU "of the moment," so as to ensure TCP efficiency. Tunnels are links with variable MTU. A
> change in route could cause the tunnel MTU to drop from 9K to 1.5K.

We don't want that - we want the tunnel MTU to remain fixed even if the path
within the tunnel is subject to route changes. Routing changes are a fact of life
in some tunneling systems, where we don't want to see the tunnel MTUs
continuously bouncing up and down. That would be a nightmare scenario for
hosts that use RFC4821 in an attempt to cope with dynamic path changes. 

> The two classic strategies for tunnel MTU is to ensure a lowest common denominator, or to use a reassembly buffer and go for some

Need to define high. Let's start with 1500 bytes and see if we can address that
size, then only consider higher MTUs once we are satisfied with the 1500.

With a 1500 tunnel MTU, and with a minimum link MTU of 1280 bytes,
fragmentation would only be applied for packets between (1280-ENCAPS)
and 1500. For packets in that size range, we would send a 2-fragment tunnel
packet. In that case, especially if the probability that a single packet will arrive
is very high, then the probability that both packets will arrive is still high.

Also, when you say "lowest common denominator MTU", that depends
very much on the environment within which the tunnel is configured. If
the LCD MTU is assured to be something like 1500, then sure you can
set the tunnel MTU to (1500-ENCAPS) without fragmentation. But, if
there is no assurance, then the LCD MTU has to be assumed to be 1280.
In that case, even 1280 original packets would need to be fragmented
into a 2-fragment because of the ENCAPS overhead.

> Neither is very satisfactory. The lowest common denominator approach foregoes the advantage of large MTU when
> they are actually available. The reassembly approach relies on intermediate fragmentation, with the corresponding retransmission
> inefficiency.
> 
> My preferred solution would have the tunnel endpoints perform MTU discovery. They could track the tunnel MTU as it evolves over
> time, send ICMP "too big" messages when packets are too long, and effectively enable end-to-end MTU discovery.

PTB messages may be lost in the network, spoofed by an adversary, or not
have enough information to translate an L2 PTB into an L3 PTB. Especially
due to the loss issue, we therefore cannot rely on RFC1191/RFC1981
PMTUD. Also, if a routing change causes the path MTU to go low, it may
be some time before a larger MTU can be discovered if the path MTU
again goes high. And, that is only if the source has the smarts to retest
for larger MTUs after it has received a PTB reporting a smaller MTU.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Oct 2016 10:29:05 -0700

Hi, Tom,

> You may have to read it twice to see my confusion.

Yes - that's a typo that should read as follows, which should be more clear:

   o  Tunnel internal MTU (TIMTU): the largest message that a tunnel
      ingress can emit into a tunnel without requiring further
      fragmentation to reach the tunnel egress. 

> TMTU, PMTU, ERMTU, RMTU - make the I-D harder to follow.

The challenge is that it's nearly impossible to discuss these issues
without coining new terms because the existing ones are so poorly
understood in the context of layering.

We generally care about many different MTUs:
    - the largest IP packet that can traverse an IP path
        this is NOT the path MTU, but rather the receiver reassembly MTU
        we CANNOT transmit transport segments larger than the payload of
this IP E2E MTU

    - the largest IP packet that can traverse an IP path without needing
on-path fragmentation
        this is the path MTU
        it is critical for IP source fragmentation (or the packets won't
get there at all)
        it's an optimization for transports that frag/reassemble (to
avoid source fragmentation)

Neither of these has anything to do with the link MTU, which has similar
variants.

As we keep discussing this issue, we've come up with various notions of
what these names could be.

We could call these the:
    IP transit MTU  (defined as the source-to-dest MTU, but equal to the
IP receiver reassembly MTU)
    IP path MTU (min of the IP hop MTUs)
    IP hop MTU = link/tunnel transit MTU - link/tunnel overhead

    link (or tunnel) transit MTU (defined as source-to-dest MTU, equal
to link/tunnel receiver reassembly MTU)
    link path MTU (min of the link hop MTUs)
    link hop MTU = MTU of a single physical link

I'm glad to revise if these are more clear.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Oct 2016 17:58:11 +0000

Hi Joe,

> Sent: Monday, October 10, 2016 10:29 AM
> To: t.petch <[email-masked]>; Templin, Fred L <[email-masked]>
> Cc: [email-masked]

IMHO, your draft is getting wrapped up with too many new acronyms for no
good reason when simple English-language text would make things easier to
understand. See Section 3.12 of 'draft-templin-aerolink' for simple and easy
to understand text.

In 'draft-templin-aerolink', there are only five acronyms:

  MTU - the MTU of the tunnel as seen by the IP layer
  MFU - the maximum fragment size within the tunnel
  MRU - the maximum reassembly size supported by the egress
  ENCAPS - the size of the encapsulation headers inserted by the ingress
  PTB - an IP Packet Too Big message

Everything else is just simple English language, and does not need any
other acronyms.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Oct 2016 11:15:00 -0700

HI, Fred (et al.),

...
> understand. 
We're dealing with legacy use of the term MTU and path MTU too, though.

>   MTU - the MTU of the tunnel as seen by the IP layer

That's the term we need to specify in more detail - to some, it's
one-hop IP MTU, to others it means path MTU, to others it means what we
would call egress reassembly MTU.

We need to either avoid the term MTU in our definitions or provide an
adjective to specify each type (as I have done).

>   MFU - the maximum fragment size within the tunnel
This is confusing because the fragment could refer to the payload or the
entire IP datagram.

>   MRU - the maximum reassembly size supported by the egress
Sure.

>   ENCAPS - the size of the encapsulation headers inserted by the ingress
Sure.

>   PTB - an IP Packet Too Big message
ICMPv4 or IPv6...

You have omitted the concept of a transit MTU, which is only the ERU
when the ingress supports fragmentation and reassembly (which not all
tunnels or links do).

Therein lies the problem with "simple" English, IMO.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 10 Oct 2016 18:28:19 +0000

Hi Joe,

Please run through my text, and you will see that it works. This does not take
anything away from the fact that you are the originator of the concept (i.e., it
cites 'intarea-tunnels' as the first order of business) but it puts things in simple
terms that are easy for an implementer to understand.

Thanks - Fred

> Sent: Monday, October 10, 2016 11:15 AM
> To: Templin, Fred L <[email-masked]>; t.petch <[email-masked]>
> Cc: [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Oct 2016 18:08:23 -0700

Fred,

I'll take a look and chat with Mark about this. The goal is clarity,
regardless of how we get there.

Joe

---

From: t.petch <[email-masked]>
Date: Tue, 11 Oct 2016 11:05:58 +0100

----- Original Message -----
From: "Joe Touch" <[email-masked]>
Sent: Monday, October 10, 2016 7:15 PM

Joe

I echo Fred's point about acronyms; yes you need adjectives to qualify
MTU but when I read
' The TMTU (d) is not limited by TIMTU (e), but by ERMTU (f), the
   tunnel equivalent of RMTU (c). '
I have to pause, expand and think.  Spelling out the adjectives every
time would make it easier.

Likewise, when 'MTU' or 'protocol' or some such is unqualified, I have
to stop and think which is meant.  I think that most uses of MTU should
have an adjective or two in front of them.

I would say the same of protocol.  At times, the I-D seems specific,
that this is IP in IP and so protocol is IP but other usages seem more
generic.  After all, you can tunnel Ethernet in ATM and while I think
that most of this I-D does not cater for that, I think there are some
places where 'protocol' is not limited to IP so if IP protocol (or IPv4
protocol or IPv6 protocol) is intended, then I would write that
explicitly.

Tom Petch

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Oct 2016 08:22:14 -0700

Hi, Tom,

I do agree that the terminology in the current version is confusing and
it would be useful to be both more clear and more consistent.

As I noted, I'll discuss this with Mark further and revise.

Joe

---

From: t.petch <[email-masked]>
Date: Wed, 12 Oct 2016 10:32:07 +0100

Joe

Looking some more, I think that the carefully defined terminology is not
then used so carefully!

s.2.2 defines the term 'forwarder' but by 3.5, 'router' moves in and
forwarder is never used again.  Is this of technical significance, or
just a question of the style of the author(?s)?  I think the latter, but
see it as introducing uncertainty.

s.2.2 defines Link MTU, Path MTU, Tunnel MTU but the document then uses
link MTU, path MTU, tunnel MTU.  A common convention is for capitalised
words to have a particular meaning, lower case not (a bit like SHOULD
and should) so is this of technical significance, or just a question of
the style of the author(?s)?  I think the latter, but see it as
introducing uncertainty.

s.2.2 defines ingress as
     'the virtual link interface of a tunnel which receives
      messages within a node, encapsulates them according to the tunnel
      protocol, and transmits them into the tunnel. '
which I think of as rather a lot for an interface to do but equally not
all that needs doing, such as finding a path, selecting an interface,
finding link and path MTU, inspecting the packet for such as DF=1,
fragmenting when called for and so on.
It could be that ingress is intended to cover all these box functions.
(I would like to use the term 'node' here but that has been defined as
'a device that can act as an endpoint or forwarder, which does not seem
to fit here).

But reading on, in figure 4, I see the box that has everything to do
labelled I(ngress) which suggests that Ingress includes everything.  But
when I get to s.3.3, I encounter 'ingress nodes' (um oxymoron?) and then
'source (ingress) ' - what does 'source' add that 'ingress' does not
already have? - and then I get a mixture in s.3.6 of 'ingress' and
'tunnel ingress'  - what does 'tunnel' add that ingress is not already
defined to have?

I think you get my laboured point.  Technical writing, for me, is
boring.  Get a set of concepts, as small as is necessary, give them an
identifier, easy to read, write, say as you read, and use them over and
over again; boring, but easy to understand.

(If I look at MTU, I expect I will find the same only more so).

Tom Petch

----- Original Message -----
From: "Joe Touch" <[email-masked]>
To: "t.petch" <[email-masked]>; "Templin, Fred L"
<[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, October 11, 2016 4:22 PM

> >> one-hop IP MTU, to others it means path MTU, to others it means
what
> > we
> >> This is confusing because the fragment could refer to the payload
or
> > the

---

From: Joe Touch <[email-masked]>
Date: Wed, 12 Oct 2016 07:02:55 -0700

Hi, Tom,

> Looking some more, I think that the carefully defined terminology is not
I agree; we have been asking for this sort of feedback for several
years. I'm glad to finally receive it, though I would note that the
focus has been on converging on the details of advice first. Now that
we've done that, it's very useful to get this feedback so we can update
the doc.

> see it as introducing uncertainty.
It's used in 2.2 because of the confusion that arises when we talk about
hosts and routers. Many documents (and entire bodies of IETF work)
erroneously imply or assume that these are mutually-exclusive functions.
Most devices we call routers include all the requirements of being a
host because control protocols originate or terminate at that node.
"Forwarding" is intended to describe the exclusive functions a router
performs that a host does not.

The Internet architecture is based on the concept of a host, a
forwarder, and a link, but the Internet is implemented with devices of a
host, a router (which is both a forwarder and a host), and a link.

It's important to discuss this issue, which is why it is grounded in the
definitions. I will add text to explain why the rest of the doc refers
to the device (router) rather than the function (forwarding).

> and should) 
It's also convention that sentences start with capital letters. In this
case, all list items start with a capital letter as well.
> so is this of technical significance, or just a question of
don't think it's confusing, and all the terms defined have the same
issue, but it seems to be consistent (e.g., we don't use Path MTU
anywhere else except at the beginning of a sentence or a bullet). I'll
doublecheck that.

> fragmenting when called for and so on.

That's exactly correct - the other functions you describe are the
responsibility of the originating host or forwarder (router).

And that's not a lot for a link; that's exactly what all link layers do.

> 'tunnel ingress'  - what does 'tunnel' add that ingress is not already
Good point to clarify.

The clarification will be intended to highlight the following:
    - a tunnel ingress is an INTERFACE, not a node
    - a node where the tunnel ingress is attached is just a node using a
link

So in the diagrams, the nodes should be labeled "ingress node" and that
term defined as "a node on which the ingress interface is attached".

> over again; boring, but easy to understand.
I agree, but it should be as simple as we can while still being clear.
It's tempting to have fewer variants of the different MTU values or to
merge the idea of a tunnel ingress as interface with an ingress as node,
but then we lose exactly the distinction that lets us correctly reason
about their role in the Internet architecture.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 12 Oct 2016 16:46:05 +0000

Hi Joe,

> Sent: Wednesday, October 12, 2016 7:03 AM

Yes, the time has come - time to move this work forward.

Thanks - Fred