Subject: [trill] Fwd: Mail regarding draft-ietf-trill-over-ip

From: Donald Eastlake <[email-masked]>
Date: Fri, 1 May 2015 12:58:15 -0400

Forwarded with permission.

Thanks,
Donald
---------- Forwarded message ----------
From: Donald Eastlake <[email-masked]>
Date: Tue, Apr 28, 2015 at 9:26 AM
Subject: Re: Mail regarding draft-ietf-trill-over-ip
To: Mingui Zhang <[email-masked]>

Hi Mingui,

Thanks for these comments! See below.

On Tue, Apr 28, 2015 at 4:27 AM, Mingui Zhang <[email-masked]>
wrote:
> Hi,
>
> I read the document. It's comprehensive and well written. Below, several
comments for your information.
>
> 1.      It's not clear how the ports IPs are associated with the ports?
Maybe, we can add some words to explain that they can be got from DHCP or
manual configuration? Or we just say it is out the scope of this document.

Yes, they need to be configured. Could be DHCP or manual or maybe some
sort of orchestration thing... Seems reasonable to mention this in the
draft.

> 2.      Is it helpful to add a reference topology? Terminologies, such as
IP tunnel, port IPs, RBridges can be put onto this figure. A walk-through
example based on this reference topology can be used to explain how the IP
tunnel is set up, how does a TRILL Data packet get
encapsulated/decapsulated and transported in the IP tunnel. I think this
would be educational.

A few more network diagrams would probably be helpful. If you look at
the minutes from the Dallas TRILL WG meeting, the suggestion of having
a couple of example packets was supported...

> 3.      Both IP and TRILL have specified BFD. Since TRILL is dependent on
IP in TRILL-over-IP, it's unnecessary to have both TRILL and IP interact
with BFD. It's best to assert TRILL-over-IP will have the IP interact with
BFD. Please refer to https://tools.ietf.org/html/rfc5882#section-4.4

Well, if you are only going to use one then I agree with the section
you reference in RFC 5882 and you should do BFD over IP. But that
doesn't check the TRILL stack, just the IP and lower stacks. So we
could recommend just using IP BFD but I don't think we should try to
prohibit people from also using BFD over TRILL on the link.

> 4.      Is the IP link in this document "a single link (physical, or a
secure tunnel such as IPsec)"? Then, we can require the TTL "MUST be set to
the maximum on transmit, and checked to be equal to the maximum value on
reception (and the packet dropped if this is not the case)." See also RFC
5880 Section 9.

I don't think so. There is nothing wrong with the communication
between two TRILL IP ports being multiple IP hops. Even if IPsec is in
use, it could be integrated with the TRILL over IP port at one end but
at the other end, the IPsec implementation could be integrated with a
firewall a couple of hops from the RBridge...

> 5.      There are six tiny typos marked in the attached doc.

OK. We'll fix this up in the next version.

Maybe you should post these comments, or some of them, to the TRILL WG
mailing list. It would be good if there was more discussion of drafts
there. Or if it OK with you, I could just forward your comments and my
responses to the list...

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

> Thanks,
> Mingui

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 May 2015 12:48:00 -0700

Hi, all,

Have you considered GUE as an encapsulation layer?

Encapsulating anything in UDP directly has a number of hazards,
including support for at-rate fragmentation, IPv4 ID generation, etc.,
that GUE is intended to address.

Joe

On 5/1/2015 9:58 AM, Donald Eastlake wrote:
> From: *Donald Eastlake* <[email-masked] <mailto:[email-masked]>>
> To: Mingui Zhang <[email-masked] <mailto:[email-masked]>>
> 
> Hi Mingui,
> 
> On Tue, Apr 28, 2015 at 4:27 AM, Mingui Zhang <[email-masked]
> <mailto:[email-masked]>> wrote:
>> Hi,
>>
>> I read the document. It's comprehensive and well written. Below, several comments for your information.
>>
>> 1.      It's not clear how the ports IPs are associated with the ports? Maybe, we can add some words to explain that they can be got from DHCP or manual configuration? Or we just say it is out the scope of this document.
> 
>> 2.      Is it helpful to add a reference topology? Terminologies, such as IP tunnel, port IPs, RBridges can be put onto this figure. A walk-through example based on this reference topology can be used to explain how the IP tunnel is set up, how does a TRILL Data packet get encapsulated/decapsulated and transported in the IP tunnel. I think this would be educational.
> 
>> 3.      Both IP and TRILL have specified BFD. Since TRILL is dependent on IP in TRILL-over-IP, it's unnecessary to have both TRILL and IP interact with BFD. It's best to assert TRILL-over-IP will have the IP interact with BFD. Please refer to https://tools.ietf.org/html/rfc5882#section-4.4
> 
>> 4.      Is the IP link in this document "a single link (physical, or a secure tunnel such as IPsec)"? Then, we can require the TTL "MUST be set to the maximum on transmit, and checked to be equal to the maximum value on reception (and the packet dropped if this is not the case)." See also RFC 5880 Section 9.
> 
>  Donald E. Eastlake 3rd   +1-508-333-2270 <tel:%2B1-508-333-2270> (cell)
>  [email-masked] <mailto:[email-masked]>
> 
>> Thanks,
>> Mingui
> 
> 
> 
> _______________________________________________
> trill mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/trill
>

---

From: Donald Eastlake <[email-masked]>
Date: Fri, 1 May 2015 21:58:22 -0400

Hi Joe,

> Have you considered GUE as an encapsulation layer?

What would be the best document to look at if someone wanted to learn about GUE?

> that GUE is intended to address.

It seems that a number of encapsulations being developed. Thus far, I
believe the preference of the strong TRILL WG has been that
encapsulation(s) with fast path support should be supported. Since the
encapsulation(s) with such support is likely to change, the direction
the WG is headed in is a provision for adjacent TRILL switches to
negotiate the encapsulation used between them. I imagine it would be
reasonable for GUE to be one available option.

Thanks,
Donald

---

From: Donald Eastlake <[email-masked]>
Date: Fri, 1 May 2015 22:03:15 -0400

This time with (most?) typos and grammatical errors fixed...

 Hi Joe,

> that GUE is intended to address.

It seems that a number of encapsulations are being developed. Thus
far, I believe the strong preference of the TRILL WG has been that
encapsulation(s) with fast path support should be available. Since the
encapsulation(s) with such support are likely to change, the direction
the WG is headed is a provision for adjacent TRILL switches to
negotiate the encapsulation used between them. I imagine it would be
reasonable for GUE to be one available option.

 Thanks,
 Donald

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 4 May 2015 03:19:19 +0000

Hi Joe,

I'm wondering whether your proposal as below is also applicable to other UDP-based encapsulation approaches which have not yet considered doing fragmentation on the tunnel layer, such as GENEVE, VXLAN-GPE, GRE-in-UDP and NSH-UDP.

Best regards,
Xiaohu

> -----Original Message-----
> From: trill [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Saturday, May 02, 2015 3:48 AM
> To: Donald Eastlake; [email-masked]
> Subject: Re: [trill] Fwd: Mail regarding draft-ietf-trill-over-ip
> 
> Hi, all,
> 
> Encapsulating anything in UDP directly has a number of hazards, including
> support for at-rate fragmentation, IPv4 ID generation, etc., that GUE is intended
> to address.
> 
> Joe
> 
> >> 1.      It's not clear how the ports IPs are associated with the ports? Maybe,
> we can add some words to explain that they can be got from DHCP or manual
> configuration? Or we just say it is out the scope of this document.
> >
> >> 2.      Is it helpful to add a reference topology? Terminologies, such as IP
> tunnel, port IPs, RBridges can be put onto this figure. A walk-through example
> based on this reference topology can be used to explain how the IP tunnel is set
> up, how does a TRILL Data packet get encapsulated/decapsulated and
> transported in the IP tunnel. I think this would be educational.
> >
> >> 3.      Both IP and TRILL have specified BFD. Since TRILL is dependent on IP
> in TRILL-over-IP, it's unnecessary to have both TRILL and IP interact with BFD. It's
> best to assert TRILL-over-IP will have the IP interact with BFD. Please refer to
> https://tools.ietf.org/html/rfc5882#section-4.4
> >
> >> 4.      Is the IP link in this document "a single link (physical, or a secure
> tunnel such as IPsec)"? Then, we can require the TTL "MUST be set to the
> maximum on transmit, and checked to be equal to the maximum value on
> reception (and the packet dropped if this is not the case)." See also RFC 5880
> Section 9.
> >
> >  155 Beaver Street, Milford, MA 01757 USA  [email-masked]
> > <mailto:[email-masked]>
> >
> >> Thanks,
> >> Mingui
> >
> >
> >

---

From: Joe Touch <[email-masked]>
Date: Mon, 04 May 2015 09:08:09 -0700

Someone else just posted the needed info...

---

From: Joe Touch <[email-masked]>
Date: Mon, 04 May 2015 09:12:29 -0700

On 5/3/2015 8:19 PM, Xuxiaohu wrote:
> Hi Joe,
> 
> I'm wondering whether your proposal as below is also applicable to
> other UDP-based encapsulation approaches which have not yet considered
> doing fragmentation on the tunnel layer, such as GENEVE, VXLAN-GPE,
> GRE-in-UDP and NSH-UDP.

Again:

We know of at least four things that tunnels need that IP-in-UDP ignores:

	- stronger checksums

	- fragmentation support
	
	- signalling support (e.g., to test whether a tunnel is up or
	to measure MTUs)

	- support for robust ID fields (related to fragmentation,
	e.g., to overcome the limits of IPv4 ID as per RFC 6864)

I haven't scrubbed GUE to ensure that all four are addressed, but it may
be more than the above, and certainly it's important not to start from
scratch needlessly.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 5 May 2015 02:23:04 +0000

Hi Joe,

> From: Joe Touch [mailto:[email-masked]]
> Sent: Tuesday, May 05, 2015 12:12 AM
> To: Xuxiaohu; Donald Eastlake; [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]

I'm just wondering whether the above is only applicable to IP-in-UDP or applicable to all the other UDP-based encapsulations as well.

> I haven't scrubbed GUE to ensure that all four are addressed, but it may be more
> than the above, and certainly it's important not to start from scratch needlessly.

I don't intend addressing items 1), 2) and 3).  As for item 3), I would add the following text:
  
"Ingress AFBRs MUST NOT fragment I-IP [RFC5565] packets (i.e., UDP encapsulated IP packets), and when the outer IP header is IPv4, ingress AFBRs MUST set the DF bit in the outer IPv4 header. It is strongly RECOMMENDED that I-IP [RFC5565] transit core be configured to carry an MTU at least large enough to accommodate the added encapsulation headers. Meanwhile, it is strongly RECOMMENDED that Path MTU Discovery [RFC1191][RFC1981] is used to prevent or minimize fragmentation." 

Ahead of the following existing text in Section 4:

   "If an ingress AFBR performs fragmentation on
   an E-IP packet before encapsulating, it MUST use the same source UDP
   port for all fragmented packets so as to ensures these fragmented
   packets are always forwarded on the same path."

In a word, IP-in-UDP is just intended for those network environments where fragmentation on the tunnel layer and strong checksums are not desired. For those network environments where fragmentation on the tunnel layer and stronger checksums are required, GUE should be considered instead.

Best regards,
Xiaohu

> >> including support for at-rate fragmentation, IPv4 ID generation,
> >> etc., that GUE is intended to address.
> >>
> >> Joe
> >>
> >>> On Tue, Apr 28, 2015 at 4:27 AM, Mingui Zhang
> >>> <[email-masked] <mailto:[email-masked]>> wrote:
> >>>> Hi,
> >>>>
> Maybe,
> >> we can add some words to explain that they can be got from DHCP or
> >>> Yes, they need to be configured. Could be DHCP or manual or maybe
> >>> some sort of orchestration thing... Seems reasonable to mention this
> >>> in the draft.
> >>>
> >> tunnel, port IPs, RBridges can be put onto this figure. A
> >> walk-through example based on this reference topology can be used to
> >> explain how the IP tunnel is set up, how does a TRILL Data packet get
> >>> A few more network diagrams would probably be helpful. If you look
> >>> at the minutes from the Dallas TRILL WG meeting, the suggestion of
> >>> having a couple of example packets was supported...
> >>>
> IP
> >> in TRILL-over-IP, it's unnecessary to have both TRILL and IP interact
> >> with BFD. It's best to assert TRILL-over-IP will have the IP interact
> >> with BFD. Please refer to
> >> tunnel such as IPsec)"? Then, we can require the TTL "MUST be set to
> >> the maximum on transmit, and checked to be equal to the maximum value
> >> on reception (and the packet dropped if this is not the case)." See
> >> also RFC 5880 Section 9.
> >>>
> >>> between two TRILL IP ports being multiple IP hops. Even if IPsec is
> >>> in use, it could be integrated with the TRILL over IP port at one
> >>> end but at the other end, the IPsec implementation could be
> >>> integrated with a firewall a couple of hops from the RBridge...
> >>>
> >>> Maybe you should post these comments, or some of them, to the TRILL
> >>> WG mailing list. It would be good if there was more discussion of
> >>> drafts there. Or if it OK with you, I could just forward your
> >>> comments and my responses to the list...
> >>>
> >>> Thanks,
> >>> Donald

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 14:54:13 +0000

Hi,

> From: Int-area [mailto:[email-masked]] On Behalf Of Xuxiaohu
> Sent: Monday, May 04, 2015 7:23 PM
> To: Joe Touch; Donald Eastlake; [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Int-area] [trill] Fwd: Mail regarding draft-ietf-trill-over-ip
> 
> Hi Joe,
> 
> "Ingress AFBRs MUST NOT fragment I-IP [RFC5565] packets (i.e., UDP encapsulated IP packets), and when the outer IP header is IPv4,
> ingress AFBRs MUST set the DF bit in the outer IPv4 header. It is strongly RECOMMENDED that I-IP [RFC5565] transit core be
> configured to carry an MTU at least large enough to accommodate the added encapsulation headers. Meanwhile, it is strongly
> RECOMMENDED that Path MTU Discovery [RFC1191][RFC1981] is used to prevent or minimize fragmentation."
> 
> In a word, IP-in-UDP is just intended for those network environments where fragmentation on the tunnel layer and strong checksums
> are not desired. For those network environments where fragmentation on the tunnel layer and stronger checksums are required,
> GUE should be considered instead.

I thought we determined the IP-in-UDP is just GUE with header compression?

Thanks - Fred
[email-masked]

> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 09:32:51 -0700

On 5/4/2015 7:23 PM, Xuxiaohu wrote:
> In a word, IP-in-UDP is just intended for those network environments
> where fragmentation on the tunnel layer and strong checksums are not
> desired.

That's insufficient. They are only applicable where fragmentation and a
strong checksum are not *needed*.

Once you run IP in IP (IP in UDP in IP qualifies as this), you have only
two choices:

	- support fragmentation

	- use in networks that are engineered so that
	fragmentation is never needed

As to the strong checksum, similarly you have to either support one or
deploy the result where that checksum isn't needed - either because you
know that all apps will have strong enough checksums of their own or
because you know enough about the kinds of errors that will occur that
strong checksums aren't needed.

But the key there is to define a use case where these properties are
true AND to limit the document solution to uses in those case ONLY.

> For those network environments where fragmentation on the
> tunnel layer and stronger checksums are required, GUE should be
> considered instead.

Agreed.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 09:34:58 -0700

> I thought we determined the IP-in-UDP is just GUE with header compression?

IP in UDP adds only port numbers and an Internet checksum.

That doesn't address fragmentation; if outer fragmentation is assumed,
IPv4 needs to be rate-limited to avoid ID collisions and the Internet
checksum is insufficient to correct those collisions.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 16:39:39 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 9:35 AM
> To: Templin, Fred L; Xuxiaohu; Donald Eastlake; [email-masked]

Right - that is why we have GUE. But, when these functions are not
needed GUE can perform header compression and the result looks
exactly like IP in UDP.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 10:53:32 -0700

> Hi Joe,
..
> needed GUE can perform header compression and the result looks

That seems impossible.

The outer IP header indicates UDP as next-protocol, and GUE based on the
port number.

You can't then compress the GUE header to nothing. You still need at
least one bit somewhere to indicate "compressed GUE header", and there's
nothing left.

And no, I don't think "compressed GUE" qualifies as an independently
useful service that warrants a separate UDP port.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 10:58:16 -0700

On 5/5/2015 10:18 AM, Dave Dolson wrote:
> Joe,
> I think this discussion is about fragmenting the outer IP packet.

That's feasible for IPv6, but extremely limiting for IPv4 because of the
limited ID field size (it would rate-limit the tunnel significantly).

> There is a 3rd choice: to fragment the inner datagram,

That is not valid for IPv6 or IPv6 DF=1, so unless you support only IPv4
DF=0 packets, it won't work.

> and/or use ICMP "datagram is too big" messages to facilitate
> PMTU discovery of the tunnel.

That won't work for several reasons:
	- ICMPs are often blocked
	- ICMP might not return enough of the nested headers
	- ICMP cannot validly say "1280 is too big" for IPv6
	(which it would need to do if the path supported only
	1280, because there wouldn't be space for the additional
	IP header)

> This certainly seems to have been the intent, when one reads about
> the IPv4 DF flag. (And IPv6 always requires DF).

See above; while the DF issue might be useful in some contexts, it
doesn't magically get around the need to fragment in ways that are not
supported by the inner header.

Joe

> From: sfc [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Tuesday, May 05, 2015 12:33 PM
> Subject: Re: [sfc] [trill] Fwd: Mail regarding draft-ietf-trill-over-ip
> 
> 
> 
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 18:04:51 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 10:54 AM

Not impossible - Tom Herbert provided the solution:

http://www.ietf.org/mail-archive/web/int-area/current/msg04593.html
 
> The outer IP header indicates UDP as next-protocol, and GUE based on the

Yes.

> least one bit somewhere to indicate "compressed GUE header", and there's

False and True. False that you can't compress the GUE header to nothing
(you can) and True that there is one bit to indicate "compressed". But,
that bit coincides with the direct IP encapsulation header.

> useful service that warrants a separate UDP port.

Right - there is only one UDP port number (GUE).

Thanks - Fred

---

From: Tom Herbert <[email-masked]>
Date: Tue, 5 May 2015 11:06:13 -0700

> least one bit somewhere to indicate "compressed GUE header", and there's
As I described previously, the first two bits of GUE header are
version number. If we reserve version 0x1, then an IPv6 or IPv4 can be
directly encapsulated on the same port with GUE (version 0)-- this is
what Fred means by GUE header compression. No new port is needed for
this and it is backwards compatible with GUE.

> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 11:26:09 -0700

> http://www.ietf.org/mail-archive/web/int-area/current/msg04593.html

That is allocating bits (or bit patterns) from the IP header.

The solution provided - to check for 0x01 - is incorrect. IP can have
versions that include 0x10 and 0x11.

The only solution would be to say that if the first three bits were 0,
then it's not an IP packet - but that would require reassigning 0x0000
and 0x0001 for GUE purposes.

Although that's possible, I don't see why we would allocate IP versions
to GUE message types.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 5 May 2015 11:45:50 -0700

> versions that include 0x10 and 0x11.
It is correct as we defined it-- this is a solution to support direct
encapsulation of only IPv4 and IPv6. This optimizes encapsulation
those two IP protocols, and does not effect encapsulation of other
protocols (including new versions of IP) using the GUE and header and
protocol field. Whether this optimization is worth it is a fair
question, but if this obviates the need for a separate port number in
UDP-IP maybe it would be.

> Although that's possible, I don't see why we would allocate IP versions
We're not proposing that at all.

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 18:47:30 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 11:26 AM

The version field in both IPv4 and IPv6 have that bit set to 1. If GUE
then deems that bit to indicate "direct IP encapsulation, then there
is no need for a GUE header of length greater than 0.

You may say that future IP protocol versions might not have that bit
set in the version field. But, the version bits for IPv4 and IPv6 will
never change (by definition) and we do not see a new IP protocol
version replacing IPv4 or IPv6 on the near-term horizon.

Even if a new IP protocol version emerged with the "direct IP
encapsulation" bit set to 0, that version can still be accommodated
by GUE. It's just that direct encapsulation cannot be used and a
non-zero-length GUE header is needed.

Thanks - Fred

---

From: Tom Herbert <[email-masked]>
Date: Tue, 5 May 2015 11:53:21 -0700

> non-zero-length GUE header is needed.
Or just define a simple version translation as part of encapsulation.
So for IPv8:

0x1000->0x0101 on encapsulation
0x0101->0x1000 on decapsualtion

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 12:29:28 -0700

> We're not proposing that at all.

Then the solution is incorrect. As noted above, if you don't know you
"own" the patterns, you don't know you can use them to indicate
"non-compressed GUE header" because they are valid uncompressed IP packets.

A field you don't reserve isn't yours to assume.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 12:33:17 -0700

> version replacing IPv4 or IPv6 on the near-term horizon.

0x01 is valid for four version numbers:
	4
	5
	6
	7

You cannot assume either that 0x01 always means IPv4 and IPv6 only OR
that any other pattern is NOT an IP packet UNLESS GUE is assigned the
values you are assuming (e.g., 0x00, 0x10, 0x11), and that's not going
to happen.

As I said on the other post:

	Codepoints you aren't assigned aren't yours to assume.

> non-zero-length GUE header is needed.

It's more than that - you need to ensure that all other IP values are
NOT interpreted as uncompressed.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 12:34:31 -0700

> 0x0101->0x1000 on decapsualtion

And what happens to 0x0101 WHEN it shows up?

You need more patterns than you have because IP is allowed to use any of
them.

That's why you need an extra bit, and that's why you can't assume these
bits are "yours".

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 12:41:33 -0700

> You need more patterns than you have because IP is allowed to use any of

FWIW, you're essentially arguing for bit-stuffing, i.e., if a pattern
you think will be common shows up, don't stuff, and if something else
shows up, then do.

That works only if you do true stuffing, e.g.,:

	0x01** = uncompressed
(might as well add 0x11** and 0x10** to that)
	0x00** = compressed

But then if 0x0000, 0x0001, 0x0010 or 0x0011 show up, you need to decide
how to represent them - they CANNOT be uncompressed without at least two
more bits somewhere else that indicates they're uncompressed and their
value.

So now your GUE methods are very sensitive to IP version numbers, which
IMO defeats the "G" in the name. Never mind how this complicates
hardware when (not if) other IP versions are used (for whatever reason).

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 19:42:41 +0000

Hi Joe,

> 0x01 is valid for four version numbers:

Sure.

> values you are assuming (e.g., 0x00, 0x10, 0x11), and that's not going

You are making this way more complicated than it really is. 0xX1XX means
that this is an IP header - at this moment, either IPv4 or IPv6. But, having
checked the bit the next step is to test the entire nibble to ensure that it
is indeed either 0x0100 or 0x0110. And, if it is neither, the packet is
dropped due to a malformed header.

But, 0xX0XX means that the GUE header is present. There is no ambiguity
here, and it just works.

> NOT interpreted as uncompressed.

Again, you are twisting something that is very simple to make it appear
complicated when in fact it is not.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 19:49:00 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 12:42 PM
> To: Tom Herbert; Templin, Fred L
> Cc: Xuxiaohu; Donald Eastlake; [email-masked]; [email-masked]; [email-masked]; [email-masked]

No; it is still Generic. If the packet is not a direct-mapped IP packet, then there
is a GUE header that can encapsulate any (foo) protocol. True that IP protocol
versions that can be represented as 0xX1XX can be directly mapped while
other (foo) protocols cannot does not mean that we should not take
advantage of the direct mapping when possible.

About h/w complications, the test is incorporated in the same test that
hardware already uses to check the IP version number, i.e., no new
instructions needed.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 14:07:46 -0700

> dropped due to a malformed header.

OK, you can declare that. But you are then declaring that the other IP
versions with the same pattern cannot be transmitted as uncompressed GUE.

> But, 0xX0XX means that the GUE header is present. There is no ambiguity

You can *declare* that 0xX0XX means a GUE header, but then you are also
declaring that 0xX0XX IP versions cannot occur as uncompressed GUE
encapsulations.

There are implications to these assumptions.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 14:10:58 -0700

On 5/5/2015 1:34 PM, Dave Dolson wrote:
> For IPv6, if the packet doesn't fit in the tunnel, the ICMP6 "too big" 
> message is sent, and the sender fragments it, sends it again, and notes 
> the PMTU for future packets. This is how IPv6 fragmentation is supported.

If the ICMP is received (and not blocked).

If the ICMP has enough information for the tunnel source to know what
happened.

But this also then means that the tunnel egress will need to reassemble.

> Some of your arguments are that implementations are broken so ICMP doesn't 
> work. I don't buy that. When the internet doesn't work, it gets fixed or 
> the customers go away.

See RFC 4821. This issue has been known for a long time, and that's why
the IETF developed an alternative that doesn't depend on ICMP receipt.

> Your one argument I think may be valid is the IPv6 1280 requirement.
> Let's say N levels of tunneling are required to reduce 1500 bytes to 1280 bytes.
> The person who wants to do the (N+1)th overlay either cannot do it, or they resort
> to fragmentation of the outer layer.
> 
> Nonetheless, the approach of reducing Path-MTU is a valid 3rd choice, provided it is 
> not reduced below 1280.

The better solution would be RFC 4821-style probing by the tunnel
ingress to the tunnel egress.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 21:23:52 +0000

Hi Joe,

> ingress to the tunnel egress.

As you know, I agree with this (per Section 3.13 of AERO) but it is not
exactly like RFC4821. With RFC4821, since the source of the probes is
also the original source of the data packets (and, since the probes are
actually data packets themselves) the probes are guaranteed to travel
the same network path as the data packets.

When the tunnel probes, however, it may be probing on behalf of many
original sources and it becomes more difficult to ensure that the probes
travel the same network path as the data packets. For this reason, the
tunnel needs to be meticulous about how it assigns n-tuples to the
probes and encapsulated data packets.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 14:58:40 -0700

> the same network path as the data packets.

That depends on how you implement the tunnels. You can easily use the
data packets as probes if you design a way for the egress to indicate
which probes get through.

T> When the tunnel probes, however, it may be probing on behalf of many
> travel the same network path as the data packets.

That's true only if the network looks into the tunnel packets and treats
them differently - which could have happened for any application packets
too.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 22:15:15 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 2:59 PM
> To: Templin, Fred L; Dave Dolson; Xuxiaohu; Donald Eastlake; [email-masked]
> Subject: Re: [trill] [sfc] Fwd: Mail regarding draft-ietf-trill-over-ip
> 
> 
> 

No, look at Section 5.2 of RFC4821. It acknowledges that MTU needs to be
maintained on a per-flow basis. When the source is the  originator of the
flows, it is easy to keep track of which MTUs are associated with which
flows. When the source is a tunnel ingress, however,  it may be required
to send the data packets of many flows across the tunnel to the egress
and, if the network can distinguish the different flows, the data packets
and probe packets may take different paths. 

What this means is that, for tunnels over IPv6, the ingress would need
to implement a discipline for setting the flow label in the encapsulation
header so that the probes and data packets appear to be part of the
same flow. One way is to simply set a constant value (e.g., 0), but then
ECMP in the path between the ingress and egress would be disabled.

A compromise would be to have the ingress set a finite number of flow
label values (e.g., 4) selected, e.g., based on a hash of the payload
packet headers. But then, each such flow label value would need to
be probed separately.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 15:22:56 -0700

On 5/5/2015 3:15 PM, Templin, Fred L wrote:
> flows. When the source is a tunnel ingress, however, 

Then the tunnel is the source of the flows. Full stop.

Any association between the tunnel flows and the flows arriving at the
tunnel ingress is the responsibility of the tunnel to maintain.

> it may be required
> and, if the network can distinguish the different flows,

Then what you're saying is that the tunnel ingress - as an app - has
flows that are distinguishable by the net. That can happen with any
application.

> the data packets

That can happen with any application.

> same flow. 

The probe packets can be the data packets. The egress can simply send
information back about what data packets succeed.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 May 2015 22:43:44 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 3:23 PM

The tunnel is the source of the encapsulated packets, which may include
the original packets of may flows (produced by the original sources).

> tunnel ingress is the responsibility of the tunnel to maintain.

Right, which is why I said the ingress would need to be meticulous about
setting the flow label.

> flows that are distinguishable by the net. That can happen with any

But, that's just it - when the original source does the probing the data
packets are always part of the same application - always. When the
tunnel ingress probes, it may be asked to encapsulate the data
pacekts of many applications. So, RFC4821-style probing at the
tunnel ingress is *different* from RFC4821-style probing at the
original source.

> That can happen with any application.

When the original source does the probing, the probes and application
data packets are indistinguishable. That is not true for tunnels.

> information back about what data packets succeed.

Yes, the probes can be data packets but the question is, which data packets?
If there are many flows being multiplexed across the same tunnel, there
would need to be a way to make sure all of the flows are properly
associated with corresponding probes.

I have already said many times that this does not make me happy but
it does need to be dealt with. Should we just set the flow label to 0 in
all tunneled packets and hope for the best?

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 May 2015 17:27:32 -0700

On 5/5/2015 3:43 PM, Templin, Fred L wrote:
> pacekts of many applications.

An "application" is a source of packets.

At a tunnel, the ingress arriving packets are that source - they ARE the
"application".

To correlate flow IDs to behavior within that flow - i.e., to the user
programs that generate the packets that arrive at the ingress - requires
the ingress to somehow infer those flows, unless they're explicitly
marked by the user applications.

That's a known limitation of any tunnel ingress. It's equally true of
the user application if there are four people using one application, and
you want 'flow' to map back to an individual person.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 6 May 2015 02:46:32 +0000

> Sent: Wednesday, May 06, 2015 3:42 AM
> > You need more patterns than you have because IP is allowed to use any
> > of them.
> 
> FWIW, you're essentially arguing for bit-stuffing, i.e., if a pattern you think will
> be common shows up, don't stuff, and if something else shows up, then do.
> 
> more bits somewhere else that indicates they're uncompressed and their value.
> 
> So now your GUE methods are very sensitive to IP version numbers, which IMO
> defeats the "G" in the name. Never mind how this complicates hardware when

Agree. As I had mentioned to Tom in private, I do believe the additional capability of GUE (e.g., fragmentation) would be much useful in some network environments. However, since GUE is intended to be a generic UDP encapsulation approach, we'd better design it as generic and future-proof as possible from the beginning, specifically speaking, without being deeply influenced by a particular GUE payload type. If both foo-in-GUE (foo could be NSH, TRILL...) and foo-in-UDP (i.e., a compact version of foo-in-GUE) are needed someday, could you still achieve the same effect as that for IP-in-UDP (as a compact IP-in-GUE) and IP-in-GUE?

Best regards,
Xiaohu

> (not if) other IP versions are used (for whatever reason).
> 
> Joe
> 
>

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 6 May 2015 02:53:54 +0000

> Sent: Wednesday, May 06, 2015 12:33 AM
> That's insufficient. They are only applicable where fragmentation and a strong
> checksum are not *needed*.

Agree that "needed" is a more appropriate word.

> Once you run IP in IP (IP in UDP in IP qualifies as this), you have only two
> choices:
> 
> 	- support fragmentation
> 
> As to the strong checksum, similarly you have to either support one or deploy
> the result where that checksum isn't needed - either because you know that all
> apps will have strong enough checksums of their own or because you know
> enough about the kinds of errors that will occur that strong checksums aren't
> needed.
> 
> But the key there is to define a use case where these properties are true AND to
> limit the document solution to uses in those case ONLY.

The use case is the Softwire networks (including both mesh and hub-spoke modes) where IP-in-IP and IP-in-GRE are good enough to address the MTU and checksum issues.

Best regards,
Xiaohu

> > For those network environments where fragmentation on the tunnel layer
> > and stronger checksums are required, GUE should be considered instead.
> 
> Agreed.
> 
> Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 6 May 2015 14:37:00 +0000

Hi Joe,

> Sent: Tuesday, May 05, 2015 5:28 PM

We are still not reaching. The concern is that if some tunneled packets
take a different network path than others then there is a chance that
the probes will take a different path than the data. The only way I know
to prevent that is to always set the Flow Label in the encapsulation
header to some fixed value (e.g., 0).

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 6 May 2015 14:38:59 +0000

HI,

> From: Xuxiaohu [mailto:[email-masked]]
> Sent: Tuesday, May 05, 2015 7:47 PM
> To: Joe Touch; Tom Herbert; Templin, Fred L
> Cc: Donald Eastlake; [email-masked]; [email-masked]; [email-masked]; [email-masked]
> Subject: RE: [trill] Fwd: Mail regarding draft-ietf-trill-over-ip
> 
> 
> 
> Agree. As I had mentioned to Tom in private, I do believe the additional capability of GUE (e.g., fragmentation) would be much useful
> in some network environments. However, since GUE is intended to be a generic UDP encapsulation approach, we'd better design it as
> generic and future-proof as possible from the beginning, specifically speaking, without being deeply influenced by a particular GUE
> payload type. If both foo-in-GUE (foo could be NSH, TRILL...) and foo-in-UDP (i.e., a compact version of foo-in-GUE) are needed
> someday, could you still achieve the same effect as that for IP-in-UDP (as a compact IP-in-GUE) and IP-in-GUE?

I don't get why we are still talking about this. We have already shown how
IP-in-UDP can be properly accommodated by GUE with header compression.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 06 May 2015 11:15:33 -0700

> header to some fixed value (e.g., 0).

You need to track ANY packet difference that could cause a path
difference. That could include:
	- flow label
	- packet length
	- traffic class
	- next-header

The only things that all tunneled packets have in common are source and
destination address. ANY other field that varies is a potential source
of path variance.

By your logic, PLMTUD would never work at all, and PMTUD only ever tells
you about a specific set of the above fields failing.

Yes, if you want to avoid that issue, you need to minimize any
differences that could affect path. But that's *already true for all
uses of PLMTUD or PMTUD*.

The point here is that assuming otherwise is the flaw.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 6 May 2015 19:46:50 +0000

Hi Joe,

> Sent: Wednesday, May 06, 2015 11:16 AM

Right.

> 	- packet length

An Equal Cost Multipath algorithm that made forwarding decisions based
on packet length would be difficult to probe.

> 	- traffic class

Traffic Class is in the same category as Flow Label - a single tunnel
could be asked to carry multiple traffic classes, which might cause
ECMP to select different paths for carrying tunneled traffic. This
gets back to whether to copy the inner ToS codepoint into the
outer header or not, and I think there were many RFCs that talk
about this.

> 	- next-header

In this case, next header would be the same for all tunneled
packets so no need for concern here.

> destination address.

and next-header.

> ANY other field that varies is a potential source
> of path variance.

In which case, probes could take a different path than data,
which is the whole point of concern.

> By your logic, PLMTUD would never work at all,

PLMTUD works because it is based on the actual data packets of the
actual flow that is being sized.

> and PMTUD only ever tells

PMTUD (when it works) will always converge to the minimum MTU of all
paths that are part of the multipath. When it works, great; when it doesn't,
you need PLMTUD.

> uses of PLMTUD or PMTUD*.

Not true for PLMTUD and not applicable for PMTUD (see above).

> The point here is that assuming otherwise is the flaw.

The point for concern is to find a way to ensure that any path
sizing probes sent by the tunnel ingress will follow the same
path as the data packets, which may have come from many
original flows and/or many original sources. I don't think we
have yet figured out how to do that.

Thanks - Fred

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 7 May 2015 01:10:22 +0000

> From: Templin, Fred L [mailto:[email-masked]]
> Sent: Wednesday, May 06, 2015 10:39 PM
> To: Xuxiaohu; Joe Touch; Tom Herbert
> > > > You need more patterns than you have because IP is allowed to use
> > > > any of them.
> > >
> > > FWIW, you're essentially arguing for bit-stuffing, i.e., if a
> > > pattern you think will be common shows up, don't stuff, and if something
> else shows up, then do.
> > >
> > > But then if 0x0000, 0x0001, 0x0010 or 0x0011 show up, you need to
> > > decide how to represent them - they CANNOT be uncompressed without
> > > at least two more bits somewhere else that indicates they're uncompressed
> and their value.
> > >
> > > So now your GUE methods are very sensitive to IP version numbers,
> > > which IMO defeats the "G" in the name. Never mind how this
> > > complicates hardware when
> >
> > Agree. As I had mentioned to Tom in private, I do believe the
> > additional capability of GUE (e.g., fragmentation) would be much
> > useful in some network environments. However, since GUE is intended to
> > be a generic UDP encapsulation approach, we'd better design it as
> > generic and future-proof as possible from the beginning, specifically speaking,
> without being deeply influenced by a particular GUE payload type. If both
> foo-in-GUE (foo could be NSH, TRILL...) and foo-in-UDP (i.e., a compact version
> of foo-in-GUE) are needed someday, could you still achieve the same effect as
> that for IP-in-UDP (as a compact IP-in-GUE) and IP-in-GUE?
> 

Show us how TRILL-in-UDP and NSH-in-UDP can be properly accommodated by GUE with header compression as well.

Best regards,
Xiaohu