Subject: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

From: Wassim Haddad <[email-masked]>
Date: Thu, 19 May 2016 10:03:48 -0700

Dear all,

The authors of draft-xu-intarea-ip-in-udp-03 (“Encapsulating IP in UDP”) have requested that the working group adopt this work as a WG work item.
So far, WG chairs have not seen widespread support and considering that lack of opposition does not qualify as support, we’re starting a working group adoption call until June 3rd. 

If you consider that the draft should be adopted as a WG work item, please indicate the reason.

Regards,

Wassim & Juan Carlos

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Thu, 19 May 2016 17:52:34 +0000

I, for one, am opposed. I have two issues. One is summarized fairly succinctly in RFC 1326; there are technical issues in mutual encapsulation. The other is in the general form of "good grief". I don't think the draft adequately argues for yet-another-tunnel format.It tells how to do it and asks for a port number, but the arguments for doing it don't make sense to me.

> On May 19, 2016, at 10:03 AM, Wassim Haddad <[email-masked]> wrote:
> 
> Dear all,
> 
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Joe Touch <[email-masked]>
Date: Thu, 19 May 2016 14:03:10 -0700

+1

I provided numerous comments on flaws in this doc over one year ago,
which remain unaddressed:

----

We know of at least four things that tunnels need that IP-in-UDP ignores:

	- stronger checksums

	- fragmentation support
	
	- signalling support (e.g., to test whether a tunnel is up or
	to measure MTUs)

	- support for robust ID fields (related to fragmentation,
	e.g., to overcome the limits of IPv4 ID as per RFC 6864)

----

Additional issues:
- fails to address the need for this approach vs GUE
- fails to consider the recommendations in draft-ietf-intarea-tunnels
- recommends PMTUD, rather than the much more robust PLPMTUD
- fails to address interactions with IPv6 flow labels, both inside and
outside the encapsulation

Joe

On 5/19/2016 10:52 AM, Fred Baker (fred) wrote:
> I, for one, am opposed. I have two issues. One is summarized fairly
> succinctly in RFC 1326; there are technical issues in mutual
> encapsulation. The other is in the general form of "good grief". I
> don't think the draft adequately argues for yet-another-tunnel
> format.It tells how to do it and asks for a port number, but the
> arguments for doing it don't make sense to me.
>
>> On May 19, 2016, at 10:03 AM, Wassim Haddad
>> <[email-masked] <mailto:[email-masked]>> wrote:
>>
>> Dear all,
>>
>> The authors of draft-xu-intarea-ip-in-udp-03 (“Encapsulating IP in
>> UDP”) have requested that the working group adopt this work as a WG
>> work item.
>> So far, WG chairs have not seen widespread support and considering
>> that lack of opposition does not qualify as support, we’re starting a
>> working group adoption call until June 3rd. 
>>
>> [email-masked] <mailto:[email-masked]>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 20 May 2016 13:17:12 +1200

As a famous physicist once said when the discovery of the muon was announced, "Who ordered that?"

In other words, I don't understand the use case that motivated this.
In the Introduction, I find:

  "By encapsulating the Softwire
   service traffic into an UDP tunnel and using the source port of the
   UDP header as an entropy field, the existing load-balancing
   capability as mentioned above can be leveraged to provide fine-
   grained load-balancing of Softwire service traffic traffic over IP
   networks."

What is meant by "the Softwire service traffic"? Does it just mean a
bunch of traffic from a single customer? Why does it all need a
common header for load balancing? The individual microflows in the
traffic will all get load-balanced anyway.

I trust we are only talking about IPv6. If we're talking about IPv4, we
certainly shouldn't be developing new methods. If we're talking about
IPv6, we have the flow label for this (and it avoids tricky problems
like finding the transport header in the presence of extension headers).
That's already covered in RFC 6438 for any kind of tunnel, although
IP in IP seems a lot simpler than IP in UDP in IP.

(If you really must do IPv4, IPv4 in IPv6 could still be load-balanced
as per RFC 6438, I guess.)

   Brian

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Fri, 20 May 2016 03:39:18 +0000

Do equivalent arguments apply also to <https://datatracker.ietf.org/doc/draft-ietf-tsvwg-gre-in-udp-encap/>? Was this one reviewed by Int-area?

Thumb typed by Carlos Pignataro.
Excuze typofraphicak errows

> On May 19, 2016, at 21:17, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 20 May 2016 16:46:19 +1200

On 20/05/2016 15:39, Carlos Pignataro (cpignata) wrote:
> Do equivalent arguments apply also to <https://datatracker.ietf.org/doc/draft-ietf-tsvwg-gre-in-udp-encap/>? 

I don't know. I've never found GRE a very interesting topic so I've never looked at that draft.

> Was this one reviewed by Int-area?

Not as far as I know, but it's still technically in TSVWG last call if you have comments.

    Brian

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 20 May 2016 09:47:28 +0000

Hi Fred,

Thanks for your comment. Please see my response inline.

From: Int-area [mailto:[email-masked]] On Behalf Of Fred Baker (fred)
Sent: Friday, May 20, 2016 1:53 AM
To: Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

I, for one, am opposed. I have two issues. One is summarized fairly succinctly in RFC 1326; there are technical issues in mutual encapsulation. The other is in the general form of "good grief". I don't think the draft adequately

[Xiaohu] I’m confused by your intention of raising the technical issues in mutual encapsulation as described in RFC1326. Are you against the necessity of Softwire services (e.g., IPv6-over-IPv4)?

argues for yet-another-tunnel format.It tells how to do it and asks for a port number, but the arguments for doing it don't make sense to me.

[Xiaohu] The argument for doing IP-in-UDP is clearly described in the Introduce section as follows:

   To fully utilize the bandwidth available in IP networks and/or
   facilitate recovery from a link or node failure, load balancing of
   traffic over Equal Cost Multi-Path (ECMP) and/or Link Aggregation
   Group (LAG) across IP networks is widely used.  [RFC5640<https://tools.ietf.org/html/rfc5640>] describes a
   method for improving the load balancing efficiency in a network
   carrying Softwire Mesh service [RFC5565<https://tools.ietf.org/html/rfc5565>] over Layer Two Tunneling
   Protocol - Version 3 (L2TPv3) [RFC3931<https://tools.ietf.org/html/rfc3931>] and Generic Routing
   Encapsulation (GRE) [RFC2784<https://tools.ietf.org/html/rfc2784>] encapsulations.  However, this method
   requires core routers to perform hash calculation on the "load-
   balancing" field contained in tunnel encapsulation headers (i.e., the
   Session ID field in L2TPv3 headers or the Key field in GRE headers),
   which is not widely supported by existing core routers.

   Most existing routers in IP networks are already capable of
   distributing IP traffic "microflows" [RFC2474<https://tools.ietf.org/html/rfc2474>] over ECMP paths and/or

Xu, et al.               Expires August 3, 2016                 [Page 2]
________________________________
<https://tools.ietf.org/html/draft-xu-intarea-ip-in-udp-03#page-3>
Internet-Draft           Encapsulating IP in UDP            January 2016

   LAG based on the hash of the five-tuple of User Datagram Protocol
   (UDP) [RFC0768<https://tools.ietf.org/html/rfc0768>] and Transmission Control Protocol (TCP) packets
   (i.e., source IP address, destination IP address, source port,
   destination port, and protocol).  By encapsulating the Softwire
   service traffic into an UDP tunnel and using the source port of the
   UDP header as an entropy field, the existing load-balancing
   capability as mentioned above can be leveraged to provide fine-
   grained load-balancing of Softwire service traffic traffic over IP
   networks.

In addition, the use of UDP tunnels with source port being used as an entropy field has been widely accepted in the IETF community (see the following examples).

p LISP [RFC6830]
p VXLAN [RFC7348]
p MPLS-in-UDP [RFC7510]
p TRILL-in-UDP [draft-ietf-trill-over-ip-05]
p NSH-in-UDP [draft-kumar-sfc-nsh-udp-transport]
p VXLAN-GPE [draft-ietf-nvo3-vxlan-gpe<https://tools.ietf.org/wg/nvo3/draft-ietf-nvo3-vxlan-gpe/> ]
p GENEVE [draft-ietf-nvo3-geneve<https://tools.ietf.org/wg/nvo3/draft-ietf-nvo3-geneve/> ]
GUE [draft-ietf-nvo3-gue<https://tools.ietf.org/wg/nvo3/draft-ietf-nvo3-gue/> ]

It seems very straightforward to use the UDP tunnels to carry IP traffic as well in the Softwire scenarios.

Best regards,
Xiaohu
On May 19, 2016, at 10:03 AM, Wassim Haddad <[email-masked]<mailto:[email-masked]>> wrote:

Dear all,

Regards,

Wassim & Juan Carlos

_______________________________________________
Int-area mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/int-area

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 20 May 2016 12:09:37 +0200

Locally we are using IP-in-UDP as provided by Mobile IPv4 specification 
for NAT Traversal, RFC 5265.  I guess it is sufficient, why another 
document?

(I have not read the doc).

Alex

Le 19/05/2016 à 19:03, Wassim Haddad a écrit :
> Dear all,
>
> The authors of draft-xu-intarea-ip-in-udp-03 (“Encapsulating IP in UDP”)
> have requested that the working group adopt this work as a WG work item.
> So far, WG chairs have not seen widespread support and considering that
> lack of opposition does not qualify as support, we’re starting a working
> group adoption call until June 3rd.
>

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 20 May 2016 10:13:41 +0000

Hi Joe,

Thanks for your comment. Note that it's WG adoption call rather than WGLC. If I understand it correctly, as long as it's worthwhile to provide fine-grained load-balancing of Softwire service traffic by leveraging the UDP tunnels, the WG should adopt it and then work on it, e.g., addressing those issues as you mentioned.

Best regards,
Xiaohu

From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
Sent: Friday, May 20, 2016 5:03 AM
To: Fred Baker (fred); Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

+1

I provided numerous comments on flaws in this doc over one year ago, which remain unaddressed:

----

       - stronger checksums

       - fragmentation support

       to measure MTUs)

Additional issues:
- fails to address the need for this approach vs GUE
- fails to consider the recommendations in draft-ietf-intarea-tunnels
- recommends PMTUD, rather than the much more robust PLPMTUD
- fails to address interactions with IPv6 flow labels, both inside and outside the encapsulation

Joe

Dear all,

The authors of draft-xu-intarea-ip-in-udp-03 ("Encapsulating IP in UDP") have requested that the working group adopt this work as a WG work item.
So far, WG chairs have not seen widespread support and considering that lack of opposition does not qualify as support, we're starting a working group adoption call until June 3rd.

Regards,

Wassim & Juan Carlos

Int-area mailing list

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 20 May 2016 10:24:50 +0000

Hi Brain,

Thanks for your comments. Please see my response inline.

> -----Original Message-----
> From: Int-area [mailto:[email-masked]] On Behalf Of Brian E
> Carpenter
> Sent: Friday, May 20, 2016 9:17 AM
> To: Wassim Haddad; [email-masked]
> What is meant by "the Softwire service traffic"? Does it just mean a bunch of
> traffic from a single customer? Why does it all need a common header for load
> balancing? The individual microflows in the traffic will all get load-balanced
> anyway.

I don't understand what you meant. You don't know what the Softwire service traffic is? If so, please refer to RFC5565.

> I trust we are only talking about IPv6. If we're talking about IPv4, we certainly
> shouldn't be developing new methods. If we're talking about IPv6, we have the

It's not a new load-balancing method at all. Instead, as mentioned in the draft

"Most existing routers in IP networks are already capable of
   distributing IP traffic "microflows" [RFC2474] over ECMP paths and/or
   LAG based on the hash of the five-tuple of User Datagram Protocol
   (UDP) [RFC0768] and Transmission Control Protocol (TCP) packets
   (i.e., source IP address, destination IP address, source port,
   destination port, and protocol)."

> flow label for this (and it avoids tricky problems like finding the transport header
> in the presence of extension headers).
> That's already covered in RFC 6438 for any kind of tunnel, although IP in IP
> seems a lot simpler than IP in UDP in IP.
> 
> (If you really must do IPv4, IPv4 in IPv6 could still be load-balanced as per RFC
> 6438, I guess.)

It said in the draft that 

" IPv6 flow label has been proposed as an entropy field for load
   balancing in IPv6 network environment [RFC6438].  However, as stated
   in [RFC6936], the end-to-end use of flow labels for load balancing is
   a long-term solution and therefore the use of load balancing using
   the transport header fields would continue until any widespread
   deployment is finally achieved.  As such, IP-in-UDP encapsulation
   would still have a practical application value in the IPv6 networks
   during this transition timeframe.
 "
If you believe the following statement quoted from RFC6438 is not correct, please correct it by evidences.

"the end-to-end use of flow labels for load balancing is
   a long-term solution and therefore the use of load balancing using
   the transport header fields would continue until any widespread
   deployment is finally achieved."

Best regards,
Xiaohu

> > The authors of draft-xu-intarea-ip-in-udp-03 (“Encapsulating IP in UDP”) have
> requested that the working group adopt this work as a WG work item.
> > So far, WG chairs have not seen widespread support and considering that lack
> of opposition does not qualify as support, we’re starting a working group
> adoption call until June 3rd.
> >

---

From: Ted Lemon <[email-masked]>
Date: Fri, 20 May 2016 08:31:58 -0400

On Fri, May 20, 2016 at 6:13 AM, Xuxiaohu <[email-masked]> wrote:

> Thanks for your comment. Note that it’s WG adoption call rather than WGLC.
> If I understand it correctly, as long as it’s worthwhile to provide
> fine-grained load-balancing of Softwire service traffic by leveraging the
> UDP tunnels, the WG should adopt it and then work on it, e.g., addressing
> those issues as you mentioned.
>

The WG shouldn't adopt it unless there is a clear motivation for doing so,
and no existing solutions to the same problem.   That is the case you need
to make.   What various people are saying is that they don't believe you
have made that case.   That is how it appears to me as well.

---

From: "Black, David" <[email-masked]>
Date: Fri, 20 May 2016 14:30:30 +0000

-- As GRE/UDP draft shepherd

> ietf-tsvwg-gre-in-udp-encap/>?

Not exactly ;-).

The motivating GRE in UDP encap use cases involves operator use of GRE for
traffic management, as summarized in the draft introduction (Section 1 - the
word "transit" is crucial):

   ...  the source port field in the UDP header may
   be used to provide additional entropy that may be used for load
   balancing GRE traffic in transit networks using existing Equal-Cost
   Multi-Path (ECMP) mechanism.

An interesting consequence is that the GRE in UDP encap draft specifies that
the GRE and UDP tunnels MUST share endpoints - the GRE and UDP headers
are applied at the same network node and are removed at the same network
node.

So, yes, we did pay attention to use cases over in TSVWG ...

-- As an individual

Like, Brian, I don't understand the softwire motivation for IP-in-UDP tunnels,
and hence oppose intarea WG adoption of this draft.

Xiaohu's response to Brian is unhelpful (IMHO):

> I don't understand what you meant. You don't know what the Softwire
> service traffic is? If so, please refer to RFC5565.

That suggests to me that the draft is missing a brief overview of Softwires,
and probably also lacks an applicability statement  limiting the scope of this
draft to Softwires.

Section 8 of RFC 5565 on "Selecting a Tunneling Technology" is not helpful in
explaining why IP-in-IP tunnels are wanted (which would then set up the
discussion in this draft about why IP-in-UDP-in-IP tunnels are wanted courtesy
of deployed equipment ECMP limitations on IP-in-IP tunnels).

Thanks, --David

> From: Int-area [mailto:[email-masked]] On Behalf Of Brian E Carpenter
> Sent: Friday, May 20, 2016 12:46 AM
> To: Carlos Pignataro (cpignata)
> Cc: [email-masked]
> lack of opposition does not qualify as support, we’re starting a working group

---

From: Tom Herbert <[email-masked]>
Date: Fri, 20 May 2016 09:13:59 -0700

> have made that case.   That is how it appears to me as well.
There is an existing solution to the same problem. GUE allows
encapsulation of IPv4 and IPv6, as well as other IP protocols (the GUE
header indicates encapsulated protocol by IP number). The only
material between GUE encapsulation of IP and IP in UDP is additional
four byte header and associated processing of that. I don't think
we've seen a use case where avoiding that overhead is critical
motivation.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 20 May 2016 10:11:24 -0700

> Thanks for your comment. Note that it’s WG adoption call rather than WGLC.

I gave reasons for not adopting it already, including key flaws and the
fact that this doc has not evolved based on feedback in the past (thus
my confidence in expecting future changes is low).

You need to do the work *first*. The WG is a place to polish, not for
basic development.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 23 May 2016 01:58:22 +0000

Hi Ted,

From: Ted Lemon [mailto:[email-masked]]
Sent: Friday, May 20, 2016 8:32 PM
To: Xuxiaohu
Cc: Joe Touch; Fred Baker (fred); Wassim Haddad; [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

On Fri, May 20, 2016 at 6:13 AM, Xuxiaohu <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for your comment. Note that it’s WG adoption call rather than WGLC. If I understand it correctly, as long as it’s worthwhile to provide fine-grained load-balancing of Softwire service traffic by leveraging the UDP tunnels, the WG should adopt it and then work on it, e.g., addressing those issues as you mentioned.

The WG shouldn't adopt it unless there is a clear motivation for doing so, and no existing solutions to the same problem.   That is the case you need to make.   What various people are saying is that they don't believe you have made that case.   That is how it appears to me as well.

There are existing solutions to the same problem (i.e., how to improve the load-balancing efficiency for softwire traffic) and they are described in RFC5640. However, as it has been mentioned in the draft (see below), the existing solutions is are not adequate for efficient load balancing of Softwire service traffic across IP networks.

“[RFC5640<https://tools.ietf.org/html/rfc5640>] describes a
   method for improving the load balancing efficiency in a network
   carrying Softwire Mesh service [RFC5565<https://tools.ietf.org/html/rfc5565>] over Layer Two Tunneling
   Protocol - Version 3 (L2TPv3) [RFC3931<https://tools.ietf.org/html/rfc3931>] and Generic Routing
   Encapsulation (GRE) [RFC2784<https://tools.ietf.org/html/rfc2784>] encapsulations.  However, this method
   requires core routers to perform hash calculation on the "load-
   balancing" field contained in tunnel encapsulation headers (i.e., the
   Session ID field in L2TPv3 headers or the Key field in GRE headers),
   which is not widely supported by existing core routers.”

The motivation has been explained clearly in the draft as follows,

“Most existing routers in IP networks are already capable of
   distributing IP traffic "microflows" [RFC2474<https://tools.ietf.org/html/rfc2474>] over ECMP paths and/or

   LAG based on the hash of the five-tuple of User Datagram Protocol
   (UDP) [RFC0768<https://tools.ietf.org/html/rfc0768>] and Transmission Control Protocol (TCP) packets
   (i.e., source IP address, destination IP address, source port,
   destination port, and protocol).  By encapsulating the Softwire
   service traffic into an UDP tunnel and using the source port of the
   UDP header as an entropy field, the existing load-balancing
   capability as mentioned above can be leveraged to provide fine-
   grained load-balancing of Softwire service traffic traffic over IP
   networks.  This is similar to why LISP [RFC6830<https://tools.ietf.org/html/rfc6830>] , MPLS-in-UDP
   [RFC7510<https://tools.ietf.org/html/rfc7510>] and VXLAN [RFC7348<https://tools.ietf.org/html/rfc7348>] use UDP encapsulation.
”
If the GRE key or L2TPv3 session ID based LB approaches as described in RFC5640, and even the flow label based approach were good enough, it seems hard to understand why the IETF community is so wild about using UDP as a tunnel technology (note that it’s not intended for IPv4 underlay case only. Instead, it’s intended for both IPv4 and IPv6 underlay cases), see the following examples:

p LISP [RFC6830]
p VXLAN [RFC7348]
p MPLS-in-UDP [RFC7510]
p GUE [draft-ietf-nvo3-gue<https://tools.ietf.org/wg/nvo3/draft-ietf-nvo3-gue/> ]
p …

If the above motivation seems still not enough to you, could you please explain the reason why you believe it’s not enough?

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 23 May 2016 02:21:41 +0000

Hi David,

> From: Int-area [mailto:[email-masked]] On Behalf Of Black, David
> Sent: Friday, May 20, 2016 10:31 PM
> To: Brian E Carpenter; Carlos Pignataro (cpignata)
> Cc: [email-masked]
> > > Do equivalent arguments apply also to
> > > <https://datatracker.ietf.org/doc/draft-
> > ietf-tsvwg-gre-in-udp-encap/>?
> 
> Not exactly ;-).
> 
> An interesting consequence is that the GRE in UDP encap draft specifies that the
> GRE and UDP tunnels MUST share endpoints - the GRE and UDP headers are
> applied at the same network node and are removed at the same network node.
> 
> That suggests to me that the draft is missing a brief overview of Softwires, and
> probably also lacks an applicability statement  limiting the scope of this draft to
> Softwires.

I had thought the Softwire use cases are well-known to the IETF folks and therefore there is no need for a brief overview of that technology.

> explaining why IP-in-IP tunnels are wanted (which would then set up the

IP-in-IP tunnels are used for carrying IPv6 traffic over IPv4 networks (see the 6rd usage as described in RFC5596 and RFC5969 and IPv6-in-IPv4 encapsulation format as described in https://tools.ietf.org/html/rfc4213#page-7) and IPv4 traffic over IPv6 networks.

BTW, the IPv4-in-IPv6 and IPv6-in-IPv4 scenarios have been described in section 3 of RFC5565. Section 8 of the same RFC just lays out the rationale for RFC5512 which describes how to advertise tunnel capabilities via BGP. 

Best regards,
Xiaohu

> > > Do equivalent arguments apply also to
> > > <https://datatracker.ietf.org/doc/draft-
> > ietf-tsvwg-gre-in-udp-encap/>?
> >
> > >> On May 19, 2016, at 21:17, Brian E Carpenter
> > >> <[email-masked]>
> > wrote:
> > >>
> > >> What is meant by "the Softwire service traffic"? Does it just mean
> > >> a bunch of traffic from a single customer? Why does it all need a
> > >> I trust we are only talking about IPv6. If we're talking about
> > >> IPv4, we certainly shouldn't be developing new methods. If we're
> > >> talking about IPv6, we have the flow label for this (and it avoids
> > >> tricky problems like finding the transport header in the presence of
> extension headers).
> > >> (If you really must do IPv4, IPv4 in IPv6 could still be
> > >> load-balanced as per RFC 6438, I guess.)
> > >>
> > >>   Brian
> > >>
> > >>
> > >>> UDP”)
> > >>> that
> > lack of opposition does not qualify as support, we’re starting a
> > >>> If you consider that the draft should be adopted as a WG work
> > >>> item, please
> > indicate the reason.
> > >>>
> > >>>
> > >>> Regards,
> > >>>
> > >>> Wassim & Juan Carlos
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 23 May 2016 02:31:18 +0000

Hi Tom,

> From: Tom Herbert [mailto:[email-masked]]
> Sent: Saturday, May 21, 2016 12:14 AM
> To: Ted Lemon
> Cc: Xuxiaohu; [email-masked]
> >> fine-grained load-balancing of Softwire service traffic by leveraging
> >> the UDP tunnels, the WG should adopt it and then work on it, e.g.,
> >> addressing those issues as you mentioned.
> >
> >
> There is an existing solution to the same problem. GUE allows encapsulation of
> IPv4 and IPv6, as well as other IP protocols (the GUE header indicates
> encapsulated protocol by IP number). The only material between GUE
> encapsulation of IP and IP in UDP is additional four byte header and associated
> processing of that. I don't think we've seen a use case where avoiding that
> overhead is critical motivation.

Before comparing GUE and IP-in-UDP (Note that both of them are built on UDP tunnels), it seems necessary to reach a rough WG consensus on the motivation of leveraging UDP tunnel for carrying softwire traffic firstly. Once the consensus is reached, we can then discuss whether we need a simple UDP tunnel header, a powerful UDP tunnel header (i.e., GUE header+UDP header) or both.

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 23 May 2016 02:33:29 +0000

Hi Alex,

> From: Int-area [mailto:[email-masked]] On Behalf Of Alexandre
> Petrescu
> Sent: Friday, May 20, 2016 6:10 PM
> To: [email-masked]
> Locally we are using IP-in-UDP as provided by Mobile IPv4 specification for NAT
> Traversal, RFC 5265.  I guess it is sufficient, why another document?
> 

If so, it'd better for you to read the doc first.

Best regards,
Xiaohu

> > The authors of draft-xu-intarea-ip-in-udp-03 ("Encapsulating IP in
> > UDP") have requested that the working group adopt this work as a WG work
> item.
> > that lack of opposition does not qualify as support, we're starting a

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 23 May 2016 02:44:07 +0000

Hi Joe,,

As for the four things that you have pointed out (see below),

        - stronger checksums

        - fragmentation support

        to measure MTUs)

I'm wondering whether those issues are specific to IP-in-UDP or not. In other words, are those issue also applicable to other X-in-UDP approaches (where X could be LISP, TRILL, VXLAN, VXLAN-GPE, GENEVE, GUE, NSH or MPLS)?

Best regards,
Xiaohu

From: Joe Touch [mailto:[email-masked]]
Sent: Saturday, May 21, 2016 1:11 AM
To: Xuxiaohu; Fred Baker (fred); Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Thanks for your comment. Note that it's WG adoption call rather than WGLC.

I gave reasons for not adopting it already, including key flaws and the fact that this doc has not evolved based on feedback in the past (thus my confidence in expecting future changes is low).

You need to do the work *first*. The WG is a place to polish, not for basic development.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 23 May 2016 09:30:52 -0700

On 5/22/2016 7:44 PM, Xuxiaohu wrote:
> I’m wondering whether those issues are specific to IP-in-UDP or not.
> In other words, are those issue also applicable to other X-in-UDP
> approaches (where X could be LISP, TRILL, VXLAN, VXLAN-GPE, GENEVE,
> GUE, NSH or MPLS)?
>

That set of issues applies to all IP-in-(xlist), where xlist includes IP
again. The issue is what IP expects and what the tunnel exports.

The other items I pointed out are specific to transport-in-UDP or this
specific approach.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 24 May 2016 02:59:01 +0000

Hi Joe,

Thanks for your response. Please see my response with [Xiaohu] inline.

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, May 24, 2016 12:31 AM
To: Xuxiaohu; Fred Baker (fred); Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

        - stronger checksums

[Xiaohu] it said in the draft that

" For IPv4 UDP encapsulation, this field is RECOMMENDED to be set
         to zero for performance or implementation reasons because the
         IPv4 header includes a checksum and use of the UDP checksum is
         optional with IPv4.  For IPv6 UDP encapsulation, the IPv6
         header does not include a checksum, so this field MUST contain
         a UDP checksum that MUST be used as specified in [RFC0768<https://tools.ietf.org/html/rfc0768>] and
         [RFC2460<https://tools.ietf.org/html/rfc2460>] unless one of the exceptions that allows use of UDP
         zero-checksum mode (as specified in [RFC6935<https://tools.ietf.org/html/rfc6935>]) applies."

In other words, the stronger checksum (i.e., non-zero checksum of UDP) is strongly recommended especially for IPv6 UDP case. Furthermore, as stated in RFC6935, "Tunnel protocols that encapsulate IP will generally be safe for
   deployment, because all IPv4 and IPv6 packets include at least one

   checksum at either the network or transport layer."  This statement is just applied to the IP-in-UDP case.

        - fragmentation support

[Xiaohu] It said in the draft that

              "   Ingress AFBRs MUST NOT fragment I-IP packets (i.e., UDP encapsulated
   IP packets), and when the outer IP header is IPv4, ingress AFBRs MUST
   set the DF bit in the outer IPv4 header.  It is strongly RECOMMENDED
   that I-IP transit core be configured to carry an MTU at least large
   enough to accommodate the added encapsulation headers.  Meanwhile, it
   is strongly RECOMMENDED that Path MTU Discovery [RFC1191<https://tools.ietf.org/html/rfc1191>] [RFC1981<https://tools.ietf.org/html/rfc1981>]
   is used to prevent or minimize fragmentation.  Once an ingress AFBR
   needs to perform fragmentation on an E-IP packet before
   encapsulating, it MUST use the same source UDP port for all
  fragmented packets so as to ensures these fragmented packets are

Xu, et al.               Expires August 3, 2016                 [Page 5]
________________________________
<https://tools.ietf.org/html/draft-xu-intarea-ip-in-udp-03#page-6>
Internet-Draft           Encapsulating IP in UDP            January 2016

   always forwarded on the same path.  In a word, IP-in-UDP is just
   applicable in those Softwire network environments where fragmentation
   on the tunnel layer is not needed."

In other words,  fragmentation on I-IP packets (i.e., UDP encapsulated IP packets) must be avoided. fragmentation on E-IP packets should be avoided by some means (e.g., I-IP transit core is configured to carry an MTU at least large enough to accommodate the added encapsulation headers or Path MTU discovery is enabled on the UDP tunnels). Fragmentation on E-IP packets is possible only when the E-IP packets are IPv4 packets and the DF bit is not set.

        to measure MTUs)

[Xiaohu] It said in the draft that
"it is strongly RECOMMENDED that Path MTU Discovery [RFC1191<https://tools.ietf.org/html/rfc1191>] [RFC1981<https://tools.ietf.org/html/rfc1981>]

   is used to prevent or minimize fragmentation."

in other words, the PMTUD signaling is recommended to be supported.

[Xiaohu] Is it enough to add a sentence like "to overcome the limits of IPv4 ID as per RFC 6864 when performing fragmentation on E-IP IPv4 packets, the specifications as described in RFC6864 MUST be followed."?

Best regards,

Xiaohu

That set of issues applies to all IP-in-(xlist), where xlist includes IP again. The issue is what IP expects and what the tunnel exports.

The other items I pointed out are specific to transport-in-UDP or this specific approach.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 23 May 2016 21:48:22 -0700

Hi Xiaohu,

On 5/23/2016 7:59 PM, Xuxiaohu wrote:
> *From:*Joe Touch [mailto:[email-masked]]
> *Sent:* Tuesday, May 24, 2016 12:31 AM
> *To:* Xuxiaohu; Fred Baker (fred); Wassim Haddad
> *Cc:* [email-masked]
> *Subject:* Re: [Int-area] Call for adoption of
> draft-xu-intarea-ip-in-udp-03
>
>  
>
>  
>
>     “ For IPv4 UDP encapsulation, this field is RECOMMENDED to be set
>
>              a UDP checksum that MUST be used as specified in [RFC0768
>     <https://tools.ietf.org/html/rfc0768>] and
>
>              [RFC2460 <https://tools.ietf.org/html/rfc2460>] unless
>     one of the exceptions that allows use of UDP
>
>              zero-checksum mode (as specified in [RFC6935
>     <https://tools.ietf.org/html/rfc6935>]) applies.”
>
>      
>
>     In other words, the stronger checksum (i.e., non-zero checksum of
>     UDP) is strongly recommended especially for IPv6 UDP case.
>     Furthermore, as stated in RFC6935, “Tunnel protocols that
>     encapsulate IP will generally be safe for
>
>        checksum at either the network or transport layer.”  This
>     statement is just applied to the IP-in-UDP case.
>

The problem is that this advice overlooks the need to account for errors
of fragmentation and reassembly (see below). Something stronger than the
Internet checksum is typically useful, e.g., as was developed for SEAL
(e.g., RFC5320).

>                   “   Ingress AFBRs MUST NOT fragment I-IP packets
>     (i.e., UDP encapsulated
>
>        is strongly RECOMMENDED that Path MTU Discovery [RFC1191
>     <https://tools.ietf.org/html/rfc1191>] [RFC1981
>     <https://tools.ietf.org/html/rfc1981>]
>
>     ------------------------------------------------------------------------
>
>        on the tunnel layer is not needed.”
>
>      
>
>     In other words,  fragmentation on I-IP packets (i.e., UDP
>     encapsulated IP packets) must be avoided. fragmentation on E-IP
>     packets should be avoided by some means (e.g., I-IP transit core
>     is configured to carry an MTU at least large enough to accommodate
>     the added encapsulation headers or Path MTU discovery is enabled
>     on the UDP tunnels). Fragmentation on E-IP packets is possible
>     only when the E-IP packets are IPv4 packets and the DF bit is not
>     set.
>

While this is a nice goal, it's not viable. In order to provide a viable
IPv6 tunnel, the payload MUST transit IPv6 payloads of at least 1280
bytes.However, there's no reason to assume that the UDP/IPv6 tunnel can
support 1280 + UDP + IPv6. So you have only one choice - you can deploy
these tunnels only where you can guarantee paths that support 1280+8+40
byte payloads over the tunnel path. However, there's no way to ensure
that solely by assuming IPv6 along that path.

I.e., you MUST support source fragmentation at the ingress at the outer
IPv6 layer (because UDP doesn't have support for fragmentation and
reassembly). If you make this requirement, you can handle IPv6 over the
tunnel.

IPv4 will work over a UDP/IPv6 tunnel fairly easily, even assuming the
common typical IPv4 MTU (576, even though that's really the reassembled
MTU, not the path MTU).

IPv4 over IPv4 is much more difficult, and not only requires
fragmentation at the outer packet but also additional reassembly
requirements at the egress above and beyond the IPv4 minimums.

>     “it is strongly RECOMMENDED that Path MTU Discovery [RFC1191
>     <https://tools.ietf.org/html/rfc1191>] [RFC1981
>     <https://tools.ietf.org/html/rfc1981>]
>
>        is used to prevent or minimize fragmentation.”
>
>      
>
>      
>

PMTUD is a recipe for black-holing. PLPMTUD is preferred for many reasons.

However, PLPMTUD only ensures proper discovery of the tunnel MTU. It
does not describe how the ICMPs inside the tunnel relate to those that
are generated by the ingress.

Again, for all of these, see draft-ietf-intarea-tunnels

Joe

>      
>
>     [Xiaohu] Is it enough to add a sentence like “to overcome the
>     limits of IPv4 ID as per RFC 6864 when performing fragmentation on
>     E-IP IPv4 packets, the specifications as described in RFC6864 MUST
>     be followed.”?
>
>      
>
>     Best regards,
>
>     Xiaohu
>
>      
>
>      
>
>     I’m wondering whether those issues are specific to IP-in-UDP or
>     not. In other words, are those issue also applicable to other
>     X-in-UDP approaches (where X could be LISP, TRILL, VXLAN,
>     VXLAN-GPE, GENEVE, GUE, NSH or MPLS)?
>
>
> That set of issues applies to all IP-in-(xlist), where xlist includes
> IP again. The issue is what IP expects and what the tunnel exports.
>

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 24 May 2016 08:54:21 +0000

Hi Joe,

The draft is only intended to introduce one additional Softwires encapsulation technology referred to as IP-in-UDP.  In other words, this encapsulation is only intended to be used within Softwires networks which are well-managed by a service provider. This encapsulation technology is not intended to be used within the Internet. As such, it seems absolutely possible to configure the I-IP transit core to carry an MTU at least large enough to accommodate the added encapsulation headers.

Although it has been said in the draft that "IP-in-UDP is just  applicable in those Softwires network environments where fragmentation on the tunnel layer is not needed." I can add a dedicated Applicability Statement section to emphasize that this Softwires encapsulation technology must only be used within Softwires networks which are well-managed by a service provider and must not be used within the Internet. Can this application statement address your concerns on fragmentation and reassembly?

Best regards,
Xiaohu

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, May 24, 2016 12:48 PM
To: Xuxiaohu; Fred Baker (fred); Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Hi Xiaohu,

Cc: [email-masked]<mailto:[email-masked]>

        - stronger checksums

The problem is that this advice overlooks the need to account for errors of fragmentation and reassembly (see below). Something stronger than the Internet checksum is typically useful, e.g., as was developed for SEAL (e.g., RFC5320).

        - fragmentation support

While this is a nice goal, it's not viable. In order to provide a viable IPv6 tunnel, the payload MUST transit IPv6 payloads of at least 1280 bytes.However, there's no reason to assume that the UDP/IPv6 tunnel can support 1280 + UDP + IPv6. So you have only one choice - you can deploy these tunnels only where you can guarantee paths that support 1280+8+40 byte payloads over the tunnel path. However, there's no way to ensure that solely by assuming IPv6 along that path.

I.e., you MUST support source fragmentation at the ingress at the outer IPv6 layer (because UDP doesn't have support for fragmentation and reassembly). If you make this requirement, you can handle IPv6 over the tunnel.

IPv4 will work over a UDP/IPv6 tunnel fairly easily, even assuming the common typical IPv4 MTU (576, even though that's really the reassembled MTU, not the path MTU).

IPv4 over IPv4 is much more difficult, and not only requires fragmentation at the outer packet but also additional reassembly requirements at the egress above and beyond the IPv4 minimums.

        to measure MTUs)

However, PLPMTUD only ensures proper discovery of the tunnel MTU. It does not describe how the ICMPs inside the tunnel relate to those that are generated by the ingress.

Joe

Best regards,

Xiaohu

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 24 May 2016 10:15:00 -0700

Two things below:

On 5/24/2016 1:54 AM, Xuxiaohu wrote:
>
> Hi Joe,
>
>  
>
> The draft is only intended to introduce one additional Softwires
> encapsulation technology referred to as IP-in-UDP. 
>

You had a similar draft that expired last summer targeted at the
Softwires WG (draft-xu-softwire-ip-in-udp). Why is this now targeted at
Intarea?

> In other words, this encapsulation is only intended to be used within
> Softwires networks which are well-managed by a service provider. This
> encapsulation technology is not intended to be used within the
> Internet. As such, it seems absolutely possible to configure the I-IP
> transit core to carry an MTU at least large enough to accommodate the
> added encapsulation headers.
>
>  Although it has been said in the draft that “IP-in-UDP is just
>  applicable in those Softwires network environments where
> fragmentation on the tunnel layer is not needed.” I can add a
> dedicated Applicability Statement section to emphasize that this
> Softwires encapsulation technology must only be used within Softwires
> networks which are well-managed by a service provider and must not be
> used within the Internet. Can this application statement address your
> concerns on fragmentation and reassembly?
>

Here's the issue -

I still do not think that this document should be a WG doc, and I
frankly don't think it's constructive for you to try to address each
flaw as it is raised.

Consider the following:

   A- you go to a restaurant and eat dinner
   B - I ask you if you like it, and you say "no"
   C- I ask why, and you say "it was too salty"

Now, does that mean that if the cook corrects the salt level that you
would now like the food?

Probably not. The same is true here.

I've given reasons I don't think it should be a WG doc. IF it is
accepted as a WG doc, I might decide how much resources I want to devote
to trying to address its deficiencies.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 25 May 2016 02:01:48 +0000

Joe,

Please see my response inline with [Xiaohu]

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, May 25, 2016 1:15 AM
To: Xuxiaohu; Fred Baker (fred); Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Two things below:

The draft is only intended to introduce one additional Softwires encapsulation technology referred to as IP-in-UDP.

You had a similar draft that expired last summer targeted at the Softwires WG (draft-xu-softwire-ip-in-udp). Why is this now targeted at Intarea?

[Xiaohu] I was told by the Softwires WG co-chairs that the Softwires WG is going to be shut down and therefore would not accept any new draft. Hence, I think the Intarea WG should be the right place for this work now.

In other words, this encapsulation is only intended to be used within Softwires networks which are well-managed by a service provider. This encapsulation technology is not intended to be used within the Internet. As such, it seems absolutely possible to configure the I-IP transit core to carry an MTU at least large enough to accommodate the added encapsulation headers.
 Although it has been said in the draft that "IP-in-UDP is just  applicable in those Softwires network environments where fragmentation on the tunnel layer is not needed." I can add a dedicated Applicability Statement section to emphasize that this Softwires encapsulation technology must only be used within Softwires networks which are well-managed by a service provider and must not be used within the Internet. Can this application statement address your concerns on fragmentation and reassembly?

Here's the issue -

I still do not think that this document should be a WG doc, and I frankly don't think it's constructive for you to try to address each flaw as it is raised.

[Xiaohu] Trying to address each flaw as it is raised is not what we IETF attendees are expected to do for any draft in the IETF?

Consider the following:

Now, does that mean that if the cook corrects the salt level that you would now like the food?

[Xiaohu] It's a good example. Hence, for those people who say no to the WG adoption of this draft due to technical reasons, please tell us those technical reasons frankly. Let's see whether those reasons are true in the target scenario. And if they are true let's see whether they are addressable.

I've given reasons I don't think it should be a WG doc. IF it is accepted as a WG doc, I might decide how much resources I want to devote to trying to address its deficiencies.
[Xiaohu] I had expressed clearly that this Softwires encapsulation technology is just targeted for Softwires networks which are well managed. Therefore, fragmentation on the tunnel layer is not needed at all. I don't understand why you are still concerned about those things that would not be issues at all in the target scenario.
Best regards,
Xiaohu

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 25 May 2016 02:24:18 +0000

Hi Joe,

I wonder whether you want to tell me the following truth by the example that you gave: no matter whatever improvements we had done with this draft, those persons who dislike it by the light of nature would dislike it in the end.

Best regards,
Xiaohu

Here's the issue -

Consider the following:

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 24 May 2016 19:42:10 -0700

On Tue, May 24, 2016 at 7:01 PM, Xuxiaohu <[email-masked]> wrote:
> You had a similar draft that expired last summer targeted at the Softwires
> WG (draft-xu-softwire-ip-in-udp). Why is this now targeted at Intarea?
>
>
>
> [Xiaohu] I was told by the Softwires WG co-chairs that the Softwires WG is
> going to be shut down and therefore would not accept any new draft. Hence, I
> think the Intarea WG should be the right place for this work now.
>
>
>
> encapsulation technology is not intended to be used within the Internet. As
> such, it seems absolutely possible to configure the I-IP transit core to
> carry an MTU at least large enough to accommodate the added encapsulation
> headers.
>
>  Although it has been said in the draft that “IP-in-UDP is just  applicable
> in those Softwires network environments where fragmentation on the tunnel
> layer is not needed.” I can add a dedicated Applicability Statement section
> to emphasize that this Softwires encapsulation technology must only be used
> within Softwires networks which are well-managed by a service provider and
> must not be used within the Internet. Can this application statement address
> your concerns on fragmentation and reassembly?
>
>
> Here's the issue -
>
> I still do not think that this document should be a WG doc, and I frankly
> don't think it's constructive for you to try to address each flaw as it is
> raised.
>
> [Xiaohu] Trying to address each flaw as it is raised is not what we IETF
> attendees are expected to do for any draft in the IETF?
>
>
> Consider the following:
>
> [Xiaohu] It’s a good example. Hence, for those people who say no to the WG
> adoption of this draft due to technical reasons, please tell us those
> technical reasons frankly. Let’s see whether those reasons are true in the
> target scenario. And if they are true let’s see whether they are
> addressable.
>
>
>
> I've given reasons I don't think it should be a WG doc. IF it is accepted as
> a WG doc, I might decide how much resources I want to devote to trying to
> address its deficiencies.
>
> [Xiaohu] I had expressed clearly that this Softwires encapsulation
> technology is just targeted for Softwires networks which are well managed.
> Therefore, fragmentation on the tunnel layer is not needed at all. I don’t
> understand why you are still concerned about those things that would not be
> issues at all in the target scenario.
>
Xiaohu,

The technical issue has already been pointed out: there are already
existing protocols that provide the same functionality, are generic
and not restricted for a narrow use case, and have already seen
significant effort into resolving all the issue with UDP encapsulation
that Joe mentioned. This has been pointed out several times now and
you haven't provided any explanation why these protocols are not
sufficient for your use case. Hence this is why you're not seeing
support to make it WG item.

Tom

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 25 May 2016 03:48:54 +0000

Tom,

> Sent: Wednesday, May 25, 2016 10:42 AM
> To: Xuxiaohu
> > [Xiaohu] I was told by the Softwires WG co-chairs that the Softwires
> > WG is going to be shut down and therefore would not accept any new
> > draft. Hence, I think the Intarea WG should be the right place for this work
> now.
> >
> >
> >
> > applicable in those Softwires network environments where fragmentation
> > on the tunnel layer is not needed.” I can add a dedicated
> > Applicability Statement section to emphasize that this Softwires
> > encapsulation technology must only be used within Softwires networks
> > which are well-managed by a service provider and must not be used
> > within the Internet. Can this application statement address your concerns on
> fragmentation and reassembly?
> >
> >
> > Here's the issue -
> >
> > [Xiaohu] Trying to address each flaw as it is raised is not what we
> > IETF attendees are expected to do for any draft in the IETF?
> >
> >
> > Consider the following:
> >
> > [Xiaohu] It’s a good example. Hence, for those people who say no to
> > the WG adoption of this draft due to technical reasons, please tell us
> > those technical reasons frankly. Let’s see whether those reasons are
> > true in the target scenario. And if they are true let’s see whether
> > they are addressable.
> >
> >
> >
> > accepted as a WG doc, I might decide how much resources I want to
> > devote to trying to address its deficiencies.
> >
> > Therefore, fragmentation on the tunnel layer is not needed at all. I
> > don’t understand why you are still concerned about those things that
> > would not be issues at all in the target scenario.
> >
> Xiaohu,
> 
> The technical issue has already been pointed out: there are already existing
> protocols that provide the same functionality, are generic and not restricted for
> a narrow use case, and have already seen significant effort into resolving all the
> issue with UDP encapsulation that Joe mentioned. This has been pointed out
> several times now and you haven't provided any explanation why these
> protocols are not sufficient for your use case. Hence this is why you're not
> seeing support to make it WG item.

What's the existing and generic protocols that provide the same functionality in your mind? GUE? If so, it seems reasonable for those X-in-UDP (X could be VXLAN, VXLAN-GPE, GEVENE, NSH, TRILL) proposals to move forward to being built on GUE rather than UDP since the former has resolved all the issues with UDP encapsulation. However, have you seen any X that is moving towards that direction?

Xiaohu

---

From: Tom Herbert <[email-masked]>
Date: Tue, 24 May 2016 21:04:31 -0700

> What's the existing and generic protocols that provide the same functionality in your mind? GUE? If so, it seems reasonable for those X-in-UDP (X could be VXLAN, VXLAN-GPE, GEVENE, NSH, TRILL) prop…

The existing protocols that provide the same functionality are GUE and
GRE/UDP. GUE allows encapsulation of any IP protocol over UDP, GRE
will allow encapsulation of any Ethertype of UDP. Both of these
provide a means to encapsulate of IPv4 and IPv6 in UDP.

Tom

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 25 May 2016 04:08:19 +0000

> Sent: Wednesday, May 25, 2016 12:05 PM
> To: Xuxiaohu
> >> > [Xiaohu] I was told by the Softwires WG co-chairs that the
> >> > Softwires WG is going to be shut down and therefore would not
> >> > accept any new draft. Hence, I think the Intarea WG should be the
> >> > right place for this work
> >> now.
> >> >
> >> >
> >> >
> >> > In other words, this encapsulation is only intended to be used
> >> > within Softwires networks which are well-managed by a service
> >> > provider. This encapsulation technology is not intended to be used
> >> > within the Internet. As such, it seems absolutely possible to
> >> > configure the I-IP transit core to carry an MTU at least large
> >> > enough to accommodate the added encapsulation headers.
> >> >
> >> > Softwires encapsulation technology must only be used within
> >> > Softwires networks which are well-managed by a service provider and
> >> > must not be used within the Internet. Can this application
> >> > statement address your concerns on
> >> > frankly don't think it's constructive for you to try to address
> >> > each flaw as it is raised.
> >> >
> >> > the WG adoption of this draft due to technical reasons, please tell
> >> > us those technical reasons frankly. Let’s see whether those reasons
> >> > are true in the target scenario. And if they are true let’s see
> >> > whether they are addressable.
> >> >
> >> >
> >> >
> >> > Therefore, fragmentation on the tunnel layer is not needed at all.
> >> > I don’t understand why you are still concerned about those things
> >> > that would not be issues at all in the target scenario.
> >> >
> >> Xiaohu,
> >>
> >> significant effort into resolving all the issue with UDP
> >> encapsulation that Joe mentioned. This has been pointed out several
> >> times now and you haven't provided any explanation why these
> > What's the existing and generic protocols that provide the same functionality
> in your mind? GUE? If so, it seems reasonable for those X-in-UDP (X could be
> VXLAN, VXLAN-GPE, GEVENE, NSH, TRILL) proposals to move forward to being
> built on GUE rather than UDP since the former has resolved all the issues with
> UDP encapsulation. However, have you seen any X that is moving towards that
> direction?
> 
> The existing protocols that provide the same functionality are GUE and GRE/UDP.
> GUE allows encapsulation of any IP protocol over UDP, GRE will allow
> encapsulation of any Ethertype of UDP. Both of these provide a means to
> encapsulate of IPv4 and IPv6 in UDP.

Which is more generic? If both are generic, why we need two?

Xiaohu

---

From: Joe Touch <[email-masked]>
Date: Wed, 25 May 2016 11:11:18 -0700

On 5/24/2016 7:24 PM, Xuxiaohu wrote:
> Hi Joe,
> 
> I wonder whether you want to tell me the following truth by the example
> that you gave: no matter whatever improvements we had done with this
> draft, those persons who dislike it by the light of nature would dislike
> it in the end.

The only improvements that would make this doc useful would be to add
capabilities already in GRE/UDP or GUE/UDP, which we already have.

It is not our obligation to find a way for your document to proceed -
that onus is on you.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 25 May 2016 11:11:30 -0700

> attendees are expected to do for any draft in the IETF?

Yes, addressing flaws is what you're supposed to do - in a draft update
(which hasn't happened in the year since they were raised).

However, you ignored the other issues that come up when this doc is
considered for WG adoption - do we need it?

You haven't made that case. If this were critical to Softwires, they
wouldn't have shut down before they had a solution. You already note
that this approach is more limited in use than the more general
solutions that are already deployed.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 25 May 2016 11:15:01 -0700

> Which is more generic? If both are generic, why we need two?

GRE and GUE represent steps in a direction, as does SEAL. That direction
is to support increased checksums, fragmentation support, control
channels, etc.

This document represents a step backwards, is useful only in more
limited environments, etc.

Again, not a good reason for the WG to adopt it.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 26 May 2016 08:34:59 +0000

> Sent: Thursday, May 26, 2016 2:11 AM
> > I wonder whether you want to tell me the following truth by the
> > example that you gave: no matter whatever improvements we had done
> > with this draft, those persons who dislike it by the light of nature
> > would dislike it in the end.
> 

Let's go over the four things you mentioned earlier in GRE/UDP and GUE/UDP:

	- stronger checksums

In GRE/UDP, in order to use UDP-zero-checksum, it gave the following restrictions:
" 6. UDP Checksum Handling

   6.1. UDP Checksum with IPv4

   For UDP in IPv4, the UDP checksum MUST be processed as specified in
   [RFC768] and [RFC1122] for both transmit and receive. The IPv4

Yong, Crabber, Xu, Herbert                                    [Page 12]
--------------------------------------------------------------------------------
 
Internet-Draft          GRE-in-UDP Encapsulation             March 2016

   header includes a checksum which protects against mis-delivery of
   the packet due to corruption of IP addresses. The UDP checksum
   potentially provides protection against corruption of the UDP header,
   GRE header, and GRE payload. Disabling the use of checksums is a
   deployment consideration that should take into account the risk and
   effects of packet corruption.

   When a decapsulator receives a packet, the UDP checksum field MUST
   be processed. If the UDP checksum is non-zero, the decapsulator MUST
   verify the checksum before accepting the packet. By default a
   decapsulator SHOULD accept UDP packets with a zero checksum. A node
   MAY be configured to disallow zero checksums per [RFC1122]; this may
   be done selectively, for instance disallowing zero checksums from
   certain hosts that are known to be sending over paths subject to
   packet corruption. If verification of a non-zero checksum fails, a
   decapsulator lacks the capability to verify a non-zero checksum, or
   a packet with a zero-checksum was received and the decapsulator is
   configured to disallow, the packet MUST be dropped and an event MAY
   be logged.

   6.2. UDP Checksum with IPv6

   For UDP in IPv6, the UDP checksum MUST be processed as specified in
   [RFC768] and [RFC2460] for both transmit and receive.

   When UDP is used over IPv6, the UDP checksum is relied upon to
   protect both the IPv6 and UDP headers from corruption. As such, A
   default GRE-in-UDP Tunnel MUST perform UDP checksum; A TMCE GRE-in-
   UDP Tunnel MAY be configured with the UDP zero-checksum mode if the
   traffic-managed controlled environment or a set of closely
   cooperating traffic-managed controlled environments (such as by
   network operators who have agreed to work together in order to
   jointly provide specific services) meet at least one of following
   conditions:

   a. It is known (perhaps through knowledge of equipment types and
      lower layer checks) that packet corruption is exceptionally
      unlikely and where the operator is willing to take the risk of
      undetected packet corruption.

   b. It is judged through observational measurements (perhaps of
      historic or current traffic flows that use a non-zero checksum)
      that the level of packet corruption is tolerably low and where
      the operator is willing to take the risk of undetected packet
      corruption.

Yong, Crabber, Xu, Herbert                                    [Page 13]
--------------------------------------------------------------------------------
 
Internet-Draft          GRE-in-UDP Encapsulation             March 2016

   c. Carrying applications that are tolerant of mis-delivered or
      corrupted packets (perhaps through higher layer checksum,
      validation, and retransmission or transmission redundancy) where
      the operator is willing to rely on the applications using the
      tunnel to survive any corrupt packets.

   The following requirements apply to a TMCE GRE-in-UDP tunnel that
   use UDP zero-checksum mode:

     a. Use of the UDP checksum with IPv6 MUST be the default
        configuration of all GRE-in-UDP tunnels.

     b. The GRE-in-UDP tunnel implementation MUST comply with all
        requirements specified in Section 4 of [RFC6936] and with
        requirement 1 specified in Section 5 of [RFC6936].

     c. The tunnel decapsulator SHOULD only allow the use of UDP zero-
        checksum mode for IPv6 on a single received UDP Destination
        Port regardless of the encapsulator. The motivation for this
        requirement is possible corruption of the UDP Destination Port,
        which may cause packet delivery to the wrong UDP port. If that
        other UDP port requires the UDP checksum, the mis-delivered
        packet will be discarded.

     d. It is RECOMMENDED that the UDP zero-checksum mode for IPv6 is
        only enabled for certain selected source addresses. The tunnel
        decapsulator MUST check that the source and destination IPv6
        addresses are valid for the GRE-in-UDP tunnel on which the
        packet was received if that tunnel uses UDP zero-checksum mode
        and discard any packet for which this check fails.

     e. The tunnel encapsulator SHOULD use different IPv6 addresses for
        each GRE-in-UDP tunnel that uses UDP zero-checksum mode
        regardless of the decapsulator in order to strengthen the
        decapsulator's check of the IPv6 source address (i.e., the same
        IPv6 source address SHOULD NOT be used with more than one IPv6
        destination address, independent of whether that destination
        address is a unicast or multicast address). When this is not
        possible, it is RECOMMENDED to use each source IPv6 address for
        as few UDP zero-checksum mode GRE-in-UDP tunnels as is feasible.

     f. When any middlebox exists on the path of a GRE-in-UDP tunnel,
        it is RECOMMENDED to use the default mode, i.e. use UDP
        checksum, to reduce the chance that the encapsulated packets to
        be dropped.

Yong, Crabber, Xu, Herbert                                    [Page 14]
--------------------------------------------------------------------------------
 
Internet-Draft          GRE-in-UDP Encapsulation             March 2016

     g. Any middlebox that allows the UDP zero-checksum mode for IPv6
        MUST comply with requirement 1 and 8-10 in Section 5 of
        [RFC6936].

     h. Measures SHOULD be taken to prevent IPv6 traffic with zero UDP
        checksums from "escaping" to the general Internet; see Section
        8 for examples of such measures.

     i. IPv6 traffic with zero UDP checksums MUST be actively monitored
        for errors by the network operator. For example, the operator
        may monitor Ethernet layer packet error rates.

     j. If a packet with a non-zero checksum is received, the checksum
        MUST be verified before accepting the packet. This is
        regardless of whether the tunnel encapsulator and decapsulator
        have been configured with UDP zero-checksum mode.

   The above requirements do not change either the requirements
   specified in [RFC2460] as modified by [RFC6935] or the requirements
   specified in [RFC6936].

   The requirement to check the source IPv6 address in addition to the
   destination IPv6 address, plus the strong recommendation against
   reuse of source IPv6 addresses among GRE-in-UDP tunnels collectively
   provide some mitigation for the absence of UDP checksum coverage of
   the IPv6 header. A traffic-managed controlled environment that
   satisfies at least one of three conditions listed above in this
   section provides additional assurance.

   A GRE-in-UDP tunnel is suitable for transmission over lower layers
   in the traffic-managed controlled environments that are allowed by
   the exceptions stated above and the rate of corruption of the inner
   IP packet on such networks is not expected to increase by comparison
   to GRE traffic that is not encapsulated in UDP.  For these reasons,
   GRE-in-UDP does not provide an additional integrity check except
   when GRE checksum is used when UDP zero-checksum mode is used with
   IPv6, and this design is in accordance with requirements 2, 3 and 5
   specified in Section 5 of [RFC6936].

   Generic Router Encapsulation (GRE) does not accumulate incorrect
   state as a consequence of GRE header corruption. A corrupt GRE
   packet may result in either packet discard or forwarding of the
   packet without accumulation of GRE state. Active monitoring of GRE-
   in-UDP traffic for errors is REQUIRED as occurrence of errors will
   result in some accumulation of error information outside the
   protocol for operational and management purposes. This design is in
   accordance with requirement 4 specified in Section 5 of [RFC6936].

Yong, Crabber, Xu, Herbert                                    [Page 15]
--------------------------------------------------------------------------------
 
Internet-Draft          GRE-in-UDP Encapsulation             March 2016

   The remaining requirements specified in Section 5 of [RFC6936] are
   not applicable to GRE-in-UDP.  Requirements 6 and 7 do not apply
   because GRE does not include a control feedback mechanism.
   Requirements 8-10 are middlebox requirements that do not apply to
   GRE-in-UDP tunnel endpoints (see Section 7.1 for further middlebox
   discussion).

   It is worth mentioning that the use of a zero UDP checksum should
   present the equivalent risk of undetected packet corruption when
   sending similar packet using GRE-in-IPv6 without UDP [RFC7676] and
   without GRE checksums.

   In summary, a TMCE GRE-in-UDP Tunnel is allowed to use UDP-zero-
   checksum mode for IPv6 when the conditions and requirements stated
   above are met. Otherwise the UDP checksum need to be used for IPv6
   as specified in [RFC768] and [RFC2460]. Use of GRE checksum is
   RECOMMENED when the UDP checksum is not used.
"

In GUE, to support UDP-checksum-zero, it said 

" Therefore, when GUE is used over
    IPv6, either the UDP checksum must be enabled or the GUE header
    checksum must be used.  An encapsulator MAY set a zero UDP checksum
    for performance or implementation reasons, in which case the GUE
    header checksum MUST be used or applicable requirements for using
    zero UDP checksums in [GREUDP] MUST be met. If the UDP checksum is
    enabled, then the GUE header checksum should not be used since it is
    mostly redundant."

It's easy for me to add the similar words to the IP-in-UDP draft like "the applicable requirements for using zero UDP checksum in [GREUDP] MUST be met when zero UDP checksum is used by the tunnel ingress". However, the major goal for disabling the UDP checksum is to improve the performance. When GUE header checksum is used and/or the bunch of applicable requirements as described in GRE/UDP are verified, is the goal of improving performance still achievable? If not, why not directly enable the UDP-checksum instead?

	- fragmentation support

In GRE/UDP, it said 

" 4.1. MTU and Fragmentation

   Regarding packet fragmentation, an encapsulator/decapsulator SHOULD
   be compliant with [RFC7588] and perform fragmentation before the
   encapsulation. The size of fragments SHOULD be less or equal to the
   PMTU associated with the path between the GRE ingress and the GRE
   egress tunnel endpoints minus the GRE and UDP overhead ..."

in GUE, it said

" 4.9. MTU and fragmentation

    Standard conventions for handling of MTU (Maximum Transmission Unit)
    and fragmentation in conjunction with networking tunnels
    (encapsulation of layer 2 or layer 3 packets) should be followed.
    Details are described in MTU and Fragmentation Issues with In-the-
    Network Tunneling [RFC4459]... "
	
It seems that the only missing thing in the IP-in-UDP draft is to allow the outer fragmentation. However, as it said in (https://tools.ietf.org/html/draft-ietf-intarea-tunnels-02#page-13), " ...IPsec performs only Outer Fragmentation; this distinguishes it from IP-in-IP, which performs only Inner Fragmentation. " Note that IP-in-IP is the dominant encapsulation choice within Softwires networks. In other words, performing only inner fragmentation works very well in practice. Furthermore, the outer fragmentation issue (e.g., reassembly cost for the egress) would become even worse since the fragments of X-in-UDP packets are more likely to be forwarded across different paths than those of X-in-IP and X-in-GRE packets. Hence, I'm wondering whether it's worthwhile to support the outer fragmentation on UDP encapsulated packets which seems useless in practice.

I haven't found any description of this in both GRE/UDP and GUE. Did you?

I haven't found any description of this in both GRE/UDP and GUE. Did you?

Xiaohu

> It is not our obligation to find a way for your document to proceed - that onus is
> on you.
> 
> Joe

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 26 May 2016 08:45:46 +0000

> Sent: Thursday, May 26, 2016 2:12 AM
> Yes, addressing flaws is what you're supposed to do - in a draft update (which
> hasn't happened in the year since they were raised).
> 
> However, you ignored the other issues that come up when this doc is considered
> for WG adoption - do we need it?

Not ignored. I just need to confirm whether it's worthwhile to be fixed in practice (i.e., in the real target network environment) before fixing them.

> You haven't made that case. If this were critical to Softwires, they wouldn't have
> shut down before they had a solution. You already note that this approach is
> more limited in use than the more general solutions that are already deployed.

There is no free lunch. More general means more complex. That's why both fruit knives and Swiss Army knives are sold in the market. BTW, what are the more general solutions that are already deployed in my mind?

Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 26 May 2016 08:55:46 +0000

> Sent: Thursday, May 26, 2016 2:15 AM
> To: Xuxiaohu; Tom Herbert
> Cc: Fred Baker (fred); Wassim Haddad; [email-masked]
> GRE/UDP.
> >> > GUE allows encapsulation of any IP protocol over UDP, GRE will
> >> > allow encapsulation of any Ethertype of UDP. Both of these provide
> >> > a means to encapsulate of IPv4 and IPv6 in UDP.
> >
> GRE and GUE represent steps in a direction, as does SEAL. That direction is to
> support increased checksums, fragmentation support, control channels, etc.

GRE or GRE/UDP?

> This document represents a step backwards, is useful only in more limited
> environments, etc.

Thanks for confirming that this encapsulation is useful in limited environment (i.e., Softwires networks). As said before, both fruit knives and Swiss Army knives are sold in the market. In other words, both generic and simple approaches have their own applicable scenarios.

Xiaohu

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 07:47:22 -0700

On 5/26/2016 1:55 AM, Xuxiaohu wrote:
> Thanks for confirming that this encapsulation is useful in limited environment (i.e., Softwires networks).
I never said what environment it was useful for, but clearly Softwires
doesn't *need* this solution (they already have two that work).

> Not ignored. I just need to confirm whether it's worthwhile to be fixed in practice (i.e., in the real target network environment) before fixing them.
There's no point in fixing a flat on a car with a broken engine.

There's also no point in continuing to discuss why.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 14:56:06 +0000

Hi, please see the following drafts regarding fortified IDs and fragmentation support:

https://datatracker.ietf.org/doc/draft-templin-intarea-grefrag/
https://datatracker.ietf.org/doc/draft-herbert-gue-fragmentation/

Thanks - Fred
[email-masked]

> From: Int-area [mailto:[email-masked]] On Behalf Of Xuxiaohu
> Sent: Thursday, May 26, 2016 1:35 AM
> To: Joe Touch <[email-masked]>; Fred Baker (fred) <[email-masked]>; Wassim Haddad <[email-masked]>
> Cc: [email-masked]
> It's easy for me to add the similar words to the IP-in-UDP draft like "the applicable requirements for using zero UDP checksum in
> [GREUDP] MUST be met when zero UDP checksum is used by the tunnel ingress". However, the major goal for disabling the UDP
> checksum is to improve the performance. When GUE header checksum is used and/or the bunch of applicable requirements as
> described in GRE/UDP are verified, is the goal of improving performance still achievable? If not, why not directly enable the UDP-
> checksum instead?
> 
> 	- fragmentation support
> 
> In GRE/UDP, it said
> 
> It seems that the only missing thing in the IP-in-UDP draft is to allow the outer fragmentation. However, as it said in
> (https://tools.ietf.org/html/draft-ietf-intarea-tunnels-02#page-13), " ...IPsec performs only Outer Fragmentation; this distinguishes it
> from IP-in-IP, which performs only Inner Fragmentation. " Note that IP-in-IP is the dominant encapsulation choice within Softwires
> networks. In other words, performing only inner fragmentation works very well in practice. Furthermore, the outer fragmentation
> issue (e.g., reassembly cost for the egress) would become even worse since the fragments of X-in-UDP packets are more likely to be
> forwarded across different paths than those of X-in-IP and X-in-GRE packets. Hence, I'm wondering whether it's worthwhile to
> support the outer fragmentation on UDP encapsulated packets which seems useless in practice.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 15:12:31 +0000

Hi Tom,

> From: Int-area [mailto:[email-masked]] On Behalf Of Tom Herbert
> Sent: Friday, May 20, 2016 9:14 AM
> To: Ted Lemon <[email-masked]>
> Cc: [email-masked]

I thought at one time we had come up with an idea for omitting the GUE
header when the payload is a plain IPv4/IPv6 packet. There was a check
of the first four bits following the UDP header to see if they encoded the
value '4' or '6'. Did that not make it into the draft?

Thanks - Fred

---

From: Tom Herbert <[email-masked]>
Date: Thu, 26 May 2016 08:59:21 -0700

> value '4' or '6'. Did that not make it into the draft?
Yes, we had come up with the idea and I have implemented the
prototype. It is not in the draft. I believe the only discernible
benefit we could identify was that it saves 4 bytes of overhead. The
major drawback is that this only works specifically for IPv4 and IPv6.

Tom

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 16:16:55 +0000

Hi Tom,

> Sent: Thursday, May 26, 2016 8:59 AM
> To: Templin, Fred L <[email-masked]>
> Cc: Ted Lemon <[email-masked]>; [email-masked]

I don't see the drawback; I think those two IP protocol versions could
carry us forward into the forseeable future. Are you thinking there
could be another IP version on the near term horizon?

Thanks - Fred

---

From: Tom Herbert <[email-masked]>
Date: Thu, 26 May 2016 09:24:08 -0700

> could be another IP version on the near term horizon?
It's not about another IP version, we are using GUE to encapsulate
various IP protocols-- GRE, MPLS, EtherIP, TCP and SCTP (in
draft-herbert-transports-over-udp-00), etc.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 09:33:58 -0700

> could be another IP version on the near term horizon?

IMO, this is a bad idea unless the protocol is specifically limited to
having IP as a payload. Otherwise, there's no way to ensure that some
other protocol won't start off with those values.

In particular, ICMP for IPv4 isn't an IP packet but has no rule about
the first 4 bits. That field ought to be large enough to handle any
protocol that might need to share the path with IP.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 16:40:02 +0000

Hi Joe,

> Sent: Thursday, May 26, 2016 9:34 AM
> To: Templin, Fred L <[email-masked]>; Tom Herbert <[email-masked]>
> Cc: [email-masked]

Tom's proposal unambiguously differentiates raw IPv4 and IPv6 from
all other encapsulations. If it is raw IPv4 or IPv6, the GUE 4-byte header
is not present. Otherwise, the GUE 4-byte header is present.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 09:56:56 -0700

> is not present. Otherwise, the GUE 4-byte header is present.
The only way the proposal "unambiguously" knows the difference is when
the first 4 bits are 4 or 6, but those are valid configurations of the
first 4 bits of other protocols (notably ICMPv4).

So if you see 4 or 6, you can't just assume that the GUE header is not
present.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 17:08:55 +0000

Hi Joe,

> Sent: Thursday, May 26, 2016 9:57 AM

There is more to it than that - Tom's proposal can unambiguously distinguish
raw IPv4/IPv6 from all others.

> So if you see 4 or 6, you can't just assume that the GUE header is not

You don't look for 4 or 6; you look for the 2nd-most significant bit being
set to 1. If it is set to 1, then you examine the 3rd and 4th MSBs and
act only of the combined 4 MSBs are either 4 or 6.

Tom can state this more eloquently and accurately than I just did...

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 10:41:21 -0700

> act only of the combined 4 MSBs are either 4 or 6.
Here's the problem:

The first 4 bits are either part of the GUE header or IPv4 or IPv6.

In the diagrams in draft-ietf-nvo3-gue and RFC791 they're indicated in
the following bit order: 0,1,2,3

In GUE, these are 0,0,x,x

In IPv4, these are 0,0,1,0

In IPv6, these are 0,1,1,0

So you absolutely know the difference between GUE and IPv6, but not so
much for IPv4. For IPv4, you still then have no idea whether the GUE
4-byte extension is used or not.

You'd have to redesign GUE to move the 00 bits to the high-part of the
first nibble for this to work.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 26 May 2016 10:52:32 -0700

On Thu, May 26, 2016 at 10:41 AM, Joe Touch <[email-masked]> wrote:
> In the diagrams in draft-ietf-nvo3-gue and RFC791 they're indicated in the
> following bit order: 0,1,2,3
>
IPv4 is 0,1,0,0. The first two bits of the GUE header are version
number which is currently is 00, so if we declare that GUE version 01
is direct IP encapsulation this scheme does work. Normal GUE has
version 00, direct IP is version 01.

> So you absolutely know the difference between GUE and IPv6, but not so much
> for IPv4. For IPv4, you still then have no idea whether the GUE 4-byte
> extension is used or not.
>
> You'd have to redesign GUE to move the 00 bits to the high-part of the first
> nibble for this to work.
>
It's a matter of defining a new version that happens to work due to
happenstance. It's feasible, but I am still worried that it's a
limited use case and saving only four bytes of overhead is
inconsequential.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 10:56:48 -0700

On 5/26/2016 10:52 AM, Tom Herbert wrote:
> IPv4 is 0,1,0,0. 
Not LSB to MSB, which is how both GUE and RFC791 define the header:

GUE:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Source port            |      Destination port         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Length              |          Checksum             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0x0|C|   Hlen  |  Proto/ctype  |            Flags            |E|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

IPv4:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Version|  IHL  |Type of Service|          Total Length         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 26 May 2016 11:22:45 -0700

>From RFC791:

"Whenever an octet represents a numeric quantity the left most bit in
the diagram is the high order or most significant bit.  That is, the
bit
labeled 0 is the most significant bit."

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 11:26:13 -0700

> labeled 0 is the most significant bit."
Arrgh.

Yup. OK, so the reason this would work is only because we no longer use
IP versions 0..3.

Got it.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 18:35:40 +0000

Hi Joe and Tom,

> Sent: Thursday, May 26, 2016 11:26 AM
> To: Tom Herbert <[email-masked]>
> Cc: Templin, Fred L <[email-masked]>; [email-masked]

I for one like it, and I am using it in my AERO implementation. As Joe points out
however it does not account for fragmentation, identification etc. But, as long
as you use it in a carefully controlled environment it should be OK.

Can we have this added back to the GUE spec?

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 11:42:38 -0700

> as you use it in a carefully controlled environment it should be OK.

Nothing accounts for fragmentation except to reassemble the contents if
you have to do DPI on-path that examines the body of ANY IP packet,
though. That's worth noting, but my advice would be to not examine the
packet that way on-path. We really need to push back on that.

> Can we have this added back to the GUE spec?
I wonder why we would bother - it just saves 4 bytes in the default
case, but it also disables use of extension headers and the magic number
trick in section 4.6.2. Are losing both of those features really worth
that few bytes of overhead AND the special-case processing?

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 26 May 2016 11:52:06 -0700

> Can we have this added back to the GUE spec?
Since it technically defines a new version of GUE and other than the
two bit version number there is nothing else in common with GUEv0
format, I would say this should be a new spec (I need to go back and
see if I wrote it up this in a draft).

Fred, do you have any objective data showing the benefits? I think we
need something tangible to justify complexity (i.e. burning 25% of the
GUE version number space  ;-) )

Thanks,
Tom

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 26 May 2016 18:58:17 +0000

Hi Tom and Joe,

> Sent: Thursday, May 26, 2016 11:52 AM
> Cc: Joe Touch <[email-masked]>; [email-masked]

I want to use it over low-end wireless data links such as for aviation. In that
domain, it is common to see links with 1Mbps or less bandwidth. So, any
extra encapsulation is expensive. I can dampen this to some extent using
techniques such as lzo message compression, but still the four bytes would
contribute to eating scarce available bandwidth.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Thu, 26 May 2016 12:59:39 -0700

FWIW:

Another caveat about using the header check to avoid the GUE header for
IPv4 and IPv6:

    (A)- looking for only bit 01=1 would expect raw IP encapsulation for
versions 4-7 and 12-15, leaving out 8-11 (of which 10 and 11 are still
possible)

    (B)- looking for either 01, 10, or 11 would allow all future IP
versions but prevent GUE from having versions of its own

Of these two, IMO, (A) is preferable, but I would strongly suggest:

    (C)- match "01" means native IP, supporting only currently-used
versions 4 and 6
    - define "10" and "11" as Reserved for future use

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 27 May 2016 01:36:41 +0000

> Sent: Thursday, May 26, 2016 10:47 PM
> I never said what environment it was useful for, but clearly Softwires doesn't
> *need* this solution (they already have two that work).

Then what's the limited environment in your mind?

> > Not ignored. I just need to confirm whether it's worthwhile to be fixed in
> practice (i.e., in the real target network environment) before fixing them.

Please don't waste your time on giving meaningless examples while escaping from technical questions on your arguments.

Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 27 May 2016 10:08:28 +0000

<Note that I have changed the subject of the email hence it has nothing to do with the WG adoption call now. It's just a discussion on a particular issue which is related to those WGs which are working on UDP tunnels. The reason for containing the old email is to use it as a background which may be useful for better understanding of this particular issue>

The possible side-effect of performing fragmentation on UDP encapsulated packets is to worsen the reassembly burden on tunnel egress since fragments of UDP encapsulated packets are more likely to be forwarded across different paths towards the tunnel egress than those of IP or GRE encapsulated packets.

It seems that most X-over-UDP proposals choose to prohibit the tunnel ingress from performing fragmentation on UDP encapsulated packets. See the following quoted text regarding fragmentation from those X-over-UDP drafts:

LISP:

When an ITR receives a packet from a site-facing interface and adds H
   octets worth of encapsulation to yield a packet size greater than L
   octets, it resolves the MTU issue by first splitting the original
   packet into 2 equal-sized fragments.  A LISP header is then prepended
   to each fragment.

VXLAN:

VTEPs MUST NOT fragment VXLAN packets.  Intermediate routers may
   fragment encapsulated VXLAN packets due to the larger frame size.
   The destination VTEP MAY silently discard such VXLAN fragments.

VXLAN-GPE:

VTEPs MUST never fragment an encapsulated VXLAN GPE packet, and when
   the outer IP header is IPv4, VTEPs MUST set the DF bit in the outer
   IPv4 header.

GEVENE:

   To prevent fragmentation and maximize performance, the best practice
   when using Geneve is to ensure that the MTU of the physical network
   is greater than or equal to the MTU of the encapsulated network plus
   tunnel headers.

GUE:

    If a packet is fragmented before encapsulation in GUE, all the
    related fragments must be encapsulated using the same source port
    (inner flow identifier). An operator may set MTU to account for
    encapsulation overhead and reduce the likelihood of fragmentation.

GRE/UDP

   encapsulation.

However, the above choice seems conflict with the requirements as described in https://tools.ietf.org/html/draft-ietf-intarea-tunnels-02

I wonder whether the IETF should reach a consensus on whether or not the fragmentation on UDP encapsulated packets should be allowed.
  
Best regards,
Xiaohu

> From: Xuxiaohu
> Sent: Thursday, May 26, 2016 4:35 PM
> To: 'Joe Touch'; Fred Baker (fred); Wassim Haddad
> Cc: [email-masked]
> Subject: RE: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03
> 
> 
> 
> It's easy for me to add the similar words to the IP-in-UDP draft like "the
> applicable requirements for using zero UDP checksum in [GREUDP] MUST be
> met when zero UDP checksum is used by the tunnel ingress". However, the
> major goal for disabling the UDP checksum is to improve the performance.
> When GUE header checksum is used and/or the bunch of applicable
> requirements as described in GRE/UDP are verified, is the goal of improving
> performance still achievable? If not, why not directly enable the UDP-checksum
> instead?
> 
> 	- fragmentation support
> 
> In GRE/UDP, it said
> 
> (https://tools.ietf.org/html/draft-ietf-intarea-tunnels-02#page-13), " ...IPsec
> performs only Outer Fragmentation; this distinguishes it from IP-in-IP, which
> performs only Inner Fragmentation. " Note that IP-in-IP is the dominant
> encapsulation choice within Softwires networks. In other words, performing
> only inner fragmentation works very well in practice. Furthermore, the outer
> fragmentation issue (e.g., reassembly cost for the egress) would become even
> worse since the fragments of X-in-UDP packets are more likely to be forwarded
> across different paths than those of X-in-IP and X-in-GRE packets. Hence, I'm
> wondering whether it's worthwhile to support the outer fragmentation on UDP
> encapsulated packets which seems useless in practice.
>

---

From: [email-masked]
Date: Fri, 27 May 2016 12:50:02 +0200

> I wonder whether the IETF should reach a consensus on whether or not the fragmentation on UDP encapsulated packets should be allowed.

Having just implemented fragmentation and reassembly support in a MAP BR...

MAP does IPv4 over IPv6 tunnels with shared IPv4 address (aka routing on ports).

- Inner IPv4 fragmentation is relatively easy, cause you don't need to reassemble you can do virtual reassembly
- Outer IPv6 reassembly is much harder, cause you have to do it on the tunnel egress.

It is not possible to implement reassembly complying with IETF RFCs.
The IPv4 reassembly buffer is specified in IPv4 as 15 seconds, in IPv6 as 60 seconds.
My implementation does upwards of 100Mpps, you do the numbers.

If you can:
 - Guarantee always in sequence
 - Maximum fragment chain of 2
 - Maximum time and packet gap between first and last fragment of _very_small_

Then sure it is implementable, but if not then you're just setting yourself up for a DOS attack.
And if you have that much control over the environment, just increase the MTU in the tunnel domain.

Code is here:
https://git.fd.io/cgit/vpp/tree/vnet/vnet/map/ip6_map.c#n530

So in short, IETF can say whatever they like, that's not going to change reality.

Best regards,
Ole

---

From: Tom Herbert <[email-masked]>
Date: Fri, 27 May 2016 08:06:18 -0700

Xiaohu,

I don't understand why UDP encapsulation would make things worse than
other encapsulations. Fragmentation would be needed at tunnel ingress
when packet size exceeds MTU of the tunnel. RFC4459 describes the
issues and general solutions that apply to the different techniques of
IP tunneling.

> It seems that most X-over-UDP proposals choose to prohibit the tunnel ingress from performing fragmentation on UDP encapsulated packets. See the following quoted text regarding fragmentation from t…
Please look a draft-herbert-gue-fragmentation-02. This is a method
where the fragmentation/reassembly is performed in the encapsulation
layer instead of (outer or inner) IP layer.

Tom

---

From: Xuxiaohu <[email-masked]>
Date: Sat, 28 May 2016 01:26:55 +0000

Hi Ole,

> From: lisp [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Friday, May 27, 2016 6:50 PM
> To: Xuxiaohu
> Cc: Softwires WG; [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> Subject: Re: [lisp] [Softwires] Is it feasible to perform fragmentation on UDP
> encapsulated packets.
> 
> > <Note that I have changed the subject of the email hence it has nothing to do
> with the WG adoption call now. It's just a discussion on a particular issue which is
> related to those WGs which are working on UDP tunnels. The reason for
> containing the old email is to use it as a background which may be useful for
> better understanding of this particular issue>
> >
> > The possible side-effect of performing fragmentation on UDP encapsulated
> packets is to worsen the reassembly burden on tunnel egress since fragments of
> UDP encapsulated packets are more likely to be forwarded across different
> paths towards the tunnel egress than those of IP or GRE encapsulated packets.
> >
> > It seems that most X-over-UDP proposals choose to prohibit the tunnel ingress
> from performing fragmentation on UDP encapsulated packets. See the following
> quoted text regarding fragmentation from those X-over-UDP drafts:
> >
> > LISP:
> >
> > However, the above choice seems conflict with the requirements as described
> in https://tools.ietf.org/html/draft-ietf-intarea-tunnels-02
> >
> >
> > I wonder whether the IETF should reach a consensus on whether or not the
> fragmentation on UDP encapsulated packets should be allowed.
> 

Thanks a lot for sharing your implementation experience!

> And if you have that much control over the environment, just increase the MTU

I couldn't agree more.

> So in short, IETF can say whatever they like, that's not going to change reality.

Wouldn't it be better if the IETF guidelines on tunnel fragmentation could be more in conformity with the reality?

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Sat, 28 May 2016 02:59:00 +0000

Hi Tom,

> Sent: Friday, May 27, 2016 11:06 PM
> To: Xuxiaohu
> Cc: [email-masked]; Softwires WG; [email-masked]; [email-masked];
> [email-masked]
> Subject: Re: [Int-area] Is it feasible to perform fragmentation on UDP
> encapsulated packets.
> 
> I don't understand why UDP encapsulation would make things worse than other
> encapsulations. Fragmentation would be needed at tunnel ingress when packet

When performing fragmentation after UDP encapsulation (a.k.a., outer fragmentation), the first fragment (still a UDP packet) and the subsequent fragments (not a UDP packet anymore) of a given UDP encapsulated packet are more likely to be routed over different paths in contrast to IP or GRE encapsulated packets. As a result, the reordering issue on the tunnel egress which is required to do reassembly would become even worse.

> size exceeds MTU of the tunnel. RFC4459 describes the issues and general
> solutions that apply to the different techniques of IP tunneling.

Partially agree. The use of UDP as a tunneling protocol was not popular at the time when RFC4459 was published. Otherwise, the worse recording issue associated with the outer fragmentation of UDP encapsulated packets may need more attention in that RFC.

By the way, RFC4459 has illustrated the risk of performing [outer] fragmentation on tunneled packets clearly and said clearly that "...   So, if reassembly could be made to work sufficiently reliably, this would be one acceptable fallback solution but only for IPv6." However, I just occasionally noticed that in section 3.3.2.1 (Tunneling GRE over IPv4) of RFC7588, it said "By default, the GRE ingress node does not fragment delivery packets. However, the GRE ingress node includes a configuration option that allows delivery packet fragmentation." Does it mean there is a conflict between those two RFCs regarding whether outer fragmentation is needed for GRE over IPv4?

> Please look a draft-herbert-gue-fragmentation-02. This is a method where the
> fragmentation/reassembly is performed in the encapsulation layer instead of
> (outer or inner) IP layer.

It's an interesting idea. However, since it needs a dedicated fragmentation field in the GUE header, it seems that this approach is not suitable for the compressed GUE packet (a.k.a., in the same format of IP-in-UDP) where the fragmentation field and even the whole GUE header is gone.

Best regards,
Xiaohu

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Sat, 28 May 2016 14:34:14 +0000

Wasim, Juan Carlos,

Back to your original request, I do not support adoption of draft-xu-intarea-ip-in-udp. I also did not support draft-xu-softwire-ip-in-udp.

I do not believe there’s a case for this new tunnel type. I also believe that a deeper look at the potential problem space can yield better solutions, as opposed to a solution looking for a problem.

Thanks,

— Carlos Pignataro.

On May 19, 2016, at 1:03 PM, Wassim Haddad <[email-masked]<mailto:[email-masked]>> wrote:

Dear all,

Regards,

Wassim & Juan Carlos

---

From: joel jaeggli <[email-masked]>
Date: Mon, 30 May 2016 01:23:48 +0200

> reassembly). If you make this requirement, you can handle IPv6 over the

Yeah I don't support it for this reason. getting IP fragments back
together in the same place a reassembled is hard is in some cases
especially when you hash. (see frag drop) given alternatives that better
address such situations it seems hard to justify.

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 30 May 2016 10:09:12 +0000

Carlos

From: Int-area [mailto:[email-masked]] On Behalf Of Carlos Pignataro (cpignata)
Sent: Saturday, May 28, 2016 10:34 PM
To: Wassim Haddad
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Wasim, Juan Carlos,

The problem associated with the LB approach as proposed in [RFC5640<https://tools.ietf.org/html/rfc5640>] is obvious. In other words, there is no need for looking for at all. With regard to whether or not the IP-in-UDP encapsulation is the best solution to that problem, it is another thing.

Xiaohu

Thanks,

— Carlos Pignataro.

Dear all,

Regards,

Wassim & Juan Carlos

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Mon, 30 May 2016 14:56:56 +0000

Xiaohu,

Please see inline.

On May 30, 2016, at 6:09 AM, Xuxiaohu <[email-masked]<mailto:[email-masked]>> wrote:

Carlos

Wasim, Juan Carlos,

The problem associated with the LB approach as proposed in [RFC5640<https://tools.ietf.org/html/rfc5640>] is obvious.

Obviousness implies some qualitative form of value judgement. As a technical term, it falls largely short. If you feel it is obvious, and you feel t’s important, make it so, because it is not.

You mention RFC 5640. We seem to have managed since 2009 (publishing date) without a new tunnel type. But more importantly, RFC 5640 concerns itself with a different problem space: augmenting BGP signaling, not defining a new tunnel type and UDP port.

Thanks,

— Carlos.

In other words, there is no need for looking for at all. With regard to whether or not the IP-in-UDP encapsulation is the best solution to that problem, it is another thing.

Xiaohu

Thanks,

— Carlos Pignataro.

Dear all,

Regards,

Wassim & Juan Carlos

---

From: Tom Herbert <[email-masked]>
Date: Mon, 30 May 2016 09:30:38 -0700

> When performing fragmentation after UDP encapsulation (a.k.a., outer fragmentation), the first fragment (still a UDP packet) and the subsequent fragments (not a UDP packet anymore) of a given UDP e…
Presumably you are referring to ECMP and devices that parse the
transport header to use port information in the hash. There are two
ways this can be "fixed" in devices that do ECMP. 1) If the packet is
a fragment don't parse the transport header. This way all fragments
will be ECMP routed based on just L3 information so they all take the
same path. 2) Use non-zero IPv6 flow label and don't parse L4 header
to do ECMP. This means all fragments and all non-fragments of a flow
will follow the same path. The flow label approach also works with
protocols other than TCP and UDP as well as any arbitrary list of
extension headers. Both of these techniques have been implement in
Linux stack.

> Partially agree. The use of UDP as a tunneling protocol was not popular at the time when RFC4459 was published. Otherwise, the worse recording issue associated with the outer fragmentation of UDP e…
Yes, but UDP was popular long before it was used for tunneling and the
problems with it of fragmentation/reassembly were well known.
Considering tunneling with UDP doesn't really change anything, the
same solutions (like those listed in RFC4459) still apply.

> It's an interesting idea. However, since it needs a dedicated fragmentation field in the GUE header, it seems that this approach is not suitable for the compressed GUE packet (a.k.a., in the same f…
Different header formats can be used for different packets in the same
flow. The outer UDP header and flow label are sufficient for ECMP, as
long devices aren't parsing into the GUE payload to do ECMP then all
packets for a flow (fragmented or not) should follow the same path.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 30 May 2016 11:12:24 -0700

> address such situations it seems hard to justify.

If you intend to support recursive IP tunneling* and believe that IP has
a minimum MTU, then you have to accept reassembly.

Joe

* where "recursive IP tunneling" is IP in [zero or more other protocols]
in IP.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 31 May 2016 08:45:39 +1200

And being pedantic...
> a minimum MTU, then you have to accept reassembly.

If you intend to support recursive datagram tunneling and believe that any path has
a minimum MTU, then you have to accept reassembly.

This is physics, and nothing to do with design details.

(Something I discovered in about 1983, when implementing OSI/CLNP
at CERN over a homebrew network with 128 byte packets.)

   Brian

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 31 May 2016 07:14:33 +0000

From: Carlos Pignataro (cpignata) [mailto:[email-masked]]
Sent: Monday, May 30, 2016 10:57 PM
To: Xuxiaohu
Cc: Wassim Haddad; [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Xiaohu,

Please see inline.

Carlos

Wasim, Juan Carlos,

RFC7510 proposes to use UDP tunnels instead of GRE or L2TPv3 tunnels to carry MPLS traffic so as to provide fine-grained load balancing of MPLS traffic over well-managed IP networks (by leveraging the widely supported UDP-based LB capability). It seems that you are a proponent of the above rationale as you are a contributor of that RFC. Now draft-xu-intarea-ip-in-udp just extends that rationale to another well-managed IP network environment(i.e., Softwires network). Why that rationale becomes “a solution looking for a problem” suddenly?  Does the efficiency of GRE key or L2TPv3 session ID based LB approaches as proposed by RFC5640 depend on the payload type of those tunnels?

Best regards,

Xiaohu

Thanks,

— Carlos.

Xiaohu

Thanks,

— Carlos Pignataro.

Dear all,

Regards,

Wassim & Juan Carlos

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 31 May 2016 07:51:27 +0000

> From: Brian E Carpenter [mailto:[email-masked]]
> Sent: Tuesday, May 31, 2016 4:46 AM
> To: Joe Touch
> Cc: joel jaeggli; Xuxiaohu; Fred Baker (fred); Wassim Haddad; [email-masked]
> >>>> reassembly). If you make this requirement, you can handle IPv6 over
> >>>> the tunnel.
> >> especially when you hash. (see frag drop) given alternatives that
> >> better address such situations it seems hard to justify.
> >
> > If you intend to support recursive IP tunneling* and believe that IP
> > has a minimum MTU, then you have to accept reassembly.
> 
> If you intend to support recursive datagram tunneling and believe that any path
> has a minimum MTU, then you have to accept reassembly.

Let's take the Softwires network as an concrete example. What's the motivation for SPs to deploy recursive IP tunneling?

Xiaohu

> (Something I discovered in about 1983, when implementing OSI/CLNP at CERN
> over a homebrew network with 128 byte packets.)
> 
>    Brian
> 
> >
> > Joe
> >
> > * where "recursive IP tunneling" is IP in [zero or more other
> > protocols] in IP.
> >

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 31 May 2016 08:13:10 +0000

> over a homebrew network with 128 byte packets.)

Reassembly on the tunnel egress may be acceptable at that old time. However, due to the considerable improvement in network bandwidth capability, the practice acceptable at the old time may have become outdated today. See the MAP implementation experience shared by Ole recently.

Xiaohu

---

From: "Black, David" <[email-masked]>
Date: Tue, 31 May 2016 14:09:48 +0000

> RFC7510 proposes to use UDP tunnels instead of GRE or L2TPv3 tunnels to carry MPLS traffic
> so as to provide fine-grained load balancing of MPLS traffic over well-managed IP networks
> (by leveraging the widely supported UDP-based LB capability). It seems that you are a proponent
> of the above rationale as you are a contributor of that RFC. Now draft-xu-intarea-ip-in-udp just
> extends that rationale to another well-managed IP network environment(i.e., Softwires network).

Not so fast ;-).

As another author of RFC 7510, I’ll note that it was an MPLS WG draft that was strongly supported by the MPLS WG to solve some problems for which existing techniques such as GRE, L2TP and MPLS entropy labels did not suffice.   RFC 7510 has nothing to say about softwires, and the SOFTWIRE WG concluded without adopting this IP-in-UDP draft.

RFC 7510 on MPLS-in-UDP does not justify this sort of reduce-to-previous-case reasoning for softwires - the softwire need for IP-in-UDP needs to be described in the draft, not left as an exercise to the reader.  It is possible that existing techniques do not suffice here, but it is the draft’s responsibility to make that case.

Thanks, --David

From: Int-area [mailto:[email-masked]] On Behalf Of Xuxiaohu
Sent: Tuesday, May 31, 2016 3:15 AM
To: Carlos Pignataro (cpignata)
Cc: [email-masked]
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

From: Carlos Pignataro (cpignata) [mailto:[email-masked]]
Sent: Monday, May 30, 2016 10:57 PM
To: Xuxiaohu
Cc: Wassim Haddad; [email-masked]<mailto:[email-masked]>
Subject: Re: [Int-area] Call for adoption of draft-xu-intarea-ip-in-udp-03

Xiaohu,

Please see inline.

Carlos

Wasim, Juan Carlos,

Best regards,

Xiaohu

Thanks,

— Carlos.

Xiaohu

Thanks,

— Carlos Pignataro.

Dear all,

Regards,

Wassim & Juan Carlos

---

From: Joe Touch <[email-masked]>
Date: Tue, 31 May 2016 08:33:30 -0700

> a minimum MTU, then you have to accept reassembly.

Agreed- at least one of the layers between the message tunnel and when
it recurses must support fragmentation and reassembly.

Those who deploy or sell systems otherwise are in denial, not a
counter-proof. Again, this is where compliance validation would be useful.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 31 May 2016 16:29:45 +0000

Brian and Joe are correct - tunnels within tunnels means that fragmentation and
reassembly are inevitable. Wish it weren't so, but that is the reality.

Thanks - Fred

> Sent: Tuesday, May 31, 2016 8:34 AM
> To: Brian E Carpenter <[email-masked]>
> Cc: [email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 1 Jun 2016 08:01:17 +1200

On 31/05/2016 20:13, Xuxiaohu wrote:
> Reassembly on the tunnel egress may be acceptable at that old time. However, due to the considerable improvement in network bandwidth capability, the practice acceptable at the old time may have become outdated today.

I don't understand what network capacity has to do with the physical and mathematical
fact that packets larger than N bytes will not fit into a packet limited to N bytes.
That was true in 1983 and will still be true in 2083.

   Brian

> See the MAP implementation experience shared by Ole recently.
> 
> Xiaohu
> 
>>    Brian
>>
>>>
>>> Joe
>>>

---

From: Joe Touch <[email-masked]>
Date: Tue, 31 May 2016 13:10:54 -0700

On 5/31/2016 1:01 PM, Brian E Carpenter wrote:
> I don't understand what network capacity has to do with the physical
> and mathematical fact that packets larger than N bytes will not fit
> into a packet limited to N bytes. That was true in 1983 and will still
> be true in 2083. Brian

Agreed.

Further, we always hit the "hardware can't do it yet" problem whenever
we roll out new technologies.

We have to resist the urge to prune our protocols down to only what we
profitably implement today.

Joe

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 1 Jun 2016 02:31:52 +0000

> From: Int-area [mailto:[email-masked]] On Behalf Of Templin, Fred L
> Sent: Wednesday, June 01, 2016 12:30 AM
> To: Joe Touch; Brian E Carpenter
> Cc: [email-masked]

If so, could you give a concrete example in real network environment where outer fragmentation is widely enabled and works very well? It would be better if the reassembly buffer size needed in the tunnel egress could be given as well.

Best regards,
Xiaohu 

> > > If you intend to support recursive datagram tunneling and believe
> > > that any path has a minimum MTU, then you have to accept reassembly.
> >

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 1 Jun 2016 02:43:39 +0000

> Sent: Wednesday, June 01, 2016 4:01 AM
> To: Xuxiaohu; Joe Touch
> Cc: joel jaeggli; Fred Baker (fred); Wassim Haddad; [email-masked]
> >>>>>> IPv6 layer (because UDP doesn't have support for fragmentation
> >>>>>> and reassembly). If you make this requirement, you can handle
> >>>>>> IPv6 over the tunnel.
> >> (Something I discovered in about 1983, when implementing OSI/CLNP at
> >> CERN over a homebrew network with 128 byte packets.)
> >
> > Reassembly on the tunnel egress may be acceptable at that old time. However,
> due to the considerable improvement in network bandwidth capability, the
> practice acceptable at the old time may have become outdated today.
> 
> I don't understand what network capacity has to do with the physical and
> mathematical fact that packets larger than N bytes will not fit into a packet
> limited to N bytes.

In the above case, there are three FEASIBLE ways: 1) Configure the transit core to carry an MTU at least large enough to accommodate the added encapsulation headers. Note that this is the dominant and mature choice in most MPLS networks; 2) Fragment the packet before encapsulation if allowed; 3) Drop it if inner fragmentation is not allowed.

> That was true in 1983 and will still be true in 2083.

Could you give a concrete example in real network environment where outer fragmentation is widely enabled and works very well before making such an assertion? It would be better if the reassembly buffer size needed in the tunnel egress could be given as well.

Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 1 Jun 2016 04:09:38 +0000

> mathematical fact that packets larger than N bytes will not fit into a packet

This article (http://learning.nil.com/assets/Tips-/The-Never-Ending-Story-of-IP-Fragmentation.pdf) may be useful for you to understand why network capability is a key factor to be considered for fragmentation and reassembly on routers (here routers are not software routers or CPU-only routers which were dominant in the old time). Of course, you could also have a look at the current fragmentation and reassembly implementations of major router vendors if you believed that article is a little bit old.

Xiaohu

---

From: Joe Touch <[email-masked]>
Date: Tue, 31 May 2016 22:03:08 -0700

On 5/31/2016 7:43 PM, Xuxiaohu wrote:
> In the above case, there are three FEASIBLE ways: 
Let's take these in order then:

> 1) Configure the transit core to carry an MTU at least large enough to accommodate the added encapsulation headers. Note that this is the dominant and mature choice in most MPLS networks;

So you are deploying IPv6 over an IPv6 tunnel. Even without UDP, you
still need 40 bytes for the encapsulation header.

So in order to deploy a compliant IPv6 1280-byte MTU tunnel, you need to
use a core that you know can transit IPv6 packets that are 1320 bytes.

Unless you examine every physical link to ensure that the technology
supports a larger MTU, you're stuck assuming that your IPv6 equipment
can only guarantee 1280. Which means you can't know this will work.

Even if you do know this for a physical path, every path you *think* is
physical will be virtualized at some point, and at some point the number
of layers of tunneling will end up eating the whole of any "headroom"
you engineer.

>  2) Fragment the packet before encapsulation if allowed;

That covers only IPv4 with DF=0. It won't cover DF=1 or IPv6.

>  3) Drop it if inner fragmentation is not allowed.
You've now black-holed your network, all because you didn't want to use
the 1500-byte fragmentation and reassembly that every IPv6 source and
sink are already required to support (per RFC2460).

Note that every tunnel ingress is a IPv6 source and every IPv6 egress is
an IPv6 sink, so if you claim that many vendors don't support this,
you're just pointing out that they don't satisfy existing IPv6 requirements.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 1 Jun 2016 17:14:08 +1200

I think I detest fragmentation, or rather re-assembly, as much as anybody,
whether designing hardware or software solutions. And of course, in a closely
managed environment, you can ensure that the outer MTU is sufficient to contain
the inner MTU. That isn't the point. The point is that designing a protocol
proposed as a general-purpose protocol for the Internet, that might be used
for a hundred years, we can't guarantee that it will always be deployed in such
a managed environment. In fact, we can pretty much guarantee that it will be
used in completely unmanaged (or badly managed) ways.

This isn't theory. We've seen it a lot where IPv6 has been tunneled across
IPv4 islands, with lots of MTU and fragmentation failures. That's even simpler
than IP in UDP in IP.

Regards
   Brian

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 1 Jun 2016 06:26:34 +0000

> Sent: Wednesday, June 01, 2016 1:14 PM
> To: Xuxiaohu; Joe Touch
> managed environment, you can ensure that the outer MTU is sufficient to
> contain the inner MTU. That isn't the point. The point is that designing a protocol

It said in the draft that "... this specification defines an IP-in-UDP encapsulation method dedicated for Softwire service (including both mesh and hub-spoke modes). " Softwire networks are well-managed SP networks. I would like to add a dedicated Applicability Statement section to emphasize it if the current statement seems not enough.

Best regards,
Xiaohu

> proposed as a general-purpose protocol for the Internet, that might be used for
> a hundred years, we can't guarantee that it will always be deployed in such a
> managed environment. In fact, we can pretty much guarantee that it will be

> >>>>>> Yeah I don't support it for this reason. getting IP fragments
> >>>>>> back together in the same place a reassembled is hard is in some
> >>>>>> cases especially when you hash. (see frag drop) given
> >>>>>> alternatives that better address such situations it seems hard to justify.
> >>>>>
> >>>>> If you intend to support recursive IP tunneling* and believe that
> >>>>> IP has a minimum MTU, then you have to accept reassembly.
> >>>>
> >> due to the considerable improvement in network bandwidth capability,
> >> the practice acceptable at the old time may have become outdated today.
> >>
> >> into a packet limited to N bytes.
> >
> > This article
> (http://learning.nil.com/assets/Tips-/The-Never-Ending-Story-of-IP-Fragmentati
> on.pdf) may be useful for you to understand why network capability is a key
> factor to be considered for fragmentation and reassembly on routers (here
> routers are not software routers or CPU-only routers which were dominant in
> the old time). Of course, you could also have a look at the current fragmentation
> and reassembly implementations of major router vendors if you believed that
> article is a little bit old.
> >
> > Xiaohu
> >