Subject: [nvo3] Comments on http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03

From: Tom Herbert <[email-masked]>
Date: Mon, 14 Jul 2014 12:43:33 -0700

Hi VXLAN-gpe authors,

Abstract: technically this is not extending a VXLAN but defines a new
protocol that looks similar to VXLAN (demonstrated by need for new UDP
port assignment).

Section 3.1: yet another protocol numbering scheme is defined. Why not
use IP protocol numbering space. e.g. 41 for IPv6, 94 for IPv4, 97 for
Ethernet. NSH would need an protocol number allocation (maybe that is
the intent so these can be used at L3? ).

Section 3.1: P-bit seems unnecessary to me, is more complex to
process, and there is no reason to be compatible with VXLAN. Without
P-bit we can always do simple indirect look-up to get protocol handler
(e.g. handler = proto_handlers[protocol]), but with the P-bit we need
to do an additional conditional.

Also, since this now has a protocol and version in the header, the
only thing fundamentally missing is a header length field. Please
consider adding that. See GUE
(http://tools.ietf.org/html/draft-herbert-gue-01) or geneve for the
justification of why this is critical.

Thanks,
Tom

---

From: Marc Binderberger <[email-masked]>
Date: Tue, 15 Jul 2014 13:42:06 -0700

Hello Tom, Paul et al.,

> protocol that looks similar to VXLAN (demonstrated by need for new UDP

(?) ... (!) interesting, the abstract was not mentioning it although it's a 
relevant change IMHO when you refer to "VxLAN".

Paul, few comments/questions:

(1) I think the abstract should mention the requirement for a new UDP port. 
Especially as this suddenly showed up (it wasn't there in -02)

(2) maybe a motivation why a new UDP port is needed?

(3) propose to remove figure 3 and use figure 4 throughout the document as 
the new proposed header

(4) does the whole P bit and "backward compatibility" makes sense? E.g. in 
4.2 what you are doing is simply sending out a VxLAN packet according to 
draft-mahalingam-dutt-dcops-vxlan. Not sure there is any gain explaining how 
to set P, O and protocol field. And I don't see the point in 4.1, VxLAN will 
send to it's own port, not the new one.

(5) could you provide a short motivation why you extend your flags, version 
field to the right side of the "I" flag when there is so much room on the 
left?  Sure, there is LISP - is there any problem mentioning this? Elephant 
in the room? ;-)

(6) Maybe the authors of draft-quinn-vxlan-gpe had just the same idea, 
independently but I have seen the OAM flag before: 
draft-singh-nvo3-vxlan-router-alert. If you borrowed a good idea, why not 
referencing?

Re Tom:

> Without P-bit we can always do simple indirect look-up to get protocol 
handler
> (e.g. handler = proto_handlers[protocol])

agree. As this will go to a new UDP port one could define the packet with a 
protocol field, always. No confusion. One precious flag saved in a fixed 
header.

> justification of why this is critical.

Well, ASIC/FPGA/NP people like fixed field headers, I think. I also see an 
advantage to have an fixed 8 byte shim with similarities between VxLAN, 
VxLAN-gpe and LISP (or LISP-gpe) from an implementation point of view. The 
hardware just needs to add 64bit from pre-programmed memory, the details how 
to fill the memory is done by the control plane. Parsing in hardware when 
receiving a packet can also be easier (with the right layout) and be shared.

Regards, Marc

---

From: Tom Herbert <[email-masked]>
Date: Tue, 15 Jul 2014 16:03:04 -0700

> send to it's own port, not the new one.
Right, once the protocol is using a different UDP port, there is no
need to maintain backwards compatibility so all the deficiencies of
VXLAN could be addressed in the "new" protocol (like the location of
the first used flag bit, or the shift/size of the VNID).

I also noticed the ambiguity if the P and O bit are simultaneously
set. This ambiguity arises from the fact that the O bit is more a
1-bit type field than a flag. I would suggest donating the the O bit
to the version field space to allow eight version/types. So val==0
indicates normal data packet (protocol field is present), val==1
indicates OAM packet. All the other fields (even flags) can be
interpreted based on the version/type.

> left?  Sure, there is LISP - is there any problem mentioning this? Elephant
It's seems pretty standard to put version first in the header since
the rest of the fields are interpreted based on that (e.g. IP).

> receiving a packet can also be easier (with the right layout) and be shared.
Yes, it seems that fixed headers have become an implicit requirement
in protocol design, however this conflicts with the requirement that
protocols that are extensible. I think the answer is to define
extensible protocols (use VLH) but assume implementations may
optimized only a fix set of combinations (conceptually how routers
optimize for zero length IP options, but allow packets with IP options
in slow path). In practice, an encapsulation protocol like VXLAN is
likely deployed in a closed network so that the encapsulation is
uniform (only a small number of variants in encapsulation would be
used). Introducing new fields would be a rare event, but we do need
this capability and an preferably it should not require a new protocol
(UDP port number) and definitely shouldn't require new HW. The ability
to program HW with a small number of combinations of the protocol to
parse would be awesome! :-)

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Tue, 15 Jul 2014 23:32:53 +0000

Hi Tom,

Thanks for the questions and comments!  Please see inline.

> protocol that looks similar to VXLAN (demonstrated by need for new UDP

We are trying to balance re-use of the VXLAN format and the need to support existing non-GPE hardware that might already be deployed.  We looked at using the same port, and the new one, and decided, at this point that a new port is easier for migration but since the packet format is essentially VXLAN to keep the VXLAN name.

> the intent so these can be used at L3? ).

We can certainly use the IP protocol numbers for the protocols that have one.  However, there are protocols that might be encapsulated that don’t have an IP protocol number (not just NSH) so we still need a registry for those.

> to do an additional conditional.

The P bit ensures that we have forwarding logic consistent with LISP (https://datatracker.ietf.org/doc/draft-lewis-lisp-gpe/).  Also, in the case if/when gpe uses the same port as VXLAN, the p-bit helps with parsing on the receiving end.

> justification of why this is critical.

The length of the gpe header is fixed, so adding length wouldn’t buy us much?

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Jul 2014 16:47:34 -0700

> We are trying to balance re-use of the VXLAN format and the need to support existing non-GPE hardware that might already be deployed.  We looked at using the same port, and the new one, and decided…

Paul, this design seems to be going in circles. If a new port is used, why not make the new port for VXLAN mean layer-3 protocols follow? Or better yet, have a demux field after the VXLAN header so you don't have to use VXLAN header bits. Because the P-bit is using a precious bit in the VXLAN/LISP header and the nonce field that can be used for other things (we have history that shows a nonce in the header is a cheap form of obscure security).

If you do this then you have no compatibility problems with initial VXLAN and LISP implementations.

And, most importantly, there will be less confusion in the marketplace.

Dino

---

From: Tom Herbert <[email-masked]>
Date: Tue, 15 Jul 2014 17:07:48 -0700

> We can certainly use the IP protocol numbers for the protocols that have one.  However, there are protocols that might be encapsulated that don’t have an IP protocol number (not just NSH) so we sti…
NSH looks an awful lot like an IP extension header to me, have you
considered requesting a protocol number for this?

You can always use proto=47 (GRE) as an secondary encapsulation header
to encapsulate the full ETHER_TYPE range. This technique is described
in GUE draft (cost is 4 bytes and a little bit of processing for these
other protocols not represented by an IP protocol number).

> The P bit ensures that we have forwarding logic consistent with LISP (https://datatracker.ietf.org/doc/draft-lewis-lisp-gpe/).  Also, in the case if/when gpe uses the same port as VXLAN, the p-bit…
I think there's more logic in being consistent with Ethernet, IP, GRE
protocol standards where the next protocol field is always valid as an
invariant.

> The length of the gpe header is fixed, so adding length wouldn’t buy us much?

Fixed length gpe (or in VXLAN for that matter) implies no protocol
eXtensibility :-).

Thanks,
Tom

---

From: Fabio Maino <[email-masked]>
Date: Tue, 15 Jul 2014 22:47:34 -0700

Tom,
one of the design goals of GPE is to be cost effective when implemented 
together with VXLAN and LISP. We do know that those are the two most 
deployed protocols out there, and will be out there for quite some time 
while any other network virtualization protocol gets deployed.  I 
believe that sharing the parsing logic across the new and legacy 
protocols, is a significant reduction in complexity that eventually will 
help adoption of GPE.

At the same time we want to facilitate convergence to a protocol that 
can be extended with new features. Multiprotocol support and explicit 
service tagging being the two most notable.

GPE with the next protocol  and version fields provides a trade off 
between those two contrasting requirements.

When you consider the flexibility of GPE, you shoud look at the 
combination of gpe with a shim header a-la nsh, that I believe provides 
the flexibility needed to carry additional metadata. It's also my 
understanding that gpe+nsh would satisfy the requirements for NIC 
processing that you describe in the GUE draft.

Finally the P bit. It simply leaves open the opportunity to deploy GPE 
on the same port of VXLAN (or LISP). I believe that in some deployments 
this may become an advantage, that is worth the use of one bit 
(especially considering that with the compact next protocol field we 
make available further bits in the GPE header to future versions).

Thanks,
Fabio

On 7/15/14, 4:03 PM, Tom Herbert wrote:
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Fabio Maino <[email-masked]>
Date: Tue, 15 Jul 2014 22:51:29 -0700

Dino,
I believe that using a format that can share as much as possible with 
the two protocols deployed today will give a better chance to GPE to be 
implemented, as vendors may want a cost effective way to migrate to the 
new protocol while preserving compatibility with legacy implementations.

The fact that GPE provides a way to be extended, either with a shim a-la 
NSH or making availabe more bits in the header for future features, I 
think opens up to the addition of new features (explicit service 
tagging, as an example) in a more organic way than trying to use the few 
bits left in VXLAN or LISP.

Unfortunately, in the LISP case, this comes to the expense of some 
features that are in the current specification, but I believe those same 
features can be mapped on a GPE+shim header, so a new implementation 
would be able to provide the equivalent of those features (e.g. nonce).

It's hard to find the right balance between backward compatibility and 
evolution of the encapsulation, but I believe GPE gets to a decent 
compromise.

Thanks,
Fabio

---

From: Marc Binderberger <[email-masked]>
Date: Tue, 15 Jul 2014 23:22:14 -0700

Hello Tom,

> 1-bit type field than a flag.

well, you may have a different OAM behaviour per protocol. It's hard to say 
as the details of OAM are out of scope for the document, I guess. Even a 
simple "punt to control plane" is still punting to different receivers 
depending on layer-2, IPv4, IPv6 etc.

> protocols that are extensible.

well, that's where the version field comes into play.

> The ability

Agree. Assuming it's really a small number :-)

Regards, Marc

---

From: Tom Herbert <[email-masked]>
Date: Wed, 16 Jul 2014 07:39:19 -0700

Hi Fabio, thanks for your insights.

On Tue, Jul 15, 2014 at 10:47 PM, Fabio Maino <[email-masked]> wrote:
> while any other network virtualization protocol gets deployed.  I believe
> that sharing the parsing logic across the new and legacy protocols, is a
> significant reduction in complexity that eventually will help adoption of
> GPE.
>
As I pointed there is already a lot of precedent in standard protocols
that have a protocol demux and that even predate the concept of
network virtualization protocols. It's still more logical to me to be
consistent with them.

> At the same time we want to facilitate convergence to a protocol that can be
> extended with new features. Multiprotocol support and explicit service
> tagging being the two most notable.
>
> GPE with the next protocol  and version fields provides a trade off between
> those two contrasting requirements.
>
The addition of next protocols and version field is good step with
GPE, these are two of three fundamental constructs we need for
extensibility.

> When you consider the flexibility of GPE, you shoud look at the combination
> of gpe with a shim header a-la nsh, that I believe provides the flexibility
> needed to carry additional metadata. It's also my understanding that gpe+nsh
> would satisfy the requirements for NIC processing that you describe in the
> GUE draft.
>
I'm not going to rehash all the previous arguments I've made for
extensibility in the encapsulation header, please look at past
threads. Generally though, I'm skeptical that a variable number of fix
length headers is really more palatable than a single variable length
header; I know the former makes for a much more complex data path in
code and increased packet overhead. Also, certain meta data like
security cookie (a strict requirement in our deployment) is intrinsic
to the encapsulation itself and logically part of the header. I still
believe that GRE serves as the best model for a highly efficient and
extensible encapsulation protocol with the only caveat that it lacks a
header length field which makes it infeasible to add new fields
without breaking parsing of payloads in intermediate devices.

> Finally the P bit. It simply leaves open the opportunity to deploy GPE on
> the same port of VXLAN (or LISP). I believe that in some deployments this
> may become an advantage, that is worth the use of one bit (especially
> considering that with the compact next protocol field we make available
> further bits in the GPE header to future versions).
>
Well, as I also pointed out on this list, I don't see any way to
deploy GPE on the same port as VXLAN without the risk of breaking
compatibility in existing deployments. From VXLAN draft: "The other 7
bits (designated "R") are
reserved fields and MUST be set to zero on transmit and ignored on
receive." The requirement to ignore reserved bits on receive precludes
the possibility that new bits can be implemented that change how the
packet is interpreted. Note that both GUE and geneve are much more
specific in this area, requiring packet to be dropped at a
decapsulating host with unknown flag bits; this might be something you
want to revisit in GPE spec.

Thanks,
Tom

---

From: Tom Herbert <[email-masked]>
Date: Wed, 16 Jul 2014 09:21:20 -0700

On Tue, Jul 15, 2014 at 11:22 PM, Marc Binderberger <[email-masked]> wrote:
> as the details of OAM are out of scope for the document, I guess.\
>
I assume that use of OAM is mutually exclusive of carrying a data
packet (if this is assumption is not correct please let me know), so
the format of OAM header can be completely different beyond the
version/type. With this property there is no need specify any details
of OAM headers now, these can be arbitrarily defined later. You can
look at ICMP for a good example of using type to allow different
formats for a control protocol.

> well, that's where the version field comes into play.
It helps to a degree, but version numbers are an extremely finite
resource and should really only be incremented when a non-backwards
compatible protocol variant is introduced. The protocol should also
allow for private extensions which would be valuable to allow for
continued innovation-- consider that there's already been one proposal
(passing MTU), which would have consumed most of the reserved bits in
the encapsulation header.

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Jul 2014 13:05:17 -0700

Well, if you want to really care about customers, creating all these variations is not doing justice to them. If you want less combinations, you keep VXLAN the way it is right now, with no changes. You keep LISP the way it is right now with no changes. You get L2 and L3 overlays with a unified pull control-plane in draft-maino-nv03-lisp-cp (preaching to the author ;-) or use BGP as an alternative push control-plane.

As for NSH, you create a new header because it is completely new requirement and has not be deployed anywhere. Since it is brand new, you need new hardware and code to be developed to support it. So changing VXLAN and LISP to support NSH doesn't make sense because you inject change in 3 places rather than in 1 new place, creating more protocol machinery, complexity, and combinations that will frustrate customers (not to mention vendor call centers).

Less entropy please,
Dino

On Jul 15, 2014, at 10:51 PM, Fabio Maino <[email-masked]> wrote:

> Dino,
> I believe that using a format that can share as much as possible with the two protocols deployed today will give a better chance to GPE to be implemented, as vendors may want a cost effective way to migrate to the new protocol while preserving compatibility with legacy implementations.
> 
> The fact that GPE provides a way to be extended, either with a shim a-la NSH or making availabe more bits in the header for future features, I think opens up to the addition of new features (explicit service tagging, as an example) in a more organic way than trying to use the few bits left in VXLAN or LISP.
> 
> Unfortunately, in the LISP case, this comes to the expense of some features that are in the current specification, but I believe those same features can be mapped on a GPE+shim header, so a new implementation would be able to provide the equivalent of those features (e.g. nonce).
> 
> It's hard to find the right balance between backward compatibility and evolution of the encapsulation, but I believe GPE gets to a decent compromise.
> 
> Thanks,
> Fabio
> 
> 
>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 16 Jul 2014 21:55:51 +0000

Agree with Dino. 

Why can't the extra information being carried by the NSH shim? 

Linda

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Dino Farinacci
Sent: Wednesday, July 16, 2014 3:05 PM
To: Fabio Maino
Cc: [email-masked]
Subject: Re: [nvo3] Comments on http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03

Less entropy please,
Dino

On Jul 15, 2014, at 10:51 PM, Fabio Maino <[email-masked]> wrote:

>>>> Abstract: technically this is not extending a VXLAN but defines a 
>>>> new protocol that looks similar to VXLAN (demonstrated by need for 
>>>> new UDP port assignment).

---

From: Fabio Maino <[email-masked]>
Date: Wed, 16 Jul 2014 15:46:44 -0700

Dino,
GPE is not changing VXLAN nor LISP, as today's implementations cannot be 
changed. GPE  is proposing a converged extension of both protocols that 
add some new features that, I believe, are needed.

All of VXLAN features are supported in GPE.

In the case of LISP, unfortunately, two features (nonce and 
map-versioning) could not be accommodated in GPE. This is not very 
different than any of today's LISP implementations that may decide to 
not support those two features (for whatever reason: cost, complexity, 
use cases addressed by the implementation... independently from GPE).

What GPE is trying to do is to simplify next generation implementations 
by keeping GPE as close as possible to the existing encapsulations, 
adding on top of those. I maintain that an implementation supporting 
GPE, LISP and VXLAN will be more cost effective than one that supports 
VXLAN, LISP and a clean slate protocol, and may have better chances of 
being deployed.

Fabio

---

From: Fabio Maino <[email-masked]>
Date: Wed, 16 Jul 2014 15:49:27 -0700

> Why can't the extra information being carried by the NSH shim?

Right. That's what we're suggesting with GPE: use a shim header, a-la 
NSH, to carry extra metadata and keep the basic structure of the VXLAN 
(and LISP, to a lesser extent) header the same.

Fabio

---

From: Lucy yong <[email-masked]>
Date: Wed, 16 Jul 2014 23:00:25 +0000

If GPE is the VXLAN evolution, why it request a new UDP port? If GPE is a new protocol, why it has to align with VXLAN format?

Lucy

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Fabio Maino
Sent: Wednesday, July 16, 2014 5:47 PM
To: Dino Farinacci
Cc: [email-masked]
Subject: Re: [nvo3] Comments on http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03

Dino,
GPE is not changing VXLAN nor LISP, as today's implementations cannot be changed. GPE  is proposing a converged extension of both protocols that add some new features that, I believe, are needed.

In the case of LISP, unfortunately, two features (nonce and
map-versioning) could not be accommodated in GPE. This is not very different than any of today's LISP implementations that may decide to not support those two features (for whatever reason: cost, complexity, use cases addressed by the implementation... independently from GPE).

What GPE is trying to do is to simplify next generation implementations by keeping GPE as close as possible to the existing encapsulations, adding on top of those. I maintain that an implementation supporting GPE, LISP and VXLAN will be more cost effective than one that supports VXLAN, LISP and a clean slate protocol, and may have better chances of being deployed.

Fabio

---

From: "Elzur, Uri" <[email-masked]>
Date: Wed, 16 Jul 2014 23:37:32 +0000

VXLAN-gpe is as aligned to existing VXLAN as much as the existing RFC provides guidance for. I.e. given the anticipated interpretations, with proper scrutiny of what is allowed , VXLAN-gpe is attempting to allow for best backwards interoperability on the existing UDP 4789 and offer best prospects for carrying the additional headers/metadata. This leads to the need for new UDP port for new formats 

Thx
 
Uri ("Oo-Ree")
C: (949)-378-7568

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, July 16, 2014 4:00 PM
To: Fabio Maino; Dino Farinacci
Cc: [email-masked]
Subject: Re: [nvo3] Comments on http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03

Lucy

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Fabio Maino
Sent: Wednesday, July 16, 2014 5:47 PM
To: Dino Farinacci
Cc: [email-masked]
Subject: Re: [nvo3] Comments on http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03

Dino,

Fabio

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Jul 2014 16:49:03 -0700

> VXLAN-gpe is as aligned to existing VXLAN as much as the existing RFC provides guidance for. I.e. given the anticipated interpretations, with proper scrutiny of what is allowed , VXLAN-gpe is attem…

But for what features? 

If chips can change to support L3 overlays with VXLAN, which they will have to do L3 header parsing as well as doing L3 lookups during recirculation, then they could just end of doing LISP. Note, the LISP header is exactly the same as the VXLAN header so the effort is the same. Therefore there is no point in having a P-bit and a next-header field.

As for NSH, it SHOULD have its own UDP header because you may want to use it natively and NOT encapsulate.

Again, people will find that all the parameters in NSH are really not necessary and just creates more things the network adminstrator has to manage and track. A service will be defined by a 5-tuple and in many cases, all packets may have to travel across a service chain.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Jul 2014 17:23:54 -0700

> GPE is not changing VXLAN nor LISP, as today's implementations cannot be changed. GPE  is proposing a converged extension of both protocols that add some new features that, I believe, are needed.

You change header fields, you are changing the protocol. I know you can be compatible but you can do L2 and L3 overlays today so there is no point in putting a protocol demux in VXLAN to do L3 overlays and a protocol demux in LISP-4341 to do L2 overlays.

> In the case of LISP, unfortunately, two features (nonce and map-versioning) could not be accommodated in GPE. This is not very different than any of today's LISP implementations that may decide to not support those two features (for whatever reason: cost, complexity, use cases addressed by the implementation... independently from GPE).

But supplying nonces is implemented pervasively.

> What GPE is trying to do is to simplify next generation implementations by keeping GPE as close as possible to the existing encapsulations, adding on top of those. I maintain that an implementation…

I know what it is trying to do. You keep saying that but the existing protocols already do it with no changes. So I think this change is effectively a noop.

Dino

---

From: Fabio Maino <[email-masked]>
Date: Wed, 16 Jul 2014 18:07:18 -0700

This is where we disagree Dino: we keep seeing proposals to extend the 
capability of network virtualization protocols. Security, as an example, 
is one area where even you are proposing extensions. I hope this can 
become an opportunity to add to the capabilities of the existing 
protocols while we go through the next generation of HW and SW.

Fabio

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Jul 2014 18:10:37 -0700

> This is where we disagree Dino: we keep seeing proposals to extend the capability of network virtualization protocols. Security, as an example, is one area where even you are proposing extensions. I hope this can become an opportunity to add to the capabilities of the existing protocols while we go through the next generation of HW and SW.

Please don't misunderstand:

(1) We have no data-plane encryption in LISP, we are adding this.
(2) We have L3 overlays with LISP, we don't need another data-plane to do the same thing.
(3) We have L2 overlays with VXLAN, we don't need another data-plane to do the same thing.

Dino

---

From: Fabio Maino <[email-masked]>
Date: Wed, 16 Jul 2014 18:16:43 -0700

But since we're respinning HW and SW to add one feature, in 2 different 
implementations, why don't we use this chance to converge to a single 
encap that can be extended to do that (and more)?

Fabio

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Jul 2014 18:27:33 -0700

> On Jul 16, 2014, at 6:16 PM, Fabio Maino <[email-masked]> wrote:
> 
> But since we're respinning HW and SW to add one feature, in 2 different implementations, why don't we use this chance to converge to a single encap that can be extended to do that (and more)?

Because users have older products from cisco and other vendors who are deploying overlays now. So the decision is already made about which encapsulation format to use for which type of overlay. 

Focusing on NSH (only) is a good thing because that is a transport independent service chaining feature that does not already exist. 

Dino

---

From: Fabio Maino <[email-masked]>
Date: Wed, 16 Jul 2014 21:11:48 -0700

> Because users have older products from cisco and other vendors who are deploying overlays now. So the decision is already made about which encapsulation format to use for which type of overlay.

and this is why we are proposing to use a common encap that is an 
extension of the two deployed today. A new implementation, with minimal 
complexity, will be able to support VXLAN, LISP (for compatibility with 
installed base) and GPE augmented with the features that may be needed.

Fabio

---

From: Tom Herbert <[email-masked]>
Date: Wed, 16 Jul 2014 21:55:22 -0700

> would satisfy the requirements for NIC processing that you describe in the
I am very uncertain whether nsh could meet our requirements, I don't
see that nsh provides an adequate model for extensibility in this
case. The meta data we've conceived thus far-- security, congestion
control, and VNID (which to me is just an instance ofmeta data) are
primary for end to end use. The necessary consumer of this is only the
decapsulation point, but we allow the possibility that intermediate
devices inspect this data but not change it (if we implement strong
authentication of the header, intermediate nodes won't be able to
change it at all). Additionally, the data we need for this is not at
all a set of idempotent items-- for instance the purpose of a security
cookie is to validate the vnid, they go to together. A decapsulating
host must validate the security token be accepting the packet, so in
nsh it seems we would need to forward parse headers in order to
process the first header. Also, we need security (or CRC validation)
to cover all the meta data, I don't know how that could work in nsh.

The other concern I'd have is how to make this performant. This a very
low level protocol and the thought of walking these chains on every
packet is unpleasant. That we'd need to do this even in cases where we
just want to parse the encapsulated packet or find a specific field
(like security) makes it seem really unpleasant. Maybe it's possible
to make this parsing super fast in HW, but I don't see how to make
this efficient in SW at least compared to an much simpler model like
in GRE.

Admittedly, my understanding of nsh is pretty limited-- seems like
there's a lot of functionality in it but also a lot of complexity. A
real example demonstrating all the packet headers for some use with
VXLAN-gpe would be quite helpful-- maybe the formats for sending a 64
bit security cookie could be enlightening.

Thanks,
Tom

---

From: "Ken Gray (kegray)" <[email-masked]>
Date: Thu, 17 Jul 2014 13:27:00 +0000

Where Dino and I agree, and not to pick on Uri - I've seen the statement
"additional headers/metadata" a few times now and it STILL sounds like
"work around SFC", particularly in respect to the metadata part (since
passing metadata was one of SFC's points).

If you want to tilt at that windmill, I don't think that binding the
solution to a particular transport encap is the optimal way to do it (if
you can avoid doing so).

On 7/16/14 7:37 PM, "Elzur, Uri" <[email-masked]> wrote:

>VXLAN-gpe is as aligned to existing VXLAN as much as the existing RFC
>provides guidance for. I.e. given the anticipated interpretations, with
>proper scrutiny of what is allowed , VXLAN-gpe is attempting to allow for
>best backwards interoperability on the existing UDP 4789 and offer best
>prospects for carrying the additional headers/metadata. This leads to the
>need for new UDP port for new formats
>
>Thx
> 
>Uri ("Oo-Ree")
>C: (949)-378-7568
>
>
>Subject: Re: [nvo3] Comments on
>http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03
>
>If GPE is the VXLAN evolution, why it request a new UDP port? If GPE is a
>new protocol, why it has to align with VXLAN format?
>
>Lucy
>
>Subject: Re: [nvo3] Comments on
>http://tools.ietf.org/html/draft-quinn-vxlan-gpe-03
>
>Dino,
>different than any of today's LISP implementations that may decide to not
>support those two features (for whatever reason: cost, complexity, use
>cases addressed by the implementation... independently from GPE).
>
>adding on top of those. I maintain that an implementation supporting GPE,
>LISP and VXLAN will be more cost effective than one that supports VXLAN,
>LISP and a clean slate protocol, and may have better chances of being
>deployed.
>
>Fabio
>
>
>> Well, if you want to really care about customers, creating all these
>>variations is not doing justice to them. If you want less combinations,
>>you keep VXLAN the way it is right now, with no changes. You keep LISP
>>the way it is right now with no changes. You get L2 and L3 overlays with
>>a unified pull control-plane in draft-maino-nv03-lisp-cp (preaching to
>>the author ;-) or use BGP as an alternative push control-plane.
>>
>> As for NSH, you create a new header because it is completely new
>>requirement and has not be deployed anywhere. Since it is brand new, you
>>need new hardware and code to be developed to support it. So changing
>>VXLAN and LISP to support NSH doesn't make sense because you inject
>>change in 3 places rather than in 1 new place, creating more protocol
>>machinery, complexity, and combinations that will frustrate customers
>>(not to mention vendor call centers).
>>
>> Less entropy please,
>> Dino
>>
>>> The fact that GPE provides a way to be extended, either with a shim
>>>a-la NSH or making availabe more bits in the header for future
>>>features, I think opens up to the addition of new features (explicit
>>>service tagging, as an example) in a more organic way than trying to
>>>use the few bits left in VXLAN or LISP.
>>>
>>>features that are in the current specification, but I believe those
>>>same features can be mapped on a GPE+shim header, so a new
>>>implementation would be able to provide the equivalent of those
>>>features (e.g. nonce).
>>>
>>>>> We are trying to balance re-use of the VXLAN format and the need to
>>>>>support existing non-GPE hardware that might already be deployed.  We
>>>>>looked at using the same port, and the new one, and decided, at this
>>>>>point that a new port is easier for migration but since the packet
>>>>>format is essentially VXLAN to keep the VXLAN name.
>>>> Paul, this design seems to be going in circles. If a new port is
>>>>used, why not make the new port for VXLAN mean layer-3 protocols
>>>>follow? Or better yet, have a demux field after the VXLAN header so
>>>>you don't have to use VXLAN header bits. Because the P-bit is using a
>>>>precious bit in the VXLAN/LISP header and the nonce field that can be
>>>>used for other things (we have history that shows a nonce in the
>>>>header is a cheap form of obscure security).
>>>>

---

From: Tom Herbert <[email-masked]>
Date: Thu, 17 Jul 2014 08:39:04 -0700

> solution to a particular transport encap is the optimal way to do it (if
By this line of thinking TCP options should be moved to extension
headers. If the meta data is part of the transport protocol and needed
for its operation then it's should be within the transport header not
buried at some other place in the packet. There's already a lot of
precedent for this in IETF protocols and we had more than thirty years
of operational experience dealing with optional data in several core
protocols. I really don't see that the encapsulation protocol case is
so unique that a new model needs to be adopted.

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 17 Jul 2014 08:41:07 -0700

> so unique that a new model needs to be adopted.

Well stated and 100% agree.

Dino

---

From: Fabio Maino <[email-masked]>
Date: Thu, 17 Jul 2014 09:00:15 -0700

Ken,
there is no intent to tie SFC to this transport, rather to make this 
transport capable to handle those (and possibly other) metadata.

Fabio

---

From: "Ken Gray (kegray)" <[email-masked]>
Date: Thu, 17 Jul 2014 16:02:03 +0000

The other authors have graciously made that clear.  Thanks.  (Never write
before coffee Šnever)

On 7/17/14 12:00 PM, "Fabio Maino (fmaino)" <[email-masked]> wrote:

>>>not
>>>GPE,

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Fri, 18 Jul 2014 22:16:41 +0000

On 7/16/14 4:49 PM, "Dino Farinacci" <[email-masked]> wrote:

>>proper scrutiny of what is allowed , VXLAN-gpe is attempting to allow
>>for best backwards interoperability on the existing UDP 4789 and offer
>>best prospects for carrying the additional headers/metadata. This leads
>>to the need for new UDP port for new formats
>
>But for what features?
>
>If chips can change to support L3 overlays with VXLAN, which they will
>have to do L3 header parsing as well as doing L3 lookups during
>recirculation, then they could just end of doing LISP. Note, the LISP
>header is exactly the same as the VXLAN header so the effort is the same.
>Therefore there is no point in having a P-bit and a next-header field.
>
SK> Native use over UDP is part of NSH design.

Surendra.

>
>Again, people will find that all the parameters in NSH are really not
>necessary and just creates more things the network adminstrator has to
>manage and track. A service will be defined by a 5-tuple and in many
>cases, all packets may have to travel across a service chain.
>
>Dino
>
>> 
>> Thx
>>  
>> Uri ("Oo-Ree")
>> C: (949)-378-7568
>> 
>> 
>> If GPE is the VXLAN evolution, why it request a new UDP port? If GPE is
>>a new protocol, why it has to align with VXLAN format?
>> 
>> Lucy
>> 
>> GPE is not changing VXLAN nor LISP, as today's implementations cannot
>>be changed. GPE  is proposing a converged extension of both protocols
>>that add some new features that, I believe, are needed.
>> 
>>>the way it is right now with no changes. You get L2 and L3 overlays
>>>with a unified pull control-plane in draft-maino-nv03-lisp-cp
>>>(preaching to the author ;-) or use BGP as an alternative push
>>>control-plane.
>>> 
>>>requirement and has not be deployed anywhere. Since it is brand new,
>>>you need new hardware and code to be developed to support it. So
>>>changing VXLAN and LISP to support NSH doesn't make sense because you
>>>inject change in 3 places rather than in 1 new place, creating more
>>>protocol machinery, complexity, and combinations that will frustrate
>>>customers (not to mention vendor call centers).
>>> 
>>> Less entropy please,
>>> Dino
>>> 
>>>>the two protocols deployed today will give a better chance to GPE to
>>>>be implemented, as vendors may want a cost effective way to migrate to
>>>>the new protocol while preserving compatibility with legacy
>>>>implementations.
>>>> 
>>>> It's hard to find the right balance between backward compatibility
>>>>and evolution of the encapsulation, but I believe GPE gets to a decent
>>>>compromise.
>>>> 
>>>> Thanks,
>>>> Fabio
>>>> 
>>>> 
>>>> 
>>>>>>support existing non-GPE hardware that might already be deployed.
>>>>>>We looked at using the same port, and the new one, and decided, at
>>>>>>this point that a new port is easier for migration but since the
>>>>>>packet format is essentially VXLAN to keep the VXLAN name.