Subject: Re: [nvo3] comments on draft-drake-nvo3-evpn-control-plane

From: Aldrin Isaac <[email-masked]>
Date: Wed, 26 Sep 2012 00:23:39 -0400

Comments inline -- aldrin

On Tue, Sep 25, 2012 at 5:54 PM, Sunny Rajagopalan
<[email-masked]> wrote:

> 1) I would suggest *not* altering the semantics of the MPLS label in the BGP
> route. Instead, use the route distinguisher to carry the 24-bit VNID (this
> is arguably better since the semantics of the RD align better with the
> semantics of the VNID). I would suggest encoding this as a type 0 RD, with
> the VNID going into the Assigned number sub-field. In addition, call out
> that an MPLS label value of 0 in the BGP route is a valid value, and will be
> used by PEs which do not support MPLS encap.

The RD was not intended to be used to signal data plane bits. That's
what the label field is for.

>
> 2) I've separately mentioned that the draft should call out that the PE
> control plane need not be co-located with the PE forwarding plane, and that
> XMPP could be used as the messaging format between the two. Each endpoint
> update should look like this:{endpoint_mac, {endpoint_ips}, {NVE IPs}, RD,
> label}. This allows for true decoupling of the control plane from the frame
> format.

This only partially covers the case where a MAC is actually known.
There will be more than one type of update to implement the E-VPN
application.  I think the starting point is to focus on the fully
distributed implementation (E-VPN co-resident with NVE) and then work
our way up once that is defined.  The message format for that
would/should match one-for-one with the BGP fields.

>
> 3) Even though it's not called out explicitly, the model used in both
> draft-drake-nvo3-evpn-control-plane and draft-marques-l3vpn-end-system
> assumes that the PE forwarding plane is interested in having all of the
> endpoint routes in their participating VNIDs. In my view, this puts an
> unnecessary load on NVEs. Instead, we can modify XMPP so that the NVE can
> request end point resolution of specific addresses within a VNID, so it only
> ever needs to cache information about flows that are transiting it.

RT Constrain is used to limit what get sent to NVE/PE.  Furthermore
the draft already states that an implementation does not have to
install a route into FIB until it is needed.

In the directory-based model is there well established/understood ways
to suppress lookups that are bound to fail?  If you are proposing a
directory-based model, then first order of business is to write a
draft that clearly articulates the methodology, pros, cons, safety
valves, experience, tradeoffs, yada yada about such a system so we
don't have to learn the hard way (at the expense of operators).  Until
then we're talking bits and pieces and speculating quite a bit and
really has no place in the draft.  In the push based model, the NVE
just drops the packet if matching forwarding entry does not exist --
it's cold hard, clear, no crossed fingers, always works.

>
> The issue here is how to push endpoint updates to the NVE when endpoints
> move (since having the PE keep track of which update to push to which NVE is
> unreasonable), My proposal here is to place the onus of requesting updates
> on the NVE, possibly triggered by the receipt of an ICMP error message. In
> other words, mandate that when an NVE receives a packet that after
> decapsulation is found to belong to an end-host that is no longer present in
> the attached customer network, it generates an ICMP error (rate controlled)
> back to source, taking care to include in the ICMP error packet enough of
> the payload so that the remote PE can figure out the customer endpoints that
> were attempting to communicate. On receipt of such an ICMP error, the NVE
> can extract the endpoint information from the payload and request the
> control plane for an endpoint update.

MAC moves are already supported quite well.  In an implementation
where the controlling E-VPN application instance is not co-resident
with the NVE then the messages that would be passed between the app
instance and the NVE would need to be defined.  However your proposal
would not be E-VPN -- looks more like data-plane signaling.

> 4) I would also suggest not having the NVE keep track of the encapsulation
> used by the remote endpoint. (this means that the tunnel encapsulation
> attribute in the draft would be unnecessary). Instead, the onus of
> translating between encapsulation methods should be on gateways. If you
> define the XMPP format well, you should be able to communicate end point
> information in a way that is agnostic of the encap method used by the NVE,
> allowing it to do the one encap it does best. A gateway can do this
> translation without BGP control plane intervention, because it would be
> configured to have interfaces that are (for eg) NVGRE on one arm and VXLAN
> on the other, and it would be obvious as to what encap to put on a packet
> going from one arm to the other. Applying an MPLS label would involve the
> gateway participating in BGP.

Gateways = choke points.  They should be avoided.  Every time I buy
the next guys NVO3 system (because it's faster, more scalable, etc)
I'll have to figure out where/how to gateway between it and my other
ten NVO3 systems.  :-/  I prefer Inter-AS option-C where I can have
it.

> 5) The multi-homing discussed in the draft only covers the case where the CE
> devices are physically separated from the PE devices by physical links. In
> the case where the PE forwarding plane is implemented in the hypervisor, the
> more important multi-homing question is what to do if the NVE is connected
> to two or more upstream devices, (basically, the NVE has two or more IP
> addresses). What I would like to happen is have a mac address route be
> associated with multiple NVE addresses. I believe this is possible using the
> framework established in the draft-raggarwa-sajassi-l2vpn-evpn, but it might
> be worthwhile to call out this case in the
> draft-drake-nvo3-evpn-control-plane draft. This is useful because
> draft-raggarwa-sajassi-l2vpn-evpn treats multiple PE IP addresses associated
> with the same mac address as belonging to separate MESs, and assumes that
> the CE-PE links will be labelled with an ethernet segment, which is not the
> case for the hypervisor NVE/PE.

When the hosts are VM, then the ESI=0.  Multi-path support is
considered a part of common capabilities of BGP and would require use
of different RD for a route for each NVE IP where RR are present.

>
> Long email, thanks for reading!
> --
> Sunny
> p.s. John, your email address in the draft is incorrect.
>

---

From: Kireeti Kompella <[email-masked]>
Date: Wed, 26 Sep 2012 09:43:35 -0700

[trimming To/CC list]

Thanks, Aldrin!

More inline. 

On Sep 25, 2012, at 21:23, Aldrin Isaac <[email-masked]> wrote:

>> route.

Note that no one is suggesting altering the semantics of an MPLS label. What's proposed is altering the interpretation of a three-byte field in the NLRI. 

>> Instead, use the route distinguisher to carry the 24-bit VNID (this

Exactly. Worse than that, the RD is responsible for _distinguishing_ routes. If VNIDs are locally generated (an option well worth holding on to), all manner of hell will break loose. 

Kireeti

---

From: Kireeti Kompella <[email-masked]>
Date: Wed, 26 Sep 2012 09:38:24 -0700

Thanks, Aldrin!

More inline. 

Kireeti

---

From: Sunny Rajagopalan <[email-masked]>
Date: Wed, 26 Sep 2012 11:10:35 -0700

Hi, Kireeti, Aldrin
The MPLS label was originally meant to identify a link, and got overridden 
in rfc4364 as a _local_ identifier of VPNs and has now  further morphed in 
draft-drake-nvo3-evpn-control-plane to be a _global_ identifier of VPNs to 
accommodate the VNID. As a semantic conservative, this drift worries me. 
:-)
The RD is meant to distinguish between routes, just like the VNID is meant 
to distinguish between routes from different tenants. If using a type 0 
bothers RD purists, maybe we could define a new type for it? The VNID is 
typically administratively assigned, just like the RD. It's true that the 
RD has never been seen before on the wire, but the VNID has already broken 
that taboo. It's time to call the VNID what it really is - an RD.

Semantic arguments aside, the chief reason I preferred using the RD 
instead of the MPLS label is that it would allow us to encode the complete 
24 bit VNID in the BGP route, instead of truncating it to 20 bits to fit 
inside the MPLS label field.

--
Sunny

From:   Kireeti Kompella <[email-masked]>
To:     Aldrin Isaac <[email-masked]>, 
Cc:     Sunny Rajagopalan/Santa Clara/IBM@IBMUS, "[email-masked]" 
<[email-masked]>, "[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>, "[email-masked]" 
<[email-masked]>, "[email-masked]" 
<[email-masked]>, "[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>
Date:   09/26/2012 09:39 AM
Subject:        Re: [nvo3] comments on draft-drake-nvo3-evpn-control-plane

Thanks, Aldrin!

More inline. 

>> 1) I would suggest *not* altering the semantics of the MPLS label in 
the BGP

Note that no one is suggesting altering the semantics of an MPLS label. 
What's proposed is altering the interpretation of a three-byte field in 
the NLRI. 

> what the label field is for.

Exactly. Worse than that, the RD is responsible for _distinguishing_ 
routes. If VNIDs are locally generated (an option well worth holding on 
to), all manner of hell will break loose. 

Kireeti

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 11:52:24 -0700

Comments inline.

Yours irrespectively,

John

From: [email-masked] [mailto:[email-masked]] On Behalf Of Sunny Rajagopalan
Sent: Wednesday, September 26, 2012 11:11 AM
To: Kireeti Kompella
Cc: [email-masked]; Aldrin Isaac
Subject: [nvo3] use of RD vs MPLS label for VNID encoding (was Re: comments on draft-drake-nvo3-evpn-control-plane)

Hi, Kireeti, Aldrin
The MPLS label was originally meant to identify a link, and got overridden in rfc4364 as a _local_ identifier of VPNs and has now  further morphed in draft-drake-nvo3-evpn-control-plane to be a _global_ identifier of VPNs to accommodate the VNID. As a semantic conservative, this drift worries me. :-)

JD: In drake-nvo3 the MPLS label is being used as the data plane packet demux at the egress.   This is entirely consistent with its usage in all of the existing L2/L3 VPN technologies.  In VXLAN and NVGRE, the data plane packet demux has global significance.   drake-nvo3 allows this but also allows the data plane packet demux to have local significance, which is more consistent with all of the existing L2/L3 VPN technologies.

The RD is meant to distinguish between routes, just like the VNID is meant to distinguish between routes from different tenants. If using a type 0 bothers RD purists, maybe we could define a new type for it? The VNID is typically administratively assigned, just like the RD. It's true that the RD has never been seen before on the wire, but the VNID has already broken that taboo. It's time to call the VNID what it really is - an RD.

JD:  RD is strictly a control plane construct used to keep BGP happy.  It has never been used as a data plane packet demux.

Semantic arguments aside, the chief reason I preferred using the RD instead of the MPLS label is that it would allow us to encode the complete 24 bit VNID in the BGP route, instead of truncating it to 20 bits to fit inside the MPLS label field.

JD:  You did not read the text carefully enough.  The contents of the 3 octet MPLS label field are copied directly into the VNID/Tenant ID field of a  packet.

--
Sunny

From:        Kireeti Kompella <[email-masked]<mailto:[email-masked]>>
To:        Aldrin Isaac <[email-masked]<mailto:[email-masked]>>,
Cc:        Sunny Rajagopalan/Santa Clara/IBM@IBMUS, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date:        09/26/2012 09:39 AM
Subject:        Re: [nvo3] comments on draft-drake-nvo3-evpn-control-plane
________________________________

Thanks, Aldrin!

More inline.

Kireeti

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 11:59:10 -0700

Sunny,

I re-read your note more carefully and contrary to your assertion, an MPLS label was never used to identify a link.  Initially it was used to determine how to disposition a packet received over a link.  This is completely consistent with its usage in all L2/L3 VPN technologies and with its usage in drake-nvo3.

So, its usage has been remarkable consistent throughout its lifetime.

Yours irrespectively,

John

From: John E Drake
Sent: Wednesday, September 26, 2012 11:52 AM
To: 'Sunny Rajagopalan'; Kireeti Kompella
Cc: [email-masked]; Aldrin Isaac
Subject: RE: [nvo3] use of RD vs MPLS label for VNID encoding (was Re: comments on draft-drake-nvo3-evpn-control-plane)

Comments inline.

Yours irrespectively,

John

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Sunny Rajagopalan
Sent: Wednesday, September 26, 2012 11:11 AM
To: Kireeti Kompella
Cc: [email-masked]<mailto:[email-masked]>; Aldrin Isaac
Subject: [nvo3] use of RD vs MPLS label for VNID encoding (was Re: comments on draft-drake-nvo3-evpn-control-plane)

Hi, Kireeti, Aldrin

--
Sunny

Thanks, Aldrin!

More inline.

Kireeti

---

From: Sunny Rajagopalan <[email-masked]>
Date: Wed, 26 Sep 2012 12:02:24 -0700

Hi, John
Please see inline.

From:   John E Drake <[email-masked]>
To:     Sunny Rajagopalan/Santa Clara/IBM@IBMUS, Kireeti Kompella 
<[email-masked]>, 
Cc:     "[email-masked]" <[email-masked]>, Aldrin Isaac 
<[email-masked]>
Date:   09/26/2012 11:54 AM
Subject:        Re: [nvo3] use of RD vs MPLS label for VNID encoding (was 
Re: comments    on draft-drake-nvo3-evpn-control-plane)
Sent by:        [email-masked]

Comments inline.
 
Yours irrespectively,
 
John
 
From: [email-masked] [mailto:[email-masked]] On Behalf Of 
Sunny Rajagopalan
Sent: Wednesday, September 26, 2012 11:11 AM
To: Kireeti Kompella
Cc: [email-masked]; Aldrin Isaac
Subject: [nvo3] use of RD vs MPLS label for VNID encoding (was Re: 
comments on draft-drake-nvo3-evpn-control-plane)
 
Hi, Kireeti, Aldrin 
 
JD: In drake-nvo3 the MPLS label is being used as the data plane packet 
demux at the egress.   This is entirely consistent with its usage in all 
of the existing L2/L3 VPN technologies.  In VXLAN and NVGRE, the data 
plane packet demux has global significance.   drake-nvo3 allows this but 
also allows the data plane packet demux to have local significance, which 
is more consistent with all of the existing L2/L3 VPN technologies. 

JD:  RD is strictly a control plane construct used to keep BGP happy.  It 
has never been used as a data plane packet demux.  

SR: I did admit that it's not been used before, but my point is that it 
serves the same purpose as the VNID, and should be used as such. MPLS 
based implementations wouldn't use it on the wire, but VXLAN based 
implementations already have the equivalent on the wire.

JD:  You did not read the text carefully enough.  The contents of the 3 
octet MPLS label field are copied directly into the VNID/Tenant ID field 
of a  packet.

SR: I was referring to the truncation described here:
 This memo specifies that when E-VPN is to be used with a VXLAN or 
     NVGRE data plane that a VNI or Tenant ID is twenty bits in 
     length and may have either global or local significance, that 
     the remaining four bits are reserved,
-- 
Sunny 

From:        Kireeti Kompella <[email-masked]> 
To:        Aldrin Isaac <[email-masked]>, 
Cc:        Sunny Rajagopalan/Santa Clara/IBM@IBMUS, "[email-masked]" <
[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]
" <[email-masked]>, "[email-masked]" <[email-masked]>, "
[email-masked]" <[email-masked]>, "
[email-masked]" <[email-masked]>, "[email-masked]" <
[email-masked]>, "[email-masked]" <
[email-masked]>, "[email-masked]" <[email-masked]> 
Date:        09/26/2012 09:39 AM 
Subject:        Re: [nvo3] comments on draft-drake-nvo3-evpn-control-plane 

Thanks, Aldrin!

More inline. 

Kireeti
_______________________________________________
nvo3 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nvo3

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 12:15:20 -0700

Snipped.  Comments inline.

Yours irrespectively,

John

SR: I did admit that it's not been used before, but my point is that it serves the same purpose as the VNID, and should be used as such. MPLS based implementations wouldn't use it on the wire, but VXLAN based implementations already have the equivalent on the wire.

JD:  As a semantic conservative you should be ashamed of yourself for suggesting this 8->, particularly as it is completely unnecessary.

JD:  You clipped the remaining part of the sentence which reads :  "and that the value  advertised in the MPLS label field is to be treated as a three octet quantity to be placed directly in the VNI or tenant ID  field of a packet."

There is no MUST or SHOULD associated with the four bits being reserved.  If it will make folks happier I can  reword the sentence to indicate that the advertised three octet label is nominally the VNID or Tenant ID but should be considered to be opaque.

--
Sunny

Thanks, Aldrin!

More inline.

Kireeti
_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Lucy yong <[email-masked]>
Date: Wed, 26 Sep 2012 19:43:08 +0000

Hi Aldrin,

[[LY]] option C requires building a tunnel between two PEs and also another tunnel between sending PE and ASBR on top, so the packet can be transported in IGP. This does not apply here. RFC4364 does not address supporting multiple data plane encapsulations. 

Lucy

---

From: Kireeti Kompella <[email-masked]>
Date: Wed, 26 Sep 2012 13:17:39 -0700

Hi Sunny,

On Sep 26, 2012, at 11:10 , Sunny Rajagopalan <[email-masked]> wrote:

> Hi, Kireeti, Aldrin 
> The MPLS label was originally meant to identify a link,

Speaking freely for the inventors of MPLS, the semantics of a label were kept very very loose; the associated signaling told you what the label meant.  So, an MPLS label for RSVP-TE might be thought of as a link identifier, but an LDP label was a proxy for a destination (egress LSR).  A 2547/4364/EVPN label is a lookup table identifier.

> and got overridden in rfc4364 as a _local_ identifier of VPNs and has now  further morphed in draft-drake-nvo3-evpn-control-plane to be a _global_ identifier of VPNs to accommodate the VNID. As a semantic conservative, this drift worries me. :-)

Perhaps less so for me.

> The RD is meant to distinguish between routes, just like the VNID is meant to distinguish between routes from different tenants. If using a type 0 bothers RD purists, maybe we could define a new ty…

RDs have a very specific purpose: to distinguish routes in the control plane.  VNIDs and labels are to choose forwarding tables; they are a data plane construct.

However, the argument that a separate BGP attribute should have been used to signal the label has been made before, and has validity, but that's water under the bridge.

> Semantic arguments aside, the chief reason I preferred using the RD instead of the MPLS label is that it would allow us to encode the complete 24 bit VNID in the BGP route, instead of truncating it…

I've already suggested to the authors that they go back to 24 bits -- there is a 24-bit/three-octet field, just use it.

Kireeti.

> Cc:        Sunny Rajagopalan/Santa Clara/IBM@IBMUS, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>

---

From: Sunny Rajagopalan <[email-masked]>
Date: Wed, 26 Sep 2012 13:52:42 -0700

Right. Nobody likes gateways. So let's say that the preferred NVE encoding 
type on a hypervisor vswitch is vxlan. Let's say you learn (using the 
tunnel-encap attribute) that your peer only supports MPLS. What good does 
it do you? Your data center only supports VXLAN - even if your vswitch 
figured out how to make an mpls packet, your underlay wouldn't know what 
to do with it.  If the NVE and the underlay infrastructure supported all 
possible types of encodings, the underlay provider would just choose one 
everywhere. 

What typically happens is that you have one data center which is hyper-v 
based and another which is esx based. Each places restrictions on the type 
of encap that hypervisor switches can produce. To interconnect the two, 
there's no point in telling the NVEs what their peer supports, because 
they can only produce the kind of packet that's kosher with their 
virtualization provider. The solution is a gateway. 

What are the scenarios where the tunnel attribute would be useful?
--
Sunny

From:   Lucy yong <[email-masked]>
To:     Aldrin Isaac <[email-masked]>, Sunny Rajagopalan/Santa 
Clara/IBM@IBMUS, 
Cc:     "[email-masked]" <[email-masked]>
Date:   09/26/2012 12:44 PM
Subject:        RE: [nvo3] comments on draft-drake-nvo3-evpn-control-plane

Hi Aldrin,

[[LY]] option C requires building a tunnel between two PEs and also 
another tunnel between sending PE and ASBR on top, so the packet can be 
transported in IGP. This does not apply here. RFC4364 does not address 
supporting multiple data plane encapsulations. 

Lucy

---

From: Sunny Rajagopalan <[email-masked]>
Date: Wed, 26 Sep 2012 13:54:07 -0700

From:   John E Drake <[email-masked]>
To:     Sunny Rajagopalan/Santa Clara/IBM@IBMUS, 
Cc:     Kireeti Kompella <[email-masked]>, Aldrin Isaac 
<[email-masked]>, "[email-masked]" <[email-masked]>
Date:   09/26/2012 12:16 PM
Subject:        Re: [nvo3] use of RD vs MPLS label for VNID encoding (was 
Re: comments    on draft-drake-nvo3-evpn-control-plane)
Sent by:        [email-masked]

Yours irrespectively,
 
John

 
JD:  As a semantic conservative you should be ashamed of yourself for 
suggesting this 8->, particularly as it is completely unnecessary.

SR: Actually, in my mind the RD is semantically the same as the VNID, so 
the use is appropriate. The semantic conservative in me wouldn't take out 
any protest rallies if we used it as such, and might even celebrate a bit 
inwardly. 

 
JD:  You clipped the remaining part of the sentence which reads :  “and 
that the value  advertised in the MPLS label field is to be treated as a 
three octet quantity to be placed directly in the VNI or tenant ID  field 
of a packet.”
 
There is no MUST or SHOULD associated with the four bits being reserved. 
If it will make folks happier I can  reword the sentence to indicate that 
the advertised three octet label is nominally the VNID or Tenant ID but 
should be considered to be opaque. 

SR: This works for me, if using the last four bits won't lead to problems 
with BGP software parsing the packet (won't setting S=0 confuse it?). To 
be clear, the text will state that the 24-bit VNID will be carried in an 
opaque manner in the 3-octet label field, right? Specifically, I'm looking 
for removing the language that says the last four bits are reserved.

-- 
Sunny 

Thanks, Aldrin!

More inline. 

Kireeti

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 13:58:57 -0700

Snipped, again

Yours irrespectively,

John

JD:  You clipped the remaining part of the sentence which reads :  “and that the value  advertised in the MPLS label field is to be treated as a three octet quantity to be placed directly in the VNI or tenant ID  field of a packet.”

SR: This works for me, if using the last four bits won't lead to problems with BGP software parsing the packet (won't setting S=0 confuse it?). To be clear, the text will state that the 24-bit VNID will be carried in an opaque manner in the 3-octet label field, right? Specifically, I'm looking for removing the language that says the last four bits are reserved.

JD:  Fine, I will make that change in the next version.  As Aldrin has been pointing out, there are a set of applications, one for each NLRI, that make use of a common BGP infrastructure.  The BGP infrastructure has no insight into what is carried a given NLRI.

---

From: Lucy yong <[email-masked]>
Date: Wed, 26 Sep 2012 21:04:34 +0000

Hi Sunny,

Yes, you make the point. If two NVEs support different data plane encapsulation only, there is no way they can communicate each other. Period.

Because of that, should we require all the NVEs supporting all the encapsulation types or each DC may ask certain type of encapsulation from vendors. For the DC migration or transition, we can use a gateway to handle it under one control plane. I don't think requiring all NVE to support all the encapsulation types is right approach for this. What if we find a better data plane encapsulation in the future? We don't want to make a dead end here.

Lucy

From: Sunny Rajagopalan [mailto:[email-masked]]
Sent: Wednesday, September 26, 2012 3:53 PM
To: Lucy yong
Cc: Aldrin Isaac; [email-masked]
Subject: support for multiple encoding types at the NVE (was RE: [nvo3] comments on draft-drake-nvo3-evpn-control-plane)

Right. Nobody likes gateways. So let's say that the preferred NVE encoding type on a hypervisor vswitch is vxlan. Let's say you learn (using the tunnel-encap attribute) that your peer only supports MPLS. What good does it do you? Your data center only supports VXLAN - even if your vswitch figured out how to make an mpls packet, your underlay wouldn't know what to do with it.  If the NVE and the underlay infrastructure supported all possible types of encodings, the underlay provider would just choose one everywhere.

What typically happens is that you have one data center which is hyper-v based and another which is esx based. Each places restrictions on the type of encap that hypervisor switches can produce. To interconnect the two, there's no point in telling the NVEs what their peer supports, because they can only produce the kind of packet that's kosher with their virtualization provider. The solution is a gateway.

To:        Aldrin Isaac <[email-masked]>, Sunny Rajagopalan/Santa Clara/IBM@IBMUS,

Hi Aldrin,

Lucy

---

From: Aldrin Isaac <[email-masked]>
Date: Wed, 26 Sep 2012 18:03:35 -0400

A globally significant data plane VNID is lobotomized -- it can only
support one topology (i.e. full mesh) out of the gate and forever.
E-VPN allows the VNID to be globally significant or simply used as a
local context ID selected by the NVE.  This allows the operator to be
more expressive with VN topology.  The actual MPLS label is 24 bits
long -- I don't see any reason (not knowing pre-existing hw/sw
limitations) why we couldn't just use all 24 bits of it as Kireeti
suggested many emails ago.

On Wed, Sep 26, 2012 at 2:10 PM, Sunny Rajagopalan
<[email-masked]> wrote:
> accommodate the VNID. As a semantic conservative, this drift worries me. :-)
> typically administratively assigned, just like the RD. It's true that the RD
> has never been seen before on the wire, but the VNID has already broken that
> taboo. It's time to call the VNID what it really is - an RD.
>
> Semantic arguments aside, the chief reason I preferred using the RD instead
> of the MPLS label is that it would allow us to encode the complete 24 bit
> VNID in the BGP route, instead of truncating it to 20 bits to fit inside the
> MPLS label field.
>
> --
> Sunny
>
>
>
>
> <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]"
> <[email-masked]>, "[email-masked]" <[email-masked]>,
> "[email-masked]" <[email-masked]>,
> "[email-masked]" <[email-masked]>, "[email-masked]"
> <[email-masked]>, "[email-masked]"
> <[email-masked]>, "[email-masked]" <[email-masked]>
> What's proposed is altering the interpretation of a three-byte field in the
> NLRI.
>
> Exactly. Worse than that, the RD is responsible for _distinguishing_ routes.
> If VNIDs are locally generated (an option well worth holding on to), all
> manner of hell will break loose.
>
> Kireeti
>
>

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 15:05:57 -0700

I'd rather have a bottle in front of me than a frontal lobotomy

Yours irrespectively,

John

> -----Original Message-----
> Aldrin Isaac
> Sent: Wednesday, September 26, 2012 3:04 PM
> To: Sunny Rajagopalan
> Cc: Kireeti Kompella; [email-masked]
> Subject: Re: [nvo3] use of RD vs MPLS label for VNID encoding (was Re:
> > The MPLS label was originally meant to identify a link, and got
> > overridden in rfc4364 as a _local_ identifier of VPNs and has now
> > further morphed in draft-drake-nvo3-evpn-control-plane to be a
> > _global_ identifier of VPNs to accommodate the VNID. As a semantic
> > conservative, this drift worries me. :-) The RD is meant to
> > distinguish between routes, just like the VNID is meant to
> distinguish
> > between routes from different tenants. If using a type 0 bothers RD
> > purists, maybe we could define a new type for it? The VNID is
> > typically administratively assigned, just like the RD. It's true that
> > the RD has never been seen before on the wire, but the VNID has
> already broken that taboo. It's time to call the VNID what it really is
> - an RD.
> >
> > instead of the MPLS label is that it would allow us to encode the
> > complete 24 bit VNID in the BGP route, instead of truncating it to 20
> > bits to fit inside the MPLS label field.
> >
> > --
> > Sunny
> >
> >
> >
> >
> "[email-masked]"
> >>> 1) I would suggest *not* altering the semantics of the MPLS label
> in
> >>> the BGP route.
> >
> > What's proposed is altering the interpretation of a three-byte field
> > in the NLRI.
> >
> >>> is arguably better since the semantics of the RD align better with
> >>> the semantics of the VNID). I would suggest encoding this as a type
> >>> 0 RD, with the VNID going into the Assigned number sub-field. In
> >>> addition, call out that an MPLS label value of 0 in the BGP route
> is
> >>> a valid value, and will be used by PEs which do not support MPLS
> >>> encap.
> >>
> routes.
> > If VNIDs are locally generated (an option well worth holding on to),
> > all manner of hell will break loose.
> >
> > Kireeti
> >
> >

---

From: John E Drake <[email-masked]>
Date: Wed, 26 Sep 2012 15:08:28 -0700

Aldrin,

We closed earlier today on a twenty four bit globally or locally significant MPLS label for VXLAN and NVGRE encapsulations.

Yours irrespectively,

John

---

From: Thomas Nadeau <[email-masked]>
Date: Wed, 26 Sep 2012 22:16:06 +0000

That is what we've been saying all along: with this approach you have
more flexibility to handle all of the use cases.

	--Tom

On 9/26/12 6:03 PM, "Aldrin Isaac" <[email-masked]> wrote:

>>overridden
>>the RD
>>instead
>>>> 1) I would suggest *not* altering the semantics of the MPLS label in
>>>>the
>>>> BGP
>>>> route.
>>
>>the
>>>> that an MPLS label value of 0 in the BGP route is a valid value, and
>>>>will
>>>> be

---

From: Aldrin Isaac <[email-masked]>
Date: Wed, 26 Sep 2012 19:02:48 -0400

The WG is NV over L3.  When we refer to MPLS, we're talking about
using MPLS context ID labels, not MPLS LSP.  If you look at all these
encaps, they have two things in common (1) IP transport tunnel and (2)
destination context ID.  I don't see how it can be unreasonably
difficult for an NVE to support more than one encap which have similar
constructs.  However E-VPN additionally has split-horizon label for
multi-homing which draft-drake-nvo3 describes a possible alternative.

On Wed, Sep 26, 2012 at 4:52 PM, Sunny Rajagopalan
<[email-masked]> wrote:
> tunnel-encap attribute) that your peer only supports MPLS. What good does it
> do you? Your data center only supports VXLAN - even if your vswitch figured
> out how to make an mpls packet, your underlay wouldn't know what to do with
> it.  If the NVE and the underlay infrastructure supported all possible types
> of encodings, the underlay provider would just choose one everywhere.
>
> there's no point in telling the NVEs what their peer supports, because they
> can only produce the kind of packet that's kosher with their virtualization
> provider. The solution is a gateway.
>
> [[LY]] option C requires building a tunnel between two PEs and also another
> tunnel between sending PE and ASBR on top, so the packet can be transported
> in IGP. This does not apply here. RFC4364 does not address supporting
> multiple data plane encapsulations.
>
> Lucy