Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Thu, 12 Nov 2015 14:33:47 +0000

I don’t support the adoption of this draft as a WG. There is a major flaw in this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I recommend to do an informational RFC that describes this using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in RFC4364.

---

From: <[email-masked]>
Date: Thu, 12 Nov 2015 16:24:56 +0100

Hi Wim,

Thanks for your feedback.

With my co-chair hat, let met ask a few questions for the sake of well 
understanding the issue you raise.

Henderickx, Wim (Wim):
> Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS 
> IP-VPNs. VXLAN/NVGRE contains a MAC address and IP-VPNs don’t.

This comment only relates to a subset of the encaps proposed in the 
solution, right ?
(e.g. does not apply to MPLS/GRE and MPLS/UDP)

On VXLAN/NVGRE, do you challenge the fact that they would be used with a 
dummy MAC address that would be replaced by the right MAC by a sender 
based on an ARP request when needed ?

> The draft [...] introduces a lot of complexity for nothing.

Can you detail what you consider is a lot of complexity ?

>
> If we want to describe a model C VPN interconnect with a IP fabric in 
> a DC I recommend to do an informational RFC that describes this using 
> VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS 
> label we defined in RFC4364.

My understanding of this draft's value is the fact that it avoids having 
service-label related state (forwarding and BGP state) on the routers at 
the DC-WAN border.  This is one of the typical property of 4364 option 
C.    It is not obvious at all to me, based on existing spec, how to get 
the same benefit on the DC-side router in an architecture where the 
DC-side router also terminates the overlay tunnels.

Do you challenge the goal itself (not having service-label related state 
on the overlay tunnels router) or suggest that the same benefit can be 
achieved with less complexity ?

-Thomas

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Thu, 12 Nov 2015 15:37:37 +0000

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Thursday 12 November 2015 at 16:24
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

With my co-chair hat, let met ask a few questions for the sake of well understanding the issue you raise.

Henderickx, Wim (Wim):
I don’t support the adoption of this draft as a WG. There is a major flaw in this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. VXLAN/NVGRE contains a MAC address and IP-VPNs don’t.

This comment only relates to a subset of the encaps proposed in the solution, right ?
(e.g. does not apply to MPLS/GRE and MPLS/UDP)

WH> even for the encapsulations that don’t have this issue, 1 the UDP port proposal breaks some of the protocols so we should never do this, 2 for the IP addressing it is way to complex to configure and maintain. As such the proposal for me would be to do the 3 label stack as we have in standard MPLS option C which you can do with the encapsulations I mentioned.

On VXLAN/NVGRE, do you challenge the fact that they would be used with a dummy MAC address that would be replaced by the right MAC by a sender based on an ARP request when needed ?

Can you detail what you consider is a lot of complexity ?
WH> UDP ports and IP address prefix maintenance

My understanding of this draft's value is the fact that it avoids having service-label related state (forwarding and BGP state) on the routers at the DC-WAN border.  This is one of the typical property of 4364 option C.    It is not obvious at all to me, based on existing spec, how to get the same benefit on the DC-side router in an architecture where the DC-side router also terminates the overlay tunnels.

Do you challenge the goal itself (not having service-label related state on the overlay tunnels router) or suggest that the same benefit can be achieved with less complexity ?

WH> the proposal I mentioned is achieving the same goal w/o the complexity in the DC GW and using already standardised mechanisms to do so.

-Thomas

---

From: <[email-masked]>
Date: Thu, 12 Nov 2015 17:46:21 +0100

Wim,

(below)

2015-11-12, Henderickx, Wim (Wim):
> WH> even for the encapsulations that don’t have this issue, 1 the UDP 
> port proposal breaks some of the protocols so we should never do this,

Sorry to insist, but where do some protocol break when used with a 
non-standard port as dstport ?

> 2 for the IP addressing it is way to complex to configure and maintain.

I agree with that.

> As such the proposal for me would be to do the 3 label stack as we 
> have in standard MPLS option C which you can do with the 
> encapsulations I mentioned.
>
>
> On VXLAN/NVGRE, do you challenge the fact that they would be used with 
> a dummy MAC address that would be replaced by the right MAC by a 
> sender based on an ARP request when needed ?

Is the above the issue you had in mind about VXLAN and NVGRE ?

> WH> UDP ports and IP address prefix maintenance

I agree for the alternative in which multiple IPs are used.
But I don't think using multiple port would be complex, the 
configuration on the box can simply consist of a range of ports to use 
and the rest would be (quite simply) automated.

> My understanding of this draft's value is the fact that it avoids 
> having service-label related state (forwarding and BGP state) on the 
> routers at the DC-WAN border.  This is one of the typical property of 
> 4364 option C.    It is not obvious at all to me, based on existing 
> spec, how to get the same benefit on the DC-side router in an 
> architecture where the DC-side router also terminates the overlay tunnels.
>
> Do you challenge the goal itself (not having service-label related 
> state on the overlay tunnels router) or suggest that the same benefit 
> can be achieved with less complexity ?
>
> WH> the proposal I mentioned is achieving the same goal w/o the 
> complexity in the DC GW and using already standardised mechanisms to 
> do so.

Let me try to recap.

For traffic going from the DC to the WAN, the DC/WAN router terminating 
the overlay will have to translate from for instance an MPLS/UDP/IP 
encap into an MPLS/MPLS(/MPLS) encap ; assuming that the bottom label 
isn't changed the information to determine the other label(s) has to be 
derived from the UDP or IP header.  The draft we discuss relies on the 
UDP port (or IP address).

You mention a 3-label option C, are you suggesting the use of 
MPLS/MPLS/UDP where the top MPLS label would play the role of the middle 
label in an typical 3-label option-C stack, and would be used to 
determine the destination-PE label between ASBRs ?

Best,

-Thomas

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorization.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange shall not be liable if this message was modified, changed or falsified.
Thank you.

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Thu, 12 Nov 2015 20:06:03 +0000

In-line with WH>

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Thursday 12 November 2015 at 17:46
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

(below)

Henderickx, Wim (Wim):

WH> even for the encapsulations that don’t have this issue, 1 the UDP port proposal breaks some of the protocols so we should never do this,

Sorry to insist, but where do some protocol break when used with a non-standard port as dstport ?

WH> VXLAN destination port is fixed so if you don’t use this many implementations will break

I agree with that.

As such the proposal for me would be to do the 3 label stack as we have in standard MPLS option C which you can do with the encapsulations I mentioned.

WH> yes

I agree for the alternative in which multiple IPs are used.
But I don't think using multiple port would be complex, the configuration on the box can simply consist of a range of ports to use and the rest would be (quite simply) automated.

WH> many HW implementation require a fixed port for certain encapsulations, so you will break these if you don’t use these

Let me try to recap.

For traffic going from the DC to the WAN, the DC/WAN router terminating the overlay will have to translate from for instance an MPLS/UDP/IP encap into an MPLS/MPLS(/MPLS) encap ; assuming that the bottom label isn't changed the information to determine the other label(s) has to be derived from the UDP or IP header.  The draft we discuss relies on the UDP port (or IP address).

WH> yes + I did not discuss the global VNIDs that are used in some implementation

You mention a 3-label option C, are you suggesting the use of MPLS/MPLS/UDP where the top MPLS label would play the role of the middle label in an typical 3-label option-C stack, and would be used to determine the destination-PE label between ASBRs ?

WH> yes

Best,

-Thomas

---

From: Haoweiguo <[email-masked]>
Date: Fri, 13 Nov 2015 01:44:07 +0000

Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is restricted only in data center inside domain. For the traffic from data center inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload will continue to be carried into MPLS network. I can emphasize this point in my later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP encapsulation, so no inner MAC concerns.

Thanks,

weiguo

________________________________

From: BESS [[email-masked]] on behalf of Henderickx, Wim (Wim) [[email-masked]]
Sent: Thursday, November 12, 2015 22:33
To: [email-masked]
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 05:00:06 +0000

The whole draft complicates any data plane we defined so far and since there is simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <[email-masked]<mailto:[email-masked]>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Thanks,

weiguo

From: BESS [[email-masked]<mailto:[email-masked]>] on behalf of Henderickx, Wim (Wim) [[email-masked]<mailto:[email-masked]>]
Sent: Thursday, November 12, 2015 22:33
To: [email-masked]<mailto:[email-masked]>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

---

From: Thomas Morin <[email-masked]>
Date: Fri, 13 Nov 2015 09:57:56 +0100

Hi Wim,

I agree on the analysis that this proposal is restricted to 
implementations that supports the chosen encap with non-IANA ports 
(which may be hard to achieve for instance on hardware implementations, 
as you suggest), or to context where managing multiple IPs would be 
operationally viable.

However, it does not seem obvious to me how the alternative you propose 
[relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] 
addresses the issue of whether the encap behavior is supported or not 
(e.g. your typical ToR chipset possibly may not support this kind of 
encap,  and even software-based switches may not be ready to support 
that today).

My take is that having different options to adapt to various 
implementations constraints we may have would have value.

(+ one question below on VXLAN...)

-Thomas

2015-11-12, Henderickx, Wim (Wim):
>> On VXLAN/NVGRE, do you challenge the fact that they would be used 
>> with a dummy MAC address that would be replaced by the right MAC by a 

I you don't mind me asking : why do you challenge that ?

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 14:48:25 +0000

Thomas, we need a solution that is implementable and will scale in control/data-plane. As such the solution I outline is standardised today and does not need new extensions in control/data-plane. Hence informational draft and can be implemented if there is market demand.

With respect to MAC you need to address both directions and does not make sense to manage this if it is not required/doe snot add value.

From: Thomas Morin <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to implementations that supports the chosen encap with non-IANA ports (which may be hard to achieve for instance on hardware implementations, as you suggest), or to context where managing multiple IPs would be operationally viable.

However, it does not seem obvious to me how the alternative you propose [relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the encap behavior is supported or not (e.g. your typical ToR chipset possibly may not support this kind of encap,  and even software-based switches may not be ready to support that today).

My take is that having different options to adapt to various implementations constraints we may have would have value.

-Thomas

WH> yes

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 16:10:25 +0000

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP redistributes VN routes between the end NVEs in source and destination Ass, ASBRs do not maintain and distribute the VN routes; a tunnel is built between the end NVEs in source and destination Ass for traffic transport. Due to the different data planes in DC and WAN, the tunnel is stitched by several segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. However, there are many DCs that do not support MPLS data plane. This draft is to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate to use another UDP port for the VXLAN encapsulation, I don’t see it breaks anything. Not sure if hw has this restriction either. Even yes, we can consider using UDP source port for this purpose. UDP source port is used for transit ECMP and filled by flow entropy, tunnel egress can determine the flow entropy value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can maintain UDP source port and MPLS label table; when DC ASBR receives a packet from NVE, by lookup the table, it gets the label for the packet, push the label on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear any better alternative.

Regards,
Lucy

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: [email-masked]
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Thanks,

weiguo

---

From: <[email-masked]>
Date: Fri, 13 Nov 2015 17:36:54 +0100

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
> Thomas, we need a solution that is implementable and will scale in 
> control/data-plane. As such the solution I outline is standardised 
> today and does not need new extensions in control/data-plane. Hence 
> informational draft and can be implemented if there is market demand.

The above argument would fully make sense to me if support 
MPLS/MPLS/(UDP or GRE) was common place in ToRs or vswitches.

> With respect to MAC you need to address both directions and does not 
> make sense to manage this if it is not required/doe snot add value.

On VXLAN and the Ethernet header: it looks to me that addressing both 
direction is just a matter of documenting in the draft (if not already 
there) what is the expected behavior for VXLAN/NVGRE. But VXLAN and 
NVGRE are not the only encaps proposed in the specs anyway.

Compared to 3-label option C, my understanding is that the added-value 
is to not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs 
and vswitches.

The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior 
specified in this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

Note that the 3-label option C approach with an MPLS/MPLS/(UDP or GRE) 
encap also requires the introduction of a control plane in the DC 
(possibly in an 'SDN' controller) to learn and advertise the middle MPLS 
label.  Whether one consider that the best path to a working solution 
may also depend on the context.

Best,

-Thomas

> To: Wim Henderickx <[email-masked] 
> <mailto:[email-masked]>>
> Cc: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> (which may be hard to achieve for instance on hardware 
> implementations, as you suggest), or to context where managing 
> multiple IPs would be operationally viable.
>
> However, it does not seem obvious to me how the alternative you 
> propose [relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) 
> encap] addresses the issue of whether the encap behavior is supported 
> or not (e.g. your typical ToR chipset possibly may not support this 
> kind of encap,  and even software-based switches may not be ready to 
> support that today).
>
>>> with a dummy MAC address that would be replaced by the right MAC by 
>>> a sender based on an ARP request when needed ?
>>

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 17:12:07 +0000

Fully agree Tomas’s points (below).

Lucy

From: BESS [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Friday, November 13, 2015 10:37 AM
To: Henderickx, Wim (Wim)
Cc: [email-masked]
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

The above argument would fully make sense to me if support MPLS/MPLS/(UDP or GRE) was common place in ToRs or vswitches.

On VXLAN and the Ethernet header: it looks to me that addressing both direction is just a matter of documenting in the draft (if not already there) what is the expected behavior for VXLAN/NVGRE. But VXLAN and NVGRE are not the only encaps proposed in the specs anyway.

Compared to 3-label option C, my understanding is that the added-value is to not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs and vswitches.

The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support MPLS/MPLS/(UDP or GRE)

Note that the 3-label option C approach with an MPLS/MPLS/(UDP or GRE) encap also requires the introduction of a control plane in the DC (possibly in an 'SDN' controller) to learn and advertise the middle MPLS label.  Whether one consider that the best path to a working solution may also depend on the context.

Best,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 18:45:16 +0000

VXLAN has a dedicated UDP port and is very clear in the RFC7348
The proposal I gave works with an IP data plane
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <[email-masked]<mailto:[email-masked]>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>, Haoweiguo <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Regards,
Lucy

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 18:59:47 +0000

Thomas, we can discuss forever and someone need to describe requirements, but the current proposal I cannot agree to for the reasons explained.

More in-line.

Date: Friday 13 November 2015 at 17:36

Hi Wim,

WH> b depends on the use case and depending on implementation you don’t need to change any of the TOR/vswitches.

WH> we already have BGP for this.

Best,

-Thomas

From: Thomas Morin <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>
Cc: "<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

-Thomas

WH> yes

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 19:15:21 +0000

Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP port. If someone wants to stick on the dedicate UDP port, we can use the src UDP port to achieve that.
The proposal I gave works with an IP data plane
[Lucy] I am not clear what is your proposal in this piled thread. Could you state or copy/paste again?

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 19:21:52 +0000

Date: Friday 13 November 2015 at 20:15

Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP port. If someone wants to stick on the dedicate UDP port, we can use the src UDP port to achieve that.
WH> they are used for entropy, you seem to be  creating a kludge.
The proposal I gave works with an IP data plane
[Lucy] I am not clear what is your proposal in this piled thread. Could you state or copy/paste again?
WH> look at the archives it is in there

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 19:42:11 +0000

Wim,

[Lucy1] It works.
[Lucy1] Is this one?  “If we want to describe a model C VPN interconnect with a IP fabric in a DC I recommend to do an informational RFC that describes this using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in RFC4364.”

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 19:47:19 +0000

Date: Friday 13 November 2015 at 20:42

Wim,

WH2> good luck, but does not mean IETF has to adopt it. Mine also works :-)

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.
WH> it is less bits then you proposal and no changes in any CP/DP that is standardised today. Where is the complexity? Works today and is deployed in many networks

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 20:31:31 +0000

Wim,

See [lucy2]

[Lucy2] IETF does not specify how entropy value is generated yet. It can be provided by tunnel egress and tell the ingress.

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.
WH> it is less bits then you proposal and no changes in any CP/DP that is standardised today. Where is the complexity? Works today and is deployed in many networks
[Lucy2] This is not compliant with many DCs’ architecture as Thomas pointed out. To make them doing it, it becomes very complex.

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 20:48:36 +0000

Date: Friday 13 November 2015 at 21:31

Wim,

See [lucy2]

WH3> the RFC is pretty clear:
— copy —

The UDP source port number
         be calculated using a hash of fields from the inner packet --
         one example being a hash of the inner Ethernet frame's headers.
         This is to enable a level of entropy for the ECMP/load-
         balancing of the VM-to-VM traffic across the VXLAN overlay.
         When calculating the UDP source port number in this manner, it
         is RECOMMENDED that the value be in the dynamic/private port
         range 49152-65535 [RFC6335<https://tools.ietf.org/html/rfc6335>].

— copy --

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.
WH> it is less bits then you proposal and no changes in any CP/DP that is standardised today. Where is the complexity? Works today and is deployed in many networks
[Lucy2] This is not compliant with many DCs’ architecture as Thomas pointed out. To make them doing it, it becomes very complex.
WH3> it is even deployed in existing DC(s)

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 20:55:39 +0000

From: Henderickx, Wim (Wim) [mailto:[email-masked]]
Sent: Friday, November 13, 2015 2:49 PM
To: Lucy yong; Haoweiguo
Cc: [email-masked]
Subject: Re: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

See [lucy2]

— copy --

[Lucy3] Yes, if existing DC supports MPLS, it can use your solution. For those DCs that do not support MPLS, and already use VXLAN+ip tunnels for intra-DC traffic, it is pain for them to implement your solution.  Our draft is to provide a solution for those DCs that do not support MPLS.

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 21:02:31 +0000

Date: Friday 13 November 2015 at 21:55

Wim,

See [lucy2]

— copy --

WH4> works also with VXLAN GPE

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 21:03:12 +0000

Wim,

— copy –

[Lucy3]  This is one way to calculate, perhaps current practice too. But it is just recommended by RFC, not mandated.
[Lucy] I am not clear what is your proposal in this piled thread. Could you state or copy/paste again?
WH> look at the archives it is in there
[Lucy1] Is this one?  “If we want to describe a model C VPN interconnect with a IP fabric in a DC I recommend to do an informational RFC that describes this using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in RFC4364.”

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 21:06:22 +0000

Wim,

See [lucy2]

— copy --

[Lucy4] do you mean VXLAN GRE or VXLAN GRE+NSH?

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 21:12:22 +0000

Date: Friday 13 November 2015 at 22:06

Wim,

See [lucy2]

— copy --

WH5> no regular VXLAN GPE and will also work with VXLAN GRE if this ever get adopted/supported by IETF, NSH is orthogonal

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 21:14:27 +0000

— copy --

[Lucy5] Could elaborate VXLAN GPE solution for this case? I may miss some. Thx ahead.

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 21:16:40 +0000

I’ll try to write something by next IETF

Date: Friday 13 November 2015 at 22:14

— copy --

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Lucy yong <[email-masked]>
Date: Fri, 13 Nov 2015 21:19:31 +0000

Our solution also works with VXLAN-gpe encap.
Lucy

From: Henderickx, Wim (Wim) [mailto:[email-masked]]
Sent: Friday, November 13, 2015 3:17 PM
To: Lucy yong; Haoweiguo
Cc: [email-masked]
Subject: Re: draft-hao-bess-inter-nvo3-vpn-optionc

— copy --

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 13 Nov 2015 21:21:30 +0000

Lucy that’s not the point and I hope this is clear by now

Date: Friday 13 November 2015 at 22:19

— copy --

Lucy

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 13 Nov 2015 22:18:52 +0000

Wim,

If the draft adds some description to say that the VxLAN header is decapsulated, and the MAC header is terminated. It is the inner IP data frames to be passed to the MPLS IP-VPNs.

Will it clear out the confusion?

Linda

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 8:34 AM
To: [email-masked]
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Sat, 14 Nov 2015 07:12:47 +0000

Linda, first of all the draft does not say anything about this and 2nd I am not confused and 3rd I explained the reasons why this does not fly. So read them

From: Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Date: Friday 13 November 2015 at 23:18
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

Linda

---

From: Haoweiguo <[email-masked]>
Date: Sun, 15 Nov 2015 12:51:20 +0000

Hi Wim,

Thanks for your detail explanation.

In your solution, MPLSoGRE/oUDP needs to be supported in data center network. To support option-c interconnection with WAN network, TORs/NVEs need to add one BGP LSP Label in middle layer, so TORs/NVEs need to perform MPLS(VPN Layer) + MPLS (BGP LSP Label) + GRE/UDP encapsulation for the traffic from local connecting end systems to WAN network. In your solution, GRE/UDP layer is not used for transport layer stitching at ASBR-d, traditonal end to end BGP LSP is still used for transport layer.

In our solution, VXLAN/NVGRE are two of the focuses, and for VXLAN/NVGRE and MPLS VPN network option-c mode interconnection, our solution seems to be the only choice. For MPLS Over GRE/UDP, it's the debating point.We propose the same interconnecting solution for MPLSoGRE/oUDP, your proposal provides another optional solution, i think it works and is good. However, i think an independent informational draft only for this point maybe not necessary, we can describe it in same single draft which includes all possible solutions for IP overlay network(VXLAN/NVGRE/VXLAN-GPE/MPLSoGRE/MPLSoUDP) and MPLS VPN network interworking using option-c mode.

In summary, Option-c mode interconnecting solution between IP overlay network and MPLS VPN network is needed for BESS working group, and we should cover all possible solutions for all possible mainstream IP overlay encapsulations in single draft.

Thanks,

weiguo

________________________________
From: BESS [[email-masked]] on behalf of Henderickx, Wim (Wim) [[email-masked]]
Sent: Saturday, November 14, 2015 15:12
To: Linda Dunbar; [email-masked]
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

Linda

---

From: Thomas Morin <[email-masked]>
Date: Mon, 16 Nov 2015 16:37:50 +0100

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
> Thomas, we can discuss forever and someone need to describe 
> requirements, but the current proposal I cannot agree to for the 
> reasons explained.

Well, although discussing forever is certainly not the goal, the reasons 
for rejecting a proposal need to be thoroughly understood.

(more below)

> is to not require the support of an MPLS/MPLS/(UDP or GRE) encap in 
> ToRs and vswitches.
>
> WH> b depends on the use case

I don't get what you mean by "b depends on the use case".
I wrote the above based on the idea that the encap used in 
MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs and 
vswitches.
Another possibility would be service-label/middle-label/Ethernet 
assuming an L2 adjacency between vswitches/ToRs and ASBRs, but this 
certainly does not match your typical DC architecture. Or perhaps had 
you something else in mind ?

> WH> and depending on implementation you don’t need to change any of 
> the TOR/vswitches.

Does this mean that for some implementations you may not need to change 
any of the TOR/vswitches, but that for some others you may ?

Let me take a practical example : while I can quite easily see how to 
implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc based 
on current vswitch implementations of VXLAN, the lack of MPLS/MPLS/(UDP, 
GRE) support in commonplace vswitches seems to me as making that 
alternate solution you suggest harder to implement.

-Thomas

>
>>
>> From: Thomas Morin <[email-masked]>
>> Organization: Orange
>> To: Wim Henderickx <[email-masked]>
>> Cc: "[email-masked]" <[email-masked] <mailto:[email-masked]>>

---

From: Thomas Morin <[email-masked]>
Date: Mon, 16 Nov 2015 16:47:32 +0100

Wim,

> VXLAN has a dedicated UDP port and is very clear in the RFC7348

Well, having a port reserved for this use that won't be the default for 
another protocol is one thing, but that does not prevent in itself the 
same protocol to be applied on another range of ports. (Because HTTP 
specs says port 80 does not prevent the URI scheme to allow specifying 
the port in the URL)

Even reading the VXLAN (Informational) specs, we see room for 
flexibility wrt ports:

       -  Destination Port: IANA has assigned the value 4789 for the
          VXLAN UDP port, and this value SHOULD be used by default as the
          destination UDP port.  Some early implementations of VXLAN have
          used other values for the destination port.  To enable
          interoperability with these implementations, the destination
          port SHOULD be configurable.

I read "4789 SHOULD be used by default" and "SHOULD be configurable".

> Will write something on what I proposed when I get some time, not soon

The co-chair in me needs to ask the following question: should this call 
for adoption be kept on hold until an outline of an alternative solution 
is provided ?

-Thomas

>
> From: Lucy yong <[email-masked] <mailto:[email-masked]>>
> <mailto:[email-masked]>>, Haoweiguo 
> <[email-masked] <mailto:[email-masked]>>
> OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
> redistributes VN routes between the end NVEs in source and destination 
> Ass, ASBRs do not maintain and distribute the VN routes; a tunnel is 
> built between the end NVEs in source and destination Ass for traffic 
> transport. Due to the different data planes in DC and WAN, the tunnel 
> is stitched by several segments, IP tunnels are used in DCs, and MPLS 
> tunnels are used in WAN.
>
> Traditional OptionC requires end-to-end MPLS, which may fit to some 
> DCI cases. However, there are many DCs that do not support MPLS data 
> plane. This draft is to provide the solution for this use case.
>
> Although VXLAN has UDP port number, if tunnel ingress and egress can 
> negotiate to use another UDP port for the VXLAN encapsulation, I don’t 
> see it breaks anything. Not sure if hw has this restriction either. 
> Even yes, we can consider using UDP source port for this purpose. UDP 
> source port is used for transit ECMP and filled by flow entropy, 
> tunnel egress can determine the flow entropy value and inform tunnel 
> ingress. Thus, tunnel egress (i.e. DC ASBR) can maintain UDP source 
> port and MPLS label table; when DC ASBR receives a packet from NVE, by 
> lookup the table, it gets the label for the packet, push the label on 
> the packet before sending toward WAN.
>
> *From:*BESS [mailto:[email-masked]] *On Behalf Of *Henderickx, 
> Wim (Wim)
> *Sent:* Thursday, November 12, 2015 11:00 PM
> *To:* Haoweiguo
> *Cc:* [email-masked] <mailto:[email-masked]>
> *Subject:* Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
> The whole draft complicates any data plane we defined so far and since 
> there is simpler solutions I don’t support this proposal.
>
> *From: *Haoweiguo <[email-masked] <mailto:[email-masked]>>
> *Date: *Friday 13 November 2015 at 02:44
> *To: *Wim Henderickx <[email-masked] 
> *Cc: *"[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> *Subject: *RE: draft-hao-bess-inter-nvo3-vpn-optionc
>
> Hi Wim,
>
> It is used for layer 3 VPN CE device to visit IP based overlay data 
> center network. For layer 3 traffic forwarding, the MAC address in 
> VXLAN/NVGRE is restricted only in data center inside domain. For the 
> traffic from data center inside to outside,  the inner MAC 
> address(destination MAC is ASBR-d's MAC, src MAC is NVE's MAC) in 
> VXLAN/NVGRE will be dropped at ASBR-d, only IP payload will continue 
> to be carried into MPLS network. I can emphasize this point in my 
> later version, it doesn't have much impact on the whole solution.
>
> This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP 
> network to interconnect with MPLS VPN network. In VXLAN-GPE, it 
> supports IP in IP encapsulation, so no inner MAC concerns.
>
> Thanks,
>
> weiguo
>
> ------------------------------------------------------------------------
>
> *From:*BESS [[email-masked] <mailto:[email-masked]>] on 
> behalf of Henderickx, Wim (Wim) [[email-masked] 
> <mailto:[email-masked]>]
> *Sent:* Thursday, November 12, 2015 22:33
> *To:* [email-masked] <mailto:[email-masked]>
> *Subject:* [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
> IP-VPNs. VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The 
> draft does not talk about any of this and introduces a lot of 
> complexity for nothing.
>
> _______________________________________________
> BESS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bess

---

From: Thomas Morin <[email-masked]>
Date: Mon, 16 Nov 2015 16:54:44 +0100

Lucy Yong :
> [Lucy1] Is this one?  “If we want to describe a model C VPN 
> interconnect with a IP fabric in a DC I recommend to do an 
> informational RFC that describes this using VXLAN-GPE, MPLSoGRE or 
> MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
> RFC4364.”
>
> If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support 
> VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for 
> this case, requires three labels + GRE/UDP tunnel, huge overhead is 
> introduced here. This is very complex in my opinion. Our solution can 
> apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 
> 5.1.2). BTW, this may be not what you suggested.

I don't think Wim mentioned 3-labels over UDP or GRE.
Wim proposal works with 2-labels over an IP encap (the topmost label of 
the typical 3-label option C being replaced by an IP transport).

> WH> it is less bits then you proposal and no changes in any CP/DP that 
> is standardised today. Where is the complexity? Works today and is 
> deployed in many networks
>
> [Lucy2] This is not compliant with many DCs’ architecture as Thomas 
> pointed out.

( Note that I merely mentioned the problem of vswitch/ToR support for 
MPLS/MPLS/(UDP or GRE), but had mentioned nothing about overall DC 
architecture.  )

-Thomas

> *From: *Lucy yong <[email-masked] <mailto:[email-masked]>>
> *Date: *Friday 13 November 2015 at 17:10

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Mon, 16 Nov 2015 16:50:35 +0000

From: Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Date: Monday 16 November 2015 at 08:29
To: Thomas Morin <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

In-line

From: BESS <[email-masked]<mailto:[email-masked]>> on behalf of Thomas Morin <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Monday 16 November 2015 at 07:37
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Well, although discussing forever is certainly not the goal, the reasons for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane here, the use cases I have seen in this txt is always where an application behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.

(more below)

I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application behind NVE/TOR requiring model C, than all the discussion on impact on NVE/TOR is void. As such I want to have a discussion on the real driver/requirement for option c interworking with an IP based Fabric.

I wrote the above based on the idea that the encap used in MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs and vswitches.
Another possibility would be service-label/middle-label/Ethernet assuming an L2 adjacency between vswitches/ToRs and ASBRs, but this certainly does not match your typical DC architecture. Or perhaps had you something else in mind ?

WH> see above. The draft right now also requires changes in existing TOR/NVE so for me all this discussion/debate is void.

WH> and depending on implementation you don’t need to change any of the TOR/vswitches.

Does this mean that for some implementations you may not need to change any of the TOR/vswitches, but that for some others you may ?

WH> any proposal on the table requires changes, so for me this is not a valid discussion

Let me take a practical example : while I can quite easily see how to implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc based on current vswitch implementations of VXLAN, the lack of MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to me as making that alternate solution you suggest harder to implement.

WH> I would disagree to this. Tell me which switch/TOR handles multiple UDP ports for VXLAN ?

-Thomas

Cc: "<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>" <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>

Hi Wim,

-Thomas

WH> yes

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Mon, 16 Nov 2015 16:52:48 +0000

In-line

From: Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Date: Monday 16 November 2015 at 08:29
To: Thomas Morin <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

From: BESS <[email-masked]<mailto:[email-masked]>> on behalf of Thomas Morin <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Monday 16 November 2015 at 07:47
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Wim Henderickx <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

Henderickx, Wim (Wim) :

Well, having a port reserved for this use that won't be the default for another protocol is one thing, but that does not prevent in itself the same protocol to be applied on another range of ports. (Because HTTP specs says port 80 does not prevent the URI scheme to allow specifying the port in the URL)

Even reading the VXLAN (Informational) specs, we see room for flexibility wrt ports:

WH> true but it does not say to use multiple at the same time. My read on this is that it allows a non standard port to be used, but not multiple at the same time.

The co-chair in me needs to ask the following question: should this call for adoption be kept on hold until an outline of an alternative solution is provided ?

WH> I first want to understand the requirements for this use case and I don’t agree to adoption since there are major DP implications in this.

-Thomas

From: Lucy yong <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>, Haoweiguo <[email-masked]<mailto:[email-masked]>>
Cc: "<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

Regards,
Lucy

From: Haoweiguo <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>

Hi Wim,

Thanks,

weiguo

From: BESS [<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>] on behalf of Henderickx, Wim (Wim) [<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>]

_______________________________________________
BESS mailing list
[email-masked]<mailto:[email-masked]>https://www.ietf.org/mailman/listinfo/bess

---

From: Lidefeng <[email-masked]>
Date: Tue, 17 Nov 2015 00:30:35 +0000

发件人: BESS [mailto:[email-masked]] 代表 Thomas Morin
发送时间: 2015年11月16日 23:38
收件人: Henderickx, Wim (Wim)
抄送: [email-masked]
主题: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

(more below)

-Thomas

Hi Wim,

-Thomas

WH> yes

---

From: Haoweiguo <[email-masked]>
Date: Tue, 17 Nov 2015 08:12:12 +0000

Hi Wim,

WH> I first want to understand the requirements for this use case and I don’t agree to adoption since there are major DP implications in this.
[weiguo]: You proposed MPLS over GRE/UDP encapsulation for the traffic between data center inside and outside MPLS VPN network. However, for some TOR switches, they don't support MPLS Over GRE/UDP. Even if they support, they rely on internal loop and the forwarding performance will be reduced half.  For option-c scenario, TOR switches need to realize MPLS+MPLS+GRE/UDP, it's more complicated for TOR switches. Current most of the TOR switches support VXLAN encapsulation and have no half forwarding performance issue.  For ASBR-d, it is router device in most case, it has capability to realize more comlicated forwarding behavior.

Thanks,
weiguo
________________________________
From: BESS [[email-masked]] on behalf of Henderickx, Wim (Wim) [[email-masked]]
Sent: Tuesday, November 17, 2015 0:52
To: Thomas Morin; [email-masked]
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

In-line

Wim,

Henderickx, Wim (Wim) :

-Thomas

Hi Wim,

Regards,
Lucy

Hi Wim,

Thanks,

weiguo

---

From: <[email-masked]>
Date: Tue, 17 Nov 2015 10:37:21 +0100

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):
> TM> Well, although discussing forever is certainly not the goal, the 
> reasons for rejecting a proposal need to be thoroughly understood.
> WH> my point is what is the real driver for supporting a plain VXLAN 
> data-plane here, the use cases I have seen in this txt is always where 
> an application behind a NVE/TOR is demanding option c, but none of the 
> NVE/TOR elements.
>

My understanding is that the applications  are contexts where overlays 
are present is when workloads (VMs or baremetal) need to be 
interconnected with VPNs. In these contexts, there can be reasons to 
want Option C to reduce the state on ASBRs.

In these context, its not the workload (VM or baremetal) that would 
typically handle VRFs, but really the vswitch or ToR.

>> TM> The right trade-off to make may in fact depend on whether you prefer:
> WH> see my above comment. If the real use case is an application 
> behind NVE/TOR requiring model C, than all the discussion on impact on 
> NVE/TOR is void. As such I want to have a discussion on the real 
> driver/requirement for option c interworking with an IP based Fabric.

Although I can agree than detailing requirements can always help, I 
don't think one can assume a certain application to dismiss the proposal.

> MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs 
> and vswitches.
> WH> see above. The draft right now also requires changes in existing 
> TOR/NVE so for me all this discussion/debate is void.

No, the spec as it is can be implemented in its VXLAN variant with 
existing vswitches (e.g. OVS allows to choose the VXLAN destination 
port, ditto for the linux kernel stack).

(ToR is certainly another story, most of them not having a flexible 
enough VXLAN dataplane nor support for any MPLS-over-IP.)

> Does this mean that for some implementations you may not need to 
> change any of the TOR/vswitches, but that for some others you may ?
>

See above, the proposal in the draft does not necessarily need changes 
in vswitches.

> implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc 
> based on current vswitch implementations of VXLAN, the lack of 
> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to me as 
> making that alternate solution you suggest harder to implement.
>

I mentioned _v_switches, and many do support a variable destination port 
for VXLAN, which is sufficient to implement what the draft proposes.

-Thomas

>>> Cc: "[email-masked]" <[email-masked]>
>>> encap] addresses the issue of whether the encap behavior is 
>>> supported or not (e.g. your typical ToR chipset possibly may not 
>>> support this kind of encap, and even software-based switches may not 
>>> be ready to support that today).
>>>
>>>>> with a dummy MAC address that would be replaced by the right MAC 
>>>>> by a sender based on an ARP request when needed ?
>>>>

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Tue, 17 Nov 2015 17:48:30 +0000

— Snip —

No, the spec as it is can be implemented in its VXLAN variant with existing vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines solutions which should be implemented across the board HW/SW/etc. Even if some SW switches can do this, the proposal will impose so many issues in HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, we better do it in the right way and guide the industry to a solution which does not imply those complexities. Otherwise we will stick with these specials forever with all consequences (bugs, etc).

- snip -

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>, BESS <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

TM> Well, although discussing forever is certainly not the goal, the reasons for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane here, the use cases I have seen in this txt is always where an application behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.

My understanding is that the applications  are contexts where overlays are present is when workloads (VMs or baremetal) need to be interconnected with VPNs. In these contexts, there can be reasons to want Option C to reduce the state on ASBRs.

In these context, its not the workload (VM or baremetal) that would typically handle VRFs, but really the vswitch or ToR.

WH2> can it not be all cases: TOR/vswitch/Application. I would make the solution flexible to support all of these not?

Although I can agree than detailing requirements can always help, I don't think one can assume a certain application to dismiss the proposal.

WH> for me the proposal is not acceptable for the reasons explained: too much impact on the data-planes

No, the spec as it is can be implemented in its VXLAN variant with existing vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support?

See above, the proposal in the draft does not necessarily need changes in vswitches.

I mentioned _v_switches, and many do support a variable destination port for VXLAN, which is sufficient to implement what the draft proposes.

-Thomas

Hi Wim,

-Thomas

WH> yes

---

From: John E Drake <[email-masked]>
Date: Tue, 17 Nov 2015 18:49:31 +0000

Hi,

I think Wim has conclusively demonstrated that this draft has fatal flaws and I don’t support it.  I also agree with his suggestion that we first figure out what problem we are trying to solve before solving it.

Yours Irrespectively,

John

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - [email-masked]; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Lucy yong <[email-masked]>
Date: Tue, 17 Nov 2015 19:54:34 +0000

For this to work every tenant needs a different VXLAN UDP destination port/receive port.

[Lucy] this is not right. For the traffic from DC ASBR -> NVE, only one UDP dst port is used; for the traffic from NVE-> DC ASBR, DC ASBR signaled UDP port number will be used. Tenant traffic is segregated by VNID, not UDP port. At an NVE, the traffic going toward the same remote PE will be encapsulated w/ VXLAN and the same UDP dst port number even the traffic may belong to different VNs. In short, tenant and UDP port are orthogonal; the solution just uses UDP port to indicate the outgoing mpls label for the packets at DC ASBR when it receives packets from NVEs.

Lucy

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 11:49 AM
To: [email-masked]; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Lucy yong <[email-masked]>
Date: Tue, 17 Nov 2015 20:09:49 +0000

Hi John,

I don’t know what is the fatel flaw here. The solution works. It is matter of the preference. Even if the way of UDP port proposed in the draft is not acceptable, we can use IP addresses to achieve it. The key is that BESS WG needs option C solution for the use case that the applications  are contexts where overlays are present is when workloads (VMs or baremetal) need to be interconnected with VPNs. In these contexts, there can be reasons to want Option C to reduce the state on ASBRs.

Regards,
Lucy

From: BESS [mailto:[email-masked]] On Behalf Of John E Drake
Sent: Tuesday, November 17, 2015 12:50 PM
To: Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

Yours Irrespectively,

John

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - [email-masked]<mailto:[email-masked]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Tue, 17 Nov 2015 22:08:39 +0000

Lucy, does not change anything but I did a typo
I meant to say every tenant needs a different VXLAN UDP destination port for upstream + receive port for downstream on NVE

From: Lucy yong <[email-masked]<mailto:[email-masked]>>
Date: Tuesday 17 November 2015 at 11:54
To: Wim Henderickx <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, BESS <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy

From: BESS [mailto:[email-masked]] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 11:49 AM
To: [email-masked]<mailto:[email-masked]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Lucy yong <[email-masked]>
Date: Wed, 18 Nov 2015 01:10:00 +0000

Hi John,

I don’t know what is the fatel flaw here. The solution works. It is matter of the preference. Even if the way of UDP port proposed in the draft is not acceptable, we can use IP addresses to achieve it. The key is that BESS WG needs option C solution(s) for the use case that the applications are contexts where overlays are present is when workloads (VMs or baremetal) need to be interconnected with VPNs. In these contexts, there can be reasons to want Option C to reduce the state on ASBRs.
Regards,
Lucy

From: BESS [mailto:[email-masked]] On Behalf Of John E Drake
Sent: Tuesday, November 17, 2015 12:50 PM
To: Henderickx, Wim (Wim); EXT - [email-masked]<mailto:[email-masked]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

Yours Irrespectively,

John

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Lucy yong <[email-masked]>
Date: Wed, 18 Nov 2015 01:19:41 +0000

Lucy, does not change anything but I did a typo
I meant to say every tenant needs a different VXLAN UDP destination port for upstream
[Lucy] In this solution, tenant and VXLAN UDP dst. port are orthogonal.  The VXLAN UDP dst. port on an upstream packet contains the info. for the outgoing label at DC ASBR.

Lucy
+ receive port for downstream on NVE

Lucy

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Haoweiguo <[email-masked]>
Date: Wed, 18 Nov 2015 04:23:49 +0000

Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's forwarding table size, the motivation is same as traditional MPLS VPN option-C. Currently, the most common practise on ASBR-d is to terminate VXLAN encapsulation, look up local routing table, and then perform MPLS encapsulation to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only transport layer information needed to be maintained at GW/ASBR-d, the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS VPN network interworking, because VXLAN is becoming pervasive in data center,  the solution in this draft was proposed for the heterogeneous network interworking.

The advantage of this solution is that only VXLAN encapsulation is required for OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d normally is router, it has capability to realize the hardware changes. It will consume many IP addresses and the IP pool for allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the hardware changes. For OVS, it also can realize the data plane changes. For TOR switch, it normally can't realize this function.  This solution mainly focuses on pure software based overlay network, it has more scalability. In public cloud data center, software based overlay network is the majority case.

Whether using solution 1 or 2 depends on the operators real envionment.

So I think our solution has no flaws, it works fine.

Thanks,

weiguo

________________________________
From: BESS [[email-masked]] on behalf of John E Drake [[email-masked]]
Sent: Wednesday, November 18, 2015 2:49
To: Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

Yours Irrespectively,

John

— Snip —

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 19 Nov 2015 17:46:01 +0000

A concern I have with this draft is that it pre-supposes the network architecture in terms of how DCs and WAN are connected. It assumes that DCs and WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:[email-masked]] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

Thanks,

weiguo

I think Wim has conclusively demonstrated that this draft has fatal flaws and I don't support it.  I also agree with his suggestion that we first figure out what problem we are trying to solve before solving it.

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

WH> and depending on implementation you don't need to change any of the TOR/vswitches.

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Osama Zia <[email-masked]>
Date: Thu, 19 Nov 2015 18:50:39 +0000

Today many large enterprises connect to cloud providers using their existing MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:[email-masked]] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <[email-masked]>; John E Drake <[email-masked]>; Henderickx, Wim (Wim) <[email-masked]>; EXT - [email-masked] <[email-masked]>; BESS <[email-masked]>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Jim Uttaro

From: BESS [mailto:[email-masked]] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]<mailto:[email-masked]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 19 Nov 2015 18:55:40 +0000

The assumption being based on one's view of what a DC is ;) Yes when an SP and a Cloud Provider connect an inter-AS model of some form is needed.. It is doubtful that it would ever be Option C as the exposes a tremendous amount of internal information. At the end of the day network architecture/design whether internal/external to a provider is not mandated by the IETF or any standards body. This draft seems to mix encap, interconnect etc.... If the encap "requires" a specific connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Regards,
Osama

From: BESS [mailto:[email-masked]] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <[email-masked]<mailto:[email-masked]>>; John E Drake <[email-masked]<mailto:[email-masked]>>; Henderickx, Wim (Wim) <[email-masked]<mailto:[email-masked]>>; EXT - [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>; BESS <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Jim Uttaro

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Osama Zia <[email-masked]>
Date: Thu, 19 Nov 2015 19:06:18 +0000

CIL

From: UTTARO, JAMES [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <[email-masked]>; UTTARO, JAMES <[email-masked]>; Haoweiguo <[email-masked]>; John E Drake <[email-masked]>; Henderickx, Wim (Wim) <[email-masked]>; EXT - [email-masked] <[email-masked]>; BESS <[email-masked]>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and a Cloud Provider connect an inter-AS model of some form is needed.. It is doubtful that it would ever be Option C as the exposes a tremendous amount of internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a customer router in the cloud that should only expose customer routes. I had argued previously in IETF that current practice of option A is not scalable.

At the end of the day network architecture/design whether internal/external to a provider is not mandated by the IETF or any standards body. This draft seems to mix encap, interconnect etc.... If the encap "requires" a specific connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]<mailto:[email-masked]>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Regards,
Osama

Jim Uttaro

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 19 Nov 2015 19:16:16 +0000

Comments In-Line..

Jim Uttaro

From: Osama Zia [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 2:06 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <[email-masked]<mailto:[email-masked]>>; UTTARO, JAMES <[email-masked]<mailto:[email-masked]>>; Haoweiguo <[email-masked]<mailto:[email-masked]>>; John E Drake <[email-masked]<mailto:[email-masked]>>; Henderickx, Wim (Wim) <[email-masked]<mailto:[email-masked]>>; EXT - [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>; BESS <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and a Cloud Provider connect an inter-AS model of some form is needed.. It is doubtful that it would ever be Option C as the exposes a tremendous amount of internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a customer router in the cloud that should only expose customer routes. I had argued previously in IETF that current practice of option A is not scalable.
[Jim U>] So, do you mean a CSC model where the NHs are the customer CEs, if so, then that is not Classical Option C where PE LBs are exchanged as the NHs for the customers path/routes.

Jim Uttaro

Regards,
Osama

Jim Uttaro

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: Osama Zia <[email-masked]>
Date: Thu, 19 Nov 2015 19:38:13 +0000

CIL

From: UTTARO, JAMES [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 11:16 AM
To: Osama Zia <[email-masked]>; UTTARO, JAMES <[email-masked]>; Haoweiguo <[email-masked]>; John E Drake <[email-masked]>; Henderickx, Wim (Wim) <[email-masked]>; EXT - [email-masked] <[email-masked]>; BESS <[email-masked]>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Comments In-Line..

Jim Uttaro

CIL

The assumption being based on one's view of what a DC is ;) Yes when an SP and a Cloud Provider connect an inter-AS model of some form is needed.. It is doubtful that it would ever be Option C as the exposes a tremendous amount of internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a customer router in the cloud that should only expose customer routes. I had argued previously in IETF that current practice of option A is not scalable.
[Jim U>] So, do you mean a CSC model where the NHs are the customer CEs, if so, then that is not Classical Option C where PE LBs are exchanged as the NHs for the customers path/routes.
[OZ] Sorry for the confusion. My comment was not specific to the draft. I was responding to your comment  "Option C as the exposes a tremendous amount of internal information"

Jim Uttaro

Regards,
Osama

Jim Uttaro

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 19 Nov 2015 20:13:07 +0000

IMO an Option C model not CSC between to different administrative authorities where internal reachability state is exchanged is a non-starter.  But coming back to this draft it seems that tying the encap and the network architecture is a requirement or am I getting that wrong. If not, how should I interpret this draft? Should it be if I do Option C as I must have different AS domains between my internal DCs and WAN and my encap models are different in the WAN and DC, and I am using specific encaps in different places etcc... it would be in my solution space??

Jim Uttaro

From: Osama Zia [mailto:[email-masked]]
Sent: Thursday, November 19, 2015 2:38 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [email-masked]; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

Comments In-Line..

Jim Uttaro

CIL

Jim Uttaro

Regards,
Osama

Jim Uttaro

Hi,

Thanks,

weiguo

Yours Irrespectively,

John

- Snip -

- snip -

Hi Wim, WG,

-Thomas

Hi Wim,

-Thomas

WH> yes

Thank you.