Subject: Re: [nvo3] FW: New Version Notification for draft-yong-nvo3-frwk-dpreq-addition-00.txt

From: "Bitar, Nabil N" <[email-masked]>
Date: Fri, 4 Jan 2013 13:13:15 -0500

Hi,
Please, see inline.

Thanks,
Nabil

On 12/17/12 1:57 PM, "Kireeti Kompella" <[email-masked]> wrote:

>On Dec 17, 2012, at 10:18 , "NAPIERALA, MARIA H" <[email-masked]> wrote:
>
>> Intra-subnet traffic can be also handled by a layer 3 overlay.
>
>Let me expand.
>
>I see the need for E-VPN for non-IP traffic.  This is real, and is not
>met by IP VPNs (news flash!)

<NB> Agreed.
>
>For IP traffic, whether intra and inter-subnet, IP VPNs suffice.

<NB> Please, see my other emamil and interested in your opinion. If we
always route when the Ethernet payload is an IP packet, then there is no
meaning for a suvnet at least it seeems to me. In another words, why not
say every subnet is 32 in case of IPv4, and /128 in case of IPv6.
>
>The solution is simple: route if IP, bridge if not.

<NB> While I see good reasons and maybe the desire to do that, I am not
sure that the rule applies always and as it may be dependent on the
application. 

>Yes, one could do IRB, but why?  IRB brings in complications, especially
>for multicast.  I'm sure someone suggested this already, so put me down
>as supporting this view.

<NB> So, just for clarity, for unicast, implementation wise it canbe done
at least with not much complication as documented elsewhere. For instance
when you support such a mode, the VM can send inter-subnet traffic to a
secial MAC address that triggers IP lookup in a VRF (e.g., similar to be
being a default GW for a subnet).What is wrong with that?
For multicast, there could be multiple options, including using IP
multicast only when the application is IP multicast. Again, interested in
the complicated parts and options looked at as the note seems to indicate.
 
>
>A NVE that supports both E-VPN and IP VPN for a given tenant simply sends
>IP traffic to the IP VPN and sends the rest to E-VPN.  How this happens
>is implementation specific.  Note that this assumes that the NVE
>intercepts ARPs and responds to them with the same MAC.  Does anyone see
>a problem with this?
<NB> Wouldn't that ARP interception and responding with own MAC prevents
the first what is implied in the first sentence unless you are implying
that traffic from a VM can be either sent on e-VPN or an IP-VPN but not
both depending on where the traffic is destined which bring us to the
above discussion.
To the last statement, in E-VPN case the NVE should reply with the IP
destination MAC learned vua BGP and not with it own MAC, right?

>
>If there is a case for _only_ intra-subnet traffic, one may create an
>E-VPN to handle both IP and non-IP; but I suspect this is a rare case.
<NB> SO, there may be a thread on that and I know that was discussion in
other contexts. I cannot see how this cane in E-VPN unless we carry IP
routes that belong to an IP VPN in BGP advertisements, which is not pure
E-VPN. E-VPN today carries ARP entries for the Vms that are connected to
the same E-VPN, but not other routes. Multiple E-VPN instances can belong
to the same IP VPN. FIBs would have to be appropriately built as well.  I
am not sure of the work involved here. I can see logically though how what
is suggested above and elsewhere to bridge and route can work.

>
>From that point of view, I would like to see E-VPNs in the data center
>*always* coupled with IP VPNs, and only dealing with non-IP traffic.
<NB> Pleas,e see above on assuming that whenever the Ethernet payload is
IP you route.
>
>This may appear drastic, but I think operationally, this is will simplify
>things.  As always, I am open to alternate suggestions, provided they are
>presented without religion or politics.  I'm especially keen to hear from
>those deploying.
>
>Kireeti.
>
>_______________________________________________
>nvo3 mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/nvo3

---

From: "NAPIERALA, MARIA H" <[email-masked]>
Date: Fri, 4 Jan 2013 19:26:17 +0000

Hi Nabil,

> say every subnet is 32 in case of IPv4, and /128 in case of IPv6.

This is about VM configuration models. The guest VM IPv4 configuration could be:
   A) local /32 IP address + /32 point-to-point route to a default gateway + default route.
   B) local /32 IP address + /24 to local Ethernet interface + default route to an address on that /24.

Both those models are supported by draft-ietf-l3vpn-end-system-00.
  
In the model B) a set of VMs are configured to belong to the same IP subnet (which is still often the case how the VM are being configured).  Both models can be supported. In the case of B), the NVE implements proxy ARP for all the addresses on the /24. With proxy ARP, there is no difference between B) and A) with respect to forwarding. 
It is just that the virtual subnet has no locality across a data-center.

> sure that the rule applies always and as it may be dependent on the

Which IP applications (i.e., actual applications running in VMs) require bridging?

---

From: Lucy yong <[email-masked]>
Date: Fri, 4 Jan 2013 21:34:23 +0000

Maria,

Couple of clarifications.

> Both those models are supported by draft-ietf-l3vpn-end-system-00.
[Lucy] When using the solution in the draft, you create one L3VPN instances for case A.
Do you create one or more L3VPN instances for case B?
> 
> In the model B) a set of VMs are configured to belong to the same IP
> subnet (which is still often the case how the VM are being configured).
> Both models can be supported. In the case of B), the NVE implements
> proxy ARP for all the addresses on the /24. With proxy ARP, there is no
> difference between B) and A) with respect to forwarding.
[Lucy] In case A, you forward on IP address. In case B, do you also always forward on IP address?
How do you set up a policy per a subnet? 

> It is just that the virtual subnet has no locality across a data-center.
[Lucy] Do you mean that both case A and B only apply within a DC?

Thanks,
Lucy

---

From: "NAPIERALA, MARIA H" <[email-masked]>
Date: Fri, 4 Jan 2013 23:06:00 +0000

Lucy,

> > This is about VM configuration models. The guest VM IPv4
> configuration
> > could be:
> >    B) local /32 IP address + /24 to local Ethernet interface +
> default
> > route to an address on that /24.
> >

I am not sure what you mean by "L3VPN instance"? 

> [Lucy] Do you mean that both case A and B only apply within a DC?

Virtual subnet has no locality either intra- or inter-DC.

Maria

---

From: Truman Boyes <[email-masked]>
Date: Fri, 4 Jan 2013 18:14:40 -0500

Hi, comments inline:

Sent from my iPad

>> met by IP VPNs (news flash!)

<TB> True. There is also the case when IP NVE end points need to join a larger EVPN domain that is provided on traditional MPLS nodes. We might not want to stitch IPVPN to EVPN but rather have EVPN at the NVE to join these other services. 
> say every subnet is 32 in case of IPv4, and /128 in case of IPv6.

<TB> I would say the subnet becomes abstracted. That's the beauty to it. Subnet now does not define broadcast domain, but rather host limit.

---

From: Lucy yong <[email-masked]>
Date: Mon, 7 Jan 2013 23:40:22 +0000

Hi Maria,

Please see more inline.

> -----Original Message-----
> From: NAPIERALA, MARIA H [mailto:[email-masked]]
> Sent: Friday, January 04, 2013 5:06 PM
> To: Lucy yong; Bitar, Nabil N; Kireeti Kompella
> Cc: [email-masked]; Aldrin Isaac
> Subject: RE: [nvo3] FW: New Version Notification for draft-yong-nvo3-
> frwk-dpreq-addition-00.txt
> 
> Lucy,
> 
[Lucy] Original subject is about to construct a tenant network that contain multiple subnets. If all tenant systems in the tenant use IP, do you construct on IP VPN for the tenant network, or configure multiple? Can tenant systems on the same subnet run a broadcast application under this construction?  

Configuring local /32 IP address on a tenant system means that there is no longer host address, just having network address. Is this what we want to do in the network virtualization in DC. Does this mean that in the nvo3, a subnet is no longer useful? I am not expert on addressing and like to hear people's opinion on this.

> > > proxy ARP for all the addresses on the /24. With proxy ARP, there
> is
> > no
[Lucy] Thanks for the explanation. WAN IPVPN [RFC4346] provides intra and inter IP subnet routing seamlessly because PE treats CE as a router and vice versa. Do you think that IPVPN-end-system fit all tenant IP networks or applications? 

Regards,
Lucy

---

From: Lucy yong <[email-masked]>
Date: Mon, 7 Jan 2013 23:49:33 +0000

One comment inline.

> <TB> I would say the subnet becomes abstracted. That's the beauty to it.
> Subnet now does not define broadcast domain, but rather host limit.
[Lucy] Does it imply that an IP host should not send broadcast traffic? Will it a beauty or reduce the network capability. Of cause, if no such need in nvo3, removing it means simplification, which is good.'

Lucy

---

From: "NAPIERALA, MARIA H" <[email-masked]>
Date: Tue, 8 Jan 2013 02:40:14 +0000

Lucy,

> > > >    A) local /32 IP address + /32 point-to-point route to a
> default
> > > > gateway + default route.
> [Lucy] Original subject is about to construct a tenant network that
> contain multiple subnets. 

This is a complication that is not desirable as it is bad idea to have to multiple subnets per VLAN. 

> If all tenant systems in the tenant use IP, do you construct on IP VPN for the tenant network, or configure
> multiple? Can tenant systems on the same subnet run a broadcast
> application under this construction?

> Configuring local /32 IP address on a tenant system means that there is
> no longer host address, just having network address. Is this what we
> want to do in the network virtualization in DC. Does this mean that in
> the nvo3, a subnet is no longer useful? I am not expert on addressing
> and like to hear people's opinion on this.
> 
> >
> [Lucy] Thanks for the explanation. WAN IPVPN [RFC4346] provides intra
> and inter IP subnet routing seamlessly because PE treats CE as a router
> and vice versa. Do you think that IPVPN-end-system fit all tenant IP
> networks or applications?
> 
> Regards,
> Lucy
> >
> > Maria
> >

---

From: Lucy yong <[email-masked]>
Date: Tue, 8 Jan 2013 15:58:38 +0000

> to multiple subnets per VLAN.
[Lucy] Agree. That is the reason I don't see EVPN fit it and propose to have a new service type.
Do you say that IPVPN-end-system can be used to construct a tenant network that contain multiple subnets? or you say that for a virtual network overlay, there is no need for IP subnet schema. Just use IP network address /32 at all tenant systems.

Lucy
> > > > > In the model B) a set of VMs are configured to belong to the
> same
> > > IP
> > > > [Lucy] In case A, you forward on IP address. In case B, do you
> also
> > > > always forward on IP address?
> > > > > It is just that the virtual subnet has no locality across a
> data-
> > > > center.

---

From: "NAPIERALA, MARIA H" <[email-masked]>
Date: Tue, 8 Jan 2013 16:19:43 +0000

Lucy,

> > This is a complication that is not desirable as it is bad idea to
> have
> > to multiple subnets per VLAN.
> Do you say that IPVPN-end-system can be used to construct a tenant
> network that contain multiple subnets? 

You could support multiple subnets in one IPVPN-end-system VPN but this is a bad network design. As I said, this is like having multiple subnets per VLAN.

> or you say that for a virtual
> network overlay, there is no need for IP subnet schema. Just use IP
> network address /32 at all tenant systems.

As I said before, a VM belonging to a subnet is one of configuration models and should be supported. An IP subnet is a CUG that corresponds to one VPN.

Maria

---

From: Lucy yong <[email-masked]>
Date: Tue, 8 Jan 2013 16:24:44 +0000

Hi Maria,
 
> > [Lucy] Agree. That is the reason I don't see EVPN fit it and propose
> to
> > have a new service type.
> You could support multiple subnets in one IPVPN-end-system VPN but this
> is a bad network design. As I said, this is like having multiple
> subnets per VLAN.
[Lucy] Thank you to clarify that again, it is clear to me now. So what you suggest that a good design is to construct several IPVPN-end-system VPNs for a tenant network that contain several subnets. may I understand you correct?

Lucy
> As I said before, a VM belonging to a subnet is one of configuration
> models and should be supported. An IP subnet is a CUG that corresponds
> to one VPN.
> 
> Maria

---

From: "NAPIERALA, MARIA H" <[email-masked]>
Date: Tue, 8 Jan 2013 20:25:29 +0000

> [Lucy] Thank you to clarify that again, it is clear to me now. So what
> you suggest that a good design is to construct several IPVPN-end-system
> VPNs for a tenant network that contain several subnets. may I
> understand you correct?
> 

Yes. The goal of grouping VMs in an IP subnet is, naturally, to match it to a virtual network. 

Maria

---

From: Lucy yong <[email-masked]>
Date: Tue, 8 Jan 2013 20:31:01 +0000

Maria,

> Yes. The goal of grouping VMs in an IP subnet is, naturally, to match
[Lucy] understand. Furthermore you want that each virtual network uses IP forwarding because they are IP based, is that right? 

Thanks,
Lucy

---

From: Aldrin Isaac <[email-masked]>
Date: Thu, 10 Jan 2013 19:04:48 -0500

Hi Kireeti,

I believe the assessment about eliminating the role of Ethernet in an
all-IP world makes sense for many scenarios, but not all.

It certainly makes sense for a "consumer cloud" (Google, Amazon, etc) but
might not be the best choice for SPs that want to be in the "enterprise
cloud" business.

In the consumer cloud each VM is a destination and generally VM networking
is fully in the control of the SP -- here the consumer doesn't care so much
about how things connect so long as they just do.

In the enterprise cloud, the details of the stuff in between source and
destination does matter.  Many enterprises tend to customize their services
(comprised of apps, policy, security, load balancing, networking,
monitoring, etc) to meet their particular goals, challenges, differentiate
themselves from their competitors, etc.  For example, some enterprise
customers would want to operate their own virtual SRX firewall.

Using IPVPN for where enterprise customers need subnets reminds me of how
SPs pushed IPVPN on customers in the early part of the last decade when
what the customers wanted was Ethernet.  If an SP is only interested in
delivering consumer cloud, then    "route IP, bridge non-IP" makes sense --
this is a large piece of the market, but it doesn't cover all of it.  Maybe
it should be "route _known_ IP and bridge other", but then what's the point?

In addition to this, I'm perplexed when I see discussion of inefficient
routing (tromboning) in Ethernet approach and none about route scaling
challenges in /32 IPVPN approach and its solution (route aggregation) which
brings us back to the same problem.

Best regards -- aldrin

On Monday, December 17, 2012, Kireeti Kompella wrote:

> I see the need for E-VPN for non-IP traffic.  This is real, and is not met
> by IP VPNs (news flash!)
>
> The solution is simple: route if IP, bridge if not.  Yes, one could do
> IRB, but why?  IRB brings in complications, especially for multicast.  I'm
> sure someone suggested this already, so put me down as supporting this view.
>
> IP traffic to the IP VPN and sends the rest to E-VPN.  How this happens is
> implementation specific.  Note that this assumes that the NVE intercepts
> ARPs and responds to them with the same MAC.  Does anyone see a problem
> with this?
>