Subject: Could millions of ARP entries on DC gateways be reduced to one

From: Xuxiaohu <[email-masked]>
Date: Thu, 25 Aug 2011 02:47:47 +0000

Hi all,

Another obvious advantage of host route based IP-only L2VPN (e.g., Virtual Subnet) over VPLS, as a DCI solution, is to reduce the ARP table size on DC gateways by several order of magnitude. Assume there are millions of CE hosts (i.e.,VMs) within a single VLAN/subnet, if VPLS is used as a DCI solution, DC gateways (i.e., DC exit routers) would have to know millions of ARP entries corresponding to these VMs. In contrast, in the Virtual Subnet solution, DC gateways are directly connected to the PE routers of the IP-only L2VPN which act as ARP proxies, MAC addresses of those ARP entries for VMs on DC gateways are identical (i.e., ARP Proxy's MAC). Thus these millions of ARP entries can be aggregated into one entry (1.0.0.0/8->ARP proxy's MAC). That's to say, the exact-matching algorithm for ARP cache lookup is changed to the longest-matching algorithm. Of course, there is no free lunch. The side-effect of this change is that DC gateways could send out packets destined for non-existing VMs to the PE routers of that IP-only L2VPN. Fortunately, once those packets arrive at the PE routers of that IP-only L2VPN, which in turn will drop them directly since there is no matching routes for them.

Since this topic is also interesting to ARMD WG, I copy this email to ARMD.

Best regards,
Xiaohu

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 25 Aug 2011 12:03:55 -0500

Xiao Hu,

That is a very interesting proposal.
Do you mean that there is a single ARP proxy which represents millions of
VMs? All the packets destined to any of those "millions of VMs" will be
rerouted to "the single ARP proxy"? Will this create too much pressure on
this single point?
How is your proposed approach comparing with SEATTLE's distributed approach?

Linda Dunbar
On Wed, Aug 24, 2011 at 9:47 PM, Xuxiaohu <[email-masked]> wrote:

>  Hi all,****
>
> ** **
>
> Another obvious advantage of host route based IP-only L2VPN (e.g., Virtual
> Subnet) over VPLS, as a DCI solution, is to reduce the ARP table size on DC
> gateways by several order of magnitude. Assume there are millions of CE
> hosts (i.e.,VMs) within a single VLAN/subnet, if VPLS is used as a DCI
> solution, DC gateways (i.e., DC exit routers) would have to know millions of
> ARP entries corresponding to these VMs. In contrast, in the Virtual Subnet
> solution, DC gateways are directly connected to the PE routers of the
> IP-only L2VPN which act as ARP proxies, MAC addresses of those ARP entries
> for VMs on DC gateways are identical (i.e., ARP Proxy’s MAC). Thus these
> millions of ARP entries can be aggregated into one entry (1.0.0.0/8->ARP
> proxy’s MAC). That’s to say, the exact-matching algorithm for ARP cache
> lookup is changed to the longest-matching algorithm. Of course, there is no
> free lunch. The side-effect of this change is that DC gateways could send
> out packets destined for non-existing VMs to the PE routers of that IP-only
> L2VPN. Fortunately, once those packets arrive at the PE routers of that
> IP-only L2VPN, which in turn will drop them directly since there is no
> matching routes for them. ****
>
> ** **
>
> ****
> ** **
> Best regards,****
> Xiaohu****
> _______________________________________________
> armd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/armd
>
>

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 26 Aug 2011 02:00:22 +0000

发件人: Linda Dunbar [mailto:[email-masked]]
发送时间: 2011年8月26日 1:04
收件人: Xuxiaohu
抄送: [email-masked]; [email-masked]
主题: Re: [armd] Could millions of ARP entries on DC gateways be reduced to one

Xiao Hu,

That is a very interesting proposal.
Do you mean that there is a single ARP proxy which represents millions of VMs? All the packets destined to any of those "millions of VMs" will be rerouted to "the single ARP proxy"? Will this create too much pressure on this single point?

Hi Linda,

It’s just an example. In practice, if the pressure on the L2VPN PE router (if Virtual Subnet is used for DCI) within the DC is too large, you could connect the DC exit router to multiple L2VPN PE routers to achieve load-balancing. For example, the traffic within VLAN 1-1000 is forwarded through PE-1 of that L2VPN,  and the traffic within VLAN 1001-2000 is forwarded through PE-2 of that L2VPN…  Such load-balancing usage is very common no matter what special L2 technology (e.g., STP, SPB,TRILL,VPLS …) is used.

SEATTLE is one of the many MAC-routing based L2VPN solutions. With any MAC-routing based L2VPN, the gateway router should and would get the real MAC addresses of the requested hosts in ARP resolution. Since these MAC addresses are not identical, the ARP cache entries on the gateway router could not be aggregated at all.

Xiaohu

Linda Dunbar
On Wed, Aug 24, 2011 at 9:47 PM, Xuxiaohu <[email-masked]<mailto:[email-masked]>> wrote:
Hi all,

Another obvious advantage of host route based IP-only L2VPN (e.g., Virtual Subnet) over VPLS, as a DCI solution, is to reduce the ARP table size on DC gateways by several order of magnitude. Assume there are millions of CE hosts (i.e.,VMs) within a single VLAN/subnet, if VPLS is used as a DCI solution, DC gateways (i.e., DC exit routers) would have to know millions of ARP entries corresponding to these VMs. In contrast, in the Virtual Subnet solution, DC gateways are directly connected to the PE routers of the IP-only L2VPN which act as ARP proxies, MAC addresses of those ARP entries for VMs on DC gateways are identical (i.e., ARP Proxy’s MAC). Thus these millions of ARP entries can be aggregated into one entry (1.0.0.0/8-<http://1.0.0.0/8->>ARP proxy’s MAC). That’s to say, the exact-matching algorithm for ARP cache lookup is changed to the longest-matching algorithm. Of course, there is no free lunch. The side-effect of this change is that DC gateways could send out packets destined for non-existing VMs to the PE routers of that IP-only L2VPN. Fortunately, once those packets arrive at the PE routers of that IP-only L2VPN, which in turn will drop them directly since there is no matching routes for them.

Best regards,
Xiaohu

_______________________________________________
armd mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/armd

---

From: Xuxiaohu <[email-masked]>
Date: Sat, 27 Aug 2011 10:02:00 +0000

Hi all,

An updated version of Virtual Subnet can be found at http://tools.ietf.org/html/draft-xu-virtual-subnet-06. Virtual Subnet is a host route based IP-only L2VPN which can be used as a scalable approach for data center interconnect. Compared to MAC forwarding based L2VPN approaches, VS could offer you many desirable benefits which are mentioned in a section of Comparison with VPLS.

Any comments are welcome.

Best regards,
Xiaohu

发件人: Xuxiaohu
发送时间: 2011年8月26日 10:12
收件人: 'Linda Dunbar'
抄送: [email-masked]; [email-masked]
主题: 答复: [armd] Could millions of ARP entries on DC gateways be reduced to one

Xiao Hu,

Hi Linda,

Xiaohu

Linda Dunbar

Best regards,
Xiaohu