Subject: Active-active DC exits in the DCI solution

From: Xuxiaohu <[email-masked]>
Date: Tue, 23 Aug 2011 03:10:16 +0000

Hi all,

There had been a lot of discussion about active-active multi-homing for a data center (DC) site in the layer2 DCI solution before. Now I'm considering whether we could further achieve active-active DC exits when considering route optimization for traffic routing to/from the outside of DCs (e.g., the Internet). In a normal case, each DC site will be connected to a default gateway (i.e., DC exit router) which is responsible for forwarding traffic routing to/from the outside. However, since these default gateways are within a single subnet due to the layer2 DCI usage, normally there is only one default gateway router (acting as VRRP master) is allowed to forward traffic routing to/from the outside. This is obviously not optimal from the perspective of WAN bandwidth utilization.

Active-active VRRP has been proposed in the above case so that the traffic destined for the outside could be forwarded by the local DC exit router. This is workable when path symmetry is not required. However, in most cases, firewall or NAT devices are deployed at the DC exits for the traffic to/from the outside. In such cases, path symmetry is a must, as a result, active-active VRRP is not available anymore.

Is there any better idea to achieve active-active DC exits while path symmetry is guaranteed?

Best regards,
Xiaohu

---

From: Aldrin Isaac <[email-masked]>
Date: Tue, 23 Aug 2011 07:37:03 -0400

There are now firewalls that support active-active functionality
including support for virtual wire (bump-in-the-wire) functionality
that may be able to solve those issues.  Moreover, virtual-wire
firewalls are interesting in that the firewall performs only network
security role while keeping all network routing/switching functions
with routers and switches.

On 8/22/11, Xuxiaohu <[email-masked]> wrote:
> Hi all,
>
> There had been a lot of discussion about active-active multi-homing for a
> data center (DC) site in the layer2 DCI solution before. Now I'm considering
> whether we could further achieve active-active DC exits when considering
> route optimization for traffic routing to/from the outside of DCs (e.g., the
> Internet). In a normal case, each DC site will be connected to a default
> gateway (i.e., DC exit router) which is responsible for forwarding traffic
> routing to/from the outside. However, since these default gateways are
> within a single subnet due to the layer2 DCI usage, normally there is only
> one default gateway router (acting as VRRP master) is allowed to forward
> traffic routing to/from the outside. This is obviously not optimal from the
> perspective of WAN bandwidth utilization.
>
> Active-active VRRP has been proposed in the above case so that the traffic
> destined for the outside could be forwarded by the local DC exit router.
> This is workable when path symmetry is not required. However, in most cases,
> firewall or NAT devices are deployed at the DC exits for the traffic to/from
> the outside. In such cases, path symmetry is a must, as a result,
> active-active VRRP is not available anymore.
>

-- 
Sent from my mobile device

---

From: Phil Bedard <[email-masked]>
Date: Tue, 23 Aug 2011 08:36:38 -0400

Most of the newer security devices including FW, IDS, IPS have the ability
to deal with asymmetric traffic flows when deployed in pairs, either through
clustering where traffic is shunted across a dedicated inter-chassis link or
state table syncrhonization.    Active/Active gateways are being achieved
these days through either virtual chassis or multi-chassis link aggregation.
Some vendors take one approach or the other while others support both
approaches.    We have deployed equipment in a mutual (back to back) MC-LAG
configuration which balances traffic in both directions.

The old way of active/active balancing at L3 would be use multiple VRRP
groups with some hosts using one gateway and other hosts using the other
gateway, but doesn't guarantee symmetry.  If you have multilayer devices
supporting virtual L3 interfaces within a L2 domain MAC learning can take
care of the asymmetry by sending traffic across a trunk between the two
devices.  GLBP from Cisco is an enhanced version of VRRP/HSRP which hands
out multiple virtual MAC addresses which allows outbound L3 load balancing,
I can't remember it being written into an information RFC or a version of
VRRP being proposed which does the same.

Phil 
From:  Xuxiaohu <[email-masked]>
Date:  Tue, 23 Aug 2011 03:10:16 +0000
To:  "[email-masked]" <[email-masked]>
Subject:  Active-active DC exits in the DCI solution

Hi all,
 
data center (DC) site in the layer2 DCI solution before. Now I¹m considering

Is there any better idea to achieve active-active DC exits while path
symmetry is guaranteed?
 
Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 24 Aug 2011 03:00:01 +0000

Hi Phil,

If I understand it correctly, the following approaches you mentioned are almost proprietary approaches which require DC exit routers in geographically dispersed locations are provided by a single vendor.

I found there was a very simple STANDARD approach to the same goal when host route based IP-only L2VPN service (e.g., Virtual Subnet http://tools.ietf.org/html/draft-xu-virtual-subnet-05) is used for DCI. For example, when incoming traffic from the Internet enters a DC, source IP addresses of the traffic will be NATed on the DC gateway. Notes that here DC exit gateways of geographically dispersed DCs are configured with different IP address pools without any overlapping. In addition, the corresponding routes for the above address pools are advertised by the DC gateways to their connected PE routers of the IP-only L2VPN respectively. Thus, when the outgoing traffic destined to the Internet arrives at its local PE router of the IP-only L2VPN, that PE router could forward the traffic according to the matching routes for the above address pools. In this way, active-active DC exits can be achieved easily in the case where path symmetry is required.

Best regards,
Xiaohu

发件人: Phil Bedard [mailto:[email-masked]]
发送时间: 2011年8月23日 20:37
收件人: Xuxiaohu; [email-masked]
主题: Re: Active-active DC exits in the DCI solution

Most of the newer security devices including FW, IDS, IPS have the ability to deal with asymmetric traffic flows when deployed in pairs, either through clustering where traffic is shunted across a dedicated inter-chassis link or state table syncrhonization.    Active/Active gateways are being achieved these days through either virtual chassis or multi-chassis link aggregation.   Some vendors take one approach or the other while others support both approaches.    We have deployed equipment in a mutual (back to back) MC-LAG configuration which balances traffic in both directions.

The old way of active/active balancing at L3 would be use multiple VRRP groups with some hosts using one gateway and other hosts using the other gateway, but doesn't guarantee symmetry.  If you have multilayer devices supporting virtual L3 interfaces within a L2 domain MAC learning can take care of the asymmetry by sending traffic across a trunk between the two devices.  GLBP from Cisco is an enhanced version of VRRP/HSRP which hands out multiple virtual MAC addresses which allows outbound L3 load balancing, I can't remember it being written into an information RFC or a version of VRRP being proposed which does the same.

Phil
From: Xuxiaohu <[email-masked]<mailto:[email-masked]>>
Date: Tue, 23 Aug 2011 03:10:16 +0000
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Active-active DC exits in the DCI solution

Hi all,

There had been a lot of discussion about active-active multi-homing for a data center (DC) site in the layer2 DCI solution before. Now I’m considering whether we could further achieve active-active DC exits when considering route optimization for traffic routing to/from the outside of DCs (e.g., the Internet). In a normal case, each DC site will be connected to a default gateway (i.e., DC exit router) which is responsible for forwarding traffic routing to/from the outside. However, since these default gateways are within a single subnet due to the layer2 DCI usage, normally there is only one default gateway router (acting as VRRP master) is allowed to forward traffic routing to/from the outside. This is obviously not optimal from the perspective of WAN bandwidth utilization.

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 24 Aug 2011 03:07:13 +0000

Hi Aldrin,

Thanks for the following information. Please see my reply to Phil Bedard in another email.

Best regards,
Xiaohu

> -----邮件原件-----
> 发件人: Aldrin Isaac [mailto:[email-masked]]
> 发送时间: 2011年8月23日 19:37

---

From: Crist <[email-masked]>
Date: Wed, 24 Aug 2011 07:16:35 -0300

Hi Xuxiaohu,

Nice solution however a case come up  when I read the draft.

Trqceability of external ips. For example if a hacker try to get into the web server. To get info about it we have sync the nat table with web servers internal logs using the time. Even with specilized tools it is not easy in some sites with high rate of incoming conns.

Hi Phil

Solutions like GLBP achieve load balance however this kind of solution do not solve the problem of optimum path inside the DC. For example even with a load balance solution the client inside dc1 can go out using the gateway in dc2.

To solve this problem today uses hsrp blocking the control messages between the DC's.In such way the both hsrp nodes becomes the master in its LAN. It's not a elegant solution but it works.

Regards 

Cristiano

Enviado via iPhone

---

From: Aldrin Isaac <[email-masked]>
Date: Wed, 24 Aug 2011 08:39:38 -0400

Another key problem with the dedicated NAT pool per DC gateway is that
when you lose a DC gateway you get a bunch of broken TCP connections.
This is a problem for some applications where app session is tied to
TCP connection.

---

From: Phil Bedard <[email-masked]>
Date: Wed, 24 Aug 2011 10:17:49 -0400

Hi Xiaohu,  yes the approaches thus far have been largely proprietary and
generally do not perform as well when you are talking about geographically
dispersed datacenters.   Generally I'm a proponent of not extending L2
domains over large distances, eventually you'll run into some kind of
limitation and I don't consider it a best practice.

There has been a lot of talk recently in the ARMD group regarding DCI and VM
Mobility in particular.  Google has a draft published regarding using IP
tunneling and a VM to VM Host directory service to extend communication
between two VMs in the same subnet when one may have moved out of the L2
broadcast domain behind a different L3 GW.  Their  solution is host based in
that it relies on the VM host to tunnel traffic as opposed to the upstream
network equipment.    It also relies on the GW at each site to tunnel
traffic via IP to where the VM now resides.

As a L2 solution Ethernet VPN is somewhat promising in it replaces the
flooding mechanisms with local learning and dissemination of MAC information
to PEs using BGP, but does not inherently solve the problem of symmetry.

The NAT solution is one that has been around for awhile.  I once worked on a
system which used WAN accelerator devices located throughout the world which
could perform WAN acceleration/compression between any pair of WAN
accelerators.  The only issue was for the WAN acceleration to work as it
should the traffic had to be symmetric through a given pair of accelerators.
So source NAT was used on each end to maintain the symmetry.  That works
okay for specific applications where the source address range is not very
large but for a large datacenter handling internet traffic that's not
realistic.  

Phil 

From:  Xuxiaohu <[email-masked]>
Date:  Wed, 24 Aug 2011 03:00:01 +0000
To:  Phil Bedard <[email-masked]>, "[email-masked]" <[email-masked]>
Subject:  Re: Active-active DC exits in the DCI solution

Hi Phil,
 
If I understand it correctly, the following approaches you mentioned are
almost proprietary approaches which require DC exit routers in
geographically dispersed locations are provided by a single vendor.
 
I found there was a very simple STANDARD approach to the same goal when host
route based IP-only L2VPN service (e.g., Virtual Subnet
http://tools.ietf.org/html/draft-xu-virtual-subnet-05) is used for DCI. For
example, when incoming traffic from the Internet enters a DC, source IP
addresses of the traffic will be NATed on the DC gateway. Notes that here DC
exit gateways of geographically dispersed DCs are configured with different
IP address pools without any overlapping. In addition, the corresponding
routes for the above address pools are advertised by the DC gateways to
their connected PE routers of the IP-only L2VPN respectively. Thus, when the
outgoing traffic destined to the Internet arrives at its local PE router of
the IP-only L2VPN, that PE router could forward the traffic according to the
matching routes for the above address pools. In this way, active-active DC
exits can be achieved easily in the case where path symmetry is required.
 
Best regards,
Xiaohu  

 

Phil 

Hi all,
 
data center (DC) site in the layer2 DCI solution before. Now I’m considering

 
Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 25 Aug 2011 01:04:04 +0000

> 发送时间: 2011年8月24日 20:40

You can use NAT synchronization/redundancy at each DC site :)  It is easy and practical compared to deploying NAT synchronization/redundancy at geographically dispersed locations.

Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 25 Aug 2011 04:12:19 +0000

发送时间: 2011年8月24日 22:18

Hi Xiaohu,  yes the approaches thus far have been largely proprietary and generally do not perform as well when you are talking about geographically dispersed datacenters.   Generally I'm a proponent of not extending L2 domains over large distances, eventually you'll run into some kind of limitation and I don't consider it a best practice.

There has been a lot of talk recently in the ARMD group regarding DCI and VM Mobility in particular.  Google has a draft published regarding using IP tunneling and a VM to VM Host directory service to extend communication between two VMs in the same subnet when one may have moved out of the L2 broadcast domain behind a different L3 GW.  Their  solution is host based in that it relies on the VM host to tunnel traffic as opposed to the upstream network equipment.    It also relies on the GW at each site to tunnel traffic via IP to where the VM now resides.

XXH: When an inbound packet arrives at the GW, if no matching mapping is found in its local mapping cache, should the packet be dropped or queued before a mapping reply is successfully returned from the mapping directory?

Before exploring any directory-based map&encap approach, I suggest seriously looking through the open issues with that kind of approach which had been identified by LISP WG and RRG.

As a L2 solution Ethernet VPN is somewhat promising in it replaces the flooding mechanisms with local learning and dissemination of MAC information to PEs using BGP, but does not inherently solve the problem of symmetry.

XXH: Your observation is correct.

The NAT solution is one that has been around for awhile.  I once worked on a system which used WAN accelerator devices located throughout the world which could perform WAN acceleration/compression between any pair of WAN accelerators.  The only issue was for the WAN acceleration to work as it should the traffic had to be symmetric through a given pair of accelerators.   So source NAT was used on each end to maintain the symmetry.  That works okay for specific applications where the source address range is not very large but for a large datacenter handling internet traffic that's not realistic.

XXH:  Couldn’t NAPT be used in the case where the address pool is not large enough?

Best regards,
Xiaohu

Phil

From: Xuxiaohu <[email-masked]<mailto:[email-masked]>>
Date: Wed, 24 Aug 2011 03:00:01 +0000
To: Phil Bedard <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: Active-active DC exits in the DCI solution

Hi Phil,

Best regards,
Xiaohu

发件人: Phil Bedard [mailto:[email-masked]]
发送时间: 2011年8月23日 20:37
收件人: Xuxiaohu; [email-masked]<mailto:[email-masked]>
主题: Re: Active-active DC exits in the DCI solution

Phil

Hi all,

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 25 Aug 2011 07:36:15 +0000

Hi Phil,

Could you please explain why source NAT is not realistic for a large datacenter? IMHO, there should be no limitation on the size of the address pool for source NAT, in other words, you could almost use the whole Interne IP address space as the source NAT address pool as long as it has no overlapping with the DC internal address space.

Best regards,
Xiaohu

Phil

Hi Phil,

Best regards,
Xiaohu

Phil

Hi all,

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 25 Aug 2011 08:24:53 +0000

发件人: Crist [mailto:[email-masked]]
发送时间: 2011年8月24日 18:17
收件人: Xuxiaohu
抄送: Phil Bedard; [email-masked]
主题: Re: Active-active DC exits in the DCI solution

Hi Xuxiaohu,

Hi Cristiano,

Is the above constrain on incoming connection rate due to the relatively high reuse frequency of IP addresses in the source NAT address pool? If so, couldn’t the “hot” IP addresses which are just released from terminated sessions be put into a “cooling” stack/pool for a while before being ready for reuse?

Best regards,
Xiaohu

Hi Cristiano

Hi Phil

Regards

Cristiano

Enviado via iPhone

Em 24/08/2011, às 00:00, Xuxiaohu <[email-masked]<mailto:[email-masked]>> escreveu:
Hi Phil,

Best regards,
Xiaohu

Phil

Hi all,

Best regards,
Xiaohu