Subject: IS-IS based VPLS for cloud data center networks

From: Xu Xiaohu <[email-masked]>
Date: Fri, 07 Jan 2011 17:26:18 +0800

Hi all,

Cloud data centers require large L2 network which should not only be able to
support shortest path forwarding and ECMP for maximizing the bandwidth
capacity between servers, but also be able to provide a huge amount of VPN
instances for isolating hundreds of thousands of tenement within the data
centers. VPLS seems as a good option for this requirement. However, BGP and
LDP based VPLS solutions seem a bit heavy-weight and hence not much
acceptable for some cloud data center providers.

So I wonder whether it is reasonable to develop IS-IS based VPLS for those
special cloud data center providers. For example, IS-IS is run between P and
PE routers, and an extended IS-IS TLV is used by PE routers to distribute
their own VPN membership information (e.g., attached VPN instances and
associated VPN labels) among each other. The VPN label will be carried in
the inner MPLS header of L2VPN data packets (e.g., MAC-in-MPLS-in-IP). Note
that the VPN label here now is just used to identify a particular VPN
instance, but not to identify both a particular VPN instance and a
particular ingress PE router as the PW label does. The egress PE router now
could use the source IP address of the outer IP tunnel header in the
received L2VPN data packet to identify the ingress PE router.

Any comment?

Best wishes,
Xiaohu

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 7 Jan 2011 12:53:04 -0800

Hi Xu,

If I understand you right, you are talking about a way to use changes
to IS-IS to do Inter Data Center address exchange.

There is something called Overlay Transport Virtualization (OTV),
which seems to do what you are suggesting. That said I know there were
attempts to send addresses instead of doing L2 learning for VPLS,
quite a while back.

Thanks,
Vishwas

On Fri, Jan 7, 2011 at 1:26 AM, Xu Xiaohu <[email-masked]> wrote:
> _______________________________________________
> Isis-wg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/isis-wg
>

---

From: Tony Li <[email-masked]>
Date: Fri, 7 Jan 2011 14:09:04 -0800

Hi Xiaohu,

> acceptable for some cloud data center providers.

It seems like moving the complexity from BGP or LDP into IS-IS isn't going to make it go away.  In fact, if anything, it would seem like this makes it even more complicated.

Isn't this a job for TRILL and Q-in-Q?

Regards,
Tony

---

From: Phil Bedard <[email-masked]>
Date: Fri, 07 Jan 2011 20:12:51 -0500

If one wishes to stay at L2, 802.1aq uses the PBB/802.1ah I-SID value to
differentiate instances, as opposed to an MPLS label, and distributes this
information via IS-IS.  The B-MAC of the source bridge is similar to an IP
address in abstracting all service instances behind a single identifier,
so the concept is similar.  TRILL may provide the same benefits.

If one is using VPLS (L3), I'm not going to say the idea has no merit
because running a single protocol on the network may be attractive to some
providers as opposed to running an IGP plus BGP/LDP.  Using IP
encapsulation instead of MPLS has been implemented by some vendors, but
with an outer encapsulation header like GRE or L2TPv3.   The downside to
using the IGP to distribute VPN information is the P core is exposed to
all of the VPN information unnecessarily.  Also there are some VPLS drafts
which use BGP to distribute MAC information as well, which may be
beneficial as opposed to using L2 learning methods.

-Phil

---

From: xuxiaohu 41208 <[email-masked]>
Date: Sat, 08 Jan 2011 17:35:31 +0800

----- 原邮件 -----
发件人: Tony Li <[email-masked]>
日期: 星期六, 一月 8日, 2011 上午6:09
主题: Re: [Isis-wg] IS-IS based VPLS for cloud data center networks
收件人: Xu Xiaohu <[email-masked]>
抄送: [email-masked], [email-masked]

> > support shortest path forwarding and ECMP for maximizing the 
> bandwidth> capacity between servers, but also be able to provide a 
> huge amount of VPN
> It seems like moving the complexity from BGP or LDP into IS-IS 
> isn't going to make it go away.  In fact, if anything, it would 
> seem like this makes it even more complicated.
> 

Hi Tony,

TRILL and SPB not only invent brand-new mechanisms for shortest path forwarding between PE routers (i.e., Rbridges and SPB device), but also specify VPN membership discovery mechanism. In contrast, IS-IS based VPLS is just aimed to provide a light-weight option of VPLS. it only needs an extended ISIS TLV for PE routers to advertise their own VPLS membership. There is no need to introduce totally new packet encapsulation formats and packet f
orwarding mechanism on the data plane or complex ECMP routing computation on the control plane.

Best regards,
Xiaohu
 
> define

---

From: Paul Unbehagen <[email-masked]>
Date: Sun, 9 Jan 2011 08:44:55 -1000

Hi Xiaohu,

To be clear, since SPB is a IEEE protocol it doesn't invent new encapsulations, it simply takes control of forwarding the native Ethernet encaps (PBB & Q) with IS-IS. The ISID is used like a VPN ID to know where endpoints are in the network and pre-populate the fibs. The same FIBs that would be created in a traditional Ethernet environment, but without and reverse learning or flooding needed. This also allows years of deployed Ethernet OAM to just work without need for modification.

Unless I've misunderstood your intention, with both PBB/VPLS and SPB you can solve everything you are mentioning without inventing something new to do the same thing.  PE's can do what they've always done and DC switches can be better at what they've always done and the two can work natively together to connect DC's together from TOR to TOR.

--
Paul Unbehagen

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 10 Jan 2011 09:52:37 +0800

Hi Phil,

> -----邮件原件-----
> 发件人: Phil Bedard [mailto:[email-masked]]
> 发送时间: 2011年1月8日 9:13
> 收件人: Xu Xiaohu; [email-masked]
> 抄送: [email-masked]
> 主题: Re: IS-IS based VPLS for cloud data center networks
> 

Agree. However, for shortest path routing/forwarding, neither of the above
two options is more mature and more familiar to operators than IP.

> providers as opposed to running an IGP plus BGP/LDP.  Using IP

Agree. That's the reason why I proposed IS-IS based VPLS :)

> with an outer encapsulation header like GRE or L2TPv3.   The downside to

Fully agree.

> all of the VPN information unnecessarily.  Also there are some VPLS drafts

Agree. IMHO, this should be an affordable cost of achieving the
configuration simplicity. In addition, the extended IS-IS TLV for VPN
membership could be partially transparent to P routers since the P routers
don't need to process VPN reachability information contained in that TLV. 

By the way, in TRILL and SPB (802.1aq) cases, the VPN membership information
is also exposed to core devices.

> beneficial as opposed to using L2 learning methods.

If the CE MAC reachability information is desired to be distributed across
PE routers on the control plane, such information should be fully
transparent to P routers as if the packets containing that information are
normal data packets.

Best wishes,
Xiaohu

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 10 Jan 2011 10:19:27 +0800

Hi Paul,

> -----邮件原件-----
> 发件人: Paul Unbehagen [mailto:[email-masked]]
> 发送时间: 2011年1月10日 2:45
> 收件人: xuxiaohu 41208
> 抄送: Tony Li; [email-masked]; [email-masked]
> To be clear, since SPB is a IEEE protocol it doesn't invent new
encapsulations,
> it simply takes control of forwarding the native Ethernet encaps (PBB & Q)
with
> IS-IS. The ISID is used like a VPN ID to know where endpoints are in the
network
> and pre-populate the fibs. The same FIBs that would be created in a
traditional
> Ethernet environment, but without and reverse learning or flooding needed.
This
> also allows years of deployed Ethernet OAM to just work without need for
> modification.

Yes, I agree that the data plane of SPB is good. However, the control plane
is a bit complex (e.g., the TE-alike ECMP function).

> Unless I've misunderstood your intention, with both PBB/VPLS and SPB you
can
> solve everything you are mentioning without inventing something new to do
the
> same thing.  

Yes, IS-IS based VPLS aims to perform the same function, but it is a
light-weight option.

Best wishes,
Xiaohu

> PE's can do what they've always done and DC switches can be better
> at what they've always done and the two can work natively together to
connect
> DC's together from TOR to TOR.
> 
> --
> Paul Unbehagen
> 
> 
> > TRILL and SPB not only invent brand-new mechanisms for shortest path
> forwarding between PE routers (i.e., Rbridges and SPB device), but also
specify
> VPN membership discovery mechanism. In contrast, IS-IS based VPLS is just
aimed
> to provide a light-weight option of VPLS. it only needs an extended ISIS
TLV
> for PE routers to advertise their own VPLS membership. There is no need to
> introduce totally new packet encapsulation formats and packet f
> > orwarding mechanism on the data plane or complex ECMP routing
computation
> on the control plane.

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 10 Jan 2011 10:30:13 +0800

> -----邮件原件-----
> 发件人: Tony Li [mailto:[email-masked]]
> 发送时间: 2011年1月8日 6:09
> 收件人: Xu Xiaohu
> 抄送: [email-masked]; [email-masked]
> > Cloud data centers require large L2 network which should not only be
able
> to
> It seems like moving the complexity from BGP or LDP into IS-IS isn't going
to
> make it go away.  In fact, if anything, it would seem like this makes it
even
> more complicated.

Hi Tony,

Did you mean that TRILL should scale its VPN instance amount by using
Q-in-Q?

Best wishes,
Xiaohu

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 10 Jan 2011 11:33:26 +0800

> transparent to P routers as if the packets containing that information are

I should rephrase the above as "...as if the packets containing that
information were normal ENCAPSULATED CE data packets, because the CE MAC
reachability information of a given VPN should only be received by the PE
routers attached to that VPN."

Best wishes,
Xiaohu

able
> > >special cloud data center providers. For example, IS-IS is run between
P
> > >and

---

From: Vishwas Manral <[email-masked]>
Date: Sun, 9 Jan 2011 19:09:58 -0800

Hi Xaiohu,

I reread the thread.

The first difference you need to consider is that IS-IS is a link
state protocol, which means that every device in the network will have
the same information. I am unsure how it will be easier to move
complexity to IS-IS, than have it on a non-link state protocol. The
information exchange required for VPN is not symmetric.

Using it we will be exchanging information about all other PE devices
to all other PE devices, which may not be required.

With the model you are talking about there are a few other things you
need to consider:

1. In case of Virtualized servers there will be multiple virtual
servers within a physical server, each of which belong to a separate
VPN.
2. Most of the virtual switches are Layer-2 capable, not MPLS or PW capable.
3. I think extensions like attaching a Route Distinguisher (RD), will
be required to IS-IS. So a lot of extensions will be required for the
same.

That said like I mentioned there is work on ways to exchange MAC
Address over IS-IS called OTV, which you can have a look at.

Thanks,
Vishwas

---

From: Vishwas Manral <[email-masked]>
Date: Sun, 9 Jan 2011 19:13:47 -0800

Hi,

To add the extensions like RD etc are required for L3VPN and not L2
VPN's, so would not apply to the VPLS case.

Thanks,
Vishwas

---

From: Peter Ashwood-Smith <[email-masked]>
Date: Mon, 10 Jan 2011 15:08:58 -0500

> The downside to
> all of the VPN information unnecessarily.

In the context of 802.1aq/SPB the 'P' core can instantiate true multicast
state through simple computation .. so its not 'unnecessary' ;) and in the
case where you don't want true multicast and can live with head end
replication its just silently ignored at fairly low cost but its not
intended for the kinds of scale/heavy weight deployments as VPLS and for the
small to medium sized network, i.e. < 500 or so nodes it works rather well.

You really need to compare 802.1aq SPB to VPLS + all the mp 2 mp signalling
protocols to see the proper trade-offs because all that signalling adds a
lot of overhead that is completely absent in .1aq.

Anyway if I understand the proposal correctly it takes IS-IS and advertises
a network wide MPLS label for a particular L2 VPN as opposed to source / VPN
specific as VPLS would do. I've seen this proposed a few times over the
years in L2VPN but it always got trounced because of the global nature of
the label.

Peter

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Phil Bedard
Sent: Friday, January 07, 2011 8:13 PM
To: Xu Xiaohu; [email-masked]
Cc: [email-masked]
Subject: Re: [Isis-wg] IS-IS based VPLS for cloud data center networks

-Phil

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 11 Jan 2011 09:02:43 +0800

Hi Peter,

> -----邮件原件-----
> 发件人: Peter Ashwood-Smith [mailto:[email-masked]]
> 发送时间: 2011年1月11日 4:09
> 收件人: 'Phil Bedard'; 'Xu Xiaohu'; [email-masked]
> 抄送: [email-masked]
> 主题: RE: [Isis-wg] IS-IS based VPLS for cloud data center networks
> 
> 
> > The downside to

> years in L2VPN but it always got trounced because of the global nature of

The MPLS VPLS label I proposed to identify different VPLS instances in IS-IS
based VPLS doesn't need to be globally unique. It is only required that a PE
router should assign the same label for a given VPLS instance to all other
PE routers. That's to say, this label is now a VPN label, rather than a PW
label.

The following is a demonstration of the IS-IS TLV for VPLS:

+-----------+
|Type=VPLS  | (8 bits)
+-----------+
|Length     | (8 bits)
+-----------+-----------------------------------------+
|    Sub_TLV for PE's IPv4 or IPv6 Address            |
+--------------------------------+--------------------+
|    VPLS_ID (20 bits)           |
+--------------------------------+
|    MPLS Label (20 bits)        |
+--------------------------------+
|                                |
|     ......                     |
+--------------------------------+
|    VPLS_ID (20 bits)           |
+--------------------------------+
|    MPLS Label (20 bits)        |
+--------------------------------+

Best wishes,
Xiaohu

---

From: "Shahram Davari" <[email-masked]>
Date: Mon, 10 Jan 2011 17:32:12 -0800

Gi Xu,

There are other ways of doing this such as using GRE tunnels. Also in data-center the network is centrally managed, meaning that a global VPN identifier can be used as opposed to VPLS.

Thx
Shahram

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Xu Xiaohu
Sent: Friday, January 07, 2011 1:26 AM
To: [email-masked]
Cc: [email-masked]
Subject: IS-IS based VPLS for cloud data center networks

Hi all,

Any comment?

Best wishes,
Xiaohu

---

From: Kris Price <[email-masked]>
Date: Tue, 11 Jan 2011 15:18:33 +1300

> acceptable for some cloud data center providers.

I agree there could be value in a simplified VPLS for use in data 
centres and campuses. Some might claim TRILL is candidate for these 
applications, but TRILL was not scoped for these kinds of data centres 
or large campuses, and so made decisions that affected its suitability 
for it. There was an early TRILL proposal to use MPLS forwarding 
(http://tools.ietf.org/html/draft-bryant-perlman-trill-pwe-encap-00).

> received L2VPN data packet to identify the ingress PE router.

Having to tunnel MPLS in IP seems a drawback to me, or is that intended 
so as to do away with the label signalling in the control plane? I'd be 
quite interested in a way to this without the obligatory IP layer.

In that case certainly IS-IS could be used for the auto-discovery 
mechanism, but the signalling and data-plane learning become tricky. If 
recievers simply send their VPN labels in IS-IS nodes can send packets, 
but the recievers can't learn the source without that outer tunnel IP 
header. Perhaps the mechanism used for BGP based signalling could be 
adapted? Or perhaps each node globally allocates themselves a label (in 
the vein of TRILLs nickname algorithm) then distribute this in IS-IS, 
and use it as the bottom label to identify themselves as the source?

Cheers
Kris

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 11 Jan 2011 11:44:26 +0800

Hi Kris,

> -----邮件原件-----
> 发件人: Kris Price [mailto:[email-masked]]
> 发送时间: 2011年1月11日 10:19
able
> so as to do away with the label signalling in the control plane? I'd be

Yes. The PE-PE tunnel is IP based tunnel, rather than MPLS LSP tunnel.

> quite interested in a way to this without the obligatory IP layer.

What is the preferred encapsulation method for the PE-PE tunnel in your
mind?

> but the recievers can't learn the source without that outer tunnel IP

IMHO, IP is the most simple, most mature and most familiar technology for
the shortest path routing/forwarding. Hence there is such problem as you
mentioned once IP tunnel is used between PE routers of a data center
network.

> and use it as the bottom label to identify themselves as the source?

It works. However, I guess the globally unique MPLS label is not much
acceptable to most people since it requires additional cost on the
management plane or the control plane.

Best wishes,
Xiaohu

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 11 Jan 2011 12:10:49 +0800

Hi Shahram,

> -----邮件原件-----
> 发件人: Shahram Davari [mailto:[email-masked]]
> 发送时间: 2011年1月11日 9:32
> 主题: RE: IS-IS based VPLS for cloud data center networks
> 
> Gi Xu,
> 
> There are other ways of doing this such as using GRE tunnels. Also in
data-center
> the network is centrally managed, meaning that a global VPN identifier can
be
> used as opposed to VPLS.

The following ISIS TLV for VPLS (demo) provides you enough flexibility.
That's to say, for a given VPLS instance, PE routers could either assign it
a globally identical label or not. The former option is only useful in the
scenario where a single multicast distribution tree in the data center
network backbone is shared by more than one VPLS instance. Such a tree is
referred to as an "Aggregate Inclusive Tree" (see
draft-ietf-l3vpn-2547bis-mcast). For unicast and ingress-replication
multicast cases, there is no need to use the former option. To achieve the
former option, the MPLS label for a given VPLS instance equals the VPLS ID
of that VPN instance.

+-----------+
|Type=VPLS  | (8 bits)
+-----------+
|Length     | (8 bits)

Best wishes,
Xiaohu

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Xu

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 11 Jan 2011 12:15:48 +0800

> -----邮件原件-----
> 发件人: [email-masked] [mailto:[email-masked]] 代表 Xu
> Xiaohu
> 发送时间: 2011年1月11日 11:44
> 收件人: 'Kris Price'; [email-masked]
> 主题: re: IS-IS based VPLS for cloud data center networks
> 
> Hi Kris,
> 
> > -----邮件原件-----
> > > centers. VPLS seems as a good option for this requirement. However,
BGP
P
> > > particular ingress PE router as the PW label does. The egress PE
router
> now

Sorry, I wanted to say: ...Hence there would be no such problem as you
mentioned once the IP based tunnels were used between PE routers of a data
center network.

Best wishes,
Xiaohu

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 11 Jan 2011 14:46:33 +0800

> 发送时间: 2011年1月11日 12:11
> 收件人: 'Shahram Davari'; [email-masked]
> 抄送: [email-masked]

By the way, the former option is not necessary either in the case where a
single multicast distribution tree in the data center network backbone is
exclusively used by one VPLS instance, because the multicast group address
itself in the IP tunnel header could identify a particular VPLS instance. In
this case, the customer multicast/broadcast or unknown unicast MAC frame
would be encapsulated directly with an IP tunnel header (e.g., MAC-in-IP).

Best wishes,
Xiaohu

able

---

From: Peter Ashwood-Smith <[email-masked]>
Date: Tue, 11 Jan 2011 02:08:43 -0500

Yes, sorry I meant to say network wide unique. 

Peter

-----Original Message-----
From: Xu Xiaohu [mailto:[email-masked]] 
Sent: Monday, January 10, 2011 8:03 PM
To: 'Peter Ashwood-Smith'; 'Phil Bedard'; [email-masked]
Cc: [email-masked]
Subject: re: [Isis-wg] IS-IS based VPLS for cloud data center networks

Hi Peter,

+-----------+
|Type=VPLS  | (8 bits)
+-----------+
|Length     | (8 bits)

Best wishes,
Xiaohu