Subject: [rbridge] network topology constraints in draft-tissa-trill-cmt-00

From: Santosh Rajagopalan <[email-masked]>
Date: Wed, 28 Mar 2012 15:00:55 -0700

This is a clever draft, but I wanted to point out some network topology 
constraints in the proposal:

1) It looks like you can only use this proposal if the CE switches are 
*not* interconnected in any fashion outside of the trill network. This is 
because sending packets into the CE network strips off information needed 
to prevent loops. Let me illustrate using the example from the draft: RB1 
receives a multidestination packet from the trill campus on tree 1, and it 
also has an "affinity" for that tree. So it decaps it and send the packet 
into the CE network (basically, a copy gets sent to each of CE1..CEn). 

Let's assume that the CE network is composed of interconnected switches, 
instead of the isolated switches shown in the picture. This is reasonable, 
because it avoids needing to take the extra hop to the aggregation layer 
for end-systems on the same chassis or rack. This means that a broadcast 
packet would be replicated by the CE network to each of its switches, 
including CE1...CEn. So CE1..CEn just got their first duplicate. Each of 
these switches looks at the attached rbridges as edge ports, so it sends 
them a copy. Now, rbridges RB1..RBk each label the ingressing packet with 
their respective "affinity" tree labels and sends them into the TRILL 
network, where it gets to the edge of the trill network, and the cycle 
repeats. You now have a loop.

In addition, if you had interconnectivity between the CE switches, then 
the edge rbridges would be able to exchange LSPs with each other, which 
will result in one of them being elected the AF. The others will then not 
encap or decap CE packets. So the affinity based approach would conflict 
with  RFC 6325. All in all, we need the CE switches to be isolated here.

2) Applying an STP-based solution like the one described in RFC 6325 ("The 
Spanning Tree Solution") to  break the connectivity between the CE 
switches won't work here, because this will render certain switches 
unreachable on some trees. In figure 13 ("wiring closet topology") in rfc 
6325, if RB1 has an affinity for tree k, then packets coming in from the 
trill cloud on that tree will need to get to B2 through RB1, but since STP 
has blocked B1-B2 this won't happen. This just reiterates that no form of 
interconnect whatsoever between the CE switches is permissible, and "The 
Spanning Tree Solution" will not work here.

3) In addition to the above constraint, each CE switch needs to be 
connected to every rbridge, and the consequence of any of the LAG links 
going down are catastrophic. Also, each rbridge needs to have a vLAG each 
to each CE switch in the LAN. This is necessary because the entire CE 
network has been "emulated" by the pseudo-rbridge (RBv) in the draft. 

Let's say a packet arrives from the trill core at a certain rbridge on a 
tree that it has an affinity for. The assumption is that by decapsulating 
the packet and sending it to each of the attached CE links, all the 
stations in the CE network will get the packet. So if there's a certain CE 
switch which isn't connected to this rbridge, it will not get the packet 
(the packet can't get to the CE switch through another CE switch, because 
of the constraint in 1) above).

This means that a) each rbridge needs to have n vLAGs, one for each CE 
switch, and b) each CE switch needs to have k ports in its LAG, one for 
each rbridge. Note that most switches have scalability constraints on the 
number of LAG members and on the number of VLAGs. For small networks this 
may not be a problem, however, this may still be a problem if one of your 
links on the LAG goes down. In that case, that CE switch will get 
permanently blackholed for some trees. (Essentially, the upstream rbridge 
on the other end of the down link no longer has any way of reaching the CE 
switch on the x trees it has an affinity for)

At the very least, this proposal needs a way for an rbridge to 
"relinquish" its affinity trees when any VLAG link goes down, and a way 
those trees to either be retired or be picked up by other rbridges. In 
addition, the rbridge will need to bring all of its CE-facing links down, 
so that the CE bridges don't try to use that rbridge to inject packets 
into the TRILL network.

4) Because of the constraint imposed by 1), you cannot interconnect two 
trill clouds using an intermediate CE cloud - the trill clouds will need 
to be merged using p2p trill links. This could be a problem if you plan to 
incrementally upgrade your switches to trill, as opposed to a fork-lift 
upgrade of your whole data center.

Note that the existing version of RFC 6325 does not have constraints on 
interconnectivity of CE switches or rbridges as described above.

Thoughts?

--
Sunny Rajagopalan

_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Wed, 28 Mar 2012 16:17:29 -0700

The key application of this draft is for multi-homed devices that are
servicing as end stations, e.g. servers on data centers. Please see
Reference topology 2 and it was discussed in Taipei and stated in the
draft that this solution is applicable to multi-homed active-active
edge.

From: [email-masked] [mailto:[email-masked]] On
Behalf Of Santosh Rajagopalan
Sent: Wednesday, March 28, 2012 3:01 PM
To: [email-masked]
Subject: [rbridge] network topology constraints in
draft-tissa-trill-cmt-00

1) It looks like you can only use this proposal if the CE switches are
*not* interconnected in any fashion outside of the trill network. This
is because sending packets into the CE network strips off information
needed to prevent loops. Let me illustrate using the example from the
draft: RB1 receives a multidestination packet from the trill campus on
tree 1, and it also has an "affinity" for that tree. So it decaps it and
send the packet into the CE network (basically, a copy gets sent to each
of CE1..CEn). 

Let's assume that the CE network is composed of interconnected switches,
instead of the isolated switches shown in the picture. This is
reasonable, because it avoids needing to take the extra hop to the
aggregation layer for end-systems on the same chassis or rack. This
means that a broadcast packet would be replicated by the CE network to
each of its switches, including CE1...CEn. So CE1..CEn just got their
first duplicate. Each of these switches looks at the attached rbridges
as edge ports, so it sends them a copy. Now, rbridges RB1..RBk each
label the ingressing packet with their respective "affinity" tree labels
and sends them into the TRILL network, where it gets to the edge of the
trill network, and the cycle repeats. You now have a loop. 

In addition, if you had interconnectivity between the CE switches, then
the edge rbridges would be able to exchange LSPs with each other, which
will result in one of them being elected the AF. The others will then
not encap or decap CE packets. So the affinity based approach would
conflict with  RFC 6325. All in all, we need the CE switches to be
isolated here. 

2) Applying an STP-based solution like the one described in RFC 6325
("The Spanning Tree Solution
<http://tools.ietf.org/html/rfc6325#appendix-A.3.3> ") to  break the
connectivity between the CE switches won't work here, because this will
render certain switches unreachable on some trees. In figure 13 ("wiring
closet topology") in rfc 6325, if RB1 has an affinity for tree k, then
packets coming in from the trill cloud on that tree will need to get to
B2 through RB1, but since STP has blocked B1-B2 this won't happen. This
just reiterates that no form of interconnect whatsoever between the CE
switches is permissible, and "The Spanning Tree Solution
<http://tools.ietf.org/html/rfc6325#appendix-A.3.3> " will not work
here. 

3) In addition to the above constraint, each CE switch needs to be
connected to every rbridge, and the consequence of any of the LAG links
going down are catastrophic. Also, each rbridge needs to have a vLAG
each to each CE switch in the LAN. This is necessary because the entire
CE network has been "emulated" by the pseudo-rbridge (RBv) in the draft.

Let's say a packet arrives from the trill core at a certain rbridge on a
tree that it has an affinity for. The assumption is that by
decapsulating the packet and sending it to each of the attached CE
links, all the stations in the CE network will get the packet. So if
there's a certain CE switch which isn't connected to this rbridge, it
will not get the packet (the packet can't get to the CE switch through
another CE switch, because of the constraint in 1) above). 

This means that a) each rbridge needs to have n vLAGs, one for each CE
switch, and b) each CE switch needs to have k ports in its LAG, one for
each rbridge. Note that most switches have scalability constraints on
the number of LAG members and on the number of VLAGs. For small networks
this may not be a problem, however, this may still be a problem if one
of your links on the LAG goes down. In that case, that CE switch will
get permanently blackholed for some trees. (Essentially, the upstream
rbridge on the other end of the down link no longer has any way of
reaching the CE switch on the x trees it has an affinity for) 

At the very least, this proposal needs a way for an rbridge to
"relinquish" its affinity trees when any VLAG link goes down, and a way
those trees to either be retired or be picked up by other rbridges. In
addition, the rbridge will need to bring all of its CE-facing links
down, so that the CE bridges don't try to use that rbridge to inject
packets into the TRILL network. 

4) Because of the constraint imposed by 1), you cannot interconnect two
trill clouds using an intermediate CE cloud - the trill clouds will need
to be merged using p2p trill links. This could be a problem if you plan
to incrementally upgrade your switches to trill, as opposed to a
fork-lift upgrade of your whole data center. 

Thoughts? 

-- 
Sunny Rajagopalan

---

From: Santosh Rajagopalan <[email-masked]>
Date: Thu, 29 Mar 2012 10:15:29 -0700

Tissa,
Are you saying that the solution is only applicable to multi-homed hosts 
or that it's also applicable to edge switches? (the draft says the example 
is valid for "either End Stations and/or Legacy bridges")? If you're 
expanding the scope to include the latter, then the inability to 
interconnect them  is a serious deficiency, as are the other points listed 
below. Anyway if the solution if being proposed only for a narrow case, it 
should be mentioned clearly in the draft - it's not enough for it to be 
"discussed in Taipei". The draft talks about a "typical deployment 
scenario" without stating that this is the only possible deployment 
scenario.

Even if you narrow the scope considerably, you still need to address the 
issue of LAG link failure - (see point # 3) below. In this draft, a single 
link failure leads to us bringing down all southbound ports on the 
upstream device, which is quite drastic. This draft needs to address this 
issue for it to be workable.

--
Sunny Rajagopalan

From:   "Tissa Senevirathne (tsenevir)" <[email-masked]>
To:     Santosh Rajagopalan/Santa Clara/IBM@IBMUS, <[email-masked]>
Date:   03/28/2012 04:17 PM
Subject:        RE: [rbridge] network topology constraints in 
draft-tissa-trill-cmt-00

The key application of this draft is for multi-homed devices that are 
servicing as end stations, e.g. servers on data centers. Please see 
Reference topology 2 and it was discussed in Taipei and stated in the 
draft that this solution is applicable to multi-homed active-active edge.

Thoughts?

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Thu, 29 Mar 2012 10:46:15 -0700

Sunny

Please see my other email to the list on the scope of this draft. This
draft is focusing only on RPF issue at the edge of the reference
scenario. How all the other operations are handles such as devices not
fully connected to all RBridges is separate work. 

From: [email-masked] [mailto:[email-masked]] On
Behalf Of Santosh Rajagopalan
Sent: Thursday, March 29, 2012 10:15 AM
To: [email-masked]
Subject: Re: [rbridge] network topology constraints
indraft-tissa-trill-cmt-00

Tissa, 
Are you saying that the solution is only applicable to multi-homed hosts
or that it's also applicable to edge switches? (the draft says the
example is valid for "either End Stations and/or Legacy bridges")? If
you're expanding the scope to include the latter, then the inability to
interconnect them  is a serious deficiency, as are the other points
listed below. Anyway if the solution if being proposed only for a narrow
case, it should be mentioned clearly in the draft - it's not enough for
it to be "discussed in Taipei". The draft talks about a "typical
deployment scenario" without stating that this is the only possible
deployment scenario. 

Even if you narrow the scope considerably, you still need to address the
issue of LAG link failure - (see point # 3) below. In this draft, a
single link failure leads to us bringing down all southbound ports on
the upstream device, which is quite drastic. This draft needs to address
this issue for it to be workable. 

-- 
Sunny Rajagopalan 

From:        "Tissa Senevirathne (tsenevir)" <[email-masked]> 
To:        Santosh Rajagopalan/Santa Clara/IBM@IBMUS,
<[email-masked]> 
Date:        03/28/2012 04:17 PM 
Subject:        RE: [rbridge] network topology constraints in
draft-tissa-trill-cmt-00 

________________________________

From: [email-masked] [mailto:[email-masked]
<mailto:[email-masked]> ] On Behalf Of Santosh Rajagopalan
Sent: Wednesday, March 28, 2012 3:01 PM
To: [email-masked]
Subject: [rbridge] network topology constraints in
draft-tissa-trill-cmt-00 

Thoughts? 

-- 
Sunny Rajagopalan

---

From: Mingui Zhang <[email-masked]>
Date: Thu, 29 Mar 2012 20:37:30 +0000

Hi Sunny,

I agree with your judgement about the LAG link failure.

I have another concern with 5.6.1. Edge RBridge RBk failure. I think that CMT should have the chance to offer an incremental recalculation of distributions trees under nodal failures. In other words, a nodal failure should affect only one distribution tree while others should remain unchanged. But paragraph 3 of this section makes me believe that there is an overall recalculation.

Thanks,
Mingui

________________________________
发件人: [email-masked] [[email-masked]] 代表 Santosh Rajagopalan [[email-masked]]
发送时间: 2012年3月30日 1:15
到: [email-masked]
主题: Re: [rbridge] network topology constraints in draft-tissa-trill-cmt-00

Tissa,
Are you saying that the solution is only applicable to multi-homed hosts or that it's also applicable to edge switches? (the draft says the example is valid for "either End Stations and/or Legacy bridges")? If you're expanding the scope to include the latter, then the inability to interconnect them  is a serious deficiency, as are the other points listed below. Anyway if the solution if being proposed only for a narrow case, it should be mentioned clearly in the draft - it's not enough for it to be "discussed in Taipei". The draft talks about a "typical deployment scenario" without stating that this is the only possible deployment scenario.

Even if you narrow the scope considerably, you still need to address the issue of LAG link failure - (see point # 3) below. In this draft, a single link failure leads to us bringing down all southbound ports on the upstream device, which is quite drastic. This draft needs to address this issue for it to be workable.

--
Sunny Rajagopalan

Subject:        RE: [rbridge] network topology constraints in draft-tissa-trill-cmt-00

The key application of this draft is for multi-homed devices that are servicing as end stations, e.g. servers on data centers. Please see Reference topology 2 and it was discussed in Taipei and stated in the draft that this solution is applicable to multi-homed active-active edge.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Santosh Rajagopalan
Sent: Wednesday, March 28, 2012 3:01 PM
To: [email-masked]
Subject: [rbridge] network topology constraints in draft-tissa-trill-cmt-00

This is a clever draft, but I wanted to point out some network topology constraints in the proposal:

1) It looks like you can only use this proposal if the CE switches are *not* interconnected in any fashion outside of the trill network. This is because sending packets into the CE network strips off information needed to prevent loops. Let me illustrate using the example from the draft: RB1 receives a multidestination packet from the trill campus on tree 1, and it also has an "affinity" for that tree. So it decaps it and send the packet into the CE network (basically, a copy gets sent to each of CE1..CEn).

Let's assume that the CE network is composed of interconnected switches, instead of the isolated switches shown in the picture. This is reasonable, because it avoids needing to take the extra hop to the aggregation layer for end-systems on the same chassis or rack. This means that a broadcast packet would be replicated by the CE network to each of its switches, including CE1...CEn. So CE1..CEn just got their first duplicate. Each of these switches looks at the attached rbridges as edge ports, so it sends them a copy. Now, rbridges RB1..RBk each label the ingressing packet with their respective "affinity" tree labels and sends them into the TRILL network, where it gets to the edge of the trill network, and the cycle repeats. You now have a loop.

In addition, if you had interconnectivity between the CE switches, then the edge rbridges would be able to exchange LSPs with each other, which will result in one of them being elected the AF. The others will then not encap or decap CE packets. So the affinity based approach would conflict with  RFC 6325. All in all, we need the CE switches to be isolated here.

2) Applying an STP-based solution like the one described in RFC 6325 ("The Spanning Tree Solution<http://tools.ietf.org/html/rfc6325#appendix-A.3.3>") to  break the connectivity between the CE switches won't work here, because this will render certain switches unreachable on some trees. In figure 13 ("wiring closet topology") in rfc 6325, if RB1 has an affinity for tree k, then packets coming in from the trill cloud on that tree will need to get to B2 through RB1, but since STP has blocked B1-B2 this won't happen. This just reiterates that no form of interconnect whatsoever between the CE switches is permissible, and "The Spanning Tree Solution<http://tools.ietf.org/html/rfc6325#appendix-A.3.3>" will not work here.

3) In addition to the above constraint, each CE switch needs to be connected to every rbridge, and the consequence of any of the LAG links going down are catastrophic. Also, each rbridge needs to have a vLAG each to each CE switch in the LAN. This is necessary because the entire CE network has been "emulated" by the pseudo-rbridge (RBv) in the draft.

Let's say a packet arrives from the trill core at a certain rbridge on a tree that it has an affinity for. The assumption is that by decapsulating the packet and sending it to each of the attached CE links, all the stations in the CE network will get the packet. So if there's a certain CE switch which isn't connected to this rbridge, it will not get the packet (the packet can't get to the CE switch through another CE switch, because of the constraint in 1) above).

This means that a) each rbridge needs to have n vLAGs, one for each CE switch, and b) each CE switch needs to have k ports in its LAG, one for each rbridge. Note that most switches have scalability constraints on the number of LAG members and on the number of VLAGs. For small networks this may not be a problem, however, this may still be a problem if one of your links on the LAG goes down. In that case, that CE switch will get permanently blackholed for some trees. (Essentially, the upstream rbridge on the other end of the down link no longer has any way of reaching the CE switch on the x trees it has an affinity for)

At the very least, this proposal needs a way for an rbridge to "relinquish" its affinity trees when any VLAG link goes down, and a way those trees to either be retired or be picked up by other rbridges. In addition, the rbridge will need to bring all of its CE-facing links down, so that the CE bridges don't try to use that rbridge to inject packets into the TRILL network.

4) Because of the constraint imposed by 1), you cannot interconnect two trill clouds using an intermediate CE cloud - the trill clouds will need to be merged using p2p trill links. This could be a problem if you plan to incrementally upgrade your switches to trill, as opposed to a fork-lift upgrade of your whole data center.

Note that the existing version of RFC 6325 does not have constraints on interconnectivity of CE switches or rbridges as described above.

Thoughts?

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Thu, 29 Mar 2012 13:57:54 -0700

When an RBrdige goes down, based on different scenarios today all of the RBridges in the network recalculate the SPF.   We are not optimizing IS-IS protocol calculation methods and there can be many different way to optimize such things, that is not the intention of a base draft.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mingui Zhang
Sent: Thursday, March 29, 2012 1:38 PM
To: Santosh Rajagopalan; [email-masked]
Subject: 答复: [rbridge] network topology constraints in draft-tissa-trill-cmt-00

Hi Sunny, 

Thanks,

Mingui

Tissa, 

-- 
Sunny Rajagopalan 

  
From: [email-masked] [mailto:[email-masked] <mailto:[email-masked]> ] On Behalf Of Santosh Rajagopalan
Sent: Wednesday, March 28, 2012 3:01 PM
To: [email-masked]
Subject: [rbridge] network topology constraints in draft-tissa-trill-cmt-00 

2) Applying an STP-based solution like the one described in RFC 6325 ("The Spanning Tree Solution <http://tools.ietf.org/html/rfc6325#appendix-A.3.3> ") to  break the connectivity between the CE switches won't work here, because this will render certain switches unreachable on some trees. In figure 13 ("wiring closet topology") in rfc 6325, if RB1 has an affinity for tree k, then packets coming in from the trill cloud on that tree will need to get to B2 through RB1, but since STP has blocked B1-B2 this won't happen. This just reiterates that no form of interconnect whatsoever between the CE switches is permissible, and "The Spanning Tree Solution <http://tools.ietf.org/html/rfc6325#appendix-A.3.3> " will not work here. 

Thoughts? 

-- 
Sunny Rajagopalan