Subject: [trill] TRILL Active-Active Edge Using Multiple MAC Attachments

From: Mingui Zhang <[email-masked]>
Date: Fri, 4 Apr 2014 08:58:33 +0000

Hi,

As promised, we bring the detail design of TRILL Active-Active Edge Using Multiple MAC Attachments.
The major updates from 01 to 02 are on how to realize split-horizon and how to achieve hardware independency.
Several TLVs are defined to make it deployable.

Welcome to your comments.

Thanks,
Mingui

>-----Original Message-----
>From: I-D-Announce [mailto:[email-masked]] On Behalf Of
>[email-masked]
>Sent: Friday, April 04, 2014 4:19 PM
>To: [email-masked]
>Subject: I-D Action: draft-zhang-trill-aa-multi-attach-02.txt
>
>
>A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>
>        Title           : TRILL Active-Active Edge Using Multiple MAC
>Attachments
>        Authors         : Mingui Zhang
>                          Radia Perlman
>                          Hongjun Zhai
>                          Mukhtiar Shaikh
>                          Muhammad Durrani
>	Filename        : draft-zhang-trill-aa-multi-attach-02.txt
>	Pages           : 16
>	Date            : 2014-04-04
>
>Abstract:
>   TRILL active-active service is to provide end stations with flow
>   level load balance and resilience against link failures at the edge
>   of TRILL campuses.
>
>   This draft proposes that member RBridges in an active-active edge
>   RBridge group use their own nicknames as ingress RBridge nicknames to
>   encapsulate frames from attached end systems. Thus, remote edge
>   RBridges are required to learn multiple locations of one MAC address
>   in one Data Label. Design goals of this proposal are discussed in the
>   document.
>
>
>The IETF datatracker status page for this draft is:
>https://datatracker.ietf.org/doc/draft-zhang-trill-aa-multi-attach/
>
>There's also a htmlized version available at:
>http://tools.ietf.org/html/draft-zhang-trill-aa-multi-attach-02
>
>A diff from the previous version is available at:
>http://www.ietf.org/rfcdiff?url2=draft-zhang-trill-aa-multi-attach-02
>
>
>Please note that it may take a couple of minutes from the time of submission
>until the htmlized version and diff are available at tools.ietf.org.
>
>Internet-Drafts are also available by anonymous FTP at:
>ftp://ftp.ietf.org/internet-drafts/
>
>_______________________________________________
>I-D-Announce mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/i-d-announce
>Internet-Draft directories: http://www.ietf.org/shadow.html
>or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 8 Apr 2014 03:37:58 +0000

Hi Sujay,

Thanks for your comments! My reply are inserted inline below.

>From: Sujay Gupta [mailto:[email-masked]]
>Sent: Monday, April 07, 2014 6:35 PM
>To: Mingui Zhang
>Subject: Re: [trill] TRILL Active-Active Edge Using Multiple MAC Attachments
>
>Hi Zhang,
>
>Indeed a good idea on solving the problem.
>But we have additional overhead of esadi to work quickly
>and send the information.

Two points here: additional and quickly

When a MAC address is learned from the mc lag side for the first time, it need be
advertised in ESADI-LSP. After that, this MAC need not be advertised for a certain time
until it is to be updated or forgotten. Compared to the ESADI under the non Active/Active 
scenario, I think the proposal does not introduce "additional" overhead.

Yes, the Active/Active edge RBridges have to send ESADI-LSP "quickly" in order to reduce 
unknown unicast. But they need to do so only when the MAC is learned for the first time. 

>Further until Esadi has advertised the mac bindings, the unknown
>mac will take a d-tree to ingress (from the mc lag side).
>Here you would have multiple entry points as all the ingress
>nodes (rbridges) are AF's for all the vlans,
>How do we stop multiple duplicate packets from ingressing?

The duplication from the ingressing won't happen since the MC-LAG always takes
only one uplink to send a frame. So only one member RBridge will get the frame 
even it is multicast or broadcast. Afterwards, the receiver RBridge will encapsulate 
this frame as a multicast packet. Then it will be sent out along the d-tree.

>
>The other concern I have is wrt load balancing, as you would
>know traditional Lag based hashing is done on the basis of
>multiple fields like mac da,sa, ip etc. 

Yes, correct.

>All this is done in hardware
>now with your solution we move this responsibility to the ingress RB
>whereas in reality it better to do it closest to the exit point, such
>that localized failures are immediately detected and the correct
>output interface is chosen. 

The proposal was supposed not to change anything of MC-LAG itself. It's possible
that traffic from distant D to S in some endnode clump attached to the AAE takes 
RB3 as the egress (See Section 5.3.1) while traffic from S to D takes RB1 as the ingress. 
This is the scenario you assumed that local failure recovery does not work well? 
But this is a issue related to hashing function of mc lag, I believe it should have
been addressed.

>Also for Trill ingressing, in the ingress
>RB traditional L2 lookup does not do load balancing based on
>flows like mac and ip field etc.
>
>What is your idea?

I suppose the Trill ingressing you mentioned here happens at the remote RBridge. 
As I know, this issue has been discussed in Section 5.3 of [ESADI]
( http://tools.ietf.org/html/draft-ietf-trill-esadi-06#section-5.3). A hashing function
for load spreading is documented there. 

Thanks,
Mingui

>
>Regds,
>-Sujay
>
>Sent from my iPad2
>
>> On Apr 4, 2014, at 2:28 PM, Mingui Zhang <[email-masked]>
>wrote:
>>
>> Hi,
>>
>> trill mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/trill

---

From: Sujay Gupta <[email-masked]>
Date: Tue, 08 Apr 2014 12:40:17 +0530

Hi Zhang,

My responses inline;

Regds,
-Sujay

> unknown unicast. But they need to do so only when the MAC is learned for the first time.
Sujay>> ESADI is definitely an additional mandatory support required to 
go with this solution.
Secondly, the solution suffers with any possible overheads and drawbacks 
of ESADI, while agreeing that
your solution does not add much additional overhead over ESADI.

> this frame as a multicast packet. Then it will be sent out along the d-tree.
Sujay>> Ok
> But this is a issue related to hashing function of mc lag, I believe it should have
Sujay>> As an engineering solution wise we are shifting the load 
balancing from the egress nodes to the ingress nodes.
This using ESADI as a mechanism, which if gives troubles in an actual 
deployment ( convergence etc.) my load balancing
is not achieved, optionally causing situations where there is 
unnecessary flooding at the MLAG end, if the information is
not updated via ESADI to the ingress end.
All this can be avoided by the MLAG ( most proprietary implementations 
will do it), locally.
Secondly your ingress node L2 forwarding table size increases, which is 
not really required as the MLAG nodes at the tail
end have been designed and equipped to do so.
I could potentially devise a similar solution having another tunneling 
mechanism(over TRILL, or not use TRILL as the transport
mechanism) to egress nodes which are bunched as MLAG's and then 
load-balance between them in the ingress node.
> for load spreading is documented there.
Sujay>> Yes, while it is documented, the available non-content aware 
forwarding logic's do not look beyond
mac and vlan for L2 forwarding. Which leads to the question, can it be 
deployed at a wide-scale?

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 8 Apr 2014 09:20:04 +0000

Hi Sujay,

My responses inline below. Thanks!

>Sujay>> ESADI is definitely an additional mandatory support required to

MINGUI>Yes, option C requires that ESADI is mandatorily supported by Active/Active RBridges.

>load-balance between them in the ingress node.

MINGUI>For the short lived situation when ESADI LSDB are not synchronized, remote RBridge (say RB4)
may send out unknown unicast instead of known unicast to the AAE RBridges. It seems this
situation also exists before we introduce the proposal.

MINGUI> Although it's possible to devise a mechanism to shift the load spreading job back 
to the egress RBridges, this mechanism requires new silicon since it uses tunneling. So, I don't 
think we should reinvent or shift the load-balancing mechanism of MC-LAG into TRILL. But I keep
an open mind on this point.

MINGUI> However, when we talks about load balancing, we are here actually talking about load balancing
of TRILL unicast. For Option C, the remote ingress RBridge (say RB4) always takes a single RBridges out of the
AAE (say RB3) as the egress. After this egress RBridge decapsulates this unicast packet, the load-balancing 
mechanism of MC-LAG will be followed.

>deployed at a wide-scale?

MINGUI> I don't think it's proper to have a TRILL switch to read beyond the TRILL header. For the documented 
hashing algorithm itself, I think it can be adopted in situations not limited to L2 forwarding. 

Cheers,
Mingui

---

From: Sujay Gupta <[email-masked]>
Date: Tue, 8 Apr 2014 16:55:54 +0530

Thanks Mingui for your responses.

A few thoughts inline.

Regds,
-Sujay

Sent from my iPad2

> On Apr 8, 2014, at 2:50 PM, Mingui Zhang <[email-masked]> wrote:
> 
> Hi Sujay,
> 
Sujay- As you would know,fundamentally MCLAG load balancing is not dependent upon whether the node knows the macda ip etc. The moment it ingresses a packet it hashes and distributes. Now with our solution the traffic balances happen post learning the macda, so our solution is not making
a better job in solving the problem.
> mechanism of MC-LAG will be followed.
Sujay- going with this logic it is best to always advertise(via esadi) a one macda to oneRB,
mapping, as the load balancing would be done post decapsulation by the mclag logic.And 
stop all the ingress RBs from becominf AF's, which gives me a solution much like CMT. So 
we really dont want the CMT solution here, for which you have share load at the ingress alone.
Which brings us back to the basic issues I discussed about.

> hashing algorithm itself, I think it can be adopted in situations not limited to L2 forwarding. 
Sujay- I would not totally agree that the hashing can be tuned in L2 forwarding, IMO the fundamental ingress trill RB L2 logic don't seem to have a flow based hashing logic, which
is of course subject to change.

---

From: Mingui Zhang <[email-masked]>
Date: Wed, 9 Apr 2014 03:20:41 +0000

Hi Sujay,

Thanks for your further comments!

>Sujay- As you would know, fundamentally MCLAG load balancing is not
>dependent upon whether the node knows the macda ip etc. The moment it
>ingresses a packet it hashes and distributes. Now with our solution the traffic
>balances happen post learning the macda, so our solution is not making

MINGUI> I agree. It's not supposed to do so.

MINGUI>Let me share my understanding of load-balancing in MC-LAG. There is a link (physical or logical) 
interconnecting any two nodes in MC-LAG. When the port identified by the output of the hashing function 
is not on the receiver node, the frame will be redirected to the target node via the interconnecting link. In 
this way, the load balancing mechanism takes effect. My point is that the MC-LAG doesn't care about what 
kind of load-balancing has been adopted in the network, it always behave according to its local knowledge. 

MINGUI>While the load-spreading function of TRILL had already been filed in RFC6325 and ESADI. This 
function at the remote TRILL ingress is not designed to replace the existing MC-LAG load balancing. It is 
used to improve the load distribution on TRILL links. In a word, load balancing on the MC-LAG and load 
spreading at the ingress RBridge are supposed to take effect simultaneously.

>mapping, as the load balancing would be done post decapsulation by the mclag logic. And
>we really don't want the CMT solution here, for which you have share load at the ingress alone.

MINGUI> I don't want to introduce any configuration either. So the proposal is designed to let the remote 
ingress to make the choice automatically.

Cheers,
Mingui

>From: trill [mailto:[email-masked]] On Behalf Of Sujay Gupta
>Sent: Tuesday, April 08, 2014 7:26 PM
>To: Mingui Zhang
>Cc: [email-masked]
>Sujay- As you would know,fundamentally MCLAG load balancing is not
>dependent upon whether the node knows the macda ip etc. The moment it
>ingresses a packet it hashes and distributes. Now with our solution the traffic
>balances happen post learning the macda, so our solution is not making
>Sujay- I would not totally agree that the hashing can be tuned in L2 forwarding,
>IMO the fundamental ingress trill RB L2 logic don't seem to have a flow based
>hashing logic, which