Subject: [trill] Thoughts on active-active edge

From: Radia Perlman <[email-masked]>
Date: Tue, 11 Dec 2012 12:53:06 -0800

I'm not too happy about any of the proposed solutions to the active-active
edge issue, and wondering if there might be a simpler approach.

To review the problem:

Suppose R1, R2, and R3 are all attached to a set of endnodes (say, through
a hypervisor).  Let's say "E" is one of the endnodes multihomed through R1,
R2, R3.  Let's say "R8" is a distant RBridge (one several hops away from
R1, R2, and R3).

a) If R1, R2, and R3 use their own nickname when encapsulating a frame from
E, then R8 will see frequent changes of E's location (between R1, R2, and
R3).  Furthermore, it would be nice if the path from R8 to E could go via
the shortest path (if, for instance, R2 is much closer to R8 than R1).

b) If R1, R2, and R3 use a pseudonode nickname (say P) for all the
multihomed endnodes, then the RPF check doesn't work when the "wrong"
RBridge injects the encapsuated packet into the campus.  (the tree will
have only one of R1, R2, or R3 attached to the pseudonode, say R2, so when
R2 injects using the pseuodnode as ingress nickname, the RPF check will
work, but it will not work if R1 or R3 inject).

--------------

So...this might have been discussed before, but it might not hurt to look
at it with fresh eyes again.

*******************
Proposed rule: R1, R2, and R3 should use the pseudonode nickname when
encapsulating unicast, but use their own nickname when encapsulating
multidestination.
***********************

Result:  R8 will see endnode changes for E, but not very frequent;  R8 will
only see changes when traffic from E alternates between multidestination
and unicast.

Question: Are perceived endnode moves a problem?  Why?

If endnode changes are, for some reason, a problem, we could have R1, R2,
and R3 announce in LSPs, that they are attached to pseudonode P as an
active/active group, and therefore, endnode moves between P and anything in
the set {R1, R2, R3} should not raise alarms.  Better yet, would be to say
if endnode E is attached to P, then don't move it to {R1, R2, or R3}.

Radia

---

From: Sunny Rajagopalan <[email-masked]>
Date: Tue, 11 Dec 2012 16:23:30 -0800

Hi, Radia
I like this idea a lot better than the other proposals floating around. 
The +ve is that it can be done with today's ISIS. The -ve is that you'll 
need a minor hardware change -  today's hardware generally doesn't have 
the ability to use a different source rbridge depending on whether the 
packet is unicast or multicast. 

ISIS on  an rbridge that is emulating a pseudonode nickname can send out 
mac reachability information for endnodes attached to it, associating the 
mac addresses with the pseudonode nickname. Any multicast group with local 
receivers must also be advertised as belonging to the pseudonode.

At the other end, hardware already has the ability to "fix" mac entries so 
that they don't get rewritten when new information comes in. I do think 
this is necessary, because I believe we will see mac flips not just in the 
unicast-multicast interleave case, but also in the pure multicast case 
(load balancing could be using the layer 4 header, and that could cause 
different links to be used, even when the smac and dmac stays the same).

This means, of course, that if the mac moves to an rbridge that's not part 
of the pseudonode group then the new rbridge will have to send out an 
update so that everyone else can fix up their entries and remove the 
"fixed" property from the entry in hardware.

The approaches we've seen so far are complex, require the maintenance of 
multiple trees, use up the nickname space, and have constraints in the 
external non-trill topologies that can be used. This approach has none of 
the problems, and has my vote.
--
Sunny

From:   Radia Perlman <[email-masked]>
To:     [email-masked], 
Date:   12/11/2012 12:54 PM
Subject:        [trill] Thoughts on active-active edge
Sent by:        [email-masked]

To review the problem:

Suppose R1, R2, and R3 are all attached to a set of endnodes (say, through 
a hypervisor).  Let's say "E" is one of the endnodes multihomed through 
R1, R2, R3.  Let's say "R8" is a distant RBridge (one several hops away 
from R1, R2, and R3).

a) If R1, R2, and R3 use their own nickname when encapsulating a frame 
from E, then R8 will see frequent changes of E's location (between R1, R2, 
and R3).  Furthermore, it would be nice if the path from R8 to E could go 
via the shortest path (if, for instance, R2 is much closer to R8 than R1).

--------------

*******************

Result:  R8 will see endnode changes for E, but not very frequent;  R8 
will only see changes when traffic from E alternates between 
multidestination and unicast.  

If endnode changes are, for some reason, a problem, we could have R1, R2, 
and R3 announce in LSPs, that they are attached to pseudonode P as an 
active/active group, and therefore, endnode moves between P and anything 
in the set {R1, R2, R3} should not raise alarms.  Better yet, would be to 
say if endnode E is attached to P, then don't move it to {R1, R2, or R3}.

Radia_______________________________________________
trill mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/trill

---

From: Radia Perlman <[email-masked]>
Date: Tue, 11 Dec 2012 16:57:07 -0800

On Tue, Dec 11, 2012 at 4:23 PM, Sunny Rajagopalan <
[email-masked]> wrote:

> need a minor hardware change -  today's hardware generally doesn't have the
> ability to use a different source rbridge depending on whether the packet
> is unicast or multicast.

But can R1  use a different ingress nickname depending on the port from
which it is received?   For instance, if R1 has 4 ports:
a) to pseudonode p1
b) to a different shared link, pseudonode p2
c) to a port with just a bunch of endnodes, so there is no pseudonode
d) a link towards the rest of the TRILL campus

Can R1 use nickname "p1" when ingressing from port a, nickname "p2" when
ingressing from port b, nickname "R1" when ingressing from port c?

Thanks,

Radia

---

From: Sunny Rajagopalan <[email-masked]>
Date: Tue, 11 Dec 2012 18:22:42 -0700

Yes, most hardware can do what you've described below, but cannot pick  for port a, source rbridge p1 for multicast and p2 for unicast. 
--
Sunny

On Dec 11, 2012, at 5:02 PM, "Radia Perlman" <[email-masked]> wrote:

On Tue, Dec 11, 2012 at 4:23 PM, Sunny Rajagopalan <[email-masked]> wrote:
> Hi, Radia 
> I like this idea a lot better than the other proposals floating around. The +ve is that it can be done with today's ISIS. The -ve is that you'll need a minor hardware change -  today's hardware generally doesn't have the ability to use a different source rbridge depending on whether the packet is unicast or multicast. 

But can R1  use a different ingress nickname depending on the port from which it is received?   For instance, if R1 has 4 ports:
a) to pseudonode p1
b) to a different shared link, pseudonode p2
c) to a port with just a bunch of endnodes, so there is no pseudonode
d) a link towards the rest of the TRILL campus

Can R1 use nickname "p1" when ingressing from port a, nickname "p2" when ingressing from port b, nickname "R1" when ingressing from port c?

Thanks,

Radia

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Wed, 12 Dec 2012 05:27:21 +0000

Please see comments in-line

From: [email-masked] [mailto:[email-masked]] On Behalf Of Radia Perlman
Sent: Tuesday, December 11, 2012 12:53 PM
To: [email-masked]
Subject: [trill] Thoughts on active-active edge

I'm not too happy about any of the proposed solutions to the active-active edge issue, and wondering if there might be a simpler approach.

To review the problem:

Suppose R1, R2, and R3 are all attached to a set of endnodes (say, through a hypervisor).  Let's say "E" is one of the endnodes multihomed through R1, R2, R3.  Let's say "R8" is a distant RBridge (one several hops away from R1, R2, and R3).

a) If R1, R2, and R3 use their own nickname when encapsulating a frame from E, then R8 will see frequent changes of E's location (between R1, R2, and R3).  Furthermore, it would be nice if the path from R8 to E could go via the shortest path (if, for instance, R2 is much closer to R8 than R1).

b) If R1, R2, and R3 use a pseudonode nickname (say P) for all the multihomed endnodes, then the RPF check doesn't work when the "wrong" RBridge injects the encapsuated packet into the campus.  (the tree will have only one of R1, R2, or R3 attached to the pseudonode, say R2, so when R2 injects using the pseuodnode as ingress nickname, the RPF check will work, but it will not work if R1 or R3 inject).

--------------

So...this might have been discussed before, but it might not hurt to look at it with fresh eyes again.

*******************
Proposed rule: R1, R2, and R3 should use the pseudonode nickname when encapsulating unicast, but use their own nickname when encapsulating multidestination.
***********************

Result:  R8 will see endnode changes for E, but not very frequent;  R8 will only see changes when traffic from E alternates between multidestination and unicast.

[Answer] This depends on the traffic Radia. You cannot assume all multicast hash to Node R1 and all unicast to R2 and R3 and hence multicast unicast address ping pong is rare. In the contrary, you could have heavy multicast stream and a unicast stream both hashing in to the same RBridge RB1.

[Answer] Yes it is big time.  Unlike ancient devices modern devices to hardware based learning. When address association start moving, it will constantly bombard the CPU with new address or address move notifications.

If endnode changes are, for some reason, a problem, we could have R1, R2, and R3 announce in LSPs, that they are attached to pseudonode P as an active/active group, and therefore, endnode moves between P and anything in the set {R1, R2, R3} should not raise alarms.  Better yet, would be to say if endnode E is attached to P, then don't move it to {R1, R2, or R3}.

[Answer] This will complicate hardware learning. Or least need to re-do the hardware implementations different than today.

We discussed this approach many times with you before and the pain points are the same.

Radia

---

From: Radia Perlman <[email-masked]>
Date: Tue, 11 Dec 2012 21:59:40 -0800

Tissa, there are always technical tradeoffs between approaches.

Tissa said:
>>[Answer] This depends on the traffic Radia. You cannot assume all
multicast hash to Node R1 and all unicast to >>R2 and R3 and hence
multicast unicast address ping pong is rare. In the contrary, you could
have heavy >>multicast stream and a unicast stream both hashing in to the
same RBridge RB1.

I'm not sure what relevance there is to whether unicast hashes to R1, R2,
R3, or alternates between them, since the rule is that they all use the
pseudonode nickname when ingressing from E. The only way there will be
frequent endnode moves for E is if E frequently alternates sending
multicast and unicast.  All unicast from E, whether it is sent to R1, R2,
or R3, will be perceived by R8 as coming from pseudonode P.  Multicast, on
the other hand, with the proposed rule, could bounce between R1, R2, and
R3.  Though as an optimization, if the pseudonode P is attached to R2, then
R2 could use P when ingressing either unicast or multicast, so multicast
that happens to hash to R2 will not be perceived as an endnode move,
whereas multicast hashing to R1 or R3 would.

Tissa said about why endnode moves are a problem:

>>[Answer] Yes it is big time.  Unlike ancient devices modern devices to
hardware based learning. When address >>association start moving, it will
constantly bombard the CPU with new address or address move notifications.

Your description sounds to me like it's only a problem if there are a lot
of moves.  Since it seems unlikely that E will keep alternating between
sending multicast and unicast, or even if there were one node that was
doing that for some reason, that there would be lots of nodes acting this
way, it seems like the amount of moves seen, relative to the total amount
of traffic received by R8, should be really small.

So, the disadvantages of this proposal, as Sunny pointed out, are
1) the active-active RBs might need a hardware change in order to
selectively use a different ingress nickname depending on whether the
packet is multicast or unicast  (which implementations is this true for?)
2) distant RBs (say R8), will perceive E moving when E switches from
sending unicast to multicast.

But as I said, all approaches have pluses and minuses, so the minuses have
to be weighed against the pluses.

The advantages of this proposed approach relative to the Affinity TLV are:

a) you only need one tree for the campus, and you can have arbitrarily many
active-active nodes.  Or to put it a different way, you can have more
active-active RBs than there are trees.

b) furthermore, it allows the active-active RBs to do load splitting of
multicast (choosing more than one tree), without requiring n*k trees, where
n is the number of active-active RBs, and k is the number of trees each
might want to use for load splitting.

c) you don't need all RBs to change at once in the campus (as you would
with the Affinity proposal).  If R8 is unmodified, R8 will see E changing
when E changes from multicast to unicast, but will not have incorrect
behavior.  With affinity, absolutely every RB in the campus must be
modified or there will be incorrect (not just suboptimal) behavior.

Radia

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Wed, 12 Dec 2012 06:10:52 +0000

Radia

Please see in line

From: Radia Perlman [mailto:[email-masked]]
Sent: Tuesday, December 11, 2012 10:00 PM
To: Tissa Senevirathne (tsenevir)
Cc: [email-masked]
Subject: Re: [trill] Thoughts on active-active edge

Tissa said:
>>[Answer] This depends on the traffic Radia. You cannot assume all multicast hash to Node R1 and all unicast to >>R2 and R3 and hence multicast unicast address ping pong is rare. In the contrary, you could have heavy >>multicast stream and a unicast stream both hashing in to the same RBridge RB1.

I'm not sure what relevance there is to whether unicast hashes to R1, R2, R3, or alternates between them, since the rule is that they all use the pseudonode nickname when ingressing from E. The only way there will be frequent endnode moves for E is if E frequently alternates sending multicast and unicast.  All unicast from E, whether it is sent to R1, R2, or R3, will be perceived by R8 as coming from pseudonode P.  Multicast, on the other hand, with the proposed rule, could bounce between R1, R2, and R3.  Though as an optimization, if the pseudonode P is attached to R2, then R2 could use P when ingressing either unicast or multicast, so multicast that happens to hash to R2 will not be perceived as an endnode move, whereas multicast hashing to R1 or R3 would.

[Answer-2] Are you saying E will not simultaneously send multicast and unicast ? are you also suggesting there is either a single multicast stream or all multicast streams are hashing to the same RBridge that E is attached to ?

Tissa said about why endnode moves are a problem:

>>[Answer] Yes it is big time.  Unlike ancient devices modern devices to hardware based learning. When address >>association start moving, it will constantly bombard the CPU with new address or address move notifications.

Your description sounds to me like it's only a problem if there are a lot of moves.  Since it seems unlikely that E will keep alternating between sending multicast and unicast, or even if there were one node that was doing that for some reason, that there would be lots of nodes acting this way, it seems like the amount of moves seen, relative to the total amount of traffic received by R8, should be really small.

[Answer-2] It is not lots of switches it is how many ping-pong, with a single address and two stream you can overload learning buffers when traffic is coming at high rates.

So, the disadvantages of this proposal, as Sunny pointed out, are
1) the active-active RBs might need a hardware change in order to selectively use a different ingress nickname depending on whether the packet is multicast or unicast  (which implementations is this true for?)
2) distant RBs (say R8), will perceive E moving when E switches from sending unicast to multicast.

[Answer-2] #2 is exactly what I am saying as a big problem. But if you feel it is not a problem, I am not going to argue any further. You should speak to few more people and get the validation. ***Address move is a major problem**

But as I said, all approaches have pluses and minuses, so the minuses have to be weighed against the pluses.

a) you only need one tree for the campus, and you can have arbitrarily many active-active nodes.  Or to put it a different way, you can have more active-active RBs than there are trees.

b) furthermore, it allows the active-active RBs to do load splitting of multicast (choosing more than one tree), without requiring n*k trees, where n is the number of active-active RBs, and k is the number of trees each might want to use for load splitting.

c) you don't need all RBs to change at once in the campus (as you would with the Affinity proposal).  If R8 is unmodified, R8 will see E changing when E changes from multicast to unicast, but will not have incorrect behavior.  With affinity, absolutely every RB in the campus must be modified or there will be incorrect (not just suboptimal) behavior.

Radia

---

From: Mingui Zhang <[email-masked]>
Date: Wed, 12 Dec 2012 06:46:55 +0000

Hi,

I think the MAC move affects the path selection at the remote RBridge 'R8', which may bring frame disorder at the endnode 'E' connected to the active-active edge. 

The new proposal suggests that one group occupies one pseudo-node nickname. If "use up the nickname space" is really a problem, the new proposal suffers from this problem, right? Having said that, I want to argue that we have to identify each active-active edge with a pseudo-node nickname in either proposal AND the consumption of nicknames on active-active edge is not a big problem.

Thanks,
Mingui 

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of Tissa
>Senevirathne (tsenevir)
>Sent: Wednesday, December 12, 2012 2:11 PM
>To: Radia Perlman
>Cc: [email-masked]
>>>[Answer] This depends on the traffic Radia. You cannot assume all multicast
>hash to Node R1 and all unicast to >>R2 and R3 and hence multicast unicast
>address ping pong is rare. In the contrary, you could have heavy >>multicast
>stream and a unicast stream both hashing in to the same RBridge RB1.
>
>
>
>I'm not sure what relevance there is to whether unicast hashes to R1, R2, R3, or
>alternates between them, since the rule is that they all use the pseudonode
>nickname when ingressing from E. The only way there will be frequent endnode
>moves for E is if E frequently alternates sending multicast and unicast.  All
>unicast from E, whether it is sent to R1, R2, or R3, will be perceived by R8 as
>coming from pseudonode P.  Multicast, on the other hand, with the proposed
>rule, could bounce between R1, R2, and R3.  Though as an optimization, if the
>pseudonode P is attached to R2, then R2 could use P when ingressing either
>unicast or multicast, so multicast that happens to hash to R2 will not be
>perceived as an endnode move, whereas multicast hashing to R1 or R3 would.
>
>
>
>
>
>[Answer-2] Are you saying E will not simultaneously send multicast and unicast ?
>are you also suggesting there is either a single multicast stream or all multicast
>streams are hashing to the same RBridge that E is attached to ?
>
>
>
>Your description sounds to me like it's only a problem if there are a lot of moves.
>Since it seems unlikely that E will keep alternating between sending multicast
>and unicast, or even if there were one node that was doing that for some reason,
>that there would be lots of nodes acting this way, it seems like the amount of
>moves seen, relative to the total amount of traffic received by R8, should be
>really small.
>
>
>
>[Answer-2] It is not lots of switches it is how many ping-pong, with a single
>address and two stream you can overload learning buffers when traffic is coming
>at high rates.
>
>
>
>1) the active-active RBs might need a hardware change in order to selectively
>use a different ingress nickname depending on whether the packet is multicast
>or unicast  (which implementations is this true for?)
>
>[Answer-2] #2 is exactly what I am saying as a big problem. But if you feel it is
>not a problem, I am not going to argue any further. You should speak to few
>more people and get the validation. ***Address move is a major problem**
>
>
>
>b) furthermore, it allows the active-active RBs to do load splitting of multicast
>(choosing more than one tree), without requiring n*k trees, where n is the
>number of active-active RBs, and k is the number of trees each might want to
>use for load splitting.
>
>
>
>c) you don't need all RBs to change at once in the campus (as you would with the
>Affinity proposal).  If R8 is unmodified, R8 will see E changing when E changes
>from multicast to unicast, but will not have incorrect behavior.  With affinity,
>absolutely every RB in the campus must be modified or there will be incorrect
>(not just suboptimal) behavior.
>
>
>
>
>
>Radia

---

From: Thomas Narten <[email-masked]>
Date: Wed, 12 Dec 2012 14:57:10 -0500

"Tissa Senevirathne (tsenevir)" <[email-masked]> writes:

> > Proposed rule: R1, R2, and R3 should use the pseudonode nickname
> >  when encapsulating unicast, but use their own nickname when
> >  encapsulating multidestination.
>  
 
>  multicast hash to Node R1 and all unicast to R2 and R3 and hence
>  multicast unicast address ping pong is rare. In the contrary, you
>  could have heavy multicast stream and a unicast stream both hashing
>  in to the same RBridge RB1.

I'm with Tissa on this one. I'm wary about making assumptions about
"likely traffic patterns" and then making architectural decisions
based on that. We could easily get burned down the road.

> [Answer] Yes it is big time.  Unlike ancient devices modern devices
>  to hardware based learning. When address association start moving,
>  it will constantly bombard the CPU with new address or address move
>  notifications.

One thing that would help I think would be to actually list the issues
associated "MAC flapping" (i.e., where the FDB entries for a given
destination Etherent MAC addr keeps flip flopping) in terms of best
egress RB.

Are the issues exactly the same as for regular bridges? Or are there
additional nuances? What breaks or causes pain when entries flap?

Possible issues:

1) the paths packets take for a given destination could switch between
2 (or more) paths. This could lead to out-of-order packet
delivery. TCP doesn't like this. OAM may not like it either. (It's one
thing to have occasional router flaps, its another to have them be a
steady-state feature.)

2) Some implementations require software "help" to update FDB
entries. I.e., changes are not done 100% by the ASICs. While that may
change in the future, this could be an issue today for some
implemntations. How common is this problem? Do we have a sense?

What else? 

Thomas

---

From: Sam Aldrin <[email-masked]>
Date: Wed, 12 Dec 2012 12:17:52 -0800

I agree with Mingui and Tissa on this.

Intentionally creating architecture change to have MAC move scenario is something not at all a good idea.
Also, reasoning that only edge or participating RBridges requires hardware changes is not a good argument for advantage either, because, that depends on topology and deployment scenario, not architecturally better.
Eventually, that change has to be on all RB's anyway, because of silicon and other changes.
I am more concerned the load balancing of the traffic, which is a potential issue, and cannot be ignored.

One point to consider with any solution is from OAM angle. Operationally, how this (any solution) could impact the behavior and the cost associated with this.

I believe we have discussed these scenarios (different solution types) previously. 
WG should reach consensus and move forward with the agreed solution. 
Atleast from the IETF WG sessions, I presumed there was a consensus on the solution. 
Am I wrong in that assumption?

cheers
-sam
On Dec 11, 2012, at 10:46 PM, Mingui Zhang <[email-masked]> wrote:

> _______________________________________________
> trill mailing list
> [email-masked]

---

From: Thomas Narten <[email-masked]>
Date: Wed, 12 Dec 2012 15:25:48 -0500

> I believe we have discussed these scenarios (different solution
> types) previously.  WG should reach consensus and move forward with
> the agreed solution.  Atleast from the IETF WG sessions, I presumed
> there was a consensus on the solution.  Am I wrong in that
> assumption?

what "solution" are you referring above as maybe having consensus?

Thomas

---

From: Sam Aldrin <[email-masked]>
Date: Wed, 12 Dec 2012 12:41:30 -0800

Cmt draft

Sent from my iPhone

---

From: Thomas Narten <[email-masked]>
Date: Wed, 12 Dec 2012 17:57:39 -0500

> Cmt draft

If you mean draft-ietf-trill-cmt-01.txt, I at least am not in
agreement.

I have several issues with the document.

For starters, it's incomplete. It needs to be paired with another
document to get an actual "standard" you can implement. We don't have
such a companion document yet.  It's been suggested that
draft-hu-trill-pseudonode-nickname-04.txt is one such document. But
that document is not a WG document and I'm not sure what its status
is. (And it needs work...)

As a general rule (not TRILL specific) I don't like WGs doing "partial
work" where they standardize one document, that is incomplete by
itself, independent of other documents that are needed to produce a
complete implementable solution to some real problem.

I've raised issues on the list about CMT and its unclarity
wrt. LAG/MC-LAG, etc. I did not really get my questions answered.

What I'd really like to start with on CMT is a clear problem
statement. I.e., exactly what problem/use case it is trying to
solve. I have not been able to get that out of the two  documents.

As for draft-hu-trill-pseudonode-nickname-04.txt, I have other
concerns. E.g., it talks about tunneling at one point, but that is
completely unspecified.

What I don't want to do is spend a lot of time on the *solution*
details. I first want a clear problem statement/use case. Without
that, it is difficult to assess how well potential solutions address
the stated problem.

Thomas

---

From: "Tissa Senevirathne (tsenevir)" <[email-masked]>
Date: Wed, 12 Dec 2012 23:14:16 +0000

I think we should answer the question in hand first.

This thread is all about "saying it is ok to have MAC move in TRILL to solve active-active edge"

I am overwhelmingly against that statement and the argument for it is all based on whole series of assumptions and ignoring the modern networking devices and how they operate.

Once that is resolved and behind we can have a separate discussion of CMT, hu draft and future approach etc.

Thanks
Tissa

-----Original Message-----
From: Thomas Narten [mailto:[email-masked]] 
Sent: Wednesday, December 12, 2012 2:58 PM
To: Sam Aldrin
Cc: Mingui Zhang; Tissa Senevirathne (tsenevir); Radia Perlman; [email-masked]
Subject: Re: [trill] Thoughts on active-active edge

> Cmt draft

If you mean draft-ietf-trill-cmt-01.txt, I at least am not in agreement.

For starters, it's incomplete. It needs to be paired with another document to get an actual "standard" you can implement. We don't have such a companion document yet.  It's been suggested that draft-hu-trill-pseudonode-nickname-04.txt is one such document. But that document is not a WG document and I'm not sure what its status is. (And it needs work...)

As a general rule (not TRILL specific) I don't like WGs doing "partial work" where they standardize one document, that is incomplete by itself, independent of other documents that are needed to produce a complete implementable solution to some real problem.

I've raised issues on the list about CMT and its unclarity wrt. LAG/MC-LAG, etc. I did not really get my questions answered.

What I'd really like to start with on CMT is a clear problem statement. I.e., exactly what problem/use case it is trying to solve. I have not been able to get that out of the two  documents.

As for draft-hu-trill-pseudonode-nickname-04.txt, I have other concerns. E.g., it talks about tunneling at one point, but that is completely unspecified.

What I don't want to do is spend a lot of time on the *solution* details. I first want a clear problem statement/use case. Without that, it is difficult to assess how well potential solutions address the stated problem.

Thomas

---

From: Sam Aldrin <[email-masked]>
Date: Wed, 12 Dec 2012 15:17:45 -0800

Comments inline.

-sam

> If you mean draft-ietf-trill-cmt-01.txt, I at least am not in
%sam- that is the draft.
> I have several issues with the document.
%sam - Good time to raise them again and discuss within WG as it is WG doc.

> complete implementable solution to some real problem.
%sam - In this context, I am not aligned with you. The reason the document was adopted as WG, is because the problem was identified and agreed upon.
Partial problem or full problem depends on how one sees it. One can also claim to have one singe 'TRILL' solution document, instead of multiple drafts targeting subset of TRILL problems.
> wrt. LAG/MC-LAG, etc. I did not really get my questions answered.
%sam - Authors could clarify. Sorry I am not co-author of any of these docs :D
> that, it is difficult to assess how well potential solutions address
%sam - In regards to CMT draft, I understood the problem scope. If something is not clear, do outline your question and authors have responsibility to clarify.

-sam

---

From: Mingui Zhang <[email-masked]>
Date: Thu, 13 Dec 2012 06:03:18 +0000

Hi Thomas,

The whole problem is how to provide end-nodes (end-stations or LAGs) with a mechanism of active-active access into the TRILL campus. The baseline requirement for this mechanism is that traffic duplication or loops MUST be avoided. However, as the mechanism is being developed, new problems appear. I'd like to list those problems as follows. 

1.	RPFC issue. 
The CMT draft addresses this issue. 

2.	Nickname configuration
It is necessary to denote the active-active edge group with a virtual RBridge, say RBv. The member RBridges act as the agents of the nickname allocation for RBv and they MUST act in consistence.

As defined in RFC6325, pseudonode nickname is actually allocated by the DRB. DRB is elected through exchanging HELLOs. The LAG or end-station is not a LAN anymore. As a result, no HELLOs can be exchanged on it. No HELLO no DRB. So, a new pseudonode nickname allocation method is needed.

3.	MAC sharing 
The MAC addressed learnt by one member RB should be synchronized up with other RBs in the same group. Or else, if a frame destined to such a MAC is via other RBs, it will cause unnecessary unknown unicast.

4.	Failure recovery
When a member RBridge or link fails, how does the whole active-active edge group response? A method is needed here to automate that process. 

I am OK with the idea to compose a problem statement draft. However, IMO, it is impossible to provide _one_ solution draft to solve all these problems.

Thanks,
Mingui

>Sent: Thursday, December 13, 2012 6:58 AM
>To: Sam Aldrin

---

From: Radia Perlman <[email-masked]>
Date: Wed, 12 Dec 2012 23:13:20 -0800

I think it would be good to have a document that explains the problem...I
certainly don't believe I know all the cases that need to be solved.  I
think I understand the hypervisor case...where the hypervisor decides which
uplink to send things to, and never forwards between the up-links.

This case is scary because the RBridges on the uplink cannot see Hellos
from each other, so if misconfigured, at the very least I could imagine
multiple RBridges decapsulating multicast from the campus to the hypervisor.

Anyway...how many uplinks do we need to support?  Do we care about problems
due to misconfiguration?

In cases like this, is it common to also have pt-to-pt links between all
the RBs attaching to the hypervisor?  If so, then it seems like it would be
possible for them to coordinate to at least detect misconfiguration, and
possibly play games with forwarding messages to each other (e.g., if one of
them is not attached to a tree and needs to encapsulate a multidestination
frame).

How many trees does the campus need?

Are there cases where there are lots of hypervisors, where they attach to
different subsets of edge RBs?  In that case, we might eat up a lot of
nicknames, since if one hypervisor is attached to {R1, R2}, and another is
attached to {R1, R2, R3}, they cannot use the same pseudonode nickname.

Are there cases other than hypervisors?  I think there are cases of bridges
that have this behavior (a port with a bunch of endnodes, and several
up-links, where the bridge does not forward between the up-links.

If this has been written down anywhere, can anyone point me to it?  If not,
it seems really prudent to answer these (and I'm sure other) questions
before arguing about specific solutions.

Radia

---

From: [email-masked]
Date: Thu, 13 Dec 2012 16:13:16 +0800

I am OK with a new document to state the problem.

As for the problems listed by Mingui, pls see in line.

Best Regards,
Zhai Hongjun
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
 Protocol Development Dept.VI, Central R&D Institute, ZTE Corporation
 No. 68, Zijinghua Road, Yuhuatai District, Nanjing, P.R.China, 210012
 
 Zhai Hongjun
 
 Tel: +86-25-52877345
 Email: [email-masked]
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

Mingui Zhang <[email-masked]> 
发件人:  [email-masked]
2012-12-13 14:03

收件人
Thomas Narten <[email-masked]>, Sam Aldrin <[email-masked]>
抄送
"Tissa Senevirathne \(tsenevir\)" <[email-masked]>, Radia Perlman 
<[email-masked]>, "[email-masked]" <[email-masked]>
主题
Re: [trill] Thoughts on active-active edge

Hi Thomas,

The whole problem is how to provide end-nodes (end-stations or LAGs) with 
a mechanism of active-active access into the TRILL campus. The baseline 
requirement for this mechanism is that traffic duplication or loops MUST 
be avoided. However, as the mechanism is being developed, new problems 
appear. I'd like to list those problems as follows. 

1.               RPFC issue. 

2.               Nickname configuration
It is necessary to denote the active-active edge group with a virtual 
RBridge, say RBv. The member RBridges act as the agents of the nickname 
allocation for RBv and they MUST act in consistence.

As defined in RFC6325, pseudonode nickname is actually allocated by the 
DRB. DRB is elected through exchanging HELLOs. The LAG or end-station is 
not a LAN anymore. As a result, no HELLOs can be exchanged on it. No HELLO 
no DRB. So, a new pseudonode nickname allocation method is needed.

[Hongjun Zhai --Begin]
I don't think the nickname allocation for RBv is a big issue. So it is not 
discussed in PN draft.

In real deployment, RBv is configured on RBridge ports an identified with 
a nickname. When the first member port is associated with an RBv, we can 
choose one of the avaialble nicknames suggested by the member RBridge 
based on its LSDB as the pseudo-nickname of the RBv. Then when other 
RBridge ports needs to join this RBv, we can use the choosen nickname to 
identify thie RBv.
[Hongjun Zhai --End]

3.               MAC sharing 
The MAC addressed learnt by one member RB should be synchronized up with 
other RBs in the same group. Or else, if a frame destined to such a MAC is 
via other RBs, it will cause unnecessary unknown unicast.

[Hongjun Zhai --Begin] 
MAC sharing is really needed but not difficult. We can use ESADI 
extenstion for this purpose. 1) allow the VLAN-scoped ESADI instance to 
advertise learned remote end station MAC addresses as well as local end 
station MAC addresses, Or 2) extend ESADI to support RB-Group scoped 
instances (please refer draft-zhai-extenstion-for-RBv for more details).
[Hongjun Zhai --End]

4.               Failure recovery
When a member RBridge or link fails, how does the whole active-active edge 
group response? A method is needed here to automate that process.

[Hongjun Zhai  --Begin]
I think it is really a problem slove in RBv. The link failure issue is 
discussed in Section 6 of PN draft. Please click the following URL for 
more details.
http://www.ietf.org/internet-drafts/draft-hu-trill-pseudonode-nickname-04.txt
.
[Hongjun Zhai  --End]

I am OK with the idea to compose a problem statement draft. However, IMO, 
it is impossible to provide _one_ solution draft to solve all these 
problems.

Thanks,
Mingui

---

From: [email-masked]
Date: Thu, 13 Dec 2012 20:03:01 +0800

Hi Radia

> This case is scary because the RBridges on the uplink cannot see Hellos 
from each other, 
> so if misconfigured, at the very least I could imagine multiple RBridges 
decapsulating 
> multicast from the campus to the hypervisor.

> Anyway...how many uplinks do we need to support?  Do we care about 
problems due to misconfiguration?

I don't know what the misconfiguration refers to. Is it the set of VLANs 
for which an Rbridge acts as AF?

> Are there cases where there are lots of hypervisors, where they attach 
to different subsets of edge RBs?  
> In that case, we might eat up a lot of nicknames, since if one 
hypervisor is attached to {R1, R2}, 
> and another is attached to {R1, R2, R3}, they cannot use the same 
pseudonode nickname.

I don't know why the two sets of RBridges can not use the same 
pseudo-nickname. If the learned MAC addresses 
can be shared among member Rbridges of an RBv and TRILL data frames can be 
tunneled to another member Rbridge
that can egress the frame, I think they can use the same pseudo-nickname.

If I am wrong, please correct me.

Best Regards,
Zhai Hongjun
 
 Zhai Hongjun
 
 Tel: +86-25-52877345

Radia Perlman <[email-masked]> 
发件人:  [email-masked]
2012-12-13 15:13

收件人
Mingui Zhang <[email-masked]>
抄送
Thomas Narten <[email-masked]>, Sam Aldrin <[email-masked]>, 
"Tissa Senevirathne \(tsenevir\)" <[email-masked]>, "[email-masked]" 
<[email-masked]>
主题
Re: [trill] Thoughts on active-active edge

I think it would be good to have a document that explains the problem...I 
certainly don't believe I know all the cases that need to be solved.  I 
think I understand the hypervisor case...where the hypervisor decides 
which uplink to send things to, and never forwards between the up-links.

This case is scary because the RBridges on the uplink cannot see Hellos 
from each other, so if misconfigured, at the very least I could imagine 
multiple RBridges decapsulating multicast from the campus to the 
hypervisor.

Anyway...how many uplinks do we need to support?  Do we care about 
problems due to misconfiguration?

In cases like this, is it common to also have pt-to-pt links between all 
the RBs attaching to the hypervisor?  If so, then it seems like it would 
be possible for them to coordinate to at least detect misconfiguration, 
and possibly play games with forwarding messages to each other (e.g., if 
one of them is not attached to a tree and needs to encapsulate a 
multidestination frame).

Are there cases other than hypervisors?  I think there are cases of 
bridges that have this behavior (a port with a bunch of endnodes, and 
several up-links, where the bridge does not forward between the up-links. 

If this has been written down anywhere, can anyone point me to it?  If 
not, it seems really prudent to answer these (and I'm sure other) 
questions before arguing about specific solutions.

Radia

---

From: Radia Perlman <[email-masked]>
Date: Thu, 13 Dec 2012 10:10:38 -0800

Answering Zhai Hongjun's questions:

Why it is necessary to have a different pseuonode nickname if the upllnk is
to different sets of RBridges:

If hypervisor H1 has uplinks to R1, R2, and R3, and uses pseudonode
nickname P1, and hypervisor H2 has uplinks to R1 and R2 (or even had
uplinks to R1, R2, and R3, but its link to R3 fails), then if H1 and H2 use
the same nickname, say P1, then traffic for H2's MAC addresses might get
sent to R3 (since R3 has to claim to be connected to P1 because it is, for
H1).  But it is no longer attached to H2 because H2's uplink to R3 failed.

The safest thing would be for every hypervisor to have a nickname.

So, how many hypervisors are there likely to be?  How usual would it be for
all of them to attach to the same set of uplinks, so that we can use the
same pseuodnode nickname?  Do we care about the case of one of a
hypervisor's uplinks failing, in which case, would the RBridges know?  If
R3 (the one to which the uplink failed) know?  Would R1 and R2 know?  Even
if R3 knew, how could it alert R1 and R2 to now use a different pseudonode
nickname for H2?  Would it be obvious to them which of the hypervisors that
have uplinks to R1, R2, and R3 they are referring to?

All of this must be configured, I assume, and if the configuration is
wrong, then who knows what happens..presumably that traffic may or may not
get delivered to a hypervisor.  And even if configured properly, what
happens when uplinks fail?  Again, presumably, traffic may or may not get
delivered to the hypervisor whose uplink fails.

-------------
As for VLANs...that actually is not a problem here, since we're not using
AFs.  The hypervisor determines which uplink to send something to.  And
which tree is being used for distribution determines which of R1, R2, or R3
will decapsulate the packet.

So the main sort of configuration that I can think of, off the top of my
head, is which pseudonodes go with which hypervisors.  And I do think
configuration is scary, especially if there's no "sanity check" whereby the
RBs can compare notes.  I don't know how R1 can know that a particular port
is to "H1" so that it can inform R2 (via LSPs?) that R1 is attached to H1,
and R2 can notice that it, indeed, is also attached to H1.

--------
And part of the description of the problem would be answering questions
like how many uplinks would need to be supported.  Two at most?  30?  If a
lot, then solutions that require a tree for every uplink would be
problematic if implementations don't want to support that many trees.  Or
is it OK to require lots of trees?

So I think there are lots of things that should be written down, as part of
describing the problem.

Radia

On Thu, Dec 13, 2012 at 4:03 AM, <[email-masked]> wrote:

>  *Radia Perlman <[email-masked]>*
> "Tissa Senevirathne \(tsenevir\)" <[email-masked]>, "[email-masked]" <
> [email-masked]>
> 主题
> not, it seems really prudent to answer these (and I'm sure other) questions

---

From: [email-masked]
Date: Fri, 14 Dec 2012 10:31:54 +0800

There is only one RBridge(that is AF) to do the TRILL encapuslation in the 
shared LAN link scenario.So we can introduce the pseudonode nickname to 
identy the shared link, and avoid the potential data loss when AF changed. 
While it is different for the active-active edge scenario: all the 
RBridges in the LAG-group should do the TRILL encapsulation, and the 
RBridge cannot see Hellos from each other, there is no AF in this case 
actually. So pseudonode nickname is not fit for active-active edge.

We could use the control protocol(for example, ESADI )to learn the MAC and 
nickname, and avoid the MAC flip by adding a new ID(link-ID). If the 
remote receives two records with the same Link-ID,it should think both of 
them are Ok. 

 R1 and R2 tell R8 that H1 is multi-homing to the trill campus, they use 
the same Link-ID to identify the LAG-group. When R8 receives the 
information, it keeps two records for H1, one is R1, and the other is R2. 
The back data frame could choice either R1 or R2 as the egress nickname.

If the link between H1 and R2 fails, the record R2 is clear.

If there is another link for H1, say R3, and R3 uses the same link-id, so 
R8 keeps another record for H1.

Best regards.

2012-12-14 02:10

收件人
[email-masked]
抄送
Thomas Narten <[email-masked]>, [email-masked], Sam Aldrin 
<[email-masked]>, Mingui Zhang <[email-masked]>, 
"[email-masked]" <[email-masked]>, "Tissa Senevirathne \(tsenevir\)" 
<[email-masked]>
主题
Re: [trill] Thoughts on active-active edge

Why it is necessary to have a different pseuonode nickname if the upllnk 
is to different sets of RBridges:

If hypervisor H1 has uplinks to R1, R2, and R3, and uses pseudonode 
nickname P1, and hypervisor H2 has uplinks to R1 and R2 (or even had 
uplinks to R1, R2, and R3, but its link to R3 fails), then if H1 and H2 
use the same nickname, say P1, then traffic for H2's MAC addresses might 
get sent to R3 (since R3 has to claim to be connected to P1 because it is, 
for H1).  But it is no longer attached to H2 because H2's uplink to R3 
failed.

So, how many hypervisors are there likely to be?  How usual would it be 
for all of them to attach to the same set of uplinks, so that we can use 
the same pseuodnode nickname?  Do we care about the case of one of a 
hypervisor's uplinks failing, in which case, would the RBridges know?  If 
R3 (the one to which the uplink failed) know?  Would R1 and R2 know?  Even 
if R3 knew, how could it alert R1 and R2 to now use a different pseudonode 
nickname for H2?  Would it be obvious to them which of the hypervisors 
that have uplinks to R1, R2, and R3 they are referring to?

-------------
As for VLANs...that actually is not a problem here, since we're not using 
AFs.  The hypervisor determines which uplink to send something to.  And 
which tree is being used for distribution determines which of R1, R2, or 
R3 will decapsulate the packet.

So the main sort of configuration that I can think of, off the top of my 
head, is which pseudonodes go with which hypervisors.  And I do think 
configuration is scary, especially if there's no "sanity check" whereby 
the RBs can compare notes.  I don't know how R1 can know that a particular 
port is to "H1" so that it can inform R2 (via LSPs?) that R1 is attached 
to H1, and R2 can notice that it, indeed, is also attached to H1.

--------

So I think there are lots of things that should be written down, as part 
of describing the problem.

Radia

Hi Radia 

from each other, 
decapsulating 

pseudonode nickname. 

Best Regards,
Zhai Hongjun

Zhai Hongjun

Tel: +86-25-52877345

收件人

Radia

---

From: [email-masked]
Date: Fri, 14 Dec 2012 11:17:13 +0800

Hi Radia

Thanks for your answering.

> Why it is necessary to have a different pseuonode nickname if the upllnk 
is to 
> different sets of RBridges:

> If hypervisor H1 has uplinks to R1, R2, and R3, and uses pseudonode 
nickname P1, 
> and hypervisor H2 has uplinks to R1 and R2 (or even had uplinks to R1, 
R2, and R3, 
> but its link to R3 fails), then if H1 and H2 use the same nickname, say 
P1, 
> then traffic for H2's MAC addresses might get sent to R3 (since R3 has 
to claim to 
> be connected to P1 because it is, for H1).  But it is no longer attached 
to H2 
> because H2's uplink to R3 failed.

I think you are right if member RBriges of an RBv do not share their 
learned addresses 
(of local attathed end nodes and remote nodes) among the RBv. If they 
share the learned 
addresses, RB3 will know it can reach H2 via RB1 or RB2. Since H2 has 
uplinks to RB1 
and RB2, the two RBridges can learn H2's MAC address by oberving H2's 
native frames. 
Then RB1 and RB2 can share H2's MAC within the RBv, so RB3 will know (from 
the shared 
information) that it can reach H2 via RB1 or RB2. On receipt of traffic 
for H2's 
MAC addresses, RB3 will tunnel the traffic to RB1 or RB2 and the latter RB 
egress 
the traffic to H2.

As for the traffic tunneling, it has been specified in 
draft-ietf-trill-clear-correct-06.txt
(Section 2.4.2.1 in Page 9 and Section 2.4.2.3 in Pag2 10). The tunneling 
is simple, 
RB3 replaces the egress nickname in the TRILL header of the traffic with 
RB1 or RB2's nickname, 
then transmits the re-encapsultion traffic to RB1 or RB2.

Therefore, After the MAC sharing among member Rbridges of an RBv and the 
traffic tunneling 
are support, it is not necessary one nickname per hypervisor.

> So, how many hypervisors are there likely to be?  How usual would it be 
for all of 
> them to attach to the same set of uplinks, so that we can use the same 
pseuodnode nickname?

I do not know how many hypervisors there likely be, but I know one RBridge 
usually has 
24 or 48 down-links that can acts as uplinks for several hypervisors. So 
about 10 or 20 
hypervisors can be dual-homed to the same two of such Rbridges. If those 
hypervisors use 
same a pseudo-nickname, it not only saves nicknames but also decreases the 
RPFC entries 
on the RBriges in the TRILL campus scope.

> Do we care about the case of one of a hypervisor's uplinks failing, in 
which case, 
> would the RBridges know? If R3 (the one to which the uplink failed) 
know?  Would R1 and R2 know?

I think the member RBridge SHOULD know the uplink failure and tell other 
member RBridges 
the failure, if that Rbridge is directly connected to hypervisor which 
uplink fails. Otherwise, 
we can make sure the multi-destination traffic can be properly egressed to 
the hypervisor.

> So the main sort of configuration that I can think of, off the top of my 
head, 
> is which pseudonodes go with which hypervisors. 

If hypervisors do not make TRILL-encapsulation/decapsulation, they do not 
need to know 
which pseudonodes go with them. But if they do the 
encapsulation/decapsultion, they really need to know.

> I don't know how R1 can know that a particular port is to "H1" so that 
it can inform R2 
> (via LSPs?) that R1 is attached to H1, and R2 can notice that it, 
indeed, is also attached to H1.

If I not misunderstood your meaning, I think R1 can learn H1's MAC 
addresses by observing H1's 
native frames. Then it will know which particular port is to H1. R2 can 
also do the learning. 
If R2 has not learned H1's MAC addresses, R1 can share the addressed with 
it via ESADI PDU. 

> And part of the description of the problem would be answering questions 
like 
> how many uplinks would need to be supported.  Two at most?  30?  If a 
lot, 
> then solutions that require a tree for every uplink would be problematic 
if 
> implementations don't want to support that many trees.  Or is it OK to 
require 
> lots of trees?

In my mind, some vendors say their proprietary MC-LAG technologies can 
support at most 8 member 
devices in theory. But in practical deployment, two or three member 
devices in a MAC-LAG 
group are OK. So I think 8 trees is supported is OK at current. It can 
meet the practical 
requirements and does not make mass RPFC entries.

If I am wrong, please correcte me.

Best Regards,
Zhai Hongjun
 
 Zhai Hongjun
 
 Tel: +86-25-52877345

收件人
[email-masked]
抄送

-------------

--------

Radia

Hi Radia 

from each other, 
decapsulating 

pseudonode nickname. 

Best Regards,
Zhai Hongjun

Zhai Hongjun

Tel: +86-25-52877345

收件人

Radia

---

From: Mingui Zhang <[email-masked]>
Date: Fri, 14 Dec 2012 08:12:08 +0000

Hi Radia,

>If this has been written down anywhere, can anyone point me to it?  If not, it
>seems really prudent to answer these (and I'm sure other) questions before
>arguing about specific solutions.

I volunteer to compose that draft. There was a draft we once brought to the TRILL site. (http://tools.ietf.org/id/draft-zhang-trill-aggregation-02.txt). However, it needs a big change.  
I plan to remove all the staff about specific solutions and include issues which were discussed on the list.

Mingui