Subject: [trill] RBridge Channel versus ESADI for MAC address flush

From: Donald Eastlake <[email-masked]>
Date: Thu, 11 Feb 2016 11:59:48 -0500

At the last TRILL WG meeting in Yokohama, towards the end of the
second session, there was a discussion of the best mechanism to
implement a remote RBridge MAC address flush mechanism.
draft-hao-trill-address-flush proposes to use RBridge Channel messages
[RFC7178] but some people suggest the use of ESADI messages [RFC7357].
There was no resolution of this at the meeting so the Chair indicated
that it should be taken to the mailing list. This message is to start
the discussion. I'll post separately with my personal opinion.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

---

From: Haoweiguo <[email-masked]>
Date: Mon, 15 Feb 2016 01:21:38 +0000

Hi,
I think RBridge channnel based solution is better. RBridge channel based solution is a stateless solution, ESADI is a stateful solution which relies on periodic state synchronization on edge RBridges.
RBridge channel is more lightweight and can acts as a MAC flush building block for data plane and control plane MAC learning solution. If we use ESADI for MAC flush, when MAC learning is through data plane, the MAC flush solution is too heavy.
My two cents.
Thanks,
weiguo
________________________________________
From: Donald Eastlake [[email-masked]]
Sent: Friday, February 12, 2016 0:59
To: [email-masked]
Subject: [trill] RBridge Channel versus ESADI for MAC address flush

Thanks,
Donald

---

From: Donald Eastlake <[email-masked]>
Date: Sun, 14 Feb 2016 23:25:38 -0500

These are my thoughts on this question:

The goal of the MAC address flush facility is to be able to clear MAC
addresses that have been learned from the data plane.

For example, if an edge RBridge RB1 knows that an end station with MAC
address ES1 in VLAN V1 is no longer attached to any of its ports, it
would want to clear any learning in the TRILL campus that associates
ES1&V1 with RB1.

Another case would be if RB2 has a port to a link with a bunch of end
stations in V1 but RB2 losses appointed forwarder for V1 on that port.
RB2 would then want to clear all learning at other RBridges in the
campus associating any of the MAC addresses out that port with RB2.
(This is actually only useful if RB2 is still appointed forwarder for
V1 out some other port. If RB2 were no longer appointed forwarder for
V1 for any port, then RB2 would use the core IS-IS instance to
withdraw the Interested VLANs sub-TLV by which RB2 declared interest
in V1. All the other RBridges in the campus can see this withdrawal
and would know to clear any V1 MAC address learning for RB2, as
specified in RFC 6325 Section 4.8.3, next to last paragraph on page
70.)

ESADI uses a mechanism patterned after the IS-IS reliable link state
flooding mechanism. The ESADI idea is that, if RB3 is using ESADI for
V1, RB3 advertises all the locally attached MAC addresses RB3 has
learned about (through the receipt of native frames or whatever) in V1
through ESADI. All other RBridges in the campus that are participating
in ESADI will reliably learn that those MAC addresses are associated
with RB3 if those other RBridges are participating in ESADI for V1.
RB3 can easily and reliably withdraw one, several, or all of these MAC
addresses by sending updated link state information through the ESADI
instance using the already existing ESADI mechanisms.

The current proposal is to use an RBridge Channel message to clear out
MAC addresses learned from the data plane.

RBridge Channel messages and ESADI messages have a number of
similarities. Both use packets structured as TRILL Data packets with
an Inner.MacDA of All-Egress-RBridges (this MAC address was formerly
called All-ESADI-RBridges). The use of this MAC address as the
Inner.MacDA assures that the packets will not be forwarded to an end
station. The difference is that after the inner MAC addresses, an
ESADI packet has the L2-IS-IS Ethertype and the rest of the frame is
structured as an IS-IS PDU, while an RBridge Channel packet has the
RBridge-Channel Ethertype and the rest of the frame is structured as
specified in RFC 7178 to provide a general extensible format for typed
messages between RBridges. (There is also a way to send RBridge
Channel messages, called “native RBridge Channel messages”, to end
stations but that is not what would be used here.)

To me, it does not make sense to try to figure out how to somehow
encode a message to clear data plane learning into ESADI. In fact,
RBridges that are not participating in ESADI will discard ESADI
messages. But if an RBridge is participating in ESADI for VLAN V1 for
example, then very likely it will disable data plane learning for V1,
since ESADI can replace data plane learning. So, it is not
particularly useful to provide a way to clear data plane learning for
ESADI participants that probably don't have any data plane learning
anyway.

It seems more reasonable to implement an address learning flush
message as an RBridge Channel protocol. True, an RBridge might not
implement the RBridge Channel facility or might implement it but not
implement such an address flush message. But it might not implement
ESADI either. Things still work in such a case as data plane address
learning will eventually time out. But traffic may get black holed
(sent to the wrong RBridge) temporarily, a problem that can be fixed
by the address flush facility.

I do not necessarily support all the details in the current draft. I'm
sure it can be improved. But I do that that using RBridge Channel
messages is the way to go.

Thanks,
Donald

---

From: honjun.zhai <[email-masked]>
Date: Mon, 15 Feb 2016 16:40:52 +0800 (CST)

<p>Hi Donald,</p>
<p></p>
<p>&gt; So, it is not particularly useful to provide a way to clear data plane learning for</p>
<p>&gt; ESADI participants that probably don't have any data plane learning anyway.</p>
<p></p>
<p>Agree.&nbsp; I also think the MAC addresses advertised by ESADI participants should </p>
<p>be flushed by ESADI messages rather than the RBridge Channel messages.<br /><br />&gt; I do not necessarily support all the details in the current draft. I'm sure it can be </p>
<p>&gt; improved. But I do that that using RBridge Channel messages is the way to go.</p>
<p></p>
<p>Agree.<br /><br /></p>
<p>Thanks,<br /></p>
<p>Hongjun Zhai</p>
<p>------------------------------------------------------------------------------</p>
<p>Jiangning&nbsp;District, Nanjing City, People's Republic of China</p>
<p>Mobile: +086-025-5287-7345</p>
<p>Email: <a href="mailto:[email-masked]">[email-masked]</a></p>
<p>------------------------------------------------------------------------------</p>
<p><br /></p>
<hr />
<p><br /><br />于 2016-02-15 12:25:38,Donald Eastlake&lt;[email-masked]&gt;写道:</p>
<blockquote style="PADDING-LEFT: 1ex; BORDER-LEFT: #ccc 1px solid; MARGIN: 0px 0px 0px 0.8ex"><pre>These are my thoughts on this question:
The goal of the MAC address flush facility is to be able to clear MAC
addresses that have been learned from the data plane.
For example, if an edge RBridge RB1 knows that an end station with MAC
address ES1 in VLAN V1 is no longer attached to any of its ports, it
would want to clear any learning in the TRILL campus that associates
ES1&amp;V1 with RB1.
Another case would be if RB2 has a port to a link with a bunch of end
stations in V1 but RB2 losses appointed forwarder for V1 on that port.
RB2 would then want to clear all learning at other RBridges in the
campus associating any of the MAC addresses out that port with RB2.
(This is actually only useful if RB2 is still appointed forwarder for
V1 out some other port. If RB2 were no longer appointed forwarder for
V1 for any port, then RB2 would use the core IS-IS instance to
withdraw the Interested VLANs sub-TLV by which RB2 declared interest
in V1. All the other RBridges in the campus can see this withdrawal
and would know to clear any V1 MAC address learning for RB2, as
specified in RFC 6325 Section 4.8.3, next to last paragraph on page
70.)
ESADI uses a mechanism patterned after the IS-IS reliable link state
flooding mechanism. The ESADI idea is that, if RB3 is using ESADI for
V1, RB3 advertises all the locally attached MAC addresses RB3 has
learned about (through the receipt of native frames or whatever) in V1
through ESADI. All other RBridges in the campus that are participating
in ESADI will reliably learn that those MAC addresses are associated
with RB3 if those other RBridges are participating in ESADI for V1.
RB3 can easily and reliably withdraw one, several, or all of these MAC
addresses by sending updated link state information through the ESADI
instance using the already existing ESADI mechanisms.
The current proposal is to use an RBridge Channel message to clear out
MAC addresses learned from the data plane.
RBridge Channel messages and ESADI messages have a number of
similarities. Both use packets structured as TRILL Data packets with
an Inner.MacDA of All-Egress-RBridges (this MAC address was formerly
called All-ESADI-RBridges). The use of this MAC address as the
Inner.MacDA assures that the packets will not be forwarded to an end
station. The difference is that after the inner MAC addresses, an
ESADI packet has the L2-IS-IS Ethertype and the rest of the frame is
structured as an IS-IS PDU, while an RBridge Channel packet has the
RBridge-Channel Ethertype and the rest of the frame is structured as
specified in RFC 7178 to provide a general extensible format for typed
messages between RBridges. (There is also a way to send RBridge
Channel messages, called native RBridge Channel messages, to end
stations but that is not what would be used here.)
To me, it does not make sense to try to figure out how to somehow
encode a message to clear data plane learning into ESADI. In fact,
RBridges that are not participating in ESADI will discard ESADI
messages. But if an RBridge is participating in ESADI for VLAN V1 for
example, then very likely it will disable data plane learning for V1,
since ESADI can replace data plane learning. So, it is not
particularly useful to provide a way to clear data plane learning for
ESADI participants that probably don't have any data plane learning
anyway.
It seems more reasonable to implement an address learning flush
message as an RBridge Channel protocol. True, an RBridge might not
implement the RBridge Channel facility or might implement it but not
implement such an address flush message. But it might not implement
ESADI either. Things still work in such a case as data plane address
learning will eventually time out. But traffic may get black holed
(sent to the wrong RBridge) temporarily, a problem that can be fixed
by the address flush facility.
I do not necessarily support all the details in the current draft. I'm
sure it can be improved. But I do that that using RBridge Channel
messages is the way to go.
Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]
&gt; ________________________________________
&gt; From: Donald Eastlake [[email-masked]]
&gt; Sent: Friday, February 12, 2016 0:59
&gt; To: [email-masked]
&gt; Subject: [trill] RBridge Channel versus ESADI for MAC address flush
&gt;
&gt; At the last TRILL WG meeting in Yokohama, towards the end of the
&gt; second session, there was a discussion of the best mechanism to
&gt; implement a remote RBridge MAC address flush mechanism.
&gt; draft-hao-trill-address-flush proposes to use RBridge Channel messages
&gt; [RFC7178] but some people suggest the use of ESADI messages [RFC7357].
&gt; There was no resolution of this at the meeting so the Chair indicated
&gt; that it should be taken to the mailing list. This message is to start
&gt; the discussion. I'll post separately with my personal opinion.
&gt;
&gt; Thanks,
&gt; Donald
&gt; =============================
&gt;  Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
&gt;  155 Beaver Street, Milford, MA 01757 USA
&gt;  [email-masked]
_______________________________________________
trill mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/trill
</pre></blockquote>

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 16 Feb 2016 07:22:04 +0000

Hi Donald,

Thanks for the explanation of the subtle reason why an RBridge Channel is needed for MAC flush. Please find my comments in-line.

> -----Original Message-----
> From: trill [mailto:[email-masked]] On Behalf Of Donald Eastlake
> Sent: Monday, February 15, 2016 12:26 PM
> To: [email-masked]
> Subject: Re: [trill] RBridge Channel versus ESADI for MAC address flush
> 
> The goal of the MAC address flush facility is to be able to clear MAC addresses
> that have been learned from the data plane.
> 
> address ES1 in VLAN V1 is no longer attached to any of its ports, it would want
> to clear any learning in the TRILL campus that associates
> ES1&V1 with RB1.
> 
> Another case would be if RB2 has a port to a link with a bunch of end stations in
> V1 but RB2 losses appointed forwarder for V1 on that port.
> RB2 would then want to clear all learning at other RBridges in the campus
> associating any of the MAC addresses out that port with RB2.
> V1 for any port, then RB2 would use the core IS-IS instance to withdraw the
> Interested VLANs sub-TLV by which RB2 declared interest in V1. All the other
> RBridges in the campus can see this withdrawal and would know to clear any
> V1 MAC address learning for RB2, as specified in RFC 6325 Section 4.8.3, next
> to last paragraph on page
> 70.)
> 
> ESADI uses a mechanism patterned after the IS-IS reliable link state flooding
> mechanism. The ESADI idea is that, if RB3 is using ESADI for V1, RB3 advertises
> all the locally attached MAC addresses RB3 has learned about (through the
> receipt of native frames or whatever) in V1 through ESADI. All other RBridges in
> the campus that are participating in ESADI will reliably learn that those MAC
> addresses are associated with RB3 if those other RBridges are participating in
> ESADI for V1.
> The current proposal is to use an RBridge Channel message to clear out MAC
> addresses learned from the data plane.
> 
> RBridge Channel messages and ESADI messages have a number of similarities.
> Both use packets structured as TRILL Data packets with an Inner.MacDA of
> All-Egress-RBridges (this MAC address was formerly called All-ESADI-RBridges).
> The use of this MAC address as the Inner.MacDA assures that the packets will
> not be forwarded to an end station. The difference is that after the inner MAC
> addresses, an ESADI packet has the L2-IS-IS Ethertype and the rest of the
> frame is structured as an IS-IS PDU, while an RBridge Channel packet has the
> RBridge-Channel Ethertype and the rest of the frame is structured as specified
> in RFC 7178 to provide a general extensible format for typed messages
> between RBridges. (There is also a way to send RBridge Channel messages,
> called “native RBridge Channel messages”, to end stations but that is not
> what would be used here.)
> 
> To me, it does not make sense to try to figure out how to somehow encode a
> message to clear data plane learning into ESADI. In fact, RBridges that are not
> participating in ESADI will discard ESADI messages. But if an RBridge is
> participating in ESADI for VLAN V1 for example, then very likely it will disable
> data plane learning for V1, 

Here, I am thinking about the granularity of "disabling data-plane learning". Should it be per Data Label (VLAN or Fine-Grained Label) or per Data Label plus the originating RBridge? I am afraid this RBridge cannot simple disable data-plane learning for the whole V1. It's probably that some RBridge in V1 are participating ESADI while others are not participating. For those not participating, the RBridge cannot disable the data-plane learning.

It seems your final judgment still holds for the fine-grained "disabling data-plane learning" (i.e., per Data Label + originating RBridge). In other words, using already existing ESADI mechanism, the originating RBridge may well withdraw its MAC addresses by sending updated link state information through the ESADI instance. However, please consider the following case: 

Suppose both RB1, RB2 are communicating with a remote RBridge, say RBx, in V1. RB1 and RB2 are participating ESADI while RB2 is not. So, RBx has to retain data-plane learning for RB2. Now, RB1 wants to flush the MAC addresses attached to RB2. This case could be meaningful when RB1 speaks for other RBridges (e.g., RB1 acts as a reflector or central controller). For this case, an add-on flush function of ESADI is desirable.

> since ESADI can replace data plane learning. So, it is
> not particularly useful to provide a way to clear data plane learning for ESADI
> participants that probably don't have any data plane learning anyway.
> 
> It seems more reasonable to implement an address learning flush message as
> an RBridge Channel protocol. True, an RBridge might not implement the
> RBridge Channel facility or might implement it but not implement such an
> address flush message. But it might not implement ESADI either. Things still
> work in such a case as data plane address learning will eventually time out. But
> traffic may get black holed (sent to the wrong RBridge) temporarily, a problem
> that can be fixed by the address flush facility.
> 
> I do not necessarily support all the details in the current draft. I'm sure it can
> be improved. But I do that that using RBridge Channel messages is the way to
> go.

Thanks,
Mingui

>  155 Beaver Street, Milford, MA 01757 USA  [email-masked]
> 
> >  155 Beaver Street, Milford, MA 01757 USA  [email-masked]
>

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 17 Feb 2016 23:04:32 -0500

Hi Mingui,

> Here, I am thinking about the granularity of "disabling data-plane learning". Should it be per Data Label (VLAN or Fine-Grained Label) or per Data Label plus the originating RBridge? I am afraid th…

Well, the currently required facility is just per VLAN. See Section
5.4 RFC 6325.

I would think that in most real world situations, the network manager
would either have all the RBridges that had end stations in VLAN-x
either use ESADI or use data plane learning. However, if there was a
mixed scenario, then it seems like one way to handle this within the
TRILL protocol is for all the RBridges to have data plane learning
turned on for VLAN-x but to treat what is learned from the data plane
as having lower confidence that what is received from ESADI.

> Suppose both RB1, RB2 are communicating with a remote RBridge, say RBx, in V1. RB1 and RB2[[I think you mean RBx here]] are participating ESADI while RB2 is not. So, RBx has to retain data-plane learning for RB2. Now, RB1 wants to flush the MAC addresses attached to RB2. This case could be meaningful when RB1 speaks for other RBridges (e.g., RB1 acts as a reflector or central controller). For this case, an add-on flush function of ESADI is desirable.

See double parenthesis above. You mean that RBx should do data plane
learning for TRILL Data packet is egresses that were ingressed by RB2.
It seems odd to me that you way that RB1 wants to flush an address
that is/was locally attached to RB2. I'm not sure that RB1 should be
able to do that. RBridges, it seems to me, have better and more
immediate knowledge of locally attached end station MACs than they do
for remote MAcs. So I think RB1 should only flush things that other
RBridges have learned due to frames ingressed by RB1. RB2 is the
RBridge that knows best about end stations attached to RB2 so I think
RB2 should be the one originating flushes for stations that were
attached to RB2.

Thanks,
Donald

---

From: Mingui Zhang <[email-masked]>
Date: Thu, 18 Feb 2016 07:51:47 +0000

Hi Donald,

Please see my comments in-line below.

> From: Donald Eastlake [mailto:[email-masked]]
> Sent: Thursday, February 18, 2016 12:05 PM
> To: Mingui Zhang
> Cc: [email-masked]
> > Thanks for the explanation of the subtle reason why an RBridge Channel is
> needed for MAC flush. Please find my comments in-line.
> >
> >> For example, if an edge RBridge RB1 knows that an end station with
> >> MAC address ES1 in VLAN V1 is no longer attached to any of its ports,
> >> it would want to clear any learning in the TRILL campus that
> >> associates
> >> ES1&V1 with RB1.
> >>
> >> stations in
> >> and would know to clear any
> >> V1 MAC address learning for RB2, as specified in RFC 6325 Section
> >> 4.8.3, next to last paragraph on page
> >> 70.)
> >>
> >> learned about (through the receipt of native frames or whatever) in
> >> V1 through ESADI. All other RBridges in the campus that are
> >> participating in ESADI will reliably learn that those MAC addresses
> >> are associated with RB3 if those other RBridges are participating in ESADI
> for V1.
> >> RB3 can easily and reliably withdraw one, several, or all of these
> >> MAC addresses by sending updated link state information through the
> >> ESADI instance using the already existing ESADI mechanisms.
> >>
> >> The current proposal is to use an RBridge Channel message to clear
> >> out MAC addresses learned from the data plane.
> >>
> similarities.
> >> Both use packets structured as TRILL Data packets with an Inner.MacDA
> >> of All-Egress-RBridges (this MAC address was formerly called
> All-ESADI-RBridges).
> >> The use of this MAC address as the Inner.MacDA assures that the
> >> packets will not be forwarded to an end station. The difference is
> >> that after the inner MAC addresses, an ESADI packet has the L2-IS-IS
> >> Ethertype and the rest of the frame is structured as an IS-IS PDU,
> >> while an RBridge Channel packet has the RBridge-Channel Ethertype and
> >> the rest of the frame is structured as specified in RFC 7178 to
> >> provide a general extensible format for typed messages between
> >> RBridges. (There is also a way to send RBridge Channel messages,
> >> called “native RBridge Channel messages”, to end stations but that is
> >> not what would be used here.)
> >>
> > Here, I am thinking about the granularity of "disabling data-plane learning".
>> Should it be per Data Label (VLAN or Fine-Grained Label) or per Data Label plus
>> the originating RBridge? I am afraid this RBridge cannot simple disable
>> data-plane learning for the whole V1. It's probably that some RBridge in V1 are
> >participating ESADI while others are not participating. For those not
> >participating, the RBridge cannot disable the data-plane learning.
>> 
> I would think that in most real world situations, the network manager would
> either have all the RBridges that had end stations in VLAN-x either use ESADI or
> use data plane learning. However, if there was a mixed scenario, then it seems
> like one way to handle this within the TRILL protocol is for all the RBridges to
> have data plane learning turned on for VLAN-x but to treat what is learned from
> the data plane as having lower confidence that what is received from ESADI.

Yes. Per VLAN disabling is neat. We should keep it unchanged.

> 
> > It seems your final judgment still holds for the fine-grained "disabling
>> data-plane learning" (i.e., per Data Label + originating RBridge). In other words,
>> using already existing ESADI mechanism, the originating RBridge may well
>> withdraw its MAC addresses by sending updated link state information through
>> the ESADI instance. However, please consider the following case:
>> 
> > Suppose both RB1, RB2 are communicating with a remote RBridge, say RBx,
>> in V1. RB1 and RB2[[I think you mean RBx here]] are participating ESADI while
>> RB2 is not. So, RBx has to retain data-plane learning for RB2. Now, RB1 wants
>> to flush the MAC addresses attached to RB2. This case could be meaningful
>> when RB1 speaks for other RBridges (e.g., RB1 acts as a reflector or central
>> controller). For this case, an add-on flush function of ESADI is desirable.
>> 
> See double parenthesis above. 

Yes, I meant RBx.

> You mean that RBx should do data plane
> It seems odd to me that you way that RB1 wants to flush an address that
> is/was locally attached to RB2. I'm not sure that RB1 should be able to do that.
> RBridges, it seems to me, have better and more immediate knowledge of
> locally attached end station MACs than they do for remote MAcs. So I think
> RB1 should only flush things that other RBridges have learned due to frames
> ingressed by RB1. RB2 is the RBridge that knows best about end stations
> attached to RB2 so I think
> RB2 should be the one originating flushes for stations that were attached to
> RB2.

An RBridge acting as the proxy for a Directory server is the case that one RBridge manipulates MAC addresses owned by another. However, all clients are supposed to participate ESADI in order to use the Directory service. Then, ESADI-flush still seems unnecessary.

So, before we get meaningful use cases, I tend to agree with you that we should keep this facility (one flushes MAC addresses of another) as disabled.

Thanks,
Mingui

> >> ESADI participants that probably don't have any data plane learning anyway.
> >>
> >> (sent to the wrong RBridge) temporarily, a problem that can be fixed by the
> address flush facility.
> >>
> >> I do not necessarily support all the details in the current draft.
> >> I'm sure it can be improved. But I do that that using RBridge Channel
> >> > draft-hao-trill-address-flush proposes to use RBridge Channel
> >> > messages [RFC7178] but some people suggest the use of ESADI messages
> [RFC7357].
> >> > There was no resolution of this at the meeting so the Chair
> >> > indicated that it should be taken to the mailing list. This message
> >> > is to start the discussion. I'll post separately with my personal opinion.
> >> >
> >> > Thanks,
> >> > Donald