Subject: [rbridge] Final outcome of outer VLAN tags onRBridge-RBridgepackets?

From: Radia.Perlman at sun.com ([email-masked])
Date: Mon, 29 Oct 2007 19:46:08 -0700

1) I suggested a phone call sometime this week, but it was buried in the end of my message. I'd particularly hope that (at least) you (Silvano), Dinesh, Anoop, me, and Don Eastlake attend, but nobody would be excluded. Send me email with all
your availability for this week (and time zones) and I'll try to pick a time.

2) As for the updated picture, and talking through how I think
it all would work -- I think all the three clouds are completely independent. So I think
we could just as well be talking about a single layer 2 cloud with a bunch of RBridges
attached. But I'll use your picture.

*********************

First: let's talk through what happens with the "use VLAN 1, no configuration" strategy:

The rule is that on every layer 2 cloud (R, G, and L in your picture), bridges inside each cloud must allow VLAN-1-tagged traffic to pass. That would mean that
VLAN 1 would not partitioned on that cloud. Also every RBridge port must be configured to allow
sending and receiving of VLAN 1 tagged traffic.

The way I read your picture (http://www.ip6.com/example.ppt): As configured, RB3, for
instance, is only able to send/receive frames tagged
with VLANs 2 and 3 on its port Y3, and send/receive VLANs 7 and 8 on its port X3.
(Is that a correct interpretation of your picture?)
So, assuming my interpretation is correct, if you started with an installation configured that
way, then ports X3, Y3, X4, Y4, X1, and X2 would have to be reconfigured to allow use of VLAN 1, and any
bridges in between these ports (e.g., inside cloud G or cloud R), if they were configured to partition VLAN 1,
would have to be configured to allow traffic marked with VLAN 1 to be forwarded.

Consequences of not doing the reconfiguration as just stated are:
If an RBridge R *cannot* send/receive on VLAN 1 on a port, R will not send any Hellos (which might be fine -- perhaps
there are no other RBridges on the link -- just endnodes, but my preference would
be to discourage and possibly
disallow having an RBridge port with VLAN 1 turned off). If VLAN 1 is partitioned on
the cloud, the same consequence happens. R will not
form any RBridge-RBridge adjacencies on that port, so no encapsulated traffic will be forwarded across that cloud. With
the safeguards specified in my earlier emails, R will be conservative about encapsulating/decapsulating data traffic
to/from that link, so that there won't be loops.

If VLAN 1 is not partitioned and all RBridges can send/receive VLAN 1, then everything works very smoothly. Other than
having had to reconfigure things, I don't see what functionality the customer is giving up by allowing RBridges
to talk on VLAN 1.

************************
Now let's talk about the "single VLAN, but we can configure which VLAN it is" strategy:

The customer *could* reconfigure things to allow VLAN 1 for RBridge-RBridge communication, and then
everything would work as above. But if for some reason they wanted to only allow VLANs 7 and 8 in cloud G,
and 2 and 3 in cloud R, then they could do the following:

Ports Y1 and Y2 will not require any configuration -- they already support VLAN 1 (the default).

Ports X1, X2, X3, and X4 would be configured to send Hellos on either VLAN 7 or 8. I'd recommend, if the VLAN
for Hellos is configurable, that you *accept* Hellos on any of the VLANs for which you can receive traffic. Whoever
gets elected DRB (say RB1) specifies, in its Hello, "use VLAN 8", and then X2, X3, and X4, as long as they continue
to receive Hellos from RB1, send their Hellos on VLAN 8. Furthermore, LSPs on cloud G are transmitted tagged with
outer VLAN 8. Encapsulated data traffic being forwarded across cloud G would also have outer VLAN tag of 8.

If RB2 were DRB instead of RB1, and RB2 was configured to transmit on VLAN 7, then it would tell all the RBridges to tag
their RBridge-RBridge traffic with 7. If there are different priorities, say RB1 is DRB for VLAN 7, and wants to
tag RBridge-RBridge traffic with 7,  and RB2 is DRB for VLAN 8, and wants to tag RBridge-RBridge traffic with "8", then you wind up with two DRBs (one for VLAN 7, and one for VLAN 8). If RB1 tells everyone "use VLAN 7" and RB2
tells everyone "use VLAN 8", then you'll get twice as many Hellos on the link as you'd need.
So perhaps as an optimization, the RBridges could notice this and if you are DRB for some
of the VLANs on a cloud, and some other DRB (for a different set of VLANs on the same cloud) is
telling the RBridges to send Hellos on a different, lower numbered VLAN, then I think it would
be reasonable to agree to using that VLAN.

Something that's perhaps a bit subtle, but does indeed work, is to realize what happens if you have
multiple DRBs as a consequence of load splitting the VLANs. What happens is:

a) you'll have multiple DRBs, and each DRB will create a different pseudonode for that cloud

b) the pseudonodes will have the same root bridge in the LSP, but with a different, nonoverlapping
set of VLANs

c) a multicast packet will not get sent twice on the cloud, because the multicast data packet
will be (inner) tagged with some VLAN tag, and will only get delivered to the pseudonode that
includes that VLAN tag.

Anyway, I'm optimistic at this point that we will converge on either "VLAN 1--nonconfigurable"
or "single VLAN per layer 2 cloud, but which VLAN is configurable", but that if there really
is a problem with this, I think a phone call would be the best thing at this point.

Thanks,

Radia

----- Original Message -----
From: Silvano Gai <sgai at nuovasystems.com>
Date: Monday, October 29, 2007 3:41 pm
Subject: RE: RE: [rbridge] Final outcome of outer VLAN tags onRBridge-RBridgepackets?

> Radia,
> 
> I have redrawn
> http://www.ip6.com/example.ppt
> 
> Using more terminology according to your suggestion.
> 
> Can you rewrite this email using the terminology of the updated
> > http://www.ip6.com/example.ppt
> 
> Thank You
> 
> -- Silvano
> 
> > -----Original Message-----
> > From: Radia.Perlman at sun.com [Radia.Perlman at sun.com]
> > Sent: Monday, October 29, 2007 1:14 PM
> > To: Silvano Gai
> > Cc: Anoop Ghanwani; Developing a hybrid router/bridge.
> > Subject: Re: RE: [rbridge] Final outcome of outer VLAN tags 
> onRBridge-
> > RBridgepackets?
> > 
> > Silvano,
> > Based on your message, I think a lot of the controversy here is 
> simply> miscommunication, which is actually great news.
> > The VLAN number we have been talking about is solely a local 
> matter on
> a
> > single layer 2 cloud. It
> > is only the outer VLAN number used for RBridge-RBridge 
> communication.> 
> > So looking at your picture, note that we have three separate 
> layer 2
> > clouds. Let's add "port" numbers,
> > so that for each of the RBridges, their top port will be called "x"
> and
> > their bottom port would
> > be called "y".
> > 
> > Let's call the clouds "G" for the top one (because it is green). And
> "L"
> > for the lower left, and "R" for the
> > lower right.
> > 
> > if we went with the "VLAN 1 unconfigurable" strategy, then we'd have
> to
> > tell that customer to open up VLAN 1 on the G cloud and R cloud (and
> I'm
> > sure I'm using the wrong word,
> > but by "open up", I mean make it so that the RBridges are capable of
> > sending and receiving traffic
> > on VLAN 1, and that VLAN 1 is not partitioned in the green 
> cloud). It
> > wouldn't cause
> > the VLANs to get merged or anything like that. It just means that 
> VLAN1
> > would not be allowed to
> > be partitioned on any of those 3 clouds, and that the RBridges 
> must be
> > able to use VLAN 1
> > to communicate with each other. And it means that all the traffic 
> for> RBridge-RBridge communication
> > (which consists of IS-IS Hellos, LSP forwarding, and encapsulated
> > data packet forwarding across the cloud between RBridges)
> > would be sent with an outer VLAN tag of 1.
> > 
> > If we instead went with
> > the "single VLAN, but configurable", then all the RBridges would be
> > configured to use, say, VLAN 7
> > for RBridge-RBridge communication on their "port x" which 
> connects to
> > cloud G.
> > RB1 and RB2's "port y" would not need to be configured, since 
> VLAN 1
> would
> > already work on the cloud
> > attached to that port. RB3 and RB4 would need to be configured to 
> use,> say, VLAN 3, on their "y" port (the
> > one to cloud R).
> > 
> > Would some number of people like to have a phone call early this 
> weekand
> > resolve it? I'd hope that
> > the people would include at least you (Silvano), Dinesh, me, Anoop,
> but I
> > don't mean to exclude anyone.
> > 
> > Radia
> > 
> > 
> > 
> > ----- Original Message -----
> > From: Silvano Gai <sgai at nuovasystems.com>
> > Date: Sunday, October 28, 2007 4:49 am
> > Subject: RE: [rbridge] Final outcome of outer VLAN tags onRBridge-
> > RBridgepackets?
> > 
> > >
> > > Radia,
> > >
> > > Can you explain me if your solution supports the example that I 
> have> > drawn in PowerPoint at:
> > > http://www.ip6.com/example.ppt
> > > ?
> > >
> > > Thank You
> > >
> > > Answering your email:
> > >
> > > I have seen customers deploying VLANs in a variety of ways.
> > >
> > > Some customers have VLAN 1 end-to-end and they use it for control
> and
> > > data traffic. Some customers use VLAN 1 only for control traffic.
> Some
> > > customers wants all VLANs (including VLAN 1) being local (not
> > > end-to-end). Some customers prefer to move some control 
> protocol to
> > > another VLAN. I have seen multiple different approaches.
> > >
> > > You ask for rational, "tangible reasons". Each customer has its 
> own> > reasons that don't match the ones of other customers. There 
> are no
> > > universal tangible reasons, you just need to realize that IEEE
> 802.1Q
> > > allows VLANs to be deployed in many different way.  Any solution
> that
> > > poses restrictions is going to be OK for some customers and KO for
> > > others.
> > >
> > > Over the years I have learned to live with the fact that customers
> > > havedifferent deployment approaches that are perfectly valid
> > > because allowed
> > > by IEEE 802.1Q and by the products.
> > >
> > > In TRILL we often speak about plug-and-play. If we consider
> > > plug-and-play, VLAN 1 is the only one enabled by default and
> therefore
> > > it seems a reasonable default solution.
> > >
> > > While this is a valid concept in the consumer space, it is
> absolutely
> > > not valid in Enterprise/Datacenter. Enterprise/Datacenter 
> customers> > request that all the new boxes are shipped with all 
> the ports
> > > disabled,because they realized that the switches will be connected
> > > in complex
> > > configurations in which plug-and-play is very dangerous.
> > >
> > > I expect the same to be true for RBridges, where customers will
> > > carefully design, test in a separate environment and bring-up a
> > > configuration that will match the existing VLAN deployment they
> have.
> > >
> > > Inline for you specific questions:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Radia Perlman [Radia.Perlman at sun.com]
> > > > Sent: Saturday, October 27, 2007 8:59 PM
> > > > To: Silvano Gai
> > > > Cc: Anoop Ghanwani; Developing a hybrid router/bridge.
> > > > Subject: Re: [rbridge] Final outcome of outer VLAN tags 
> onRBridge-
> > > > RBridgepackets?
> > > >
> > > > I'm anxious to close on this issue, but
> > > > Silvano...you always speak in such absolutes: words like
> > > > "unrealistic" without giving tangible reasons what the
> > > > problem is. Again, everything is engineering tradeoffs, and we
> > > have to
> > > > understand what the specific
> > > > pros and cons are (not phrases like
> > > > "won't work" "unrealistic" "not flexible enough", but instead
> "won't
> > > allow
> > > > this *particular* scenario")
> > > >
> > > > Question: Is the problem picking specifically VLAN 1 as
> > > > our chosen single VLAN number? If we picked a different
> > > (unconfigurable)
> > > > VLAN number would that be OK?
> > >
> > > I think that VLAN 1 is probably our best default choice. I don't
> > > objectto have VLAN 1 to be the default VLAN.
> > >
> > > If so, what number? If you are arguing that
> > > > it's OK to require one VLAN not to be partitioned, and that all
> > > > RBridge-RBridge
> > > > communication be done over that VLAN, then I really don't
> > > understand.
> > > I am not sure what you are asking. If you are speaking of the
> backbone
> > > between RBridges, the outer VLAN tag for sure may not be the same
> > > end-to-end.
> > >
> > > A department of a large corporation may want to deploy RBridges 
> in 5
> > > different sites. The VLAN they have available from site_1 to 
> site_2> > maybe VLAN 5, form site_1 to site_3 VLAN 7, from site_3 
> to site_4
> > > VLAN 4.
> > >
> > > If you want all this VLANs to be a single VLAN end-to-end, 
> there is
> a
> > > significant change that needs to happen in the communication
> > > infrastructure that may be considered difficult or impossible.
> > >
> > > If instead you are discussing the access to the RBridge clouds, I
> > > thinkit is reasonable to require that the RBridges that perform
> TRILL
> > > encapsulation for an Ethernet Cloud have a common VLAN. VLAN1 may
> > > be the
> > > default, but I think that the value should be programmable.
> > >
> > >
> > > > If that were the case, why can't we (TRILL) put a stake in the
> > > ground> and say "RBridges will
> > > > use VLAN 1", and if the customer wanted to use VLAN 1 for
> something
> > > > else, but was willing
> > > > to configure RBridges to use, say, VLAN 57, why can't that
> customer
> > > > renumber their own
> > > > VLANs so that they switch their uses of VLAN 57 and VLAN 1, and
> let
> > > the
> > > > rest of
> > > > the world have the simplicity of an unconfigurable choice of 
> VLAN1?
> > >
> > > I don't think this is acceptable for many customers. You are
> > > putting a
> > > stake in the ground that IEEE has never put.
> > >
> > > >
> > > > Are you saying that you don't want to require customers to have
> > > *any*> VLAN number that
> > > > isn't partitioned,
> > >
> > > I am not sure what you mean with "*any* VLAN number that isn't
> > > partitioned", are you speaking of inner VLAN or outer VLAN?
> > >
> > > In most installation the numbering spaces of the Backbone VLANs 
> and> > Access VLANs are disjoint. I don't think this is an issue 
> since the
> > > access VLANs appears in the inner VLAN tag, while the backbone 
> VLANs> > appears in the outer VLAN tag.
> > >
> > >
> > > and you want a solution that somehow finds all
> > > > possible ways of having
> > > > RBridges talk to each other using every possible VLAN?
> > >
> > > Again, when you are saying "RBridges talk to each other" are you
> > > speaking of the TRILL encapsulated frames or of the DRB 
> election on
> > > theaccess?
> > >
> > > Again see my example at:
> > > http://www.ip6.com/example.ppt
> > >
> > > -- Silvano
> > 
> 
>

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 30 Oct 2007 02:52:42 -0700

Radia,

inline

> From: Radia.Perlman at sun.com [mailto:Radia.Perlman at sun.com]
> Sent: Monday, October 29, 2007 7:46 PM
> To: Silvano Gai
> Subject: Re: RE: RE: [rbridge] Final outcome of outer VLAN tags
onRBridge-
> RBridgepackets?
> 
> 1) I suggested a phone call sometime this week, but it was buried in
the
> end of my message. I'd particularly hope that (at least) you
(Silvano),
> Dinesh, Anoop, me, and Don Eastlake attend, but nobody would be
excluded.
> Send me email with all
> your availability for this week (and time zones) and I'll try to pick
a
> time.
> we could just as well be talking about a single layer 2 cloud with a
bunch
> of RBridges

A single cloud or three separate clouds are not the same thing. 

Let's assume that cloud L is located in Manhattan (main data center),
cloud R is located in New Jersey (backup data center) and Cloud G is
some form of public/private fiber connectivity with Ethernet switches
connected at the ends. Cloud G is run by central IT. Let's also assume
that the finance department has been assigned two VLANs (7 and 8) on the
Green Cloud and can use only that two VLANs. Originally the finance
department was using routers (instead of RBridges in
http://www.ip6.com/example.ppt), but now it is thinking to replace them
with RBridges. With routers finance needs at least two separate IP
subnets on VLAN 3, one on Cloud L, one on Cloud R. Replacing the routers
with RBridges, finance can use the same IP subnet for VLAN 3 in Cloud L
and Cloud R.

> The rule is that on every layer 2 cloud (R, G, and L in your picture),
> bridges inside each cloud must allow VLAN-1-tagged traffic to pass. 

This is impossible in our scenario. Finance has only VLAN 7 and 8 on the
Green Cloud. Finance is not the only department that wants to deploy
RBridges, also Marketing and Sales want to do the same. The requests of
Finance, Marketing, and Sales to own VLAN 1 on the Green Cloud conflict.
There is a strong corporate policy that the departments cannot share
VLANs (and believe me, on Wall Street these rules are enforced).

On Cloud R also we have an issue; Finance does not own VLAN 1.

Therefore allowing VLAN 1 traffic to pass on Cloud G and R is impossible
from a deployment perspective.

That
> would mean that
> VLAN 1 would not partitioned on that cloud. Also every RBridge port
must
> be configured to allow
> with VLANs 2 and 3 on its port Y3, and send/receive VLANs 7 and 8 on
its
> port X3.

YES

> way, then ports X3, Y3, X4, Y4, X1, and X2 would have to be
reconfigured
> to allow use of VLAN 1, and any
> bridges in between these ports (e.g., inside cloud G or cloud R), if
they
> were configured to partition VLAN 1,

In this example, this is impossible, as I explained you above.

> If an RBridge R *cannot* send/receive on VLAN 1 on a port, R will not
send
> any Hellos (which might be fine -- perhaps
> form any RBridge-RBridge adjacencies on that port, so no encapsulated
> traffic will be forwarded across that cloud. With
> the safeguards specified in my earlier emails, R will be conservative
> about encapsulating/decapsulating data traffic
> If VLAN 1 is not partitioned and all RBridges can send/receive VLAN 1,
> then everything works very smoothly. Other than
> having had to reconfigure things, I don't see what functionality the
> customer is giving up by allowing RBridges
> to talk on VLAN 1.

You are assuming that a customer "owns" the whole network. This is often
not the case. Departments own pieces of the network. This often is a
complex and painful process that results from merger and acquisition and
from complex security policy inside the corporation.

> 
> ************************
> Now let's talk about the "single VLAN, but we can configure which VLAN
it
> is" strategy:
> everything would work as above. But if for some reason they wanted to
only
> allow VLANs 7 and 8 in cloud G,
> Ports Y1 and Y2 will not require any configuration -- they already
support
> VLAN 1 (the default).

OK

> Ports X1, X2, X3, and X4 would be configured to send Hellos on either
VLAN
> 7 or 8. 

OK

I'd recommend, if the VLAN
> for Hellos is configurable, that you *accept* Hellos on any of the
VLANs
> for which you can receive traffic. 

This is a detail we can work later.

Whoever
> gets elected DRB (say RB1) specifies, in its Hello, "use VLAN 8", and
then
> X2, X3, and X4, as long as they continue
> to receive Hellos from RB1, send their Hellos on VLAN 8. Furthermore,
LSPs
> on cloud G are transmitted tagged with
> If RB2 were DRB instead of RB1, and RB2 was configured to transmit on
VLAN
> 7, then it would tell all the RBridges to tag
> their RBridge-RBridge traffic with 7. If there are different
priorities,
> say RB1 is DRB for VLAN 7, and wants to
> tag RBridge-RBridge traffic with 7,  and RB2 is DRB for VLAN 8, and
wants
> to tag RBridge-RBridge traffic with "8", then you wind up with two
DRBs
> (one for VLAN 7, and one for VLAN 8). If RB1 tells everyone "use VLAN
7"
> and RB2
> tells everyone "use VLAN 8", then you'll get twice as many Hellos on
the
> link as you'd need.
> So perhaps as an optimization, the RBridges could notice this and if
you
> are DRB for some
> of the VLANs on a cloud, and some other DRB (for a different set of
VLANs
> on the same cloud) is
> telling the RBridges to send Hellos on a different, lower numbered
VLAN,
> then I think it would
> Something that's perhaps a bit subtle, but does indeed work, is to
realize
> what happens if you have
> multiple DRBs as a consequence of load splitting the VLANs. What
happens
> is:
> b) the pseudonodes will have the same root bridge in the LSP, but with
a
> different, nonoverlapping
> set of VLANs
> 
> c) a multicast packet will not get sent twice on the cloud, because
the
> multicast data packet
> will be (inner) tagged with some VLAN tag, and will only get delivered
to
> the pseudonode that
> includes that VLAN tag.
> 
> Anyway, I'm optimistic at this point that we will converge on either
"VLAN
> 1--nonconfigurable"
> or "single VLAN per layer 2 cloud, but which VLAN is configurable",
but
> that if there really
> is a problem with this, I think a phone call would be the best thing
at
> this point.

"VLAN 1--nonconfigurable" is a non-starter for me, I hope to have
explained why.

"single VLAN per layer 2 cloud" may be workable, but I need to see the
details of the algorithm to see if there are corner cases.

If we can agree to concentrate our focus on a "single VLAN per layer 2
cloud" we have made some progress.

-- Silvano

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 30 Oct 2007 04:26:22 -0700

Radia,

I have added few slides to the example and I suggest that we use it as a
possible case of study for TRILL (of course, not the only one).

Let's see if the solutions we have discussed work on this example.

The complete case of study is at:

http://www.ip6.com/acme-example-v1.ppt

-- Silvano

---

From: james.d.carlson at Sun.COM (James Carlson)
Date: Tue, 30 Oct 2007 17:37:26 -0400

> http://www.ip6.com/acme-example-v1.ppt

I have a few narrow questions that I think might help me understand
this picture a bit better.  (I probably have more questions once I
know the narrow answers.  ;-})

  1.  After the proposed fix, should VLAN 3 on Cloud L be bridged
      together with VLAN 3 on Cloud R, even though the backbone itself
      doesn't directly carry VLAN 3?  (I.e., are TRILL packets with
      outer VLAN ID 7 or 8 and with inner VLAN 3 present on the
      backbone?)

  2.  What happened to the routing that once went on?  It was
      previously possible to route between these VLANs, but in the new
      diagram, it's not.  Is routing between these networks
      unimportant, or does it take place elsewhere, or must RBridges
      both bridge and route in this scenario?

  3.  Suppose we were to configure VLAN 7 within Cloud L.  Would you
      expect H1 to be able to access nodes within the backbone Cloud G
      at that point?  (I'm trying to figure out whether the "X" and
      "Y" interfaces are of fundamentally different types or if the
      RBridges are just bridges.  I suspect they're different.)

-- 
James Carlson, Solaris Networking              <james.d.carlson at sun.com>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

---

From: jliu at Brocade.COM (Jian Liu)
Date: Tue, 30 Oct 2007 15:28:27 -0700

Silvano,

A couple of questions:

1. In the proposed solution, what is considered in the TRILL domain? It
seems TRILL domain includes RBridge 1-4 and backbone cloud G, correct?
2. Are there regular bridges or RBridges in backbone cloud G?

Regards,
Jian

-----Original Message-----
From: rbridge-bounces at postel.org [mailto:rbridge-bounces at postel.org] On
Behalf Of Silvano Gai
Sent: Tuesday, October 30, 2007 4:26 AM
To: Radia.Perlman at sun.com
Cc: Developing a hybrid router/bridge.
Subject: [rbridge] RBridge: a case of study

Radia,

-- Silvano

_______________________________________________
rbridge mailing list
rbridge at postel.org
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Tue, 30 Oct 2007 17:22:52 -0700

> cloud" we have made some progress.
I can live with "single VLAN, configurable, default=1". I'd recommend 
that regardless of
the VLAN you are configured to send a Hello on, you accept Hellos on any 
VLAN you
are capable of receiving traffic on, and that each DRB tells all the 
other RBridges on that cloud
what VLAN to send on, so that they all wind up sending RBridge traffic 
on the same VLAN.

I think the only complexity/overhead it adds over "VLAN 1, 
nonconfigurable is"

a) something configurable to document and present in the management 
interface (the VLAN on
which to send Hellos)
b) until a DRB is elected, there might be lots of RBridges transmitting 
on different VLANs because
each might be configured with a different VLAN
c) if there are multiple (say k)  DRBs (because for load splitting each 
of them is DRB for
a different (nonoverlapping) set
of VLANs), then there will be k times as many Hellos being sent
d) I think there might be a slightly higher probability that there will 
be misconfiguration such
as telling each RBridge a different set of VLANs to talk on, but the 
safeguards that I specified
in previous emails (listen to BPDUs, report root bridge in pseudonode 
LSP, don't decapsulate
from an ingress RBridge that you haven't seen all the LSPs from, wait 
until you've synchrornized
LSP databases with your neighbors when you first start up) will prevent 
loops in these cases, and
at worst orphan parts of the layer 2 cloud from the rest of the campus.

So...can everyone live with this? (Silvano -- as you said above,
you promised to think about whether there
were any corner cases to worry about. You also said you wanted to see 
"the whole algorithm
specified". I'll do that in a separate email.

Radia

---

From: anoop at brocade.com (Anoop Ghanwani)
Date: Tue, 30 Oct 2007 18:41:15 -0700

> From: rbridge-bounces at postel.org 
> [mailto:rbridge-bounces at postel.org] On Behalf Of Radia Perlman
> Sent: Tuesday, October 30, 2007 5:23 PM
> To: Silvano Gai
> Subject: Re: [rbridge] Final outcome of outer VLANtags 
> onRBridge-RBridgepackets?
> 
> Silvano Gai wrote:
> >
> I can live with "single VLAN, configurable, default=1". I'd 
> recommend that regardless of the VLAN you are configured to 
> send a Hello on, you accept Hellos on any VLAN you are 
> capable of receiving traffic on, and that each DRB tells all 
> the other RBridges on that cloud what VLAN to send on, so 
> that they all wind up sending RBridge traffic on the same VLAN.
> 
> a) something configurable to document and present in the 
> management interface (the VLAN on which to send Hellos)
> b) until a DRB is elected, there might be lots of RBridges 
> transmitting on different VLANs because each might be 
> configured with a different VLAN

The problem with this is that there may be RB's sending
hellos on a VLAN that other RB's don't see.  As a result
the adjacency creation may not be correct.  For example,
RB1 may send its hellos on VLAN 4, RB2 may send its
hellos on VLAN 5, neither RBridge is able to see the
other's hellos because of the way the bridged LAN is
configured.

I think the correct way to solve this is to say that
hellos will be sent on exactly one VLAN for any bridged
LAN - VLAN 1 would be default, but configuration is
allowed.  So in the LSP, along with things like root
bridge for the pseudonode, we advertise the "Rbridge
control VLAN" that is used.  If a conflict is detected
when an LSP is received (same root, different control
VLAN), then the RBridge with higher ID stop being a
DRB and reports a misconfiguration.

A misconfiguration could also be detected by configuring
all RBridges on the LAN to be in a maintenance association
using 802.1ag. (But that would be very configuration
intensive.  802.1ag would cause a trap to be generated
if all Rbridges don't see all the Rbridges they expect 
to see, or if they see extra ones that they haven't
been configured about.)

> c) if there are multiple (say k)  DRBs (because for load 
> splitting each of them is DRB for a different 
> (nonoverlapping) set of VLANs), then there will be k times as 
> many Hellos being sent
> d) I think there might be a slightly higher probability that 
> there will be misconfiguration such as telling each RBridge a 
> different set of VLANs to talk on, but the safeguards that I 
> specified in previous emails (listen to BPDUs, report root 
> bridge in pseudonode LSP, don't decapsulate from an ingress 
> RBridge that you haven't seen all the LSPs from, wait until 
> you've synchrornized LSP databases with your neighbors when 
> you first start up) will prevent loops in these cases, and at 
> worst orphan parts of the layer 2 cloud from the rest of the campus.
> 
> So...can everyone live with this? (Silvano -- as you said 
> above, you promised to think about whether there were any 
> corner cases to worry about. You also said you wanted to see 
> "the whole algorithm specified". I'll do that in a separate email.

---

From: anoop at brocade.com (Anoop Ghanwani)
Date: Tue, 30 Oct 2007 18:54:28 -0700

> [mailto:rbridge-bounces at postel.org] On Behalf Of James Carlson
> Sent: Tuesday, October 30, 2007 2:37 PM
> To: Silvano Gai
> Cc: Developing a hybrid router/bridge.; Radia.Perlman at Sun.COM
> Subject: Re: [rbridge] RBridge: a case of study
> 
> Silvano Gai writes:
> > Radia,
> > 
> > I have added few slides to the example and I suggest that 
> we use it as 
> > a possible case of study for TRILL (of course, not the only one).
> > 
> I have a few narrow questions that I think might help me 
> understand this picture a bit better.  (I probably have more 
> questions once I know the narrow answers.  ;-})

Let me take a shot at answering these.

>       outer VLAN ID 7 or 8 and with inner VLAN 3 present on the

Yes, that's pretty much what ends up happening.  
We have effectively extended the bridged LAN.

>       both bridge and route in this scenario?

Routing can continue to happen wherever it was happening.
The Rbridge by itself doesn't do routing, but there's no
reason why one cannot build a device that does bridging,
RBridging, and routing.

>       RBridges are just bridges.  I suspect they're different.)

This is where I think would help to introduce some
additional terminology in the spec.  Just like MPLS
LSRs have ingress, core, and egress functionality,
I think we should have similar functions for RBridging.
We could have access ports, trill ports and hybrid
ports.  From access ports, we only accept non-trill
encap'ed frames, from trill ports, we accept only
trill-encap'ed frames, and finally, from hybrid ports
we accept either.  Today we only allow hybrid ports
and perhaps that's OK as a spec.

Anyway, back to the specifics of your question: If you
did configure VLAN 7 within cloud L, then the RBridges
labeled "Router 1" and "Router 2" would have 2 of their
ports configured to be on VLAN 7 and would simply bridge
between those ports.

Anoop

---

From: zloh at fulcrummicro.com (Zhi-Hern Loh)
Date: Tue, 30 Oct 2007 19:53:58 -0700 (PDT)

> We have effectively extended the bridged LAN.

Anoop, I'm reading the RBridge protocol specification version 5 and it seems to me that your comment is inconsistent with the spec. In section, 5.2.2.3, the spec says that the outer VLAN tag of TRILL data frames has the same VID as the inner tag. Is this section going to be changed?

Futhermore, if we were to use different VIDs on outer VLAN tags then we would need to handle the multi-destination case where one would need to send a multi-destination frame per outer VID to communicate with the RBs in the distribution tree which are using different VIDs. Is this correct?

Thanks,
Hern
Fulcrum Microsystems

---

From: ddutt at cisco.com (Dinesh G Dutt)
Date: Tue, 30 Oct 2007 20:57:02 -0700

The spec is a trifle out of date and will be fixed once we all agree on 
some of the outstanding issues,

Dinesh
Zhi-Hern Loh wrote:
>>>> Let's see if the solutions we have discussed work on this
>>>>         
>> example.
>>     
>>>       together with VLAN 3 on Cloud R, even though the backbone
>>>       
>> itself
>>     

-- 
We make our world significant by the courage of our questions and by 
the depth of our answers.                               - Carl Sagan

---

From: ddutt at cisco.com (Dinesh G Dutt)
Date: Tue, 30 Oct 2007 21:01:32 -0700

Radia,

I've been swamped at work and have been travelling as well. Most of my 
issues were related to sending Hello on a single VLAN only, be it VLAN 1 
or configurable. I'll get back to you in the next couple of days after I 
have caught up on the discussions,

Dinesh

---

From: Donald.Eastlake at motorola.com (Eastlake III Donald-LDE008)
Date: Wed, 31 Oct 2007 00:22:15 -0400

Hi,

Yes, as noted in protocol draft -05, the pseudo code was unchanged from
-04 and is out of date.

As currently being discussed, the TRILL encapsulated data being
forwarded out a particular physical port would all have the same Outer
VLAN ID.

Thanks,
Donald

-----Original Message-----
From: rbridge-bounces at postel.org [mailto:rbridge-bounces at postel.org] On
Behalf Of Zhi-Hern Loh
Sent: Tuesday, October 30, 2007 10:54 PM
To: Anoop Ghanwani
Cc: Developing a hybrid router/bridge.; Radia Perlman; James Carlson
Subject: Re: [rbridge] RBridge: a case of study

> We have effectively extended the bridged LAN.

Anoop, I'm reading the RBridge protocol specification version 5 and it
seems to me that your comment is inconsistent with the spec. In section,
5.2.2.3, the spec says that the outer VLAN tag of TRILL data frames has
the same VID as the inner tag. Is this section going to be changed?

Futhermore, if we were to use different VIDs on outer VLAN tags then we
would need to handle the multi-destination case where one would need to
send a multi-destination frame per outer VID to communicate with the RBs
in the distribution tree which are using different VIDs. Is this
correct?

Thanks,
Hern
Fulcrum Microsystems

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 31 Oct 2007 01:45:00 -0700

I agree with Anoop answers

-- Silvano

> From: Anoop Ghanwani [mailto:anoop at brocade.com]
> Sent: Tuesday, October 30, 2007 6:54 PM
> To: James Carlson; Silvano Gai
> Subject: RE: [rbridge] RBridge: a case of study
> 
> 
> 
itself

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 31 Oct 2007 01:46:34 -0700

inline

> From: Jian Liu [mailto:jliu at Brocade.COM]
> Sent: Tuesday, October 30, 2007 3:28 PM
> To: Silvano Gai; Radia.Perlman at sun.com

Yes, It may as well be the whole picture.

> 2. Are there regular bridges or RBridges in backbone cloud G?

All the clouds are built with regular IEEE 802.1Q bridges

-- Silvano

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 31 Oct 2007 01:57:21 -0700

Anoop wrote:

--- snip ---

> and perhaps that's OK as a spec.

I strongly second Anoop proposal of adding terminology related to
access, core, hybrid, ingress, egress, etc.

The lack of such terminology in TRILL has been cause of confusion and it
has slow down the progress of the spec.

-- Silvano

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 31 Oct 2007 02:32:16 -0700

James,

-- snip --

>       RBridges are just bridges.  I suspect they're different.)

I have produced an updated drawing to cover the case of "hybrid ports".

http://www.ip6.com/example-v2.ppt

In this drawing I assume that H5 is able to talk with H6.

-- Silvano

---

From: eric.gray at ericsson.com (Eric Gray)
Date: Wed, 31 Oct 2007 06:16:33 -0500

Donald,

	When you say "would all have the same Outer [VID]"
- do you mean:

1) all frames MUST have the same VID within the an RBridge 
   campus, 
2) all frames forwarded from a (physical) port on one RBridge 
   to a (physical) port on another RBridge MUST have the same
   VID, or 
3) something else?

--
Eric Gray
Principal Engineer
Ericsson  

> [mailto:rbridge-bounces at postel.org] On Behalf Of Eastlake III 
> Donald-LDE008
> Sent: Wednesday, October 31, 2007 12:22 AM
> To: Zhi-Hern Loh
> [mailto:rbridge-bounces at postel.org] On
> Behalf Of Zhi-Hern Loh

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 31 Oct 2007 05:36:12 -0700

Anoop, Radia,

The problem pointed out by Anoop, i.e.

> the adjacency creation may not be correct. 

It is the key issue we need to solve: we need a ROBUST solution.

-- Silvano

---

From: james.d.carlson at Sun.COM (James Carlson)
Date: Wed, 31 Oct 2007 09:01:47 -0400

> In this drawing I assume that H5 is able to talk with H6.

That new diagram still does not show the confusing situation that I
was referring to above.  In fact, it shows a situation that is
essentially _identical_ to the previous one you showed in terms of
added bridge functionality.

To replicate the confusing functionality, consider what happens when
adding a host (call it H7) attached to Cloud P.  Without VLAN 7 in
Cloud P, H7 will not be able to participate in VLAN 7 even though all
attached RBridges are in fact transiting VLAN 7 traffic on that
network.  If VLAN 7 is provisioned onto Cloud P (exactly how does this
happen?), then H7 can play.  I see no functionality comparable to this
in the existing draft.

The core change that I think you're asserting here is that when the
user deploys RBridges, the network automatically discovers all of the
far-flung and disjoint islands of VLAN ID usage, and glues those
together by tunneling over whatever VLANs might be in the way between
them.  It no longer matters what configuration separates them.

This is quite a departure from traditional bridging, and I think it's
likely to be both confusing and dangerous, so I'm not sure it ought to
be the default behavior.  In today's bridges, if I configure a trunk
port such that VLAN 3 traffic does not traverse the link, then --
quite simply -- no packets with VLAN ID set to 3 go over that link.
In the new model, that restriction does not apply.

If I say "allow VLAN ID 3," then both an O-RBridge (original RBridge
as I understood it) and the new G-RBridge (Silvano Gai modified
RBridge) will allow VLAN ID 3 across the link.  If I say "disallow
VLAN ID 3 and allow only VLAN 7," then the O-RBridge would stop
passing VLAN ID 3 traffic, but the G-RBridge would simply switch the
outer VLAN ID number and use VLAN 7 effectively as a tunnel to pass
VLAN 3.  The TRILL-encapsulated packets will have VLAN 3 on the inside
and VLAN 7 on the outside.

I'm not at all sure that's a good thing.  I can see why folks who are
struggling with the unkempt nature of VLAN ID provisioning across
complex (and politically divided) networks would be excited about a
"do what I mean" option -- particularly if it means that I don't have
to tell my local IT group to reconfigure switches to allow separated
chunks of my new VLAN to connect together -- but the side-effects seem
extreme.

---

From: james.d.carlson at sun.com (James Carlson)
Date: Wed, 31 Oct 2007 10:14:05 -0400

>    to a (physical) port on another RBridge MUST have the same

(I'm not Donald, but ...)

My understanding was more like:

  3) Where an outer VLAN tag is present, all frames transmitted by an
     RBridge must have the same inner and outer VLAN tag numbers.
     Where an outer VLAN tag is not present, the implicit tag for that
     port must match or be compatible with the inner tag.

... which would be consistent with "traditional" bridge behavior and
would effectively rule out the type of solution that Silvano Gai has
been proposing.

The distinction with (2) in your list is that (2) appears to rule out
the use of so-called "tagged" or leaf or ingress ports, forcing all
ports to carry VLAN tags.

---

From: zloh at fulcrummicro.com (Zhi-Hern Loh)
Date: Wed, 31 Oct 2007 10:17:26 -0700 (PDT)

Donald,

In addition to Eric's questions, could you (or anyone else) explain what would happen in the following scenario?

Toplogy:

                RBa
                /
              VID a
               /
   RBn-link X-----VID b------RBb

Problem:

  RBn is connected to 2 VLANs on link/port X. Connectivity to RBa is via VLAN a, and connectivity to RBb is via VLAN b.

  Suppose that RBa and RBb are in the multi-destination distribution tree from RBn. For a multi-destination frame going out link/port X on RBn to RBa and RBb, what is the VID on the outer VLAN tag? Or could there be need to send 2 frames, one with VID a and another with VID b?

Thanks,
Hern
Fulcrum Microsystems

----- Original Message -----
From: "Eric Gray" <eric.gray at ericsson.com>
To: "Eastlake III Donald-LDE008" <Donald.Eastlake at motorola.com>, "Zhi-Hern Loh" <zloh at fulcrummicro.com>
Cc: "Developing a hybrid router/bridge." <rbridge at postel.org>
Sent: Wednesday, October 31, 2007 4:16:33 AM (GMT-0800) America/Los_Angeles
Subject: RE: [rbridge] RBridge: a case of study

Donald,

--
Eric Gray
Principal Engineer
Ericsson

---

From: eric.gray at ericsson.com (Eric Gray)
Date: Wed, 31 Oct 2007 12:27:18 -0500

James,

	That is kind of what I though, and - IMO - it seems to 
be pointless (over) specification.  Still, it's better than
other possible interpretation for what Donald was saying (see 
toward the end, below).

	Consider this simplified representation of a (specific) 
RBridge implementation:

      _________________________ _ _
     |       ________________      |
   A |    B |                |C    | D
  ---+-[*]--+    Q-Bridge    +-[?]-+---
     |      |________________|     |
     |                             |
     |            RBridge          |
     |_________________________ _ _|

	It's important to realize that this is one potential
implementation - used as an example.

	In this example:

o	A is the port connecting this RBridge to other RBridges;
o	[*] represents the process we're defining for preparing 
	a frame to be ingressed on to, or egressed off of, the
	RBridge campus;
o	B is an internal port, connecting that function to an
	internal Q-Bridge;
o	C is an internal port connecting this same internal 
	Q-Bridge to any arbitrary set of other functions within
	the implementation - possibly including other internal 
	Q-Bridges;
o	[?] represents the arbitrary (set of) internal functions
	(again, possibly including other internal Q-Bridges)
	connecting internal port C to port D;
o	D is the port connecting this RBridge to end stations 
	external to the RBridge campus.

What we would be saying - if we specify what you suggest - is 
that the VID cannot change in going from A to B (or B to A).

	That may be a very reasonable thing to say, but we have
to be careful in saying it.  It is important NOT to be saying 
that the VID cannot change in going from A to D (or D to A).  
The externally visible behavior - for this implementation may 
include changing the VID for any specific set of frames at 'B', 
at 'C', or within the set of functions represented by '[?]'.  

	This ability to change VID values is consistent with 
existing device capabilities.  There may be an administrative 
domain boundary at this device, or the device may provide a
Q-in-Q function.  If we are not saying this implementation 
will be non-conformant, it's unreasonable to disallow this.

	What I thought Donald meant by "the same" was not with
respect to the "inner" VLAN ID - but with respect to all of
the frames sent using a particular physical interface.  That
would be REALLY BAD.  It would - for example - make it very
hard to do as good a job with link utilization using RBridges
as it is already possible to do using MSTP bridges.

--
Eric Gray
Principal Engineer
Ericsson  

> From: James Carlson [mailto:james.d.carlson at sun.com] 
> Sent: Wednesday, October 31, 2007 10:14 AM
> To: Eric Gray
> Cc: Eastlake III Donald-LDE008; Zhi-Hern Loh; Developing a 
> hybrid router/bridge.
> Importance: High

---

From: anoop at brocade.com (Anoop Ghanwani)
Date: Wed, 31 Oct 2007 10:33:07 -0700

Hern,

That's one of the problems that we're trying to avoid
by forcing all RBridges on a bridged LAN to be configured
such that they are all reachable and see one another on
a single VLAN.

Anoop 

> [mailto:rbridge-bounces at postel.org] On Behalf Of Zhi-Hern Loh
> Sent: Wednesday, October 31, 2007 10:17 AM
> To: Eric Gray
> In addition to Eric's questions, could you (or anyone else) 
> explain what would happen in the following scenario?
> 
> Toplogy:
> 
>                 RBa
>                 /
>               VID a
>                /
>   RBn is connected to 2 VLANs on link/port X. Connectivity to 
> RBa is via VLAN a, and connectivity to RBb is via VLAN b.
> 
>   Suppose that RBa and RBb are in the multi-destination 
> distribution tree from RBn. For a multi-destination frame 
> going out link/port X on RBn to RBa and RBb, what is the VID 
> on the outer VLAN tag? Or could there be need to send 2 
> frames, one with VID a and another with VID b?
> 
> 
> 
> Thanks,
> Hern
> Fulcrum Microsystems
> 
> 
> To: "Eastlake III Donald-LDE008" 
> <Donald.Eastlake at motorola.com>, "Zhi-Hern Loh" <zloh at fulcrummicro.com>
> > [mailto:rbridge-bounces at postel.org] On Behalf Of Zhi-Hern Loh

---

From: eric.gray at ericsson.com (Eric Gray)
Date: Wed, 31 Oct 2007 12:47:47 -0500

Anoop,

	This is a generic VLAN bridging problem, commonly
dealt with using VLAN bridges today.  Clearly, it is not
the case that all VLAN bridges are required to use a 
common VLAN for frame forwarding.

	In addition, as I understand it (in part based on
James' reply), we are NOT saying that the common VLAN
you mention must be used for all frame forwarding - 
hence it is likely that the existence of a common VLAN
does not necessarily fix the problem.

	Just to be clear, if we were saying that a common
VLAN must be used for all forwarding between RBridges,
it quickly becomes obvious that load-sharing based on
VLAN separation becomes useless in practice.  If load
sharing based on VID is not useful, then MSTP already
provides L2 forwarding that makes more efficient use of
links than would be possible using RBridges (unless all
RBridge-to-RBridge connections are via P2P links).

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Wednesday, October 31, 2007 1:33 PM
> To: Zhi-Hern Loh; Eric Gray
> > > Yes, as noted in protocol draft -05, the pseudo code was 
> unchanged 
> > > from
> > > -04 and is out of date.
> > >

---

From: james.d.carlson at sun.com (James Carlson)
Date: Wed, 31 Oct 2007 14:38:39 -0400

> that the VID cannot change in going from A to B (or B to A).

Effectively, yes.

> will be non-conformant, it's unreasonable to disallow this.

I think there's a distinction to draw here.

If an implementation has some sort of VLAN mapping function beyond the
typical tagged/untagged distinction, that seems fine to me.  The rule
should be written to allow such a thing, because it's trivial to show
that you could construct the same device out of a series of cascaded
devices.

That's not what I was understanding Silvano Gai to say.  Instead, I
was understanding his proposal as a sort of "pernicious bridging."

By "pernicious bridging," I mean that the RBridge detects that it has
disjoint groups of interfaces in the network that have the same VLAN
ID configured.  Based on this detection alone, it then finds shortest
paths between those locations.  Where those paths cross an interface
that doesn't support the desired VLAN ID, we simply use a different
one -- perhaps chosen at random (Silvano's slides don't describe this)
-- and drive on.

If it's based on administratively-configured mappings ("VLAN 3 can
appear on this interface using VLAN ID 7"), then it doesn't sound at
all like a problem to me, because the algorithms and administrative
entries still make sense if you change all the numbers.  If it just
"happens" by virtue of discovering disjoint VLANs, then I think it's a
problem.

> as it is already possible to do using MSTP bridges.

Yes; that'd be bad.

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Wed, 31 Oct 2007 14:50:36 -0700

Eric....if I understand your concern...

Having all RBridge-RBridge traffic between neighbor
RBridges (RBridges on a single layer 2 cloud) take place on a single VLAN
(whether the VLAN used be configurable for that cloud, or whether it's 
always VLAN 1),
does not preclude load splitting. "VLAN 1" as the VLAN tag in
the outer header is just used in order to get a packet from one
RBridge to a neighbor RBridge. Layer 3 techniques for load splitting, 
traffic engineering, etc.,
can be used for having RBridges choose paths across the campus. When 
doing load
splitting, a smart RBridge, just like a router, could choose anything it 
can see in order to decide which
path to send a frame on, or which traffic to drop when there is
congestion, including the inner VLAN tag. And although some might 
consider it
architecturally impure, there isn't anything precluding an RBridge from 
looking
even further into the packet and making forwarding/traffic 
splitting/traffic dropping decisions based
on things like the diffsrv field in the IP
header, or the TCP ports.

The outer VLAN tag is just how to wrap up the frame when forwarding to 
the next hop
RBridge that the RBridge forwarding decision has selected for this frame.

Radia

Eric Gray wrote:
>>> <Donald.Eastlake at motorola.com>, "Zhi-Hern Loh" 
>>>       
>> <zloh at fulcrummicro.com>
>>     
>>>> forwarded out a particular physical port would all have the 
>>>>         
>>> same Outer 
>>>       
>>>> VLAN ID.
>>>>
>>>> Thanks,
>>>> Donald
>>>>
>>>> Cc: Developing a hybrid router/bridge.; Radia Perlman; 
>>>>         
>> James Carlson
>>     
>>>>>>> a possible case of study for TRILL (of course, not the 
>>>>>>>               
>>>> only one).
>>>>         
>>>>>>   1.  After the proposed fix, should VLAN 3 on Cloud L 
>>>>>>             
>>> be bridged
>>>       
>>>>>>       together with VLAN 3 on Cloud R, even though 
>>>>>>             
>> the backbone
>>     
>>>>> itself
>>>>>           
>>>>>>       doesn't directly carry VLAN 3?  (I.e., are TRILL 
>>>>>>             
>>>> packets with
>>>>         
>>>>>>       outer VLAN ID 7 or 8 and with inner VLAN 3 
>>>>>>             
>> present on the
>>     
>>>>>>       backbone?)
>>>>>>             
>>>> Anoop, I'm reading the RBridge protocol specification 
>>>>         
>>> version 5 and it
>>>       
>>>> the same VID as the inner tag. Is this section going to 
>>>>         
>> be changed?
>>