Subject: [rbridge] long-awaited review comments on draft-ietf-trill-rbridge-arch-05

From: James Carlson <[email-masked]>
Date: Wed, 14 May 2008 17:21:34 -0400

Here are my long-awaited review comments on the TRILL architecture
document.  I intentionally read through it as someone who wanted an
introduction to RBridges -- not someone who already knows the contents
of the protocol draft, or who has spent too many hours staring at IEEE
documents.  Many of the things I found were related to that.

In the notes below, individual notes are separated by a single "-" on
a line.  The notes are indented two spaces.  The final section has
minor editorial nits.

-

  How do STP and RBridges interact?  We need to make it clear fairly
  early on that neither regular bridges nor RBridges will forward STP
  messages, but that regular bridges will forward TRILL IS-IS traffic
  while RBridges will not.  Thus the expected model is that:

  - Directly connected STP-speakers see each other, and do tree
    computation as usual.  Those separated by RBridges are effectively
    on different networks.

  - RBridges don't see any of the STP-only speakers as part of the
    topology, and thus consider any sequence of regular bridges to be
    one hop.

  This is eventually described in section 5.5, but that's quite a ways
  down in the document.

-

  Section 2.2 page 11:

    The term "R-tree" is defined, but then never used again.

-

  3.2.1 doesn't give enough detail about the nature of the unicast
  forwarding database.  There must be entries of at least these forms:

    { Destination MAC } -> { TRILL egress address }

    { TRILL address } -> { output link, MAC next hop }

  The first represents ingress behavior, and the second is TRILL
  transit.  It's possible to compose together the first entry with the
  second, avoiding a double look-up, so that the first entry looks
  like this:

    { Destination MAC } -> { TRILL egress address, output link, MAC next hop }

  But the second type of entry must exist for TRILL forwarding within
  a campus.  It's also possible to optimize the case where the egress
  is the local system and thus normal bridge forwarding is needed.
  That case looks like:

    { Destination MAC } -> { output link }

  However, a system must always recognize its TRILL address and use
  that to select an action of decapsulation followed by normal
  bridging behavior, which means look-up based on the inner MAC header
  to find a local entry of the above form.

-

  Section 3.2.2., on page 15, the term "Egress RBridge" is defined for
  the multi-destination case in part with this text:

        o  Egress RBridge - an RBridge that is the tail end of a path 
           corresponding to a specific Multi-destination TRILL 
           Forwarding Database entry. All RBridges within a TRILL Campus 

  I think this gets a bit confusing, because there are also "Egress
  RBridges" in the unicast case, and using the same formally term for
  two potentially different things seems like a mistake.  As
  alternatives, I suggest:

  - Change "Egress RBridge" into a role that an RBridge plays in the
    network, and define it in terms of the responsibilities of that
    role outside of this section.  The section on multi-destination
    traffic can then clarify how a node 'knows' it must play that
    role.  (For unicast, it's easy.  The nickname in the destination
    field is *your* nickname.)

  - Create a new, distinct term that encompasses the specific behavior
    and role of a multi-destination egress.

-

  Section 3.2.2., on page 16, it says:

        Multi-destination TRILL Forwarding Database entries may also 
        include Multicast-Group Address specific information relative to 
        each egress RBridge that is a member of a given well-known 
        multicast group, to allow scoping of multicast forwarding by 
        multicast group. 

  Why are the words "well-known" used here?  The point of well-known
  group addresses is that the handling is already defined --
  membership isn't really needed.  Shouldn't this say "of a given
  multicast group"?  (If not, then what exactly is the significance of
  limiting these entries to *just* those for well-known group
  addresses?)

- 

  Section 4.1, page 17:

        At an architectural level, it is sufficient to note that every 
        end station attached to a TRILL Campus should have a primary 
        point of attachment to the TRILL Campus, as might be defined 
        (for example) by a Designated RBridge.  Furthermore, if it is 

  I read that several times, and then had to refer to other sections
  before the actual meaning of this text became clear.  "Primary point
  of attachment" doesn't mean to me what it must have meant to the
  author.  When I first read it, I thought it was a wire or subnet.
  Then I started thinking in terms of DLPI PPAs.  Then I got _really_
  confused.  ;-}

  The apparent meaning of this text is that, for each end station on
  each VLAN, there must be at most one RBridge that acts as the TRILL
  encapsulation/decapsulation gateway when talking to other nodes in
  the rest of the campus.  And one way to do that is to have a
  per-VLAN Designated RBridge.

  There's no actual "attachment" of any sort.

  Unfortunately, the text takes several unclear paragraphs to say
  that.  It seems that part of the reason it's so unclear is that the
  document is trying to drive far out of its way in order to be "fair"
  to other possible proposals other than having a Designated RBridge.
  Perhaps we could even do per-end-station elections.

  I think the text should be shortened up considerably and clarified,
  because this point is effectively drowned out by too many words.
  (This comment applies to similarly affected sections, such as 5.2,
  which seems to be crawling with degenerates.  ;-})

-

  In general, I think section 4.1 worries itself too much about the
  definitions of bridges (802 references and such) and far too little
  about the architectural implications for RBridges.

  We (those creating TRILL-based RBridges) don't care about bridges.
  We should not have to.  We shouldn't have to specify that bridges
  need to "be consistent" with 802.1D or 802.1Q -- they either are or
  aren't, and that's the problem of the bridge vendor.

  I note that the document didn't spend any time talking about the
  standards for repeaters.  Those have about as much bearing to the
  matter here.

  The *important* part is whether any equipment that may form a
  non-RBridge L2 data path between RBridge ports must allow TRILL
  communication between those ports such that RBridges can safely
  elect or determine a single Designated RBridge.  It doesn't matter
  how that path is formed (802.1D is one possibility), just that it
  exists.

-

  Section 5.2, page 21:

        As described previously, RBridge learning is similar to typical
        bridge learning - i.e. - all RBridges listen promiscuously to L2
        Frames on each local LAN and acquire end station location
        information associated with source MAC addresses in L2 frames
        they observe.

  All egress RBridges should also learn from the L2 frames that they
  decapsulate.  The two cases are distinct and important parts of
  learning:

  - The ingress learns on which local port the end station exists,
    just like any ordinary bridge would do.

  - The egress learns which nickname is the remote encapsulator (and
    thus per section 4.1, decapsulator) for that end station.  This
    part is unlike an ordinary bridge.

  This latter bit is crucial.  It's what requires the encapsulator
  (which fills in a source nickname) and decapsulator (which will be
  the target of return traffic) to be the same node, or at least
  requires the encapsulator to fill in the decapsulator's nickname as
  the "sender."

-

  Section 5.2, page 22:

        The trade-off is between the complexity associated with flooding 
        data verses the complexity associated with flooding reachability 
        information.   

  This is duplication of the information already in 3.2.3.  This could
  be trimmed down.

-

  Section 5.2, page 23:

        Note that an egress RBridge will - in most case - be the RBridge 
        determined to be the primary point of attachment for a 
        destination end station on the local link or VLAN accessed via 
        its egress interface(s). Exceptions to this might exist under 
        circumstances in which use of distinct RBridges for ingress and 

  I think this digression should just be removed.  Not only is it in
  conflict with the intent of section 4.1, but (like the whole "point
  of attachment" thing) it's a point of unnecessary confusion.

  If it becomes feasible for some RBridge implementation strategy to
  allow for distinct ingress/egress nodes in some cases, then I think
  it's that other document's problem to describe how the deviation
  that document describes is consistent with the overall story,
  including (particularly) the egress node's learning capability.

  By this same token, any implementation could be arbitrarily strange
  in areas not specified by the architectural document.  For instance,
  someone could implement all of this with ATM and map nicknames into
  VCIs.  It's not really possible (or even useful) to describe all the
  ways one could go strange.

  The architecture document should describe how the system is intended
  to operate and what the parts should do.  I don't see a reason to
  insert loopholes that allow for unspecified future variations.  At
  best, it's a distraction, because we don't know how to make that
  work.  (And, in fact, I suspect it does _NOT_ work in any case,
  because it breaks learning.)

-

  Sections 5.3.2-2 and 5.3.2-3, pages 28 and 29: there's a lot of
  duplication here.

-

  I'm surprised that section 5.4 doesn't discuss why IS-IS was chosen,
  or what special things need to be done with it in order to make it
  work here (such as setting a fixed "area" value).

-

  Section 5.5, page 30:

        o  Transparent Participation (Transparent-STP) 
        o  Active Participation (Participate-STP) 
        o  Blocking Participation (Block-STP) 

  I don't see that these terms are defined anywhere.  It seems
  somewhat obvious what they mean -- *if* you already understand
  RBridges -- but they're likely to confuse.

  This also looks like material that's in the same category as the 5.2
  advice about separate ingress/egress.  It's possible that someone
  could define a "new" version of RBridges that either forwards STP
  messages (!) or has each RBridge acting as an STP node in a single
  network (!!), but neither of those is really the solution we're
  trying to describe.  It's not part of the architecture.

-

  Section 5.5.1, page 32:

        Finally, note that there is a chicken-and-egg problem associated 
        with RBridge participation in STP where RBridges may themselves 
        be connected by spanning trees. 

  I'm not positive that this problem actually occurs.  If an RBridge
  runs STP, the port will be blocked until STP finishes its usual set
  of listening/learning/forwarding timers, so the RBridge network
  won't see or use the link either.

  STP is the egg, and TRILL is the chicken.  I think.

- Editorial nits

  Section 1, page 4:

        The principal objectives of this architecture is to provide an 
                                                      ^^ are

        allow some level of optimization support to be provided in 
        compliant implementations, in as many case as possible. 
                                              ^^^^ cases

  Section 3.2.1, page 14:

        for each VLAN, if this is supported by configuration. Note that 
        scaling concerns may dictate otherwise, either in specific of 
                                                          ^^^^^^^^ ?
        RBridge protocol specification, or in deployment.  The Unicast 

  Section 3.2.2., page 15:

        o  Zero or more entries grouped for each root RBridge - keyed by 
           some root RBridge identifier - used to determine forwarding 
           of broadcast, multicast, and flooded frames originally 
           RBridge encapsulated by that ingress within the TRILL Campus. 
           ^^^^^^^ TRILL

        Each entry would contain an indication of which single interface 
        a broadcast, multicast or flooded frame would be forwarded for 

	(The text suddenly jumps into subjunctive mood rather than
	staying in future tense.  It's unclear to me why this is so,
	but it looks like an error in the text.)

  Section 3.2.3, page 16:

        The Ingress TRILL Forwarding Database determines how arriving 
        traffic will be encapsulated, for forwarding toward the egress 
                                    ^
        RBridge, via the TRILL Campus. It becomes configured in much the 
               ^

  Section 4.6, page 20:

        It is the combination of the local MAC desitnation (which is for 
                                               ^^^^^^^^^^^
        a locally attached RBridge) and the TRILL encapsulation that 

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: "Eric Gray" <[email-masked]>
Date: Thu, 15 May 2008 14:45:44 -0500

James,

	Thank you for the review and comments.  Sorry to have taken 
so long to get back to you on them, but I am currently in Eilat,
Israel for an Interim meeting of IEEE 802.1 Interworking.

	While I personally agree with most of your comments, I do
not believe that I am at liberty to make changes along the lines
of many of the ones you suggest, because the existing text is the 
result of WG direction or prior consensus/decisions.

	I explicitly invite WG members to read this review and to
indicate support (or lack thereof) for making the changes you've
suggested - in particular, those relating to the fact that the
current Architecture document does not assume use of designated
RBridges.  

	There were many reasons why I had initially made the "DRB
assumption" and you have pointed out several of them.  However,
I was given very explicit direction by the WG to remove the DRB
assumption, which ultimately made it necessary to describe all
of the specific complications involved in doing anything else.

	Also, I must wait for other reviewer comments before I can
start making changes in any case.

--
Eric Gray
Principal Engineer
Ericsson  

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of James Carlson
> Sent: Wednesday, May 14, 2008 5:22 PM
> To: TRILL/RBridge Working Group
> Subject: [rbridge] long-awaited review comments 
> ondraft-ietf-trill-rbridge-arch-05
> 

There are usually structure issues with any document.  Each different
reader may find that they would prefer that some part of the content of
the document was provided earlier than another.  This is not always an
option.

For example, if the interaction information you requested above were to 
appear toward the beginning of the document, it would be necessary to
either assume the reader understands elements of the architecture that
are implicitly included in these statements, or provide the description 
of these things earlier in the document.  Pretty soon, you may find all
of the information gets cycled and the specific content you wanted to
see earlier is now pretty much where it was orignally.

Moreover, some structure decisions are a result of previous comments,  
specific requests made against earlier versions or a compromise that
was acceptable to the majority of the commenters at some time.  In the
specific case of interactions between spanning tree and trill protocol,
there was some argument earlier (from Donald, I believe) that this ID
should not include any discussion of these interactons, accept possibly
in an appendix.  However, it was also argued that the interactions were
relevant to the architecture and that - because the architecture ID is
strictly informative in any case - there is no particular reason to put 
this kind of information in an appendix.

So it is part of a tutorial instead.

Would it be sufficient to provide a pointer to section 5.5 at some point
toward the beginning of the document?  If so, where specifically would
you think it should be put?

>     The term "R-tree" is defined, but then never used again.

This was specifically agreed to in the course of terminology alignment
several meetings (and versions) ago.  Radia and Silvano requested that
the architecture document should include this definition.  However, I
argued at the time that this is specific to a particular specification
of protocol, and not a term generally applicable to the architecture.

Hence the term is defined, and not used.

>   to find a local entry of the above form.

The exact proposed additions are actually not quite complete/accurate
since the ingress and forwarding operations are 1) distinct (since the
ingress operation also includes encapsulation) and 2) discussed in 
different sections (ingress information is discussed in section 3.2.3
and an ingress RBridge would contain - at least logically - an ingress 
forwarding database, a unicast forwarding database and, possibly, a
multi-point forwarding database).

At one point, the architecture did contain information at the level of
detail suggested in this comment.  However, this level of detail was
found to be objectionable by a number of people in earlier versions.

Because of the fact that the protocol specification design team was -
at the time - in the process of evolving the details of the content of
the corresponding information base AND was also heavily represented in 
the pool of those people who were providing the strongest objections, 
one (or more) subsequent versions contained an explicit statement that
the information provided represented a logical model only and did not
constrain either protocol specifications or implementations.

This text - at least in part - remains in section 3.2 text immediately
preceding section 3.2.1.

At a later point - marked I believe by a change in the makeup of the 
protocol design team - even this qualified text was considered to be
objectionable and possibly incorrect.  Hence - by mutual consent of
all parties - all detailed text of the nature suggested in this comment
was removed and replaced by the following paragraph:

       'The Unicast TRILL Forwarding Database contains data specific to 
        RBridge forwarding for unicast traffic. The specific fields 
        contained in this table are to be defined in RBridge protocol 
        specifications. In the abstract, however, the table should 
        contain forwarding direction and encapsulation associated with 
        an RBridge encapsulated frame received - determined by the TRILL

        "shim" header destination and VLAN (if applicable).'  

>     and role of a multi-destination egress.

This comment is somewhat misleading.

Egress RBridge (as a role in a network) is defined in the terminology 
section.

Immediately preceding this (and other) definition(s) is the following
paragraph:

        'In discussing entries to be included in the Multi-destination 
        TRILL Forwarding Database, the following entities are 
        temporarily defined, or further qualified:'

This is an editorial choice, and should not produce confusion for most
readers.  In this I may be mistaken, but - in the context of where it 
is used - my 'editorial' opinion was that inventing a new term to
describe the role of a multi-destination egress RBridge, as an entry 
in a database (as opposed to an entity in the network), would be more 
confusing, rather than less. 

The principal reason for "qualifying" these definitions is to make it
clearer that (from an architectural perspective), these are still the
same entities, but they play a different role in this database than 
they do in the role of forwarding frames.  

Perhaps it would be potentially less confusing if the entire set of 
"qualified" definitions were included in a NOTE in which they were
also made to look significantly less like "definitions"?

>         include Multicast-Group Address specific information relative 
>         to each egress RBridge that is a member of a given well-known 

I will happily remove the phrase "well-known."

> 
> - 
> 
>   Section 4.1, page 17:
> 
>         At an architectural level, it is sufficient to note that
>         every end station attached to a TRILL Campus should have a
>         primary point of attachment to the TRILL Campus, as might
>         be defined (for example) by a Designated RBridge.  
>         Furthermore, if it is [...]
> 

Unfortunately, I agree with this comment completely and cannot take any
action on it - at least not on the basis of a single comment from one
reviewer.

The current wording is the result of extensive comments on the notion
that the architecture draft has no particular reason to limit protocol
specifications to use of a designated RBridge.  However, the complexity
associated with trying to be fair to other potential approaches means
spending some time and effort in describing the consequences and issues 
associated with particular choices (especially the choice not to allow
us to assume use of a DRB in the architecture).

The term "primary point of attachment" was my choice, and I was not 
very happy with it either.  In any specific case, an RBridge that is
the designated ingress/egress for any particular set of end-stations
is the RBridge to which they are "attached" (in the exact same sense
that they might be "attached" to a bridge - for learning purposes).

I discussed this usage in detail in a "status" presentation a few WG
meetings ago (I could look up exactly which one, but is it really 
worth it?).  I asked for suggestions for another term at that time
and received none.

Personally, I would have preferred to use "designated RBridge" - but
I suspected that would not have been accepted by those who argued to
remove the "DRB assumption."

I am happy to use any reasonable term instead, and provide a detailed
definition as well.  It is - IMO - not the case that there can be "at
most one" DRB-equivalent for any end-station, but explaining that in
a precise definition is bound to be even more confusing.  As sages
say, be careful what you ask for.

Also, as you undoubtedly know, "degenerate" is a precise mathematical 
term meaning "a limiting case of a mathematical system that is more 
symmetrical or simpler in form than the general case."  My favorite
example is a degenerate circle represented by the equation -

     2    2
    X  + Y  = 0

- which obviously describes a single point located at the origin of a 
cartesian coordinate system in X and Y.

I prefer not to replace either of the two uses of this word, as to
do so would be to go from a term that has at least one definition 
that is precisely correct, to another term that does not.

>   how that path is formed (802.1D is one possibility), just that it

I won't argue whether or not this true, it is not quite the point.

The issue discussed is entirely about the need to be compatible with 
bridge learning (defined for 802.1 bridges).  If - in fact - the issue
was limited to links between RBridges, my answer might be different.

If we do not need to be consistent with bridge learning (in either a
transit or ingress/egress case), a lot of things are different.  But
the key difference - for this section - is that it is important for
RBridges to provide forwarding that is consistent with the way that
bridge learning works.  In the simplest approach - where we treat a
set of (spanning tree) connected bridges as a single link between
RBridges (or a single stub connected to a single RBridge), and have
a single RBridge that provides egress/ingress to the RBridge campus
- then the specifics of the topology and the way that bridge learning
occurs would be unecessary.

However, architecturally, there is no reason to assume that this must
be the case.  For example (not as a recommendation), it is possible
for RBridges to be aware of both how bridge learning occurs and the
details of the bridge topology in such a way that load-sharing could
be done without impact on local bridge forwarding.

This may not be precisely clear to people in the IETF, generally.  It
is probably not the case that everyone immediately realizes that the
frames delivered to an ingress RBridge do not (usually) have the MAC 
DA of that RBridge, nor that frames delivered from an egress RBridge 
do not (usually) have the MAC SA of the egress RBridge.  Because these
things are true, however, it is necessary for the RBridges to behave 
in a way that is consistent with bridge learning.

Use of a designated RBridge makes this relatively trivial, but is -
as discussed previously - not a valid architectural assumption in the
opinion of several vocal members of the WG.

>   requires the encapsulator to fill in the decapsulator's nickname as

This is - IMO - a level of detail below architecture.  In particular,
the mechanistic details of RBridge learning very definitely do not 
belong in an architectural specification.

In addition, the proposed additional text is actually architecturally
incorrect.

For one thing, the choice to learn on decapsulation is the current
approach assumed as default in the protocol specification but is not
an architectural requirement at all.

>   This is duplication of the information already in 3.2.3.  This could

I am unsure how this information is in any way related to the text in
section 3.2.3 (Ingress TRILL Forwarding Database).

Also, note the text at the beginning of section 5 where the very first
sentence says:

   "This section is intended primarily to serve as a tutorial for 
    RBridge operations."

This was the result of a specific request that the architecture draft
should include such a tutorial.  

As such, it may contain text that is redundant.

In addition, this is a single sentence statement that is very relevant 
to the text in surrounding paragraphs.  The preceding paragraph talks 
about the fact that the current assumption (of the WG, in the protocol 
specification referenced) is that the ingress forwarding database is 
populated by learning from (potentially flooded) data frames on egress.

The immediately subsequent paragraph mentions at least one application 
in which this could be a poor choice (as an engineering trade off).  
The single sentence paragraph to which you refer simply describes the 
specific trade-off that applies to the decision.

It is both appropriate and necessary to the flow of the text to include
this statement at this point.

>   because it breaks learning.)

It's interesting that you did pick up on the learning problem, but did
not see that this was the point of a lot of the text in section 4.1.
Minimally, I will try to make at least that much clearer.

Again, there is no "architectural" reason to restrict implementations
- or protocol specifications - to behavioral assumptions such as the
ones you suggest.  This was not my choice, but direction from the WG.

Unhappily, this particular direction from the working group is at least
technically correct.  As I admitted at the time, this was an effort on
my part to avoid having to describe the complexity involved in making 
any other assumptions.

And - unlike the ATM suggestion you mention - this particular degree of
"architectural freedom" has everything to do with Ethernet technologies 
RBridges are expected to be compatible with.  As an example, consider
the following scenario:
                _____
    S-1 <------|     |------> RB-1
    S-2 <------| H-1 |
    S-3 <------|_____|------> RB-2

In this case, S-1, S-2 and S-3 are end stations, H-1 is a Hub, and 
RB-1 and RB-2 are RBridges.  While I am not recommending it, there
is no "architectural" reason to prohibit RB-1 from being a PPOA for
S-1 and S-3, while RB-2 is the PPOA for S-2.  There is in this case
a strong likelihood that no learning is involved that would become
confused.

How this might be supported is out of scope for architecture, and I
do not include - or explicitly hint at - any such example, nor do I
say that this scenario (or any like it) needs to be supported by 
a specific protocol solution.  I do state that there are things a
protocol specification must define in the event that it does decide
to support anything as complicated as this.

I do not deny that this is confusing.  Those WG members who read any
of the first 4 or 5 versions (at least), would be able to point out 
that I had tried to avoid this confusion by assuming that a DRB will
be used.

Since WG members argued that the architectural specification is not 
allowed to make such an assumption, the architecture document now 
needs to provide all of the nicely complicated descriptions of the 
consequences of not assuming that a DRB is going to be used.

Dinesh Dutt and Joe Touch were among the people who argued that the
"DRB assumption" should be removed.  At least one of the WG chairs
also felt that it probably should be removed.  Nobody, other than 
myself, had any objection to this argument (though I did mention 
that it would significantly complicate the architecture and I also 
tried to socialize the complexity involved with at least a few of 
the active WG participants) at the time.

>   work here (such as setting a fixed "area" value).

There is no completely unarguable reason for making the choice to 
use IS-IS.  Also, the basis for making the choice is irrelevant to
the architecture specification.  The fact that the choice was made 
is only included as a result of a process decision to discontinue
efforts to make progress on another document that was actually the
appropriate place to make such observations.

If you would care to propose specific text - and we can achieve a
degree of WG consensus - as to why IS-IS was chosen, I would be 
okay with adding it.

The reason I ask is that I am not certain that the real reasons
why IS-IS was chosen are both appropriate for inclusion in, and 
consistently likely to remain true over the life of, an RFC.

With respect to setting a 'fixed "area" value' people who've been
to most of the meetings will probably recall that I asked about 
this and was told that it was not the case.  If this has in fact
changed, I was not told about it.

Can you provide a specific reference?  There is no instance of
either the word "area" or the word "fixed" that has anything to
do with this topic in the protocol specification.  The word "area"
is mostly used in connection with the "options" portion of the
TRILL header, and the word "fixed" is used with enablement status
and the assignment of VLAN 1.

I also checked current IS-IS WG drafts, and did not see anything
obviously likely to include this.

Also, is it really appropriate to include this level of detail in
the architecture document?  This sounds like a protocol operation
specification...

>   trying to describe.  It's not part of the architecture.

An architecture does not describe a solution.  This should be clear
from the title, which is intended to indicate that this document
describes the architecture that applies to a solution to the TRILL
"problem."  The abstract further clarifies that the role of this 
document is to propose "an architecture for RBridge systems as a 
solution to the TRILL problem" and goes on to say that protocol and
mechanisms (which would be part of an actual solution) are specified
elsewhere.

Perhaps I should be more explicit in stating that actual solutions
(including protocol and mechanisms) are specified elsewhere?

The fact that a solution could be defined that includes either of
the two (currently unchosen) alternatives is a perfectly valid
reason why they are mentioned in an architecture document.

Furthermore, the fact that these terms exist - and are not included
in the Terminology section - is the result of a long-term debate 
about the possible alternatives, as well as the fact that it still
might be necessary to specify one of them to resolve one or more 
specific issues with RBridge solution(s) specification(s).  See,
for instance, the wiring closet problem in section 5.5.1.

The fact that these terms are not defined (at least in section 2)
is that there was no consensus in a terminology alignment effort to 
include these terms.  In addition, it is (as you said) somewhat
obvious what these terms mean (though I disagree that knowing much
about RBridges is required; it is actually more important to know
a little bit about STP).

Further, it seems somewhat pretentious to define terminology in the
main architecture text that is only used in a tutorial subsection
of the document.  Understading of this text is "nice-to-have" but
not necessary.

> 
> -
> 
>   Section 5.5.1, page 32:
> 
>         Finally, note that there is a chicken-and-egg problem 
>         associated with RBridge participation in STP where 
>         RBridges may themselves be connected by spanning trees. 
> 

Some negativity is only natural.  I fogive you.  :-)

If you look into the "wiring-closet" problem, you will see that it
is possible (with some potential solutions to this problem) to have
STP race conditions, or oscillations, depending on exactly how the
protocols interact.

Since the WG is apparently not interested in solving that problem
- in spite of feedback from IEEE 802.1 that doing so is important
- there is little reason to go into the details.  This is even more
true, given the fact that section 5 is now a tutorial.

>         compliant implementations, in as many case as possible. 

Thanks, these will be fixed.

>         RBridge protocol specification, or in deployment.  

I had noticed this one as well.  It either originally said, or was 
meant to say, "... specific instances of ..."

I propose to remove "specific of", making the text read "... either
in RBridge protocol specification, or in deployment."

> The Unicast 
>            some root RBridge identifier - used to determine [...]
>            RBridge encapsulated by that ingress within the [...] 
>            ^^^^^^^ TRILL

Yes, I will correct this.

> 	but it looks like an error in the text.)

Not sure I agree that there is an issue with tense here, but I
probably need to re-formulate this text to make it readable.

I propose to replace the text -

  "Each entry would contain an indication of which single interface 
   a broadcast, multicast or flooded frame would be forwarded for 
   each (root RBridge, egress RBridge) pair."

- with -

  "Entries contain an indication of the interface a broadcast, 
   multicast or flooded frame is forwarded on for all applicable
   {root RBridge, egress RBridge} pairs."

>         RBridge, via the TRILL Campus. It becomes configured 

I propose to fix the sentence structure by replacing the first 
sentence with:

  "The Ingress TRILL Forwarding Database is used to determine 
   how arriving traffic will be encapsulated for forwarding,
   toward the egress RBridge, via the TRILL Campus."

> 
>   Section 4.6, page 20:
> 
>         It is the combination of the local MAC desitnation 
>                                                ^^^^^^^^^^^

Thanks, I will fix the spelling of destination.

---

From: James Carlson <[email-masked]>
Date: Thu, 15 May 2008 17:15:07 -0400

> result of WG direction or prior consensus/decisions.

To be clear: I wasn't asking you (personally) to make any changes
without working group consensus.

During the last IETF meeting in Philadelphia, there was a call for
volunteers to read the document and offer comments on the list,
because the document shouldn't go forward if it hasn't had review.  I
offered my time up to do that review, and as a result, I was offering
comments on the document.

> of the specific complications involved in doing anything else.

Yep; understood.  If the consensus is to leave the architecture
document wishy-washy due to high level equivocation like this, then I
can certainly live with it.  I'm implementing to the protocol specs
rather than the architecture anyway.  ;-}

> the document was provided earlier than another.  This is not always an

When attempting to read the document as someone without intimate
RBridge knowlege, I think the very *first* question that reader has is
how traditional bridging and the new RBridges "see" each other.  How
does interoperability with existing equipment work?

In every single presentation I've given on our OpenSolaris RBridges
project, that question has come up.  In fact, I just gave a short talk
at our internal quarterly engineering status review today, and guess
what the only question was?

> see earlier is now pretty much where it was orignally.

I don't believe that to be true, because I'm identifying a need to set
out some boundary conditions first: the reader should not expect to
see us discussing STP interoperability (there isn't any because the
STP 'domain' ends at the RBridge doorstep) or how RBridge links are
disabled by STP (they're not).

But if the document authors don't feel they can do that without
damaging the structure of the document, then that's fine.  I've
offered my comments, and that's all I set out to do.

> you think it should be put?

Having a pointer that says "interoperability between traditional
bridges using Spanning Tree and RBridges is preserved by the rules
described in section 5.5" somewhere near the top (perhaps before or
amid the "as an overview, however" part on page 5) would help.

> Hence the term is defined, and not used.

It looks like flotsam in the architecture document.  I suggest
removing it.

> >   forwarding database.  There must be entries of at least these forms:
[...]
> found to be objectionable by a number of people in earlier versions.

In *no* place does the document describe the form or abstract
operation of these databases required for the forwarding functions.

That's important, and it is indeed architectural.  It's not an
implementation detail that can be deferred to a protocol
specification.  The database itself doesn't appear inside the
protocol.  It's a fundamental assumption of the mechanism used in
forwarding and it's necessary to understand it in order to understand
egress learning behavior.

(Honestly, given the amount of architectural information that's
present in the protocol specification, I'm not sure why I should argue
about this point -- collapsing the documents would be a better
solution -- but as long as we're trying to separate architectural
matters from protocol matters, we're not quite getting there.)

>         "shim" header destination and VLAN (if applicable).'  

That's exactly what I'm pointing out as insufficient in terms of
architecture.

> >   the multi-destination case in part with this text:
[...]
> This comment is somewhat misleading.

I don't believe it is.

>         temporarily defined, or further qualified:'

Right.  That's exactly what I'm pointing out as difficult to
understand.  Using the same term with different definitions within a
single document is dangerous.

> readers.

I guess I'm just not "most readers," then.

> also made to look significantly less like "definitions"?

Possibly, though I'm not sure how that would function.

The higher level issue that I'm pointing out is that you're using
English as though it were a programming language -- complete with
clear lexical scoping rules and unambiguous syntax.  Unfortunately,
written prose isn't like that, even in a specification, and
particularly in an IETF document.

The essence of a good RFC is clarity.  It's the ability to produce
interoperable implementations by multiple vendors and by multiple
readers who may be separated by time, distance, and native language.

I don't believe it's wise to sacrifice clarity for the sake of
precision or economy.

> action on it - at least not on the basis of a single comment from one

I guess my time was well spent.  ;-}

> term meaning "a limiting case of a mathematical system that is more 

Of course.

> example is a degenerate circle represented by the equation -

You might have noticed that the phrase "degenerate case" is used twice
in this section and appears to refer to two subtly different cases.

> occurs would be unecessary.

I'll offer that injecting explicit end-station forwarding entries into
the TRILL database (one of the options that's discussed multiple times
in the text, along with all of its disadvantages) is *NOT* consistent
with traditional bridge learning, so it's apparent that strict
consistency with 802.1D or 802.1Q isn't the point, either.

> in a way that is consistent with bridge learning.

It would be a bizarre bridge indeed that requires the end stations to
somehow "know" the local bridge MAC address in order to set the MAC DA
correctly, or that alters the MAC source address of the original frame
in transit.  I don't see how that could possibly work right.  What
you're talking about above seems to be the behavior of a router, not a
bridge.

I expect RBridges to be (first and foremost) layer two bridges.  I'm
surprised that we have to talk about consistency with 802.1D or 802.1Q
learning, as the learning that RBridges must do is actually somewhat
different (particularly the tunnel egress portion), and because the
learning that other bridges do (or don't do!) is completely invisible
to any RBridge implementation.

Even if we were somehow completely inconsistent with those other
documents, I fail to see how it would be an architectural issue for
RBridges, which _must_ be able to stand on their own.

> belong in an architectural specification.

I don't agree.  It's a crucial bit of the architecture.  It's how the
learning function operates, and it's one of the things that
distinguishes RBridges from regular bridges.

The document goes to the trouble of describing learning based on
source MAC address on the sender (encapsulator) RBridge, but then
leaves out how the destination (decapsulator) RBridge learns the
reverse path.  That seems like a hole to me.

> an architectural requirement at all.

If you do data-based learning, rather than injecting the end station
addresses into the IS-IS data, then you *MUST* do as I outline above.
There's no other option other than flooding everything, and that's not
reasonably viable.

> section 3.2.3 (Ingress TRILL Forwarding Database).

The two sections, one after the other, are almost word-for-word
identical.  This is a comment on the layout of the text.

> populated by learning from (potentially flooded) data frames on egress.

Except, as you've pointed out before, that's a "detail" of the
protocol implementation ... right?

> ones you suggest.  This was not my choice, but direction from the WG.

The architectural reason to do so is that it allows the receiver (the
egress RBridge) to *assume* that the sender's nickname is the reverse
path for the source MAC address in the decapsulated packet.

If this assumption is broken, then there's no way to do this right, as
long as the architecture allows for data-driven learning.  The section
of text I'm commenting on is this:

Note that it says "in most case" [sic].  It's allowing for egress
RBridge and ingress (PPOA) being different, and I don't think that's a
viable architecture.

> S-1 and S-3, while RB-2 is the PPOA for S-2.

Agreed.  That works fine.  That's not the problem I'm pointing out.

If RB-2 is the PPOA for S-2, and encapsulates the packets for S-2, but
the RB-2 is unable or unwilling to be the decapsulation point, how
does it know to put RB-1's nickname into the TRILL header?

That's what happens if you allow a protocol implementation to split up
the encapsulation point from the decapsulation point: there has to be
some way for the encapsulator to present the right TRILL nickname as
the source such that the return packets will be sent to the
decapsulation point.  The architecture contains nothing that would
allow this to happen in any obvious way.  (Or, really, any reason to
bother supporting such a thing.)

In other words, I see this as just a needless complication.

(It's possible that you're trying to describe a transient condition
that may occur during re-election of a new DRB.  If so, then perhaps
it should be made clear that although this *could* happen, it's not
the expected long-term behavior of the system.)

> that I had tried to avoid this confusion by assuming that a DRB will

My comment in this section wasn't actually about the DRB selection.
It was instead about split encapsulation/decapsulation by a single end
station.

> appropriate place to make such observations.

Just as the choice to use IS-IS is architectural, the high-level
changes that must be made to IS-IS in order to make it usable as the
chosen solution (i.e., how well it fits the fundamental requirements)
are *also* architectural issues.

If you don't want the fit-to-function of IS-IS to be architectural,
then I think the choice of the routing protocol needs to be ruled out
of scope for this document as well.  (No, I don't think that's a good
idea at all, but I don't see how the architectural choice of IS-IS as
a basis can be made off-hand without discussing the implications.)

> consistently likely to remain true over the life of, an RFC.

For one thing, it runs directly on the link layer, and can be
specified such that no user configuration is necessary to make it
work.  In contrast, the alternatives (such as OSPF) run on network
layer protocols that require explicit configuration of subnets and the
like.

> changed, I was not told about it.

If that's not done, then how do we ever achieve our zero-configuration
goal?

This seems obvious to me, and it's something we've discussed several
times in the working group.  It's a point on which I believe we
already have substantial consensus.

> and the assignment of VLAN 1.

See draft-eastlake-trill-rbridge-isis-00, section 2.1, bullet item 3:

   3. TRILL uses a distinct, constant IS-IS Area Address that would
      never appear as a real Layer 3 IS-IS area address. This Area
      Address is the value zero. (See Section 4.1.)

> the architecture document?  This sounds like a protocol operation

The use of a fixed area is architectural.  The exact area number and
the optional features that might be related to it are not.

> "problem."  The abstract further clarifies that the role of this 

I emphatically disagree with that position.

First of all, we already have an abstract problem statement.  It's
described in draft-ietf-trill-prob-02.

That's not something I believe this document needs to do.  Instead,
this document needs to describe the architecture of a solution to the
problem statement.  "Architecture," in this instance, means the
components, interactions, and behaviors that will be and *should be*
combined to produce the desired results.  The problem itself isn't
architecture, as far as I understand it.

That necessarily describes a solution, and I don't see how we can have
an architecture document that doesn't actually address a solution.  If
that's what you and the other wg participants really want, then please
leave my name off as a reviewer.  That's a dead end to me.

This sort of all-and-the-kitchen-sink approach, where we don't make
actual decisions in this working group, but instead allow for a range
of purported but never realized "options" is _exactly_ what will lead
to interoperability problems and a failure of the group to specify
something useful.

I fear a lack of specificity much more than I do "offending" one or
more working group participants who don't want to have a simple and
unambiguous document.  Pretending as though we'll design something
that will work utterly differently with regard to STP is, in my
opinion, a terrible mistake.  We're on better ground when we rule
_out_ useless possibilities than when we make unending allowances for
them.

It's always easier to add options and features in the future than it
is to rule out mistaken features that were added just for the sake of
flexibility.

> about RBridges is required; it is actually more important to know

I think the reader must actually understand *both*.

> of the document.  Understading of this text is "nice-to-have" but

Pretension or not, I don't see how defining the terms nowhere but
using them regardless is a solution to that particular problem.

> STP race conditions, or oscillations, depending on exactly how the

I did look into the problem, but I don't see any such modes.

If TRILL starts up first, then we end up with separate routes.  If
that STP link comes up later, then we'll have a temporary loop
(mitigated by TRILL's hop count) until the two RBridge neighbors
discover each other, and establish a new single DRB.

In the other order, TRILL will immediately detect the path, and set up
a single DRB as expected.

In either case, the situation is then stable.  TRILL does *not* cause
STP to recalculate anything.  TRILL doesn't shut down links, which are
the only things that STP really cares about here.

There's no oscillation, because there's no path that allows for
feedback -- a necessary condition of oscillation.  There is
potentially a *transient* as the topology settles, but the system is
dynamically stable.

> true, given the fact that section 5 is now a tutorial.

I think it's an interesting problem to solve, and worth mentioning,
but far from the threat that you seem to be describing.

In particular, network reconfiguration can by itself create such a
temporary loop without ever needing to invoke the "wiring closet
problem."  That's why the TRILL headers will have hop counts.  It's in
the nature of the beast when you start to deal with routing protocols.

I don't doubt that such things make 802.1 folks queasy.

> > Note that 

OK.

>    {root RBridge, egress RBridge} pairs."

OK.  The rest is still in future tense, but that corrects the odd jump
in mood.

>    toward the egress RBridge, via the TRILL Campus."

OK.

---

From: "Eric Gray" <[email-masked]>
Date: Mon, 19 May 2008 15:53:07 -0500

James,

	I did thank you for your review and comments and I
did not mean to do so in a perfunctory way.  I sincerely
appreciate your comments and the time you spent making
them.

--
Eric Gray
Principal Engineer
Ericsson  

> From: James Carlson [mailto:[email-masked]] 
> Sent: Thursday, May 15, 2008 5:15 PM
> To: Eric Gray
> Cc: TRILL/RBridge Working Group
> Subject: RE: [rbridge] long-awaited review comments 
> Importance: High