Subject: [rbridge] FTag

From: sgai at nuovasystems.com (Silvano Gai)
Date: Sun, 22 Oct 2006 12:00:45 -0700

Eric,

This is your point of view and I disagree with it.

For the High Performance market (very relevant to TRILL), the switch
port ASICs are all limited by the buffer memory bandwidth. Therefore I
reaffirm my point that having a solution that does not require payload
realignment is superior.

Your point about MPLS is invalid, because I showed you with the diagram
that MPLS, .1Q, and 1S do not require payload realignment.

Your point about address representation in implementation is invalid,
since I am talking about payload realignment and I never mentioned
anything about header storage/processing.

-- Silvano

> -----Original Message-----
> From: Gray, Eric [mailto:Eric.Gray at marconi.com]
> Sent: Friday, October 20, 2006 12:05 PM
> To: Silvano Gai
> Cc: rbridge at postel.org
> Subject: RE: [rbridge] FTag
> 
> Silvano,
> 
> 	I was working on the replies to you analysis below, thinking
> I would not complete them this week.  As you can see, I had a bit
> more time than I expected to have and was able to complete my
> response earlier than I thought.
> 
> 	As a first observation, whenever someone says "this is not
> hardware friendly", my gut reaction is to ask "for what hardware"?
> 
> 	As a second observation, although the number of MPLS labels
> may change, the Ethernet encapsulations defined for PWE3 already
> uses a 32-bit alignment assumption independent of Ethernet frame
> alignment either "below" or "above" the MPLS SHIM.  Consequently,
> there is definitely hardware in existence, in development or both
> that deals with this alleged "alignment issue."
> 
> --> -----Original Message-----
> --> From: Silvano Gai [mailto:sgai at nuovasystems.com]
> --> Sent: Wednesday, October 18, 2006 12:04 PM
> --> To: Gray, Eric
> --> Cc: rbridge at postel.org
> --> Subject: RE: [rbridge] FTag
> -->
> -->
> -->
> --> Eric,
> -->
> --> Let me depict you what happens when you insert one MPLS
> --> label, [or] a 1Q option or a 1S option
> -->
> --> Original:
> --> +--------------------------------+
> --> |               DA               |
> --> +----------------+---------------+
> --> |       DA       |     SA        |
> --> +----------------+---------------+
> --> |               SA               |
> --> +--------------------------------+
> --> |    Ethertype   |   Payload     |
> --> +--------------------------------+
> --> |                                |
> --> |         ...........            |
> -->
> -->
> --> Original + Opt (where Opt can be MPLS, 1Q or 1S)
> --> +--------------------------------+
> --> |               DA               |
> --> +----------------+---------------+
> --> |       DA       |     SA        |
> --> +----------------+---------------+
> --> |               SA               |
> --> +--------------------------------+
> --> |    Ethertype   |      Opt      |
> --> +--------------------------------+
> --> |      opt       |   Payload     |
> --> +--------------------------------+
> --> |                                |
> --> |         ...........            |
> -->
> --> The HW that perform the insertion/removal does not need to
> --> realign the payload that is in its buffer.
> -->
> 
> This is an interesting, but inefficient, storage scheme in the
> first place.  With this storage scheme, the same DA and/or SA
> values will be stored in uncompressed format hundreds or even
> thousands of times in any sizable buffer.
> 
> I think it is unrealistic to assert that this is the only way
> that this information might be stored in a buffer.  If it is
> not stored in this precise fashion, then the below observations
> are invalidated.
> 
> -->
> --> Now Let's consider the encapsulation/decapsulation function
> --> by looking at the Original frame + TRILL header (as proposed
> --> by the current draft)
> 
> Let's be clear about what is proposed by the current draft.
> 
> The encapsulation below is not - in any way - proposed by the
> current draft as it does not assume that there will be a 16-bit
> "slop factor" as you indicate with "????????" in your picture.
> 
> --> +--------------------------------+
> --> |         Outer DA               |
> --> +----------------+---------------+
> --> |    Outer DA    |  Outer SA     |
> --> +----------------+---------------+
> --> |         Outer SA               |
> --> +--------------------------------+
> --> |Ethertype=TRILL |   Trill Shim  |
> --> +--------------------------------+
> --> |  Trill Shim    |  ????????     |
> --> +--------------------------------+
> --> |               DA               |
> --> +----------------+---------------+
> --> |       DA       |     SA        |
> --> +----------------+---------------+
> --> |               SA               |
> --> +--------------------------------+
> --> |    Ethertype   |   Payload     |
> --> +--------------------------------+
> --> |                                |
> --> |         ...........            |
> 
> If this was proposed by "the current draft", then it should be
> put forward as "an example only" in later versions.  An equally
> valid example may be either of the following:
> 
>  +--------------------------------+
>  |         Outer DA               |
>  +----------------+---------------+
>  |    Outer DA    |  Outer SA     |
>  +----------------+---------------+
>  |         Outer SA               |
>  +--------------------------------+
>  |Ethertype=TRILL |   Trill Shim  |
>  +--------------------------------+
>  |  Trill Shim    |      DA       |
>  +--------------------------------+
>  |               DA               |
>  +----------------+---------------+
>  |               SA               |
>  +----------------+---------------+
>  |       SA       |   Ethertype   |
>  +--------------------------------+
>  |            Payload             |
>  +                                +
>  |          ...........           |
> 
> OR
> 
>  +--------------------------------+
>  |         Outer DA               |
>  +----------------+---------------+
>  |    Outer DA    |  Outer SA     |
>  +----------------+---------------+
>  |         Outer SA               |
>  +--------------------------------+
>  |Ethertype=TRILL |   Trill Shim  |
>  +--------------------------------+
>  |  Trill Shim    |      DA       |
>  +--------------------------------+
>  |               DA               |
>  +----------------+---------------+
>  |               SA               |
>  +----------------+---------------+
>  |       SA       |     0x8100    |
>  +--------------------------------+
>  | P |  VLAN ID   |   Ethertype   |
>  +--------------------------------+
>  |            Payload             |
>  +                                +
>  |          ...........           |
> 
> And there are others.  Note that this is a depiction of what a
> frame would look like "on the wire".  It is inappropriate for
> us to make assumptions about what it would look like in memory
> for any specific implementation.
> 
> In any case, when it goes on the wire, the 16-bit "field" shown
> in your example would not be present unless there was reason to
> have it.
> 
> -->
> --> If you don't have the 16 bits indicated by ???????, the
> --> hardware will need to realign the inner frame. The frame
> --> is not on the wire. The frame sits in the [frame] buffer
> --> and the memory bandwidth of the packet buffer is today
> --> the MOST important consideration in port ASIC design.
> 
> From the above discussions, you should be able to see the following:
> 
> 1) It is inappropriate to make assumptions about how frames are
>    stored in memory prior to being put on the wire as we are not
>    here to standardize that.
> 
> 2) It is likely that any such assumptions will be incorrect in
>    at least some implementations.
> 
> Based on this, your otherwise reasonable argument is invalidated
> as it is based on inappropriate, and incorrect, assumptions about
> implementations.
> 
> -->
> --> More inline
> -->
> --> > -----Original Message-----
> --> > From: Gray, Eric [mailto:Eric.Gray at marconi.com]
> --> > Sent: Wednesday, October 18, 2006 4:57 AM
> --> > To: Silvano Gai
> --> > Cc: rbridge at postel.org
> --> > Subject: RE: [rbridge] FTag
> --> >
> --> > Silvano,
> --> >
> --> > Let me answer your points in order, leaving out irrelevant
issues:
> --> >
> --> > <Your Point> N*32 is correct, when including Ethernet
options/shims
> --> >              (e.g. .1Q or .1S).  This is hardware friendly,
because
> --> > 		 alignment doesn't change going from one header
to
> another.
> --> >
> --> > There are a few different "Ethernet" encapsulations, once we
start
> to
> --> > include Ethernet options and the shims associated with those
> options.
> --> >
> --> > First, and I believe most important, the "shim" we are talking
about
> --> > (in TRILL) is not an Ethernet option (or associated shim).  It's
a
> --> > "shim" as a separate "layer."  Otherwise, it would be
inappropriate
> --> > for us to do this work here (in TRILL, as part of the IETF).
> -->
> --> From the above picture it should be clear that you should
> --> consider the TRILL header as a whole, and the TRILL header
> --> as only one form of Ethernet encapsulation.
> 
> Actually, what should be clear is that the intent is explicitly
> NOT to define an aggregate header type, combining a specific
> Ethernet header and TRILL SHIM.  Not only would doing so limit
> the usefulness of RBridges in general, it would also involve our
> doing work that is strictly out of scope in the IETF.
> 
> What we need to do is recognize that both tunnel, and native,
> encapsulations may be any defined Ethernet encapsulation that
> is supported by the interfaces of any specific implementation.
> 
> -->
> --> >
> --> > Second, "Ethernet" headers do not all align the same.
> --> >
> -->
> --> Agreed, but that is not my concern, My concern is to not
> --> change their alignment due to the insertion/removal of the
> --> TRILL header.
> 
> However, it is my concern.  A key reason for using a layer
> model in networking protocols is to separate functions and
> their definitions in ways that isolate the workings of one
> layer from the changes in another.
> 
> -->
> --> > Currently, we are talking about having a single Ethertype
defined to
> --> > distinguish inter-RBridge traffic from other types of Ethernet
> frames.
> --> > If your implementation produces 32-bit aligned Ethernet frames,
> using
> --> > 802.1Q, then it will similarly produce 32-bit aligned frames -
prior
> --> > to considering the TRILL shim - using the new Ethertype.
> --> >
> --> > We are not proposing a change in any Ethernet frame/header
format.
> -->
> --> Neither do I.
> 
> This is semantics.  You are clearly proposing that we should
> define a new type of Ethernet header - calling it a TRILL
> Ethernet header - and that is not what we're supposed to be
> doing.
> 
> -->
> --> >
> --> > The reference to 802.1Q is slightly disingenuous, because 802.1Q
> --> > effectively displaces the existing Ethertype by 32 bits.  802.1Q
> --> > "shims" include the Ethertype, because they replace the original
> --> > one with 0x8100 and have to include the original in the "shim."
> --> >
> --> > I assume that a similar observation could be made about 802.1S.
> -->
> --> Yes and a similar observation can be made for MPLS, but all these
> three
> --> do not change the alignment of the fields that follow.
> 
> And, as you may have noted from the above, if you're talking
> about frame alignment in a buffer, this depends a lot on the
> specifics of any implementation.
> 
> As I pointed out above, there is already a "proof of concept"
> example in PWE3 Ethernet encapsulation.
> 
> -->
> --> >
> --> > In TRILL, we are not replacing the original Ethertype, we are
adding
> --> > a new encapsulation.  The original Ethertype stays in the
original
> --> > - re-encapsulated, or tunneled - Ethernet frame.  Therefore,
there's
> --> > no reason to consider the Ethertype to be part of the TRILL
shim.
> -->
> --> As I told before you need to consider the whole TRILL header, i.e.
> --> 6 bytes of DA
> --> 6 bytes of SA
> --> 2 bytes of Ethertype
> --> 4 bytes of shim
> --> ----------------------
> --> 18 bytes
> 
> Yes, you have asserted this, and I have refuted it.
> 
> Hence we are in disagreement on that point and we should let other
> people make their own assertions (and refutations) and see how things
> turn out.
> 
> -->
> --> Which is 2 bytes short of N*32, i.e. it is not HW friendly.
> -->
> --> For this reason I propose that Ethertype + shim should be
> --> N*32 to be HW
> --> friendly.
> -->
> -->
> --> >
> --> > <Your Point> TRILL should adopt this model for its shim header -
the
> --> >              Ethertype should be included as part of computing
> "shim"
> --> >              size.
> --> >
> --> > In TRILL, we are not replacing the original Ethertype, we are
adding
> --> > a new encapsulation.  The original Ethertype stays in the
original
> --> > - re-encapsulated, or tunneled - Ethernet frame.  Therefore,
there's
> --> > no reason to consider the Ethertype to be part of the TRILL
shim.
> --> >
> --> > <Your Point> TRILL should optimize the Ethernet case that will
cover
> --> >              99% of TRILL applications.
> --> >
> --> > Coverage is slightly more complete than that.  Considering all
of
> the
> --> > layer 2 encapsulations that currently come under an Ethernet
> umbrella,
> --> > TRILL applications are 100% included.
> --> >
> --> > I think we are essentially in agreement on this point.
> --> >
> --> > <Your Point> TRILL bridges will be implemented mainly in
hardware
> and
> --> >              the encapsulation and decapsulation functions need
to
> be
> --> >              HW friendly.
> --> >
> --> > Okay.  I was not making any other point - except to say that
> alignment
> --> > is usually a software concern, especially in initial design.
> --> >
> --> > Let me be blunt, alignment does not factor into initial hardware
> design,
> --> > but is an issue with hardware re-use complexity.  Bits on a wire
are
> not
> --> > "aligned" - you simply collect them as they arrive, serially.
> -->
> --> I agree that bits on the wire are not aligned, but you need to
buffer
> --> the incoming frame (or at least part of it) until you have enough
> fields
> --> to perform your lookup and decide what to do with the frame. From
that
> --> point on the frame is not on the wire, it is in the packet buffer
and
> --> alignment is crucial at the performance we are talking about.
> 
> This is perhaps the most specious part of the argument.  While a huge
> majority of switch builders use essentially similar buffering models,
> at an architectural level, the details of how and where the frames are
> buffered is very much a part of the "secret sauce" of implementations.
> 
> Given the number of potential ways of doing input buffering, and the
> likelihood that we are going to be able to get enough data on specific
> implementations to perform any sort of meaningful statistical
analysis,
> I see no reason to assign any greater validity to the assumption that
> frame alignment is important than to the assumption that it is not.
> 
> -->
> --> I hope this time I made my point more clear
> 
> It did.  I had thought you were talking about the importance of
> alignment in framing hardware, and you clarified that you were
> talking about buffering.
> 
> This doesn't change the fact that I think you're wrong, but it
> does help me to understand exactly what you're wrong about.  :-)
> 
> -->
> --> -- Silvano
> -- [SNIP] --