Subject: Re: [rbridge] draft protocol-10 WGLC Maximum Bridge Transit Delay

From: James Carlson <[email-masked]>
Date: Mon, 1 Dec 2008 09:15:29 -0500

Donald Eastlake writes:
> > Do modern bridges actually _implement_ such a feature, or do they just
> > have a knob labeled "Maximum Bridge Transit Delay" that's connected to
> > nothing underneath?
> 
> It is my impression that low end 802.1D bridges don't implement this
> but mid and high end 802.1Q bridges do. Typically, when a frame is
> received, a control block is created with the VLAN, priority, time of
> receipt, and other control information. When a frame is de-queued for
> transmission, the time is checked and frame discarded if it is too
> old. This time check doesn't need to be, and probably isn't, extremely
> accurate.

Under what conditions would that be expected to happen?

I suspect that one good reason to avoid bothering with such a feature
on high-end switches is that the amount of buffering required to reach
a 1 second delay at high line rates becomes prohibitive -- in other
words, you'll ordinarily drop on queue entry well before that happens
in all but pathological cases, so why bother caring?

> > I'm inclined to say "no" to this, just on the basis that I cannot see
> > how it would serve any real purpose.  It looks like a spec for the
> > sake of a spec.
> 
> In bridges, I think it is required to back up the assurances of
> Spanning Tree Protocol. I think bridges also flush the output queue
> and any input queue associated with a port when the port goes down.
> Generally, it seems like good hygiene not to keep a frame for a long
> time and then release it...

We're not using STP here, though, so we don't need to back up its
assurances.

As for avoiding a too-old release, I don't think that's easy to avoid
with modern hardware due to the funky 802 flow control mechanism.  If
the other end is having trouble, you can end up with a really old
frame caught in your transmission craw.  It'd require some fancy
footwork in most implementations I've seen to detect that sort of edge
case, reset, and restart.

> It would certainly be easy enough to make this a SHOULD rather than
> being mandatory the way it is in 802.1, not that people seem to pay
> that much attention to what is mandatory in 802.1. But perhaps it
> really isn't necessary for RBridges... Anyone else have an opinion on
> this.

That 'one second delay' smells to me too much like the old IP TTL
specification, and likely dates back to the same rules of thumb.

I think that SHOULD is a fairly strong statement; it means that
implementors who choose not to implement that feature need to have a
clear explanation of why they don't, if they're going to try to follow
the spirit of the RFC.  I don't think it ought to be a SHOULD unless
it's something that's required for interoperability in most cases, but
can be avoided in a few.  As 2119 says:

3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
   may exist valid reasons in particular circumstances to ignore a
   particular item, but the full implications must be understood and
   carefully weighed before choosing a different course.

Do RBridge implementors really need to weigh the pros and cons of
implementing the delay time check in order to achieve
interoperability?

MAY is closer to the intent of saying, "here's something that's common
in this field and that may be helpful in an implementation, but that
is not known to be required for the sake of interoperability of
RBridges."

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: James Carlson <[email-masked]>
Date: Mon, 1 Dec 2008 13:27:07 -0500

Donald Eastlake writes:
> On Mon, Dec 1, 2008 at 9:15 AM, James Carlson <[email-masked]> wrote:
> Hummm, it's not clear to me what the resolution of the maximum transit
> delay is... although 1 second is the default, the resolution could be
> small allowing it to be set to some number of milliseconds or
> something...

Actually, I wasn't talking about the resolution, but rather just the
amount of buffering you'd need at a single priority level to queue
across that much delay anywhere and not begin dropping.

> As above, you could have a stream of, say, 1 Gbps higher priority
> frames going from port 1 to port 3 and one lower priority frame for
> port 3 that drifts in port 2 and this poor low priority frame could
> just sit in a queue for days and then pop out due to a hiccup in the
> high priority stream. I won't argue if you want to call that
> pathological but it doesn't seem good that it could happen.

Yes, I'd call that pathological, but, ok, point taken.

> The specific provisions in 802.1D-2004 include :
> "6.3.6 Frame lifetime
> The MAC Service mandates an upper bound to the transit delay
> experienced for a particular instance of
> communication. This maximum frame lifetime is necessary to ensure the
> correct operation of higher layer
> protocols. ..."

If there's no limit on the number of bridges that may be encountered
in transit, and no limit on the speed-of-light delays between nodes,
then I think there's little that this transit delay limit does to
protect those upper layer protocols.

It limits variance in some cases, at the cost of higher loss, but
that's about it; it can't practically set any upper limit.  (For a
worst-case scenario, consider two adjacent nodes that are unable to
use a shared link due to redundancy elsewhere in the network.  Failure
of that redundancy can bring up that low-latency link, and restoring
the far-away link tears it down again.  The variance in that case may
be predictable if you know the entire topology, but it's arbitrarily
large.)

> Similar provisions appear in 802.1Q-2005.

Yep; I saw them.  I wasn't questioning whether they're there, but
whether they do any real good.

> Well, I'm OK with saying MAY. Since this is described in the 802.1
> standards as being in the port output queue behavior, and since we now
> incorporate that by reference unless we say otherwise, one could argue
> that it is mandatory under the current draft. Unless people weigh in
> with other opinions, I'll change it to MAY.

That seems fine.  (If we're incorporating all of 802.1 by reference,
do we need to restate much of it ... ?)

---

From: "Donald Eastlake" <[email-masked]>
Date: Mon, 1 Dec 2008 14:32:46 -0500

Hi Jim,

> Donald Eastlake writes:
...
> protect those upper layer protocols.

Seems like good points. I'm not aware of any hops limit for bridges
like the 7 level limit for hubs.

> be predictable if you know the entire topology, but it's arbitrarily
...
> do we need to restate much of it ... ?)

Not much of it. But, since we are essentially incorporating a snapshot
of the 802.1Q port processing below the EISS layer, I think we need to
say how we differ from 802.1Q in that area. Currently, the protocol
spec just says we differ for BPDUs and VLAN registration protocols.
Although it seems like a minor difference in practice, if enforcement
of a maximum bridge transit delay is a MAY rather than being mandatory
as it is in 802.1, I think we should say so.

Thanks,
Donald

> MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677
=============================
 Donald E. Eastlake 3rd   +1-508-634-2066 (home)
 155 Beaver Street
 Milford, MA 01757 USA
 [email-masked]
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: "Donald Eastlake" <[email-masked]>
Date: Mon, 1 Dec 2008 12:55:23 -0500

Hi Jim,

> Under what conditions would that be expected to happen?

Actually, I think I may have misstated what happens as you would want
old frames discarded that were just sitting around in a queue, not
just when they were de-queued.

All you need is enough higher priority frames and low priority ones
could be queued forever even with no link ever going down and no flow
control impediments.

>> time and then release it...

(As I recall, the 802.1 bridging model does not really admit that
input queues can exist so any frame in an input queue is logically
considered not to have been received yet by the bridge...)

"7.1.1 Relay
A MAC Bridge relays individual MAC user data frames between the
separate MACs of the bridged LANs
connected to its Ports. The functions that support relaying frames and
maintain QoS are as follows:
...
l) Frame discard to ensure that a maximum bridge transit delay is not
exceeded (6.3.6).
..."

"7.7.3 Queuing frames
...
A frame queued for transmission on a Port shall be removed from that
queue if that is necessary to ensure
that the maximum bridge transit delay (6.3.6, Table 7-3) will not be
exceeded at the time at which the frame
would subsequently be transmitted.
..."

Thanks,
Donald

---

From: "Anoop Ghanwani" <[email-masked]>
Date: Mon, 1 Dec 2008 18:20:04 -0800

I think it may be worth adding it as a MAY (or even a SHOULD).
Bridges do actually implement this and it can happen due to 
starvation of service, either because of strict priority 
queueing (as pointed out by Donald) or because link-level 
flow control such as 802.3x is in use.

For routers, according to RFC 1812

  When a router forwards a packet, it MUST reduce the TTL by at least
  one.  If it holds a packet for more than one second, it MAY decrement
  the TTL by one for each second.

We know how many people implement the optional part :-),
but at least the issue is discussed.

Anoop

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Donald Eastlake
> Sent: Monday, December 01, 2008 9:55 AM
> To: James Carlson
> Cc: Developing a hybrid router/bridge.
> Subject: Re: [rbridge] draft protocol-10 WGLC Maximum Bridge 
> Transit Delay
> 
> Hi Jim,
>

---

From: "Donald Eastlake" <[email-masked]>
Date: Sun, 14 Dec 2008 17:12:39 -0500

Hi,

I plan to resolve this by adding a Maximum Transit Delay per RBridge
configuration parameter and state that frames which have exceeded that
time MAY be discarded.

Donald