Subject: Re: RE: [802.1] 802.1ad Customer FCS Retention

From: Stewart Bryant <[email-masked]>
Date: Tue, 25 Nov 2003 11:14:05 +0000

The IEEE 802.1 community may not be aware, but in addition to the
encapsulation protocols, PWE3 is also standardising an OAM protocol
designed to validate the PW.

http://www.ietf.org/internet-drafts/draft-ietf-pwe3-vccv-01.txt

This does not do a thorough bit integrity test, and maybe it
needs to be extended to do so.

Perhaps the best way for a customer to validate the PW is
with an OAM protocol that runs between the CEs, rather than
trying to discover the problem implicitly through the FCS?

Stewart

Mick Seaman wrote:

> There seems to be an undercurrent in this discussion that somehow an FCS-32
> is good protection against arbitrary packet mangling. Sorry, not true.
> 
> The FCS/CRC-32 is designed to protect against a limited number of changes in
> bits, and was designed against a noise error model that rated changes in
> more than a certain number of bits improbable. You can't reliably use the
> CRC to defend against a wider range of errors such as omission or
> replacement of whole bytes or words (which has been know to sporadically
> occur in DMA controllers designed close to their limits), or substitution of
> part of the frame for another frame (which might occur if frames were
> interleaved on a synchronous clock and the clock slipped a bit position once
> in a while for example). Nor does the FCS protect well against software
> errors where a relay system does something other than it is supposed to do
> when forwarding a frame.
> 
> The protection offered by a CRC-32 against such events is little better than
> the chance of randomly guessing the right number in 2**32. To put that in
> perspective if all frames on a fully utilised 10 Gb/s link were errored you
> would get a frame with a good FCS about once every 20 minutes. Of course if
> all the frames were errored the link would probably come down, but spread
> around a network or many the errored 10Gb/s aggregate is more plausible. Now
> if you factor out all the errors that are going to get caught with the exact
> same CRC algorithm used on a link by link basis then the extra CRC is pretty
> much a waste of 4 bytes. At least use a different integrity check if you are
> serious! (and not one that the existing hardware is not going to
> accidentally emulate by accident when there is a software bug!).
> 
> This discussion seems to be largely driven by philosophy, if it is going to
> go anywhere at all there better be an engineering component.
> 
> Separately you are not going to get bit level transparency for a bridged
> network. In particular we have already established the layering model will
> not be broken to allow media specific frames (PAUSE, LACP) etc. to transit
> though bridges.
> 
> Mick
> 
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Scott Kotrla
> Sent: Monday, November 24, 2003 9:40 AM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: RE: [PWE3] RE: [802.1] 802.1ad Customer FCS Retention
> 
> 
> 
> As a service provider, I want layers.  I want to be as a transparent as
> possible.  Throwing away the customer's FCS and introducing the
> possibility of undetected errors is not transparent.  VLAN stacking
> isn't the best solution here for layer encapsulation, but it can work.
> MAC-in-MAC is architecturally a better fit because the whole customer
> frame goes into the new payload field, but wasting 12 bytes for dummy
> DA/SA that won't be used for TRANS (MEF layer) applications doesn't make
> a lot of sense.  ETH layer functionality is the application of existing
> 802 technologies to service providers, so arguments about what exactly
> does or does not belong has value.
> 
> TRANS layer functionality for physical Ethernet transport is new
> functionality that 802 was not designed for.  But for service providers,
> this provides new requirements.  The TRANS layer often refers to
> transport over SONET, but there is no reason why it shouldn't also
> support the model of transport over Ethernet, in which case the
> requirements are different from ETH requirements.  If physical Ethernet
> is being used for point-to-point TRANS functionality, new features like
> a Path FCS are required.  Otherwise, Ethernet over Fiber is not a viable
> alternative to Ethernet over SONET.
> 
> In this context, it may be clearer how Stacked VLANs and PW are
> comparable.  As a provider, I am looking for a way to tunnel customer
> ports or customer VLANs from point-to-point from end-to-end
> transparently across the access network.  I am open to other
> alternatives.  Ethernet TRANS should maintain customer FCS end-to-end as
> a separate layer from the service Ethernet Link FCS.
> 
> As some background, what started MCI's quest for FCS retention was a lab
> infrastructure Ethernet switch that was corrupting packets and
> calculating a new FCS on packets where no changes should have been
> occurring.  This was one of the top Ethernet switch vendors as provided
> by Gartner in 2002.  This took days to troubleshoot, and as service
> providers, scared the Hell out of us.  The idea of VLAN stacking, VLAN
> swapping, marking as discard eligible, and messing with packets in
> general seems to open up the possibility of such things happening even
> more.
> 
> Thanks,
> 
> Scott
> 
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Friday, November 21, 2003 11:16 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: RE: [PWE3] RE: [802.1] 802.1ad Customer FCS Retention
> 
> 
> Hi Norm....We are on the same page I am sure, just using different
> words....please see in-line.
> 
> regards, Neil
> 
> 
>>Norman Finn wrote 21 November 2003 12:03
>>
>>
>>Neil,
>>
>>I think Pat Thaler's reply helps to straighten out this
>>confusion, but let me
>>try, also.
>>
>>There is a *lot* of confusion about the words "end-to-end",
>>and about "transport layers".
> 
> NH=> Agreed.  G.805 and G.809 func arch really helps sort this out...as
> I hope was clear this week from Maarten's excellent analysis of the
> ethernet OAM issues at SG13.
> 
> 
>>There are lots of ends in this picture, and there are layers
>>and there are layers.
> 
> NH=> Agreed.....just wish some more people grokked this stuff as it
> should help mitigate arch violates.
> 
>>An 802.1Q VLAN bridge, and by extension, a P802.1ad provider
>>bridge, does NOT supply a
>>"transport layer" in the same sense as IP-over-Ethernet or
>>Ethernet-over-IP.  This is
>>because, at the data forwarding layers, VLAN tags are best
>>though of as extensions to
>>the address space, not as additional protocol layers.
> 
> NH=> Understood and agree.
> 
> 
>> If you
>>want to call it a
>>"transport layer", feel free!
> 
> NH=> No, that would be completely wrong....VLANs do *not* form a layer
> network in any way, shape or form.
> 
> 
>> But don't assume that VLANs
>>are equivalent to other
>>"transport layers".  Data integrity, specifically, is
>>different in bridges.
>>
>>Routers provide "best effort" not only in frame delivery, but
>>in data integrity.  There
>>is no upper limit specified for the probability of delivery
>>of a packet with undetected
>>errors.  IP's response, if you demand a particular level of
>>data integrity, is that
>>your application must supply its own checksum.
>>
>>Bridges, however, are not routers!
>>
>>Bridges *do* guarantee a level of service.  They manage to do
>>this by guaranteeing the
>>quality of each bridge, and by *demanding* the quality of
>>each link.  Non-VLAN bridges,
>>VLAN bridges, and Provider Bridges all make that guarantee to
>>their users using this
>>method.  The Customers of a Provider Bridge service are not
>>an "802.1Q application"
>>over an "802.1ad transport layer".  They are, in the 802.1
>>model, just endstations
>>attached to a bridge.  The Customers' data is just as well
>>protected as an enterprise
>>bridges' endstations, as long as the model is maintained.
>>Adding an additional CRC
>>to end-to-end service, where the ends are Customers attached
>>to 802.1ad Provider
>>Bridges, is therefore pointless.  Changing that model is as
>>silly as turning the model
>>for routing upside down.
> 
> NH=> I agree with all this....so not quite sure what you think I am not
> understanding here.
> 
>>When a Pseudowire is used to connect two bridges,
> 
> NH=> Similar to VLANs, a PW is *not* a layer network either.  MPLS does
> create layer networks however....but I suspect that is not well
> understood by many.  One of my colleagues, who like Maarten is a real
> expert in the func modelling detail, is formally describing MPLS right
> now...but of course MPLS is a not a singular layer network, it has many
> flavours and this work will clearly bring out the different arch
> behaviours.
> 
> 
>>the
>>"end-to-end" is now just those
>>two bridges.
> 
> NH=> Agreed.  If one drew out the proper func arch model one can see
> this immediately.
> 
> 
>>Those bridges are using that PW as an Ethernet
>>to carry their traffic.
>>The bridges demand that every wire be as reliable as an
>>Ethernet.  That's the PW CRC
>>issue.
> 
> NH=> Grokked totally. A PW is a form of adaptation function that lies
> between a client (here we are discussing ethernet) and MPLS (or IP if
> one is happy with a cnls server behaviour)...and then there are further
> layer networks below this, eg the VC4 layer network of SDH, until we hit
> the duct.  The fact that a PW is not a layer network and not really
> necessary is neither here nor there....other than its just something
> else to manage.  When the port on a bridge is not connected to an
> 'ethernet shared media' (orginal case), but some technology (any) then
> one simply has to adapt the ethernet PDU (*including its FCS*, as its
> part of the client ethernet CI) to the payload of the server layer.  One
> should *not* be playing around with client layer network structures in a
> simple client->server adaptation process at all.  So I am totally
> against all the 'Martini' PW compression stuff....the BW squeezing is
> simply not worth it (compared to other technolgies that will squander
> BW). So !
>  IMO the ethernet FCS is part of the ethernet's CI and should not be
> removed by *any* server layer technology.
> 
>>-- Norm
>>
>>[email-masked] wrote:
>>
>>>Norm....As per our discussion today face-face.  I don't
>>
>>really see what the problem is here.  The client layer
>>network should not care about the server layer network (and
>>vice versa)....we need this layer independence for lots of
>>reasons.  An ethernet traffic unit has a FCS.  This should
>>simply be transported transparently in the payload area of
>>whatever the server layer is....and the server layer should
>>use its own (OAM) 'integrity mechanisms' irrespective of the
>>client.  Problems arise when people start doing things like
>>client layer compression and thus forming inter-layer dependencies.
>>
>>>Further, as we agreed today and as I posted on an earlier
>>
>>mail to this thread, the ethernet FCS is a CRC that can be
>>linerarly updated for adaptation function changes (eg VLAN ID
>>change) on each link-connection (as provided by some server
>>layer trail technology) whilst preserving its end-end
>>intergrity check of the rest of the ethernet traffic unit
>>between its trail termination points.
>>
>>>regards, Neil
>>>
>>>
>>>
>>>>-----Original Message-----
>>>>From: Norman Finn [mailto:[email-masked]]
>>>>Sent: 18 November 2003 08:56
>>>>To: mark seery
>>>>Cc: [email-masked]; [email-masked]
>>>>Subject: Re: [PWE3] RE: [802.1] 802.1ad Customer FCS Retention
>>>>
>>>>
>>>>Mark,
>>>>
>>>>You say below,
>>>>
>>>>
>>>>>but a PW will be running over a physical media itself
>>>>
>>>>already, possibly
>>>>
>>>>>even SONET.............
>>>>
>>>>Exactly!  Here, the trail gets a bit murky.  If I'm in total
>>>>control, and the PW
>>>>is running over known Ethernet links, then I can get away
>>>>without the CRC in the
>>>>PW.  I have no problem whatsoever with the current definition
>>>>of PWs without CRC.
>>>>
>>>>But, if I don't have tight control over the physical media
>>>>over which the PW runs,
>>>>then I really need a CRC on the PW for the bridge to be able
>>>>to trust the "wire"
>>>>and thus provide the same level of data integrity that
>>>>bridges offer over physical
>>>>Ethernets.  So, I claim that bridges need a CRC option on PWs.
>>>>
>>>>-- Norm
>>>>
>>>>mark seery wrote:
>>>>
>>>>
>>>>>Norman Finn wrote:
>>>>>
>>>>>
>>>>>
>>>>>><snip>
>>>>>>In other words, routers pass about $0.93 of the data
>>>>>
>>>>integrity buck to
>>>>
>>>>
>>>>>>bridges,
>>>>>>and bridges pass about $0.75 of that to the data links.
>>>>>
>>>>The buck has
>>>>
>>>>
>>>>>>to stop
>>>>>>somewhere, and the data link is where it stops.  The PW
>>>>>
>>>>has to take
>>>>
>>>>
>>>>>>the same
>>>>>>hit as the physical media.
>>>>>>
>>>>>>-- Norm
>>>>>
>>>>>
>>>>>but a PW will be running over a physical media itself
>>>>
>>>>already, possibly
>>>>
>>>>
>>>>>even SONET.............
>>>>>
>>>>>mark
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>_______________________________________________
>>>>pwe3 mailing list
>>>>[email-masked]
>>>>https://www1.ietf.org/mailman/listinfo/pwe3
>>>>
>>>
>>>
>>
>>
>>_______________________________________________
>>pwe3 mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/pwe3
>>
> 
> 
> 
> 
> 
> 
> _______________________________________________
> pwe3 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pwe3
> 
>

---

From: Scott Kotrla <[email-masked]>
Date: Tue, 25 Nov 2003 10:35:43 -0600

So are you suggesting because a CRC-32 will miss 1 in 4 billion frames
it is not useful to know?  I don't need to catch every frame to know
there is a problem.

The standard Ethernet CRC protects the link and not the internal switch,
and it is very difficult to put requirements on the internal working of
a switch.  As stated in a previous message:

"The model of operation is simply a basis for describing the
functionality of the MAC Bridge. It is in no way intended to constrain
real implementations of a MAC Bridge; these may adopt any internal model
of operation compatible with the externally visible behavior that this
standard specifies. Conformance of equipment to this standard is purely
in respect of observable protocol."

I am not worried about link errors.  I am worried about corruption
within switches.  If switches have unknown failure modes that add
unknown corruption, what besides a Path FCS will be able to determine
that there is a problem.  Once again, detection is at least 10 times as
important as diagnosis.  If we can come up with a better way to detect
when (not if) switches are corrupting packets, I would be very
interested in understanding the benefits of this approach.

The service provider requirement can be succinctly stated as:

The network shall be able to detect when (not if) switches are
corrupting packets.  Suggesting that switches will never unknowingly
corrupt packets is wishing the problem away and is not an acceptable
answer.  Never is a very long time.  Besides, out of the 100 or so
switches I have worked with, I have already seen it happen.  And I am
not all that lucky.

Thanks,

Scott

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Mick Seaman
Sent: Monday, November 24, 2003 2:05 PM
To: [email-masked]; [email-masked]
Subject: RE: [PWE3] RE: [802.1] 802.1ad Customer FCS Retention

There seems to be an undercurrent in this discussion that somehow an
FCS-32
is good protection against arbitrary packet mangling. Sorry, not true.

The FCS/CRC-32 is designed to protect against a limited number of
changes in
bits, and was designed against a noise error model that rated changes in
more than a certain number of bits improbable. You can't reliably use
the
CRC to defend against a wider range of errors such as omission or
replacement of whole bytes or words (which has been know to sporadically
occur in DMA controllers designed close to their limits), or
substitution of
part of the frame for another frame (which might occur if frames were
interleaved on a synchronous clock and the clock slipped a bit position
once
in a while for example). Nor does the FCS protect well against software
errors where a relay system does something other than it is supposed to
do
when forwarding a frame.

The protection offered by a CRC-32 against such events is little better
than
the chance of randomly guessing the right number in 2**32. To put that
in
perspective if all frames on a fully utilised 10 Gb/s link were errored
you
would get a frame with a good FCS about once every 20 minutes. Of course
if
all the frames were errored the link would probably come down, but
spread
around a network or many the errored 10Gb/s aggregate is more plausible.
Now
if you factor out all the errors that are going to get caught with the
exact
same CRC algorithm used on a link by link basis then the extra CRC is
pretty
much a waste of 4 bytes. At least use a different integrity check if you
are
serious! (and not one that the existing hardware is not going to
accidentally emulate by accident when there is a software bug!).

This discussion seems to be largely driven by philosophy, if it is going
to
go anywhere at all there better be an engineering component.

Separately you are not going to get bit level transparency for a bridged
network. In particular we have already established the layering model
will
not be broken to allow media specific frames (PAUSE, LACP) etc. to
transit
though bridges.

Mick

-----Original Message-----

Thanks,

Scott

-----Original Message-----

regards, Neil