Subject: Re: RE: [802.1] 802.1ad Customer FCS Retention

From: mark seery <[email-masked]>
Date: Thu, 13 Nov 2003 17:59:18 -0800

Hi Norman.

I certainly appreciate people with 802 experience chiming in. Matt's 
comments were very good, and you have provided much good background as well.

In your long note, I got the impression that your central argument is 
that a PW is not Ethernet grade unless it provides 32-bit CRC or better. 
I understand 802 wanting to define what meets it standard of goodness. 
By mandating this you are suggesting that a range of existing systems, 
and potentially chipsets, do not support this requirement - I'll assume 
that is a considered judgement from you having weighed all the 
implications. You are also suggesting that the things that we once did 
to support less sophisticated and reliable media must always be done, 
once again I'll assume you hve considered this and you remain convinced 
of this necessity. And lastly as Luca states, you are suggesting that 
you did not receive this email ;-) All joking aside, I am all for strong 
integrity checks, there are these other nagging issues. If the standards 
are driven in such a way that a new class of equipment is required, then 
that's OK with me too, because I am more likely to benefit from that, 
than if not. Perhaps inherent in your comments is the thought that 
service providers will deploy a new class of specialized provider 
bridges that do only that functionality. Perhaps that is possible as 
well, though that is unlikely the intent of the IETF PW group, but 
understand 802s right to pursue creating this opportunity.

On your desire to impress of your real world experience with PW and 
bridges, I don't doubt it and I am not sure what I said to inspire that 
reaction. As far as I recall all I have said is that I didn't believe 
"existing" bridges would be in the middle of a PW [because they don't 
speak PW].

On the assumption 802 makes that 802 is everywhere, I have no problem 
with that as it has served 802 so well, but I would observe that IP/IETF 
has thrived by assuming the complete opposite - such is life....

Once last comment to the group, I was forced to stay away from my 
computer to do some travel today which led me to recall that this 
industry has thrived precisely by adhering to some basic principles. One 
of which is if you a terminating a layer, you through away the 
lower-layer headers and trailers, and if you are not terminating a 
layer, you leave them alone. While I realize that observation leads to 
rat-holes about what is a layer (which the group has previously 
expressed a desire not to go down), the principle is powerful and useful 
(IMHO). On this, I have a feeling we *might* agree, but this of course 
brings up all the ugly issues, that your requirements do, as stated in 
the first paragraph.

best, and thanks for your feedback.

Mark

---

From: mark seery <[email-masked]>
Date: Thu, 13 Nov 2003 18:59:16 -0800

Norman Finn wrote:

> <snip>  End-to-end
> preservation of the Customer's original FCS is an interesting 
> proposition, but is
> not relevant to this formulation of the problem.

Norman I disagree.

First of all I don't think I stated it that way, I talked about was 
payload integrity. In an X over Y situation, if X is assuming that Y is 
maintaining the integrity of X's PDUs, and it does not, then that is a 
problem. X then processes its PDUs assuming they are not corrupt. This 
becomes a problem even with two bridges exchanging BPDUs over a PW, and 
is certainly a problem with a service provider making guarantees to 
subscribers.

The problem I suggested of X's PDUs being corrupted as Y removed a FCS 
and added a new one, is at least on the same theoretical plane as 
suggesting that modern facilities introduce errors at some significant 
level requiring a 32 bit FCS. Both might actually be false claims most 
of the time, but we protect for the times when it is not false.

So this gets back to the issue of transport vs relay vs routing, and the 
principles of terminating a layer or not, if adhered to would suggest 
that X's PDUs do not get changed because they are being transported over 
Y. X wanted Y to mess with its PDUs, it probably would have signed up 
for a routed service (where the layers below IP are terminated in every 
possible sense) rather than a transport service.

This is an issue that PW should be dealing with as well, but in the 
interest of time.....

So the draft from malis et el. address my particular concerns, but I 
respect the right of operators to assert they want certain layering 
principles adhered to, if in their opinion that is the solution they 
need, because they have certain architectural precepts they believe good 
operations are built on. As in so much of technology it is not always 
the point issue that is the driving, it is everything else that is being 
leveraged based on certain assumptions.

Mark

---

From: Norman Finn <[email-masked]>
Date: Thu, 13 Nov 2003 15:38:08 -0700

Mark,

In order to find out what PWE3 does or doesn't need to do, I'd like to restate
the problem.

Ethernet networks, consisting of IEEE 802 media and IEEE 802 bridges, today offer
a service that delivers a certain level data integrity assurance.  This is
necessarily the minimum level of assurance that we want Metro Ethernet to supply.
Any less would violate the unstated and unknown assumptions made by any number of
protocols that will be using a Metro Ethernet Service.

I take as a given (having sold them) that networks consisting of IEEE P802.1ad
Provider Bridges, Ethernets, and Pseudowires will be built.  To put it another
way, P802.1ad Provider Bridges will be an important "user" of Pseudowires.

Provider Bridges will be almost identical from IEEE 802.1Q bridges.  In particular,
they will not introduce any new mechanism, such as carrying a customer frame's
encapsulated FCS inside an outer FCS.  That would require more changes to the
IEEE 802.1Q spec that 802.1 is willing to make on this pass.

Therefore, the only model available to P802.1ad Metro Ethernets for preserving data
integrity is exactly that employed today by 802.1Q.  This model makes the extremely
important assumption that the world consists solely of 802 media and 802 bridges.  In
particular:

  1. All 802 media provide the same 32-bit Frame Check Sequence (FRC == CRC, Cyclic
     Redundancy Check).

  2. Bridges are prohibited from introducing undetected errors at a rate significantly
     greater than that of a single hop over an 802 medium.

There are a number of techniques by which a bridge can handle requirement 2.  They've
been mentioned in this thread and don't need to be gone over again.  End-to-end
preservation of the Customer's original FCS is an interesting proposition, but is
not relevant to this formulation of the problem.  What is important is that, unless a
Pseudowire can make at least as good a guarantee about data integrity as a physical
Ethernet can make, then it cannot be of any use to a P802.1ad Provider Bridge.

Now, while one can argue that a Pseudowire *might* be carried only over Ethernet
physical media, that is clearly insufficient.  The only solution is for there to be
a version of Pseudowires that incorporate the IEEE 32-bit CRC, or something even
stronger, with every Ethernet frame.  This requirement has nothing to do with whether
end-to-end CRC is employed; it depends simply on the fact that a Pseudowire is not
useful to a major class of devices, namely bridges, unless it carries at least an
Ethernet CRC.

-- Norm

Mark Seery wrote:
> Hi Ali,
>  
> */Ali Sajassi <[email-masked]>/* wrote:
> 
>     <snip>
>     a) If they don't detect, then they basically violate the basic
>     requirement
>     of "keeping the integrity of FCS"
> 
>  
> Ultimately, the issue may not be the integrity of FCS but the integrity 
> of the payload. If the payload and a FCS become separated, and the 
> payload gets corrupted before/while a new FCS is being generated, what 
> has protected the integrity of the payload. Serious concern for people 
> doing financial transactions as an example.
>  
> While I personally believe networks should be build so the layer that 
> creates the PDU does the end-to-end check (so that service providers can 
> do interworking for example without worrying about essentially the same 
> issue and remembering that Ethernet is in some people's minds a link 
> layer), it might be that service providers do not have control over 
> their subscribers, and it might be that technologies built historically 
> relying on link layer checks are at risk; and I suppose it could be 
> argued that a PW is a virtual link layer - though I think that is debatable.
>  
> So while I personally believe that we should not try to solve a PWE 
> integrity problem (perceived or otherwise) by making use of an Ethernet 
> FCS, if there is an integrity of the payload problem, that might be a 
> different issue.
>  
>  >> <snip> b) If they introduce an error and they cannot detect it and 
> the end-point detects it, then how do you know where the error occurred 
> and how do you trouble shoot <<
>  
> Historically, service providers have used sectionalized (section, path, 
> segment, end-to-end, near end, far end,.....) OAM mechanisms within the 
> transport layer to achieve ongoing measurements, and theoretically 
> recorded for retroactive inspection, to achieve such insights (not 
> wishing to ignite another round of OAM discussions , just an observation).
>

---

From: mark seery <[email-masked]>
Date: Mon, 17 Nov 2003 08:06:09 -0800

Norman Finn wrote:

> Mark,
>
> Sorry for the lag in my reply -- poor 802.11 on an airliner.  :)
> See in-line

Yeah, we really got to extend the range of 802.11 ;-)

I guess the bottom line to me is ("philisophically" speaking), if a 
bridge is merely a user of a PW, then what is the difference between 
saying a PW has to have a CRC and saying a strand of fiber or copper 
cable has to have a CRC? If the bridges simply do what they have always 
done, using their own link layer, and simply use PW as a transport, then 
seems you maintain the maximum interoperability, are not reliant on 
other standards bodies (not saying bodies can't work together as 802 and 
IETF often do very well), and simply treat PW as a "media". If you 
disagree with this point, then I understand all your other points.

I say this of course WRT the way 802 might look at it. Scott's point of 
view might be that any link has to meet his requirements because like 
you, he might feel the operator should determine whether something meets 
his company's definition of grade. While on the subject, with respect to 
your response to Scott, it might be instructive to know that operators 
have already had bad experiences with another product class that 
supported Ethernet, without any tools for doing problem determination, 
so there might be some sensitivities here on this issue - we are all 
nothing, if but human ;-)

Best,
Mark

---

From: mark seery <[email-masked]>
Date: Mon, 17 Nov 2003 08:11:33 -0800

Norman Finn wrote:

> <snip>
> In other words, routers pass about $0.93 of the data integrity buck to 
> bridges,
> and bridges pass about $0.75 of that to the data links.  The buck has 
> to stop
> somewhere, and the data link is where it stops.  The PW has to take 
> the same
> hit as the physical media.
>
> -- Norm

but a PW will be running over a physical media itself already, possibly 
even SONET.............

mark

---

From: Norman Finn <[email-masked]>
Date: Mon, 17 Nov 2003 10:49:14 +0100

>> not relevant to this formulation of the problem.

You can't disagree with my assertion about my own formulation of the problem.  :)

> problem...

Mark, I appreciate the lecture on IP philosophy, and I have no argument with
the IP philosophy.  But you're trying to apply it where it doesn't belong.
You're exactly right that it is up to the endpoints to maintain the PDU
integrity.  What you're missing is where the ends of the PW are.

As far as PW are concerned, the endpoints of the Layer 3 service are the ends
of the PW, not the Customer's ends over the Provider's Metro Ether.  The users
of the PW are the bridges, not the Customers.  And bridges require a 32-bit CRC
on their links in order to do their job properly.

-- Norm

---

From: Norman Finn <[email-masked]>
Date: Mon, 17 Nov 2003 17:00:44 +0100

Mark,

I agree with this point, but it leads me to opposite conclusion.

PW is just another medium to a bridge.  However, *all* media known to bridges,
at least as far as IEEE 802.1 are concerned, are 802 media, and all 802 media
have 32-bit CRCs.  If you connect a medium that doesn't have at least a 32-bit
CRC to a bridge, then the resultant bridged LAN will suffer a significant
degradation in its ability to transport frames without error, because bridges
depend on reliable media.  And then the routers on top of the bridges don't
work so well.

Routers can get away with their liberal assumptions about reliability only
because links and bridges take it very seriously.  Right now, the routers
depend on the links and the bridges for reliability, and router people sometimes
forget that they're taking this reliability for granted.  I've used systems
where the link-layer CRC was not working, and data dependent errors resulted.
IP doesn't work for beans in that environment.

In other words, routers pass about $0.93 of the data integrity buck to bridges,
and bridges pass about $0.75 of that to the data links.  The buck has to stop
somewhere, and the data link is where it stops.  The PW has to take the same
hit as the physical media.

-- Norm

mark seery wrote:
> Hi Norman.
> 
> I guess the bottom line to me is (philisophically speaking), if a bridge 
> is merely a user of a PW, then what is the difference between saying a 
> PW has to have a CRC and saying a strand of fiber or copper cable has to 
> have a CRC? If the bridges simply do what they have always done, using 
> their own link layer, and simply use PW as a transport, then seems you 
> maintain the maximum interoperability, are not reliant on other 
> standards bodies (not saying bodies can't work together as 802 and IETF 
> often do very well), and simply treat PW as a "media". If you disagree 
> with this point, then I understand all your other points.
> 
> Best,
> Mark
> 
> 
>

---

From: mark seery <[email-masked]>
Date: Mon, 17 Nov 2003 07:09:46 -0800

Hi Norman.

Best,
Mark

---

From: Norman Finn <[email-masked]>
Date: Mon, 17 Nov 2003 10:48:33 +0100

Mark,

See in-line.

> that a PW is not Ethernet grade unless it provides 32-bit CRC or better. 

Exactly right.

> implications.

I do not make any judgments on what requirements that various non-bridge
Ethernet endstations place on Ethernets.  That's not my concern.

What I am very concerned with is what requirements bridges make on Ethernets.
Those requirements are very firm, and a PW that does not have error detection
as good as the IEEE 32-bit CDC fails to meet them.  To proceed without that
ability is therefore risky.

 > If the standards

You've got it exactly backwards, If I read your comments correctly, Mark.
A PW that does not provide a 32-bit CRC would force bridges to behave
completely differently than they do now (e.g. end-to-end CRC), which is
exactly what is not wanted.  There are far more bridges and bridge ports
out there than there are routers and router ports.  You don't change all
the bridge ports to make up for a single new type of "Ethernet" that is
lame in the error detection department.  You fix the lame "Ethernet": PWs.

> "existing" bridges would be in the middle of a PW [because they don't 

Of course not.  They'd be at the ends.  Also, I wasn't arguing with
you in my original mail -- only using your message as a launching point
for making the case for CRC in PW.

> has thrived by assuming the complete opposite - such is life....

And I'm sure IP will continue to do so.  As will 802.

-- Norm

---

From: Nick DelRegno <[email-masked]>
Date: Mon, 17 Nov 2003 16:22:26 -0600

If the PW transits only the physical media, then the discussion is moot.
However, since PWs will likely span multiple physical media, multiple
devices, and at the end of the day looks to the 802 bridges like a physical
media, it needs to held to the reliability/tolerance standards associated
with physical media.

At the extreme, a PW FCS would provide this functionality.  But
pragmatically, reuse of the Ethernet/FR/PPP FCS provides similar
functionality without major overhaul to the standard.

A similar example would seem to be that of SONET Section/Line/Path overhead
and the PM enabled by such segmentation.  Why have a path level if we have a
bunch of serial line PMs.  Wouldn't that be good enough? Or does it make
sense to have a path level check to ensure the integrity of the path?

The problem with tunneling of any sort is the loss of visibility within the
tunnel.  This makes troubleshooting difficult, but it shouldn't prevent
end-to-end tunnel fault detection.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of mark
seery
Sent: Monday, November 17, 2003 10:12 AM
To: Norman Finn
Cc: [email-masked]; [email-masked]
Subject: Re: [PWE3] RE: [802.1] 802.1ad Customer FCS Retention

mark

_______________________________________________
pwe3 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pwe3

---

From: Norman Finn <[email-masked]>
Date: Tue, 18 Nov 2003 09:56:06 +0100

Mark,

You say below,

 > but a PW will be running over a physical media itself already, possibly

Exactly!  Here, the trail gets a bit murky.  If I'm in total control, and the PW
is running over known Ethernet links, then I can get away without the CRC in the
PW.  I have no problem whatsoever with the current definition of PWs without CRC.

But, if I don't have tight control over the physical media over which the PW runs,
then I really need a CRC on the PW for the bridge to be able to trust the "wire"
and thus provide the same level of data integrity that bridges offer over physical
Ethernets.  So, I claim that bridges need a CRC option on PWs.

-- Norm

---

From: mark seery <[email-masked]>
Date: Tue, 18 Nov 2003 11:15:35 -0500 (EST)

Hi Norman,

This is a good formulation of the problem. 

thanks,
mark