Subject: PW Availability

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Wed, 24 Mar 2004 15:42:20 -0500

Neil,

A while ago we exchanged emails on the availability of Pseudo Wires. The relevant part is attached to this email.

I agree with the observations you made, but my main question was not answered: if you emulate a service over PWs, is it really necessary to measure Network Performance (NP) of the PW itself? If a Service Provider measures NP on end-to-end ATM VCs, why is it necessary to obtain additional NP measurements on one specific segment?

Now, assuming that a Service Provider wants to measure NP of PWs, how does that work? I assume that it is based on comparing packet counts both at ingress and egress. Following the method proposed by Y.1711, performance measurements are stopped upon detection of a defect and restarted a certain period after the defects disappears and the connection is stable again. However, in general a defect exists some time before a connection end-point finds out. Therefore, a number of packets will have been dropped already. Rather than trying to minimize the effect of this time lag by specifying available/unavailable state, as Y.1711 does, would it not make more sense to sanitize the data in an off-line performance monitoring tool that can exclude counts taken just before the defect occurred?

In short: how exactly do Service Provivers measure Network Performance?

Peter

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Monday, March 01, 2004 3:57 PM

> > > 2	There is no demarcation between the available state and 
> > > unavailable state for network trail entities.  This means 
> > > there is no temporal basis for the specification of an 
> > > availability SLA or a network performance SLA, noting that 
> > > the latter is only relevant to the available state.  These 
> > > impact network element specifications and thus have 
> > > consequential impact on NMS/OSS.  NE and NMS/OSS 
> > > considerations are important for an operator to see correctly 
> > > executed.  CF Y.1711 where the unavailable state is specified 
> > > (as a consequential persistence of defects).
> > > 
> > 
> > Adding a definition of availabillity would be pretty easy to 
> > specify and implement. I would like to understand better, 
> > though, how this would be used in practice. It seems to me 
> > that SLAs apply to the services carried over PWs, not to the 
> > PWs themselves.
> NH=> Several points here.
> 
> First, there is a difference between QoS and Network 
> Performance (NP) metrics/objectives.  The former are what a 
> specific application percieves, whereas the latter are 
> application invariant (ie for a given level of NP impairment 
> different applications can have quite a different perspective 
> on QoS).  NP metrics are really what the network elements 
> should capture and SLAs specify.
> 
> Second, the control of availability and NP are complementary 
> performance requirements.  If one does not have an agreed 
> datum at which NP metrics should no longer be counted then 
> one can easily distort the NP SLA.  There is also the 
> requirement to measure the same 'apples and apples' here in 
> different networks/equipment.....and this also applies 
> *across* nested layer networks (the commonly agreed 
> integration period for assessing availability is based on 10s 
> of outage). Further, a service which has a 99% unavailability 
> can still have perfect NP......this is why these metrics are 
> complimentary.  See section 7 of Y.1711 for more info.
> 
> Third, there is a key ordering behaviour here:
> => ensure the right net arch for mode considered 1st
> => this allows the relevant defects to be identified
> => this allows appropriate OAM to be specified (and from this 
> defect entry/exit criteria and consequent actions)
> => this allows unavalability entry/exit criteria to be 
> specified based on defect persistence
> => this allows the up-time to be specified over which NP is relevent
> You can very easily make life more complex than it need be by 
> not following this ordering.  Further, and for complexity 
> reasons, I would also not advise using NP metrics within the 
> availability entry/exit criteria.....just use defect 
> persistency.  Much careful thought went into Y.1711 on these, 
> and several other, matters that may not be readily apparent 
> to someone on a cursory reading.
> >

---

From: [email-masked]
Date: Wed, 24 Mar 2004 22:53:25 -0000

Peter..
> 
> Neil,
> 
> A while ago we exchanged emails on the availability of Pseudo 
> Wires. The relevant part is attached to this email.
> 
> I agree with the observations you made, but my main question 
> was not answered: if you emulate a service over PWs,
NH=> Can you define precisely what 'emulate' means here?

> is it 
> really necessary to measure Network Performance (NP) of the 
> PW itself?
NH=> Pass.  Depends on what the definition of 'emulation' really meens (and I have no idea) and what a PW actually is....to me its some form of adaptation.  XoverIP could have different perf to XoverMPLS....the defect behaviour is for sure very different.  I think folks really ought to differentiate these 2 cases as they are obviously not the same.

> If a Service Provider measures NP on end-to-end 
> ATM VCs, why is it necessary to obtain additional NP 
> measurements on one specific segment?
NH=> To see where impairments are coming from say.  In a more general case if you have some global reference connection with metrics A, B, C that have objectives a, b, c (to be met over some period, eg month), you need some way to fairly apportion these to  partitions of the end-end path, eg to countries or operator domains.  So each partition need to check its bit is within its apportioned target.
> 
> Now, assuming that a Service Provider wants to measure NP of 
> PWs, how does that work?
NH=> I have no idea....you are starting down the very slippery slope of making a PW a networked entity....and it is not (as yet anyway).

> I assume that it is based on 
> comparing packet counts both at ingress and egress. Following 
> the method proposed by Y.1711, performance measurements are 
> stopped upon detection of a defect and restarted a certain 
> period after the defects disappears and the connection is 
> stable again. However, in general a defect exists some time 
> before a connection end-point finds out. Therefore, a number 
> of packets will have been dropped already. Rather than trying 
> to minimize the effect of this time lag by specifying 
> available/unavailable state, as Y.1711 does, would it not 
> make more sense to sanitize the data in an off-line 
> performance monitoring tool that can exclude counts taken 
> just before the defect occurred?
NH=> Y.1711 has not defined any NP collection yet.  Its more important to get the basic defects, short-breaks and unavailability bits done 1st.  NP stuff can be added later on this firm defect detection/handling basis.  I can think of some smart/simple ways of doing NP collection however.....the details are not given in Y.1711 right now. One also has to be aware of 'PDUs in flight'.  
> In short: how exactly do Service Provivers measure Network 
NH=> Badly to date is my view.  Why?  Because the ordering I mentioned in the mail below has never properly been executed in the past.

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Thu, 25 Mar 2004 15:49:17 -0500

Neil,

I would be happy to add to draft-nadeau-pwe3-oam-msg-map-04.txt that a PW is declared unavailable if a defect persists for more than 10 seconds. However, at this point I am not sure what that means since I don't know how it affects anything that goes on in the network.

You have mentioned that it is important to recognize such a state, because it determines when Performance Monitoring should be started and stopped. However, from your response to my last email, it appears that a clear and realistic approach to Performance Monitoring does not exist yet. Therefore, I wonder what the point is of introducing concepts that may or may not be relevant in the future.

Therefore, I propose that we add the following statements to draft-nadeau-pwe3-oam-msg-map-04.txt:

a) in general (i.e. for other technologies), operators consider a connection unavailable of a defect persists for more than 10 seconds

b) the interaction between PW fault management and Performance Monitoring is for further study.

Further comments in-line.

Peter

> Sent: Wednesday, March 24, 2004 5:53 PM
> To: [email-masked]; [email-masked]
> Subject: RE: PW Availability
> 
> 
> Peter..
> > 
> > Neil,
> > 

I meant "emulate" in the same way as it is used in all WG drafts and emails on this mailing list: the encapsulation of client traffic so as to transport it over a PSN.

> NH=> Pass.  Depends on what the definition of 'emulation' 
> really meens (and I have no idea) and what a PW actually 
> is....to me its some form of adaptation.  XoverIP could have 
> different perf to XoverMPLS....the defect behaviour is for 
> sure very different.  I think folks really ought to 
> differentiate these 2 cases as they are obviously not the same.

I may be naive, but I would think that Network Performance to the extend that it relates to SLAs should be independent of the network layers over which the service is carried.

In fact, you assertion that defect behavior of IP and MPLS networks is so different that XoverIP and XoverMPLS should be considered as two totally different solutions suggests that NP measurements of X would yield completely different results in the two cases. I can't see why that would be desirable to Service Providers.
 
> NH=> To see where impairments are coming from say.  In a more 
> general case if you have some global reference connection 
> with metrics A, B, C that have objectives a, b, c (to be met 
> over some period, eg month), you need some way to fairly 
> apportion these to  partitions of the end-end path, eg to 
> countries or operator domains.  So each partition need to 
> check its bit is within its apportioned target.
> > 
> NH=> I have no idea....you are starting down the very 
> slippery slope of making a PW a networked entity....and it is 
> not (as yet anyway).

I am  not sure who started down the slippery slope. I did not say there was a need for an availability state for PWs. You did. I am actually arguing for the position that it doesn't make sense to measure NP for Pseudo Wires.

> NH=> Y.1711 has not defined any NP collection yet.  Its more 
> important to get the basic defects, short-breaks and 
> unavailability bits done 1st.  NP stuff can be added later on 
> this firm defect detection/handling basis.  I can think of 
> some smart/simple ways of doing NP collection however.....the 
> details are not given in Y.1711 right now. One also has to be 
> aware of 'PDUs in flight'.  
> > 
> > In short: how exactly do Service Prodivers measure Network 
> > Performance?
> NH=> Badly to date is my view.  Why?  Because the ordering I 
> mentioned in the mail below has never properly been executed 
> in the past.

I am not sure what point you are trying to make. Service Providers (BT being one of them) have large networks in which they run ATM over SDH or SONET. I was not asking a rhetorical question. I am genuinely interested to learn more about operator experiences.

However, if you say that even ATM-over-SDH is such a mess that decent Performance Monitoring is not possible, then what does that mean for our current work? Existing networks will not go away for a long time to come. We have to work with what we have. Especially with regards to PWE, I don't see how we can work on Performance Monitoring without taking existing protocols and techniques into account.

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 26 Mar 2004 10:28:02 +0000

I am surprised that such a relatively large fixed number is specified.

What is the rational for 10s?

- Stewart

Busschbach, Peter B (Peter) wrote:

>>>>=> this allows the up-time to be specified over which NP 
>>
>>is relevent
>>
> _______________________________________________
> pwe3 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pwe3
> 
>

---

From: [email-masked]
Date: Fri, 26 Mar 2004 10:56:28 -0000

<snipped>
> What is the rational for 10s?

Hi Stewart,

This is what folks use to define availability in most telecoms networks and it is in common usage.  The value is historical (I can explain its roots if you are really intersted). You need some value specifying, and this is probably as good as any other for the purpose it serves......but it is not the full story of course. If you don't have an agreed definition of availability you cannot compare apples with apples. This affects at least 2 dimensions:
-	in a horizontally partitioned layer network, ie spanning several operators domains......this also leads into partitioned performance allocations within (longer) reference connections
-	in a vertically nested set of layer networks, ie one wants some form on uniformity of performance specification across each layer network

Further, there is an ordering here.  PM measurements need to know the state of the network.  PMs measurements are only associated with the available state for SLA purposes....so clearly we need a common basis for PM measureements, and this is dependent on a common basis for availabilty.  Unavailability is a function of defect persistence.  So one also needs a commonly agreed specification of defects (these are of course different for each mode).  And finally, though most importantly, to define the defects one needs to respect the func arch requirements of the mode considered......break that and you will have major problems doing the rest that is dependent on this anyway (as we know only too well).

Please have a read through Y.1711.....its all done in there (and more....check out the stuff on short-breaks for example).

Note - You could propose changing the definition of availability.....but we would still need some commonly agreed definition.   But I suspect you may get some kick-back from folks who want to retain the classical definition.  And to be honest I can't actually see changing it is really going to provide that much benefit....respecting the arch of the mode and having sound defect detection/handling is far more important IMO speaking as an operator.

regards, Neil

---

From: [email-masked]
Date: Fri, 26 Mar 2004 14:34:46 -0000

As a postscript to this, I have had some private mail requests to give further insight on 2 things:
-	the origins of 10s availability value
-	the origins of 50ms prot-sw value

10s availability:

This goes way back to the days when the 64k N-ISDN was being specified (ie early/mid 80s), and folks were just starting to introduce digital Tx and Sw systems.  I was a new graduate then...so its a while ago ;-)

Folks knew that they had to define 2 types of performance specifications. One type would control how much downtime a network path could exhibit and the other type would control the transfer impairments when the path was 'up'.

The dominant application at the time was telephony.  So 10s came from a guess at an upper bound on how long it would be before folks would have cleared down the call.  IMO, I suspect few folks would clear a call until maybe 4+ seconds had passed....but 10s is a bit long.  There were also factors on how fast folks could effect restoration schemes then.  Whether this figure is right/wrong is of course debateable.....and it has indeed been discussed many times since with a view to change it, but it still remains.  If you reduce it you are making things more stringent for operators to meet.....many may not like this.  It also won't harmonise with existing networks/practices.  So, for the purpose it serves I think it is fine, and I see no compelling reason for wanting to change it now.

Now here's some stuff related to availability that you may not know about......those who are not interested in this should skip the text delimited by ====> and return (if they want) to where I discuss the 50ms issue.

=====>

The error performance metrics for the up-state were defined in Rec G.821 (this also had the availability definition in too).  There were originally 3 parameters:
-	the ES (Errored Second), ie 1s period with >=1 bit errors
-	the SES (Severely Errored Second), ie 1s period with an error density >= 10^-3
-	the DM (Degraded Minute), ie 1min period with an error density >=10^-6 *after* removing SES

Folks thought these were sensible for the services envisaged at the time.  IMO only 1 of these parameters is truly valid....the ES (or more generally 'an errored interval')....and if you read on you will understand (some of the reasons) why.

The definition of entry to the unavailable state is 10 consecutive SES.  These ES/SES events are then *excluded* from the NP SLA. Similarly, the exit from the unavailable state required 10 consecutive seconds with no SES events.....they could have ES events however, and if there were any these would need including in the NP SLA.  Note also that any ES events may also give rise to DM events.  And also note that the DM might straddle a period of unavailable time.  All rather messy.

You may hear me say from time to time 'don't use NP metrics as part of the (un)availability entry/exit criteria'.....the above hints at why this is, but I need to explain more about the above metrics to make this point even clearer.

I spent a great deal of time in late 80s and early 90s trying to properly understand the behaviour of Tx networks.  We built some specialised kit to study error event distributions.  And we found some nice mathematical models of real error distributions.  Forget Poisson stuff.  Digital Tx systems are not dominated by thermal noise.  The models we found that accurately characterised Tx systems had self-similar characteristics (due to ESD, weather related, human activity, etc).  More specifically, this work was based on the work of a colleague and good friend of mine called Andy Reid (I have mentioned his name before, this is also the guy who did the early functional modelling stuff too) and a mathematician from our labs called David Bond......in fact the 'Bond Model' was a development of the stuff Mandelbrot did earlier on error event distributions whilst he was with IBM (I think).  However, all this work was somewhat dry/academic.....but a major immediate conclusion from it was 'do not try and time-scale error metric objectives to shorter periods', eg if you have some objective to meet over a 1 month period (which is what G.821 requires) then simple statistical methods of deriving the confidence bounds for what you could expect in a shorter period (eg 15min, hour, or day say) just did not hold.  This is important for folks attempting to set provsioning levels and mtce thresholds in NEs.

These studies however showed us some other very important practical things.  Most error events were a short intense burst.  These could be expanded (in a non-linear way) as the error event propagated up nested layer networks by affecting framing and justification bits for example.  I am not going to go into the detail of this as it would take me pages to explain.

Now the errors event we saw came in 2 main flavours:
(i) us/ms events of error density 0.5
(ii) >1s events of error density 0.5

Consider the type (i) events first. If these existed for >2ms they would create an ES/SES pair event (0.5 error density over 2ms = 10^-3 error density over 1s). If these existed for <2ms they would always create an ES event but they might also create an ES/DM pair (that is, an error event lasting between 0.12ms and 2ms with an error density of 0.5 would create an error density >=10^-6 over a 1 min period).  What a nonsense!  If not obvious, the metrics being used (both SES, and (much worse) DM) bore no relationship to the event they were recording.  So we were measuring all this stuff that made absolutely no sense.

I then spent several years trying to convince folks to scrap DMs.....which thankfully they eventually did.  I did not have the heart to ram home the point that the SES metric was equally flawed....esp as it was also the basis for the availability specification.

However, the *practically* key observation is that I never saw a real nice/stable 10^-3 error density event in all the years I looked for them.....such nice stable Poisson behaviour was the stuff of fiction in digital Tx systems (it only applied to the old analogue Tx systems). But folks loved talking about BERs....and they still do it today!

Real error events knocked systems over, eg caused re-frames or such.  So the important things to measure were the defects like LOS, LOF.  I never saw a SES that did not cause a defect event.  And this is one of the reasons why I urge folks not to use NP metrics in any definition of entry/exit of the available state.....you can make life unnecessarily complex for no real benefit. Now I know one can't directly translate these observations of co-cs Tx networks to co-ps/cl-ps pkt networks, but I do believe the same general principles should apply.....and you are for sure going to get the server layer co-cs networks delivering these type of error events into their client layers.

Now let me come back to the type (ii) events I noted above.  I expected type (i) events to massively dominate the population and type (ii) events to be relatively rare.  The big shock we found was that of all the SES events the split was roughly 50:50. Type (ii) events could last up to say 3-4s and then just disappear.....if they lasted more than 4s they lasted forever, unless one took action to repair/restore the failed path.  This was worrying but it also explained something we had been noticing from customer complaints......folks were seeing 'short-break' events.  Note that a type (ii) event is 3 orders of magnitude more damaging than a type (i) event.....so no wonder it was affecting applications.  What concerned me at the time was that we had no way of distinguishing/measuring these 2 types of SES event if we threw them both in the same SES bucket.

We had some massive witch-hunts to discover what was the cause of these events, and although loads of components got changed out no one ever got to the bottom of them (at least AFAIK before I left this area of work).  We asked other operators to look for these events and sure enough they found them.  And although I had moved on at that time I believe they did eventually get specified in Rec G.826 (next gen G.821 for primary rate and above Tx networks) and were called 'consecutive SES (CSES) events'.....where a CSES event was defined as 2-10 consecutive SES events.

Now for those of you who have read the stuff I wrote-up in Y.1711 you may start to notice how I have drawn on the above experiences of real networks to shape, and simplify, MPLS OAM.  The defects are all based on harmonised 3s entry/exit periods to align with the error events that may or may not self-heal.....but if they don't self heal they give us the basis to measure short-break events.  I also stop NP counters on defect entry....again to make life simpler.  Unavailability is now based on defects that persist for 10s.  If you enter the unavailable state the NP counters are only re-enabled on entry to the available state....so there is some hysteresis here to make sure the network is stable again.....again this is aimed at keeping stuff simple.

There is actually a whole load of other stuff about Y.1711 that might not be apparent to someone on a cursory read that aims to make its simple.

Anyway, that is the origins of the 10s availability definition and some of the deeper stuff I know about that relate to it.  Hope those who asked for this found it useful.

<======

50ms prot Sw target in SDH:

This one is much easier to explain.  I know the folks in BT who were active in the stds when this was originally being defined.  There is nothing magical about 50ms.  This was the figure folks thought was reasonable to assume in the largest SDH rings at the time based on propagation delay and processing requirements.  I was for sure *not* driven by any application requirements.  It also assumes one has access to the 'bottom-of-layered-network-stack' signal....so one can get a handle on real loss of signal at origin and thus accurately measure the defects.

We developed a general rule about prot-sw way back that I think still holds true today....as fast as possible as close to the duct as possible, and as slow as possible when close to the application.  If you read my explanation of the 10s availability stuff above you may have noted the significance of the type (ii) SES events here.  That is, at the network layers above the Tx networks (and I would place MPLS here) one does not want to be invoking prot-sw too fast because you could be erroneously triggering on error events that may go away.

Now its a while since I studied real error events on Tx networks so if anyone reading this has any newer information on what they look like today it would be good to hear about this.  However, if what I saw still holds true then we should be careful not to prot-sw too hastily above the Tx layer networks, eg at least make this configurable.  Of course one also must detect the defect first, and that will generally be to a lower resolution above the Tx layer networks.

Final point - I don't know any applications that need a 50ms restoration value.  Voice for sure does not.  Some old PSTN signalling protocols were a bit flaky......but then one should not penalise the whole future network behaviour on a flaky protocol, you should fix that protocol.  So I regards the folk chasing really fast restoration times above the Tx layer networks as in some 'macho arms race'.....I for sure don't see any logic or application drivers for this.

regards, Neil

> From: Harrison,N,Neil,IKL2 R 
> Sent: 26 March 2004 10:58
> To: [email-masked]'
> Cc: Busschbach, Peter B (Peter); [email-masked]
> Subject: RE: [PWE3] RE: PW Availability
> 
> 
> <snipped>
> > 
> This is what folks use to define availability in most 
> telecoms networks and it is in common usage.  The value is 
> historical (I can explain its roots if you are really 
> intersted). You need some value specifying, and this is 
> probably as good as any other for the purpose it 
> serves......but it is not the full story of course. If you 
> don't have an agreed definition of availability you cannot 
> compare apples with apples. This affects at least 2 dimensions:
> -	in a horizontally partitioned layer network, ie 
> spanning several operators domains......this also leads into 
> partitioned performance allocations within (longer) reference 
> connections
> -	in a vertically nested set of layer networks, ie one 
> wants some form on uniformity of performance specification 
> across each layer network
> 
> Further, there is an ordering here.  PM measurements need to 
> know the state of the network.  PMs measurements are only 
> associated with the available state for SLA purposes....so 
> clearly we need a common basis for PM measureements, and this 
> is dependent on a common basis for availabilty.  
> Unavailability is a function of defect persistence.  So one 
> also needs a commonly agreed specification of defects (these 
> are of course different for each mode).  And finally, though 
> most importantly, to define the defects one needs to respect 
> the func arch requirements of the mode considered......break 
> that and you will have major problems doing the rest that is 
> dependent on this anyway (as we know only too well).
> 
> Please have a read through Y.1711.....its all done in there 
> (and more....check out the stuff on short-breaks for example).
> 
> Note - You could propose changing the definition of 
> availability.....but we would still need some commonly agreed 
> definition.   But I suspect you may get some kick-back from 
> folks who want to retain the classical definition.  And to be 
> honest I can't actually see changing it is really going to 
> provide that much benefit....respecting the arch of the mode 
> and having sound defect detection/handling is far more 
> important IMO speaking as an operator.
> 
> regards, Neil
>

---

From: Sasha Vainshtein <[email-masked]>
Date: Fri, 26 Mar 2004 20:05:18 +0200

Stewart and all,
I assume that 10 seconds delay for declaring a PW unavailable
is compliant with G.826 where a circuit is declared unavailable after 10
consecutive severely errored seconds (these secondsa are then alos
re-classified as unavailable). And, if I remember correctly, a defect 
renders a second when it was observed severely errored.

This explains (hopefully) the background for the definition you 
have questioned. Whetherthis definition is or is not suitable for PWs
is another question.

Regards,
                    Sasha

-----Original Message-----
From: Stewart Bryant
To: [email-masked]'
Cc: Busschbach, Peter B (Peter); [email-masked]
Sent: 26/03/2004 12:28
Subject: Re: [PWE3] RE: PW Availability

- Stewart

Busschbach, Peter B (Peter) wrote:

> Neil,
> 
> I would be happy to add to draft-nadeau-pwe3-oam-msg-map-04.txt that a
PW is declared unavailable if a defect persists for more than 10
seconds. However, at this point I am not sure what that means since I
don't know how it affects anything that goes on in the network.
> 
> You have mentioned that it is important to recognize such a state,
because it determines when Performance Monitoring should be started and
stopped. However, from your response to my last email, it appears that a
clear and realistic approach to Performance Monitoring does not exist
yet. Therefore, I wonder what the point is of introducing concepts that
may or may not be relevant in the future.
> a) in general (i.e. for other technologies), operators consider a
connection unavailable of a defect persists for more than 10 seconds
> I meant "emulate" in the same way as it is used in all WG drafts and
emails on this mailing list: the encapsulation of client traffic so as
to transport it over a PSN.
> I may be naive, but I would think that Network Performance to the
extend that it relates to SLAs should be independent of the network
layers over which the service is carried.
> 
> In fact, you assertion that defect behavior of IP and MPLS networks is
so different that XoverIP and XoverMPLS should be considered as two
totally different solutions suggests that NP measurements of X would
yield completely different results in the two cases. I can't see why
that would be desirable to Service Providers.
> I am  not sure who started down the slippery slope. I did not say
there was a need for an availability state for PWs. You did. I am
actually arguing for the position that it doesn't make sense to measure
NP for Pseudo Wires.
> I am not sure what point you are trying to make. Service Providers (BT
being one of them) have large networks in which they run ATM over SDH or
SONET. I was not asking a rhetorical question. I am genuinely interested
to learn more about operator experiences.
> 
> However, if you say that even ATM-over-SDH is such a mess that decent
Performance Monitoring is not possible, then what does that mean for our
current work? Existing networks will not go away for a long time to
come. We have to work with what we have. Especially with regards to PWE,
I don't see how we can work on Performance Monitoring without taking
existing protocols and techniques into account.

---

From: [email-masked]
Date: Tue, 30 Mar 2004 11:45:49 +0100

Thanks Peter.....I have some responses for you in-line....sorry for the delay.  regards, Neil

> From: Busschbach, Peter B (Peter) [mailto:[email-masked]]
> Sent: 25 March 2004 20:49
> To: Harrison,N,Neil,IKR2 R; [email-masked]
> I would be happy to add to 
> draft-nadeau-pwe3-oam-msg-map-04.txt that a PW is declared 
> unavailable if a defect persists for more than 10 seconds. 
> However, at this point I am not sure what that means since I 
NH=> OK.  But please note:  
-	you will have seen my explanation of the origins of the definition of availability and how I think it should be simplified, re how Y.1711 has addressed this.
-	there is need to aim for harmonisation of NP/availability metrics and objectives across all layer networks.......this becomes even more obvious as a requirement if there is no control over client/server layer relationships (though as I have noted before some C/S relationships between the 3 modes make far more sense than others)
-	both availability and NP aspects can be sensibly associated with trails.  Only co-cs and co-ps modes have trails.  So for MPLS these metrics need to be associated with an LSP.  Sure you can try and associate them with a PW but we don't network PWs, ie we do not switch, nest as client/server, etc PWs since they are an adaptation function.  I know most folks seem to struggle to grasp this but it is/will be a real limiting problem for MPLS....that is, a PW lives in no-mans land between a given client (eg FR, ATM etc) and either a cl-ps IP server layer or a co-ps MPLS server layer.  I am really puzzled however why folks find this hard to understand technically?
> 
> You have mentioned that it is important to recognize such a 
> state, because it determines when Performance Monitoring 
> should be started and stopped. However, from your response to 
> my last email, it appears that a clear and realistic approach 
> to Performance Monitoring does not exist yet. Therefore, I 
> wonder what the point is of introducing concepts that may or 
> may not be relevant in the future.
NH=> This is not a logical argument.  If you don't get the ordering right then yes for sure you have a self-fulfilling prophecy that that there is no chance to get the NP aspects sorted.  However, if you do get the ordering right then when you are ready to measure NP you have the right basis.  NP is 2nd order in importance to defect detection/handling.  We have not discussed NP issues yet. You can also easily end up specifying way too many complex/unnecessary NP stuff that, in terms of its real benefit in a network, is an illusion of efficiency.  The 'QoS challenges' are not about having loads of classes in 1 mode, but they are all about how to deliver a suite of public/private services that are consistently meaningful across a large range of network load points, and thus allow SPs the ability to charge a fair price for doing so.  Real networks run lightly loaded for most of their working life (for lots of reasons).  Having lots of QoS classes won't help you with this goal if:
-	you can't differentiate 'urgency' and 'importance'......and this is one reason why there is a big difference between having the co-ps mode trail (eg LSP) or only the cl-ps mode pkt (eg IP) as the networked entity;
-	resource sharing is allowed between traffic classes....since this kills any sensible notion of 'QoS differentiation' dead.
> a) in general (i.e. for other technologies), operators 
> consider a connection unavailable of a defect persists for 
> more than 10 seconds
> 
> b) the interaction between PW fault management and 
> Performance Monitoring is for further study.
NH=> OK, seems this group is hell-bent of defining all the networking stuff round PWs rather than the LSP and pkt networked entities of the co-ps and cl-ps modes respectively.  Its wrong of course, but I alone can't stop you.  Some further remarks below.  
> I meant "emulate" in the same way as it is used in all WG 
> drafts and emails on this mailing list: the encapsulation of 
> client traffic so as to transport it over a PSN.
NH=> Oh yes I know, that is why I asked you to flush this point out.  Its imprecise, and is thus a meaningless statement.  This is not a good starting point for operators who regard MPLS as a key core target technology.  I think its high time folks stopped pretending MPLS is the same as IP....its wrecking the potential of MPLS IMO.  XoverMPLS is not the same thing as XoverIP.  We can manage trails in a co-ps mode like MPLS, we cannot do that in a cl-ps mode like IP.  That is simply a fact which, for whatever reason, some folks here seem unable (or more likely 'unwilling') to comprehend/accept.  Speaking as an engineer who has been brought-up to believe in honesty and accuracy, I find this 'sticking one's head in the sand' behaviour of ignoring the truth rather disturbing. {Note - this is not the same as saying I won't accept compromises....I have no problem with that *providing* they are recognised as such (eg arch violations) and we understand the implications of them.  I only have problems when folks tell me they are 'right' and won't even recognise identified problems.}  
> I may be naive, but I would think that Network Performance to 
> the extend that it relates to SLAs should be independent of 
> the network layers over which the service is carried.
NH=> The whole point of my repeated observations about the 3 networking modes and client/server inheritance is to show this is not true.  The behaviour of a cl-ps mode network is different to a that of a co-ps mode network is different to that of a co-cs mode network.  One introduces constraints to the basic 'broadcast at source, filter at sink' mode (that ethernet uses) to get these 3 wide-area modes.....and one does this because they bring benefits in scaling/design/operation/NP/mtce/etc....in other words the network becomes simpler as we move towards the co-cs mode.  If not obvious, consider the fact that the co-cs mode has *no* traffic/QoS classes at all.....and just think how much simpler routing/signalling/resource-accounting/fault-management/etc is therefore.  So if a co-cs mode is the server layer then the client layer cannot get any better *inherited* NP than this.....remember, each layer network will inherit the NP of its server layer and this is recursive to the duct.  There are 9 combinations of client/server relationships between the 3 modes.....some make more sense than others.  We cannot be prescriptive about this (and currently there are no stds that describe this or control how many times one is allowed to go up/down a network stack of different modes)....however we can use common sense ;-)
> 
> In fact, you assertion that defect behavior of IP and MPLS 
> networks is so different that XoverIP and XoverMPLS should be 
> considered as two totally different solutions suggests that 
> NP measurements of X would yield completely different results 
> in the two cases. I can't see why that would be desirable to 
> Service Providers.
NH=> Can't you?  Blimey that is of concern then.  As a hint go check real core network utilisations, esp when one has uncertainty of behaviour under failure.  But let me try and help further:  In a cl-ps network you only have one traffic entity.....the pkt.  In a co-ps network you have 2 traffic entities.....the parent trail and the child pkt.  The trail is the dominant entity here.  One can therefore provide additional constraints to the co-ps mode that don't exist in the cl-ps mode, eg you can explicitly control the directivity by signalling, and the survivability of the parent trail entity can be completely decoupled from the (child pkt) traffic it carries.....a rather important feature IMO, esp for VPNs (noting that this can be generalised to include all client/server cases since these are, if you think about it, a form of 'VPN').  Now if you really wanted to see the power the co-ps mode can unlock for you I would need to get into a discussion of what QoS/NP really means....but that is another long story however.

But hang on a minute......this should all be obvious to network engineers.  This is starting to feel like I am giving some lecture on networking basics.
> I am  not sure who started down the slippery slope.
NH=> I'll tell you then.  It was those folks who wanted to 'do' stuff like FR and ATM without deploying native technologies.  Quite understandable.  However, its one thing doing a cheap/cheerful expedient technology emulation (over either IP or MPLS), but its quite another thing to then say PWs are the basis of next gen networks for everybody.  AFAI can tell, the direction here is network convergence on PWs.......but I think most operators want a convergence on MPLS.  But PWs are not even a layer network, and the real layer networks of IP and MPLS are not the same.....if there were the same then why did MPLS come into being in the first place?  Every time I make this observation folks ignore it....I assume because its something some would rather was not recognised (though these same folks don't seem to have a problem recognising that SDH and OTNs are co-cs mode networks when talking about GMPLS....there are some very strange double-stds being applied here IMO).  This attitude is not healthy IMO.  Its not allowing the full potential of MPLS to be seen/developed, and IMO its also an attempt to hide (or maybe justify) some arch violations of MPLS IMO.  Now I know there are religious/commercial factors at play here, but I am trying to be brutally honest about 'networking truths'.....I used to think that 'technical honesty/accuracy' was something that was held in high esteem in IETF, but I have had to change my views on that.

> I did not 
> say there was a need for an availability state for PWs. You 
> did.
NH=> Yes, but it is now a logical conclusion of what has been done here.  A PW is now a unifying adaptation function above either an IP server layer network or an MPLS server layer network.....but it does not exist in its own right as a layer network interface.....you still have FR, ATM, or whatever above this.  We do not (as yet anyway) 'network' PWs but we do network IP pkts and MPLS trails.....and pkts and trails are quite different network entities.  However, given the status of this discussion I fear you either honestly don't get this (which I find hard to believe in your case) or you are unwilling to recognise it for some other reason.

>I am actually arguing for the position that it doesn't 
> make sense to measure NP for Pseudo Wires.
NH=> Good, that is something we agree on then.  I am actually arguing for an acceptance by the community that there are 3 networks modes.  SDH and OTNs belong to the co-cs mode.  IP and ethernet belongs to the cl-ps mode (though only IP has proper addressing/routing).  MPLS, ATM and FR belong to the co-ps mode.  These modes have fundamentally different behaviours....that is why they were created in the first place for goodness sake.  These are simply facts....pretending they are not true is not helping anyone in the long-term.  Now given your statement above, I therefore have to assume that you also agree that the NP needs to be associated with the real network then....so that is either IP or MPLS.  I want to associate *all* the networked attributes (ie defects, availability, NP) with the LSP in MPLS as its this that is the networked entity.  One manages trails in a co-ps mode network quite differently to how one manages pkts in a cl-ps mode network.   
> I am not sure what point you are trying to make. Service 
> Providers (BT being one of them) have large networks in which 
> they run ATM over SDH or SONET. I was not asking a rhetorical 
> question. I am genuinely interested to learn more about 
> operator experiences.
NH=> And the answer I gave is correct.  We really do not do a good job, because whilst I.610 may look superficially OK, there are several problems with it....and they started because folks initially focussed on NP/QoS and forgot about the more basic defect detection/handling first.  I can explain but its yet another long story.
> 
> However, if you say that even ATM-over-SDH
> is such a mess 
NH=> No, SDH is fine.  Its ATM's OAM that is not good in some respects.  The problem with ATM was that folks got far too interested with NP metrics and defining lots of traffic classes (we had 5 AALs at one stage).....a complexity that we can now see was largely unnecessary....and they forgot about attending to the less sexy basic stuff, like defining defects and their detection/handling.  So ATM's OAM is far from ideal.  I can explain why if you are really interested.

> that decent Performance Monitoring is not possible, then what 
> does that mean for our current work?
NH=> Which current work?  Y.1711 is fine.  It corrects the problems in I.610.

> Existing networks will 
> not go away for a long time to come.
NH=> Agreed.  That is why if we continue to screw things up by not facing up to network arch truths early its us operators who will be left paying for it for many years hence.  This is why I will not let folks off the hook who continue to violate func arch rules just because it might suit them right now in selling boxes.

> We have to work with 
> what we have. Especially with regards to PWE, I don't see how 
> we can work on Performance Monitoring without taking existing 
> protocols and techniques into account.
NH=> I would understand if you you were talking about going back and trying to redo FR and ATM say. But one should not be specifying stuff in stds that are still draft and known to be flawed.  We and other operators have not yet built our next generation networks, ie a converged core on MPLS.  PWs are not a common deployed legacy technology.  MPLS is only used today in niche applications.  We are very concerned about the arch flaws in some types of MPLS and its missing functional components.  Similarly, we are also concerned about PWs that are then based on these.  We therefore do not regard these technologies as legacy in the same way as FR or ATM.  MPLS is currently not fit for purpose as a future key convergence technology.  You appear to be insisting here that because someone has made some errors in the past with some types of MPLS that these must be visited on all future work and imposed on all operators....why?  We have the chance to correct these.  Why is it not possible to get the stds right now based on respecting the func arch requirements?

regards Neil