Subject: [Pce] Network modes and layer network relationships

From: [email-masked]
Date: Wed, 13 Jul 2005 17:58:54 +0100

Note - change of thread title

Thanks Igor, I've snipped this to where further remarks are required.  I
refer to your mail 08 July 2005 16:02, Subject: Re: [Pce] PCE Disco Reqs
v01.

> IB>> Let me ask you something. Suppose we have VC12 layer and
> VC4 layer. Is it unrealistic to expect that the same instance of the
> control plane manages both of them?
NH=> I see Ben has already responded.....however I want to say something
more here (quite a bit more in fact).

The trite answer to your question is 'yes'.  However, in order to fully
understand the reasons why I say this, we need to look to the general
case and not a specific and tightly constrained case.  Indeed, much of
the GMPLS stuff (and certainly when you see the remarks from folks like
Dimitri that caused me to react and support what you were saying
initially) have an assumption (maybe much weaker now than some time ago)
that it is possible to have a single control-plane running from IP to
optics across all layer networks.  Well, I do not buy into this general
case view at all....in fact I don't know many folks that do, but I do
know loads of folks that don't.

Now let me try and answer your specific question in this general case
context.  However, you still need to refer to what I said in previous
mails as these other factors are also important here, eg who owns which
layer?, have folks really done the cost models for assuming a BoD server
layer GoS is viable?, etc......

First, let us be very clear about the architectural situation we are
dealing with wrt the co-cs networking mode.  We should NOT dive in with
the specifics of how one technology embodiment (here SDH) of the co-cs
mode has been defined, as  our solution should be generic to all cases
of the co-cs mode.  This is because:

(i) We want a generic solution that provides maximal functional
reuse.....and this becomes particularly critical once we look at the
downstream NMS/OSS impact of network architecture.  Let me put this
point another way:  Opex costs are the most crucial costs to
control/minimise, and if the underpinning network architecture is not
right (eg violations, deficiencies....which will also add complexity
costs) then no amount of money thrown at the NMS/OSS piece later can
recover the situation....if you start with junk, massaging junk does not
really improve it.

(ii) We MUST respect the overriding requirement for functional
decoupling in a client/server layer network relationship between
different operating parties.....this is the base starting point of a
(private) network builder service, which one can then add functionality
to (eg an rfc2547-style VPN say) IFF the client wants this (and many
customers don't).  This is crucial, and I make no excuses for repeatedly
making this point.

IMPORTANT NOTE:  The above (client/server) case is VERY different to a
*public* service case where we have a peer-partition interworking
situation between independent operating parties in the SAME layer
network, ie this is an External NNI case (and it is different to an
Internal NNI case wrt to what is allowed to be advertised/controlled
across the NNI).  These cases MUST NOT be confused....and they are by
some folks IMO, cf those blundering into 'MPLS NNIs' because we cannot
create functional decoupling at the LSP level in MPLS due to the way it
has been defined, ie the wrapper.  This (wrapper) case DOES NOT apply to
the co-cs mode because the co-cs mode simply does not allow it.

We must first observe that in the co-cs mode the CI (characteristic
information, see G.805) of the trail we are networking is a constant
stream of bits with a precisely controlled temporal specification, ie
the trail rate is constrained to lie within a range of X bits/s +/-
delta ppm. I will ignore the jitter/wander impairments that can be
induced on this trail by lower layer networks (due to muxing in a
client->server adaptation function) or (right at the bottom of the stack
on the physical section layer) due to diurnal temperature cycles.
However, these must also be carefully controlled (esp if client timing
information is to be carried), and this is why certain client/server
relationships between the 3 modes will not make much sense despite what
some folks are doing in PWE3.  And I don't say this as conjecture
either, this is based on current lab testing here in BT.

Now in a co mode network (ie applies to both co-cs and co-ps) there are
well defined (ie identifiable) connection points associated with links
and subnetworks.  When these connection points are bound together in a
concatenated sequence between network access points an important
networking entity called a trail is created (strictly speaking in G.805
sense, the trail goes between a pair of trail termination points, and
between these points the trail's CI must be transported transparently).
This also implies the co mode MUST preserve traffic unit ordering....and
in turn, this implies that the ONLY allowable trail constructions in the
co mode are p2p and p2mp.  Further, because of the fixed/known bit rate
you are forced into a hierarchy based on well-defined layer networks, ie
we cannot have the wrapper behaviour in the co-cs mode.  Now although we
cannot violate the 'ordering' requirement in the co-cs mode we can in
the co-ps mode and, if we do this, there are significant implications
for traffic/performance/fault management.

At this point you may be saying 'where the heck is this leading to?'.
Well, the reason I went through the stuff above is to make the point
that this trail that we have constructed in some arbitrary co-cs mode
technology belongs to the traffic data-plane....in other words, it is
fully utilised and CANNOT carry the control-plane nor the
management-plane protocols for THIS layer network.  Thus, for THIS layer
network we must have a SEPARATE data-plane layer network (often referred
to as 'OOB'...Out-Of-Band) that conveys the control and management plane
protocols to each node (which is the smallest G.805 subnetwork of
practical interest in THIS layer network) where the trail will be
networked, ie switched.

Ok, so what does this mean?  Well, it means that the only practical way
you can do this for THIS layer network is to look for some LOWER server
layer network (or networks) to carry both the traffic data-plane network
and the control/management data-plane network of THIS layer network
system.  Now let's assume you want to make life simple, and don't want
to worry whether the performance of the server layer will have any
significant impact on the performance achievable in THIS layer network,
so you choose a co-cs mode server layer network for this
purpose....noting that any layer network will recursively inherit the
performance behaviour of ALL server layers right down to the duct.
Indeed, it's no co-incidence that when it comes to bulk transport of ANY
types of traffic in a WAN environment you will find a co-cs mode
technology at the bottom of the stack....this also applies to the
Internet traffic just in case anyone was wondering.  And also bear in
mind my earlier observation wrt the case that there is a requirement in
some client layer network for timing transport.

Now, in order to make it easy to see, let me give a well-known example
of what the above means....and which is almost identical to the SDH case
mentioned (which I will shortly return to wrt specifics here).

Consider the PSTN.  This is a 64kbit/s co-cs mode layer network wrt to
the traffic data-plane.  So how does control/management traffic get
transported between PSTN nodes?  Well, it could use a whole range of
different OOB server layer networks to do this.  But let me pick a
specific and very familiar case (for Europe at least) of an E1 server
layer network (and yes this IS a different layer network to the 64k
layer network).

When we adapt the 64k layer network client link connections (ie the
things that connect 64k layer nodes together) to an E1 server layer
trail the 1st thing we have to note is the bit rate specification of the
server, as there must be a known/fixed and carefully controlled
relationship here (Aside:  In this particular case the client and server
layer networks are synchronised and thus locked together wrt timing, but
when these are plesiochronous 'justification processes' are used to
align the client and server layer bit rates).  In other words the server
layer determines how the adaptation muxing regime works.  Now, I assume
most folks will know (which is why I picked this example) that there are
32 timeslots (TS) in an E1 frame.  Each TS represents a specific client
layer link connection, and as noted for this case, it will be carried on
a E1 server layer trail.  By convention (ie so we where know where stuff
is):
-	TS0 carries the E1 trail framing and other trail CI
OAM/integrity-checking information overhead like a CRC4 (this belongs to
the E1 layer network, and if one drew out the G.805 func model of this
you would see that it is generated/terminated at the E1 trail
termination points),
-	TSs 1-15 and 17-31 carry 64k client traffic link connections,
and
-	TS16 carries the control-plane protocols for the 64k layer
network.  So, TS16 is a logical OOB link that goes between the
subnetwork controllers of the 64k (traffic) layer network that is
carried by an E1 server layer trail between 64k nodes.

What sort of control-plane (or in a more general case also
management-plane) protocols run on this OOB network is of no concern of
the 64k traffic data-plane nor indeed the (server) E1 trail.  I can
select any appropriate signalling protocol and routing process I like
here.  HOWEVER, the design of this network is a big
concern....especially wrt to resilience, and because it is so critical
it must take its design cues directly from the duct network.  Further
note that because it is logically OOB wrt the traffic data-plane network
it controls (i) it can be made very stable (ie wrt to traffic loadings),
(ii) it can be made very secure to attack and (iii) it needs its own
addressing scheme which is NOT the same addressing scheme/space as that
used/occupied by the traffic data-plane network it controls.....and it
also has nothing to do with the addressing scheme used on the lower E1
layer network.

Now let's go back to the SDH case you mentioned.  And let's start by
considering the VC4 layer network....and think of this as analogous to
the 64k layer network of the above example.  So, as we now know, we are
going to have to find some LOWER layer network to carry the
control/management plane protocols between the VC4 subnetwork nodes.  In
the SDH networking system the most obvious choice for this (and it does
NOT have to use this) is the STM layer network.  This is a real layer
network of its own right, however it is degenerate because we do not
network STM trail constructs....these simply go p2p between (in the case
I have selected) the VC4 nodes.  So, the STM trail is the analogy for
the VC4 client layer of what the E1 trail was for the 64k client layer
in the previous example.  There are, again by convention, well known
'TSs' in the STM section overhead that could be used to carry the
control/management protocols for the VC4 layer network.....these are
often referred to as the DDC (data communications channel).  So these
are the analogy of what TS16 is for the 64k/E1 client/server case.

Now let's assume I own the VC4 layer network and I use the STM overhead
to carry my OOB control/management plane protocols for this VC4 layer
network.  If I also own the VC12 layer network (and indeed all the
possible VCx layer networks I could create within the SDH networking
system) I could also consider using this same OOB network to carry
control/management plane protocols for these too.  I would have to
devise a scheme that logically separated control/management for each of
the different VCx layer networks because they are DIFFERENT layer
networks and their (subnetwork) switching nodes and trail terminations
points will be in different places.  Note also that because they are
different layer networks their addresses belong to separate
spaces...even if they use the same structure....which is rather obvious
by noting that it clearly makes no sense to attempt to connect a VC4
trail termination point to a VC12 trail termination point.

I could, however, conceive of an alternative way of doing this where the
STM server layer overhead carried the control/management for the VC4
layer network as before, but I now use the VC4 layer network as the
server layer to the VC12 layer network.....in other words I could use
the VC4 overhead (say) to carry an OOB control/management plane network
for the VC12 layer network, or I could even postulate a scheme whereby I
carved up a VC4 into VC12s and dedicated one of the VC12 channels to
carry the control/management of the VC12 layer network. 

In all these cases the same principle that I outlined with the 64k
client and E1 server case remains, ie we need to define some way of
carrying the control/management OOB, and there are many ways this could
be achieved.  Indeed, these do not even need to be within the SDH
networking system....though it obviously makes sense to use this where
possible simply for practical convenience.

Now suppose I do not own the VC12 layer (which I suspect would be an
unusual case), or say some other non-SDH-based layer network such as
ATM, PDH, FR, whatever that was carried over my SDH VC4 layer network
(which is a far more usual case).  It should be obvious that there is no
way I could contemplate sharing my STM-based OOB VC4 control/management
plane layer network with some other operator.  In any case, for the
non-SDH-based layer networks such as ATM say its obvious that their span
would not be congruent (invariably larger) than my (server) SDH layer
network anyway.  Remember - a link connection in a client layer network
is created by a trail in a server layer network...so trail spans of
different layer networks are generally not congruent.

So, to wrap-up here....when you have a client/server layer networking
case between 2 *different* operating parties there must be full
functional decoupling between the 2 layer networks.  This is not for
debate it's a must-have starting point requirement.  What this means in
practice is that all 3 planes of the client layer network must be
carried transparently by the *traffic* data-plane component of some
other server layer network (or networks....though in most practical
cases the SAME server layer trail will carry all 3 client layer planes).
And how this is determined is a function of the client->server muxing
adaptation function of the link-connections that form the topology of
the client layer network (be this the client traffic data-plane network
or the control/management plane network).

Note - Not sure if I have done a good job at explaining all this, and I
guess to some folks this will be obvious but to many others (in my
experience at least) some of this won't be that obvious.  If anyone
wants to know more about functional architecture stuff (or control-plane
stuff) the best source of information I know of is Reid/Sexton
'Broadband networking', esp the stuff covered in the early chapters
(Andy Reid is a colleague of mine in BT/CTO, but this is not the reason
I am mentioning his book ;-)). 

> If, say, PCE fails a path 
> computation for a VC12 connection, could it suggest a 
> reconfiguration in the VC4 layer, pass this suggestion to a 
> VNT Manager (application/entity that has security, budget, etc.
> authorization) to add trails in VC4 layer and provide extra 
> flexibility in the VC12 layer?
NH=> If the same party owns both VC12 and VC4 layer networks you may be
able to do this, ie change the VC12 topology  (subject to GoS issues),
but this is a very different case to where the same party does NOT own
all the layer networks......and the latter is the more general case of
leasing capacity off some other operator.  This is why I repeatedly keep
making the point that a network builder service is THE starting
point.....this capability MUST be available.

BTW - You should note that operators are looking to phase out the lower
layer co-cs networks, eg VC12.  This creates a very important
responsibility for the co-ps mode technology to step up to this
job.....and you can probably guess what I could say next here about
certain co-ps technologies!

> Correct me if I am wrong, but 
> you sound like always require
> NMS+human in the inter-layer relationships. Why the process 
> could not be
> autometed via control plane ?
NH=> Well, it depends on who owns which layer networks.   And in all
cases you still have a the problem of designing the lower layer network
to some GoS such that client demands can be serviced in some acceptable
period and at some acceptable cost.  Further, as Alan McGuire here
pointed out (off list) if you have got to the stage where circuitous
client layer routings are required your network design/planning process
is already in rather serious trouble.  But have any folks got any good
models they are working to for this (also see aside below)?  We've
looked at this problem in some detail and concluded its a no-go area
right now.  But even if/when you do this, I am afraid you will still
find that the NMS is king here and it MUST know what actions have been
taken by the control-plane.

Aside - If, instead of just considering the private network builder
service case, you looked at the potential population of the public
network serve case for SVCs (ie mode selection in end-systems) and the
likely greater calling rates and far shorter holding times of trails
here, then there could be a more sound commercial proposition here.
Especially noting that mode selection can solve what are otherwise very
hard QoS/pricing problems that are facing operators.  I note the
proposals from Jaihyung Cho (on the 'end-to-end L2-LSP' thread) are
starting to give some hints in this direction.

<snipped>

> IB>> Nobody said that GMPLS must control all layers,
NH=> This is not how I would interpret some of the statements I have
seen in the past from some folks, nor IMO how this general topic areas
was initially 'sold' to the masses.  

> IB>> for
> instance all/some SDH layers. Network topology in lower 
> layers it does not control, of course, needs to be in place.
NH=> Agreed.  But what about higher layer networks that are NOT owned by
the same party....what do you say here?

regards, Neil

_______________________________________________
Pce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pce

---

From: "Igor Bryskin" <[email-masked]>
Date: Wed, 13 Jul 2005 15:41:14 -0400

Hi Neil,

Thanks for such a detailed information. I hope you will come to Paris (IETF)
and we can continue the discussion.

While reading your response I realized that I should've been more specific
describing the example. For instance. I never assumed using overhead bits
for the control plane connectivity for neither of layers. My example was as
follows:

1. A single transport network operator owns VC4, VC12, etc. layers, so that
a single control plane domain (signaling, routing, TE, path computation.
etc.) manages all of them. (Surely there must be commercial benefits for the
operator to own multiple layers that can establish server/client
relationships);

2. This is *not* a public service case: there are no control plane
relationships with other operators for the layers in question;

3. For the control and management plane connectivity the
*out-of-band-out-of-fiber* DCN is used - no overhead bits of neither of
layers!

4. The control plane contains a logical construct - VNT Manager (centralized
or distributed). Only this entity is entitled to perform dynamic server
layer re-configurations to satisfy client layer connection requests
(provided, that the server layer is under control of the operator in
question. The re-configuration of a server layer that does not belong to the
operator is out of scope of this discussion, although it could be an
interesting discussion in future). The VNT Manager has an access to:
a) resources available in each of the layers including their termination and
adaptation capabilities. The resources are clearly broken into two groups:
ones that could be used for dynamic re-configuration and ones that could be
used only under the management control;
b) service demand and it's dynamics (e.g. mean active time);
c) policies wrt use of the resources;
d) network planning tools for verification of validity/feasibility of
possible re-configurations

I understand that you agree that such a setup is possible. The question is
this scenario is *commercially realistic* today? 3-5 years from now?

Hope will see you, Alan or Ben in Paris.

Igor

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Wednesday, July 13, 2005 12:58 PM
Subject: Network modes and layer network relationships

<snipped>

regards, Neil