Subject: RE: Question on draft-shiomoto-ccamp-gmpls-addressing-00.txt

From: <[email-masked]>
Date: Tue, 8 Mar 2005 13:14:08 -0000

Ashok,

Thanks for the explanation.  If I understand correctly what you are saying is that the best current practice is to use an OOB control plane for the non-PSC  layer networks and an in-band control plane for the PSC layer networks.  So one must avoid using the control plane network (of a non-PSC layer network) for the transfer of data (or control) plane packets from the PSC layer network's control & data planes.  Fine, I agree that is the BCP, but I don't believe the current draft articulates that intent very well and section 7.4 probably needs expanding (IMO one or two paragraphs is probably not enough to discuss/address/detail OOB Vs in-band properly and unambiguously), for example it is not IMO clear from the current text that you are talking about 2 separate control plane networks - one OOB (for non-PSC layer networks/nodes) and one inband (for PSC layer networks/nodes).

Thanks
Ben

> -----Original Message-----
> From: Ashok Narayanan [mailto:[email-masked]]
> Sent: 08 March 2005 12:59
> To: Niven-jenkins,B,Ben,XDE73 R
> Cc: [email-masked]
> Subject: Re: Question on draft-shiomoto-ccamp-gmpls-addressing-00.txt
> 
> 
> > 
> > I am trying to understand section 7.4 (Separation of 
> Control and Data Plane Traffic) of 
> draft-shiomoto-ccamp-gmpls-addressing-00.txt better.
> > 
> > The draft states:
> > "PSC-capable nodes implementing an OOB control plane (perhaps to
> > communicate with an optical switch) MUST NOT use the OOB control
> > plane for data traffic.  For example, in the case of MPLS service
> > running on top of a GMPLS LSP, if the peer PSC device is reachable
> > via both the control plane and the data plane, control protocols
> > such as LDP MUST NOT form adjacencies over the control plane
> > interfaces.  This may be provided by a combination of implementation
> > features and deployment guidelines."
> > 
> > So, if the control plane is OOB why must control protocols (such as
> > LDP) NOT form adjacencies over the control plane interfaces?  This
> > implies that LDP (for e.g.) MUST form its adjacencies over the data
> > plane interfaces and therefore LDP is not running OOB (and therefore
> > the control plane is not entirely OOB).
> > 
> 
> That's an interesting question. The draft focuses on GMPLS/TE
> deployments and the use of an out-of-band control channel as a matter
> of necessity (i.e. because optical switches are not capable of
> handling in-band control traffic). Consider the general deployment of
> a core of optical switches surrounded by GMPLS/TE routers at the
> edge. From the perspective of this network (optical switches, edge
> routers, and the links in between), that statement translates to:
> 
>  _ The GMPLS/TE control plane between the edge routers and the optical
>    switches in the core is entirely out-of-band
> 
>  - All other protocols between the edge routers MUST be 
> entirely in-band.
> 
> The control plane network may well be a series of point-to-point GRE
> tunnels overlaid on some other low-bandwidth network (actually this is
> what the draft recommends). Ordinarily one would establish a GRE
> tunnel between each pair of neighbors so they can talk to each other.
> But this means that signalling messages going from edge router to edge
> router actually have to be routed through the control planes of
> intermediate switches. The two alternatives to avoid this (to create a
> full mesh of GRE tunnels - at least between all the edge routers - or
> to not use GRE at all if the control plane is switched Ethernet, and
> therefore to essentially permit a full mesh of OSPF peers) are not
> really palatable (and in the case of SDCC, not usable). The optical
> switches in the core should not be in the business of routing
> packets. Sometimes they have to (e.g. Notify messages), but they are
> not designed for this purpose and should not be deployed as such.
> 
> And practically speaking the control network is expected to be a
> low-bandwidth network that can "just get the job done". A single
> 10/100 Ethernet (or, worse, 192kbps SDCCs) can probably handle the
> control traffic for a GMPLS/TE network, but will probably fail if we
> want to use it to peer LDP or BGP for 10,000 prefixes/labels between
> the edge routers.
> 
> The best current practice is therefore to use the low-bandwidth
> control network between routers and switches for GMPLS/TE signaling
> (RSVP-TE, OSPF, LMP) and use the high-bandwidth optical TE LSPs
> signalled between edge routers to do other L3 protocols between them.
> 
> The issue of a separated control plane between PSC edge routers for
> different L3 protocols for administrative reasons is an interesting
> one. Nothing precludes you from creating multiple parallel LSPs
> between edge routers and using one of them to carry control traffic
> between the two edge routers on behalf of all the others; that would
> still satisfy this requirement. Alternatively, one could engineer a
> more beefy control plane network so that it can in fact carry all the
> control plane traffic (BGP, LDP etc.) but that is not the BCP today
> based on available implementations.
> 
> -Ashok
> 
> On Tue, Mar 08, 2005 at 10:33:54AM -0000, 
> [email-masked] wrote:
> > Colleagues,
> > 
> > I am trying to understand section 7.4 (Separation of 
> Control and Data Plane Traffic) of 
> draft-shiomoto-ccamp-gmpls-addressing-00.txt better.
> > 
> > The draft states:
> > "PSC-capable nodes implementing an OOB control plane 
> (perhaps to communicate with an optical switch) MUST NOT use 
> the OOB control plane for data traffic.  For example, in the 
> case of MPLS service running on top of a GMPLS LSP, if the 
> peer PSC device is reachable via both the control plane and 
> the data plane, control protocols such as LDP MUST NOT form 
> adjacencies over the control plane interfaces.  This may be 
> provided by a combination of implementation features and 
> deployment guidelines."
> > 
> > So, if the control plane is OOB why must control protocols 
> (such as LDP) NOT form adjacencies over the control plane 
> interfaces?  This implies that LDP (for e.g.) MUST form its 
> adjacencies over the data plane interfaces and therefore LDP 
> is not running OOB (and therefore the control plane is not 
> entirely OOB).
> > 
> > Thanks
> > Ben
> > 
> > 
> > -- 
> > Ben Niven-Jenkins
> > Networking Specialist, BT Exact
> > 
> > e-mail: [email-masked]
> > tel: +44(0) 1473 648225
> > mob: +44(0) 7918 077205
> > 
> > pp34/161 B81 Callisto House, Adastral Park, Martlesham, 
> Ipswich IP5 3RE, UK
> > 
> > BT Exact is a trademark of British Telecommunications plc
> > Registered office: 81 Newgate Street London EC1A 7AJ
> > Registered in England no. 1800000
> > 
> > This electronic message contains information from British 
> Telecommunications which may be privileged or confidential. 
> The information is intended to be for the use of the 
> individual(s) or the entity named above. If you are not the 
> intended recipient be aware that any disclosure, copying, 
> distribution or use of the contents of this information is 
> prohibited. If you have received this electronic message in 
> error, please notify us by telephone or email (to the numbers 
> or address above) immediately. 
> > 
> > Activity and use of the British Telecommunications plc 
> email system is monitored to secure its effective operation 
> and for other lawful business purposes. Communications using 
> this system will also be monitored and may be recorded to 
> secure effective operation and for other lawful business purposes.
> 
> -- 
> 
> 
> 
> --- Asok the Intern ----------------------------------------
> Ashok Narayanan
> IOS Network Protocols, Cisco Systems
> 1414 Mass Ave, Boxborough MA 01719
> Ph: 978-936-1608.  Fax: 978-936-2218 (Attn: Ashok Narayanan)
>

---

From: Ashok Narayanan <[email-masked]>
Date: Tue, 8 Mar 2005 08:21:22 -0500

Ben,

Thanks for the comment. We'll try to clarify this in the next version.

-Ashok

-- 

--- Asok the Intern ----------------------------------------
Ashok Narayanan

---

From: dimitri papadimitriou <[email-masked]>
Date: Tue, 08 Mar 2005 14:56:05 +0100

ashok - this section is indeed to be clarified

1. for non-PSC it should distinguish the logical from the physical 
control channel separation with respect to the data plane channels, i do 
not see for instance why DCC channels could not be used to exchange 
control plane traffic and still keep a logical separation

2. for PSC - why is there something different in terms of "practice" 
than what is currently used in MPLS networks ?

or

3. do you assume that the interconnection of a PSC with non-PSC network 
(both driven by GMPLS) could indeed introduce change from the current 
practices operators used to use for their packet networks ? that in case 
could be further applied to PSC-only networks ?

thanks,
- dimitri.

Ashok Narayanan wrote:

>> Thanks for the explanation.  If I understand correctly what you are
>> saying is that the best current practice is to use an OOB control
>> plane for the non-PSC  layer networks and an in-band control plane
>> for the PSC layer networks.  So one must avoid using the control
>> plane network (of a non-PSC layer network) for the transfer of data
>> (or control) plane packets from the PSC layer network's control &
>> data planes.  Fine, I agree that is the BCP, but I don't believe
>> the current draft articulates that intent very well and section 7.4
>> probably needs expanding (IMO one or two paragraphs is probably not
>> enough to discuss/address/detail OOB Vs in-band properly and
>> unambiguously), for example it is not IMO clear from the current
>> text that you are talking about 2 separate control plane networks -
>> one OOB (for non-PSC layer networks/nodes) and one inband (for PSC
>> layer networks/nodes).
>> 
>> Thanks Ben
>> 
>> 
>>> -----Original Message----- From: Ashok Narayanan
>>> [mailto:[email-masked]] Sent: 08 March 2005 12:59 To:
>>> Niven-jenkins,B,Ben,XDE73 R Cc: [email-masked] Subject: Re:
>>> Question on draft-shiomoto-ccamp-gmpls-addressing-00.txt
>>> 
>>> 
>>> 
>>>> The draft states: "PSC-capable nodes implementing an OOB
>>>> control plane (perhaps to communicate with an optical switch)
>>>> MUST NOT use the OOB control plane for data traffic.  For
>>>> example, in the case of MPLS service running on top of a GMPLS
>>>> LSP, if the peer PSC device is reachable via both the control
>>>> plane and the data plane, control protocols such as LDP MUST
>>>> NOT form adjacencies over the control plane interfaces.  This
>>>> may be provided by a combination of implementation features and
>>>> deployment guidelines."
>>>> 
>>>> adjacencies over the data plane interfaces and therefore LDP is
>>>> not running OOB (and therefore the control plane is not
>>>> entirely OOB).
>>>> 
>>> 
>>> deployments and the use of an out-of-band control channel as a
>>> matter of necessity (i.e. because optical switches are not
>>> capable of handling in-band control traffic). Consider the
>>> general deployment of a core of optical switches surrounded by
>>> GMPLS/TE routers at the edge. From the perspective of this
>>> network (optical switches, edge routers, and the links in
>>> between), that statement translates to:
>>> 
>>> _ The GMPLS/TE control plane between the edge routers and the
>>> optical switches in the core is entirely out-of-band
>>> 
>>> - All other protocols between the edge routers MUST be entirely
>>> in-band.
>>> 
>>> The control plane network may well be a series of point-to-point
>>> GRE tunnels overlaid on some other low-bandwidth network
>>> (actually this is what the draft recommends). Ordinarily one
>>> would establish a GRE tunnel between each pair of neighbors so
>>> they can talk to each other. But this means that signalling
>>> messages going from edge router to edge router actually have to
>>> be routed through the control planes of intermediate switches.
>>> The two alternatives to avoid this (to create a full mesh of GRE
>>> tunnels - at least between all the edge routers - or to not use
>>> GRE at all if the control plane is switched Ethernet, and 
>>> therefore to essentially permit a full mesh of OSPF peers) are
>>> not really palatable (and in the case of SDCC, not usable). The
>>> optical switches in the core should not be in the business of
>>> routing packets. Sometimes they have to (e.g. Notify messages),
>>> but they are not designed for this purpose and should not be
>>> deployed as such.
>>> 
>>> 10/100 Ethernet (or, worse, 192kbps SDCCs) can probably handle
>>> the control traffic for a GMPLS/TE network, but will probably
>>> fail if we want to use it to peer LDP or BGP for 10,000
>>> prefixes/labels between the edge routers.
>>> 
>>> control network between routers and switches for GMPLS/TE
>>> signaling (RSVP-TE, OSPF, LMP) and use the high-bandwidth optical
>>> TE LSPs signalled between edge routers to do other L3 protocols
>>> between them.
>>> 
>>> The issue of a separated control plane between PSC edge routers
>>> for different L3 protocols for administrative reasons is an
>>> interesting one. Nothing precludes you from creating multiple
>>> parallel LSPs between edge routers and using one of them to carry
>>> control traffic between the two edge routers on behalf of all the
>>> others; that would still satisfy this requirement. Alternatively,
>>> one could engineer a more beefy control plane network so that it
>>> can in fact carry all the control plane traffic (BGP, LDP etc.)
>>> but that is not the BCP today based on available implementations.
>>> 
>>> 
>>> -Ashok
>>> 
>>>> The draft states: "PSC-capable nodes implementing an OOB
>>>> control plane
>>> 
>>> (perhaps to communicate with an optical switch) MUST NOT use the
>>> OOB control plane for data traffic.  For example, in the case of
>>> MPLS service running on top of a GMPLS LSP, if the peer PSC
>>> device is reachable via both the control plane and the data
>>> plane, control protocols such as LDP MUST NOT form adjacencies
>>> over the control plane interfaces.  This may be provided by a
>>> combination of implementation features and deployment
>>> guidelines."
>>> 
>>> adjacencies over the data plane interfaces and therefore LDP is
>>> not running OOB (and therefore the control plane is not entirely
>>> OOB).
>>> 
>>>> Thanks Ben
>>>> 
>>>> 
>>>> -- Ben Niven-Jenkins Networking Specialist, BT Exact
>>>> 
>>>> e-mail: [email-masked] tel: +44(0) 1473 648225 
>>>> mob: +44(0) 7918 077205
>>>> 
>>>> Registered office: 81 Newgate Street London EC1A 7AJ Registered
>>>> in England no. 1800000
>>>> 
>>> Telecommunications which may be privileged or confidential. The
>>> information is intended to be for the use of the individual(s) or
>>> the entity named above. If you are not the intended recipient be
>>> aware that any disclosure, copying, distribution or use of the
>>> contents of this information is prohibited. If you have received
>>> this electronic message in error, please notify us by telephone
>>> or email (to the numbers or address above) immediately.
>>> 
>>> email system is monitored to secure its effective operation and
>>> for other lawful business purposes. Communications using this
>>> system will also be monitored and may be recorded to secure
>>> effective operation and for other lawful business purposes.
>>> 
>>> --
>>> 
>>> 
>>> 
>>> Ashok Narayanan IOS Network Protocols, Cisco Systems 1414 Mass
>>> Ave, Boxborough MA 01719 Ph: 978-936-1608.  Fax: 978-936-2218
>>> (Attn: Ashok Narayanan)
>>> 
> 
>

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 8 Mar 2005 14:12:54 -0000

Hi,

Ben summarized...

> saying is that the best current practice is to use an OOB control plane
> for the non-PSC  layer networks and an in-band control plane for the
> PSC layer networks.

If that is the intention of the authors of this draft, they will have to
fight me :-)

I do not see why an in-band control channel for a PSC network would be
made a RECOMMEND, SHOULD or MUST.

>  So one must avoid using the control plane network (of a non-PSC
> layer network) for the transfer of data (or control) plane packets from
> the PSC layer network's control & data planes.

Yes. But more precisely, one must avoid using the control channels for
data traffic. Even in-band control channel may use a different address
space, and those addresses must not be used for data.

Cheers,
Adrian

---

From: Ashok Narayanan <[email-masked]>
Date: Tue, 8 Mar 2005 09:29:59 -0500

Dimitri,

Thanks for the comments. Hopefully this text will get beaten into
shape with the attention you folks are paying to it.

> control plane traffic and still keep a logical separation

Sure; of course we are not saying that. DCC channels although
travelling on the same fiber are logically separated from the data
channel on the same fiber. I don't think there is any statement to the
contrary.

> than what is currently used in MPLS networks ?

We aren't recommending any. The separated L3 control plane idea I had
mentioned was my attempt to understand the kind of application that
Ben was getting at when he asked "why must control protocols (such as
LDP) NOT form adjacencies over the control plane interfaces". See below.

> could be further applied to PSC-only networks ?

I'd say the first part of this is slightly true. Let me try and
reformulate; please tell me if this is better:

"Consider a GMPLS/TE network with optical switches in the core and PSC
devices at the edge, with a separated control plane (either OOB or via
DCC). The edge PSC devices have IP reachability with each other via
both the separated control plane network and via any optical GMPLS/TE
LSPs that may be configured between them (the "data traffic
network"). For the reasons mentioned below, traffic on the separated
control network MUST be limited to GMPLS/TE signalling traffic solely
for the purpose of established optical LSPs within the core of optical
switches. All bearer traffic carried through this network MUST be
carried over GMPLS/TE LSPs and MUST NOT be carried over the separated
control network. Further, any protocols that run purely between the
PSC devices that are not used for GMPLS/TE signalling within the
optical switch core (e.g. protocols for carrying bearer traffic like
LDP and BGP) MUST communicate over GMPLS/TE LSPs and MUST NOT form
adjacencies or exchange messages over the separated control network."

So there is in fact a change from the current practice of a PSC
network *only* at the edge of a cloud of optical switches running
GMPLS. If this was a regular cloud of routers then we could say "if
you have IP reachability over link X, use it". But in the case of an
optical switch network we may well have IP reachability through the
separated control network but we don't want to use it.

Regarding the second part of that question: we are not suggesting any
techniques to be applied in PSC-only networks. The separated PSC
control plane I alluded to earlier may be a useful idea, but it merits
separate treatment and is beyond the scope of this section IMHO.

-Ashok

-- 

--- Asok the Intern ----------------------------------------
Ashok Narayanan

---

From: Ashok Narayanan <[email-masked]>
Date: Tue, 8 Mar 2005 09:51:50 -0500

> If that is the intention of the authors of this draft, they will have to

I better check out this website:
 http://www.all-karate.com/how_to_learn_karate.php

> made a RECOMMEND, SHOULD or MUST.

That is certainly not the recommendation (phew). 

I clarified in a separate email, but basically the point we are making
is that in an optical GMPLS/TE network with optical switches in the
core and PSC devices on the edge, edge-to-edge protocols not used for
GMPLS/TE signalling within the core must be kept off the separated
control network. I think the text needs to be tightened up to say
this; I sent out an alternative formulation just now, let's see how
that goes.

> space, and those addresses must not be used for data.

Which is fine, but then raises the question: what is "data"? Bearer
traffic? SIP messages transiting through a number of hops to get here?
IPv4 RSVP for QoS? BGP or LDP running between the PSCs? I claim that
for the aforementioned "GMPLS/TE-optical-core-PSC-edge" network, *all*
of the aforementioned must be kept off the separated control plane
*used for GMPLS within the optical core*. Nothing prevents these other
protocols from having their own separated control plane, just as long
as that other control plane also goes over a GMPLS/TE LSP or via some
other "data" path.

What I'm trying to say is that the optical switches in the core should
basically not have to ever see any signalling or data packets that
don't pertain to GMPLS/TE signalling within the optical core.

In summary: (is this a better way to write this in the draft?)

1) A core of optical switches with PSC edge devices running GMPLS may
use a separated control network (DCC, OOB Ethernet, or some other) to
carry GMPLS/TE signalling messages. Further, this control network may
be low-bandwidth and may pass through devices that are not well suited
for IP routing.

2) Bearer data traffic transiting this network MUST NOT be carried on
this separated control network.

3) Further, given the nature of the control network and the devices
that maintain it, protocols running between the PSC edge devices that
are not used for GMPLS/TE signalling within the core MUST NOT be
carried over this control network. (N.B. should this be relaxed to a
"SHOULD"?) This does not preclude these other protocols from running
separated control planes of their own, as long as the protocol
signalling messages do not transit the separated control network used
within the GMPLS/TE core.

-Ashok

--- Asok the Intern ----------------------------------------
Ashok Narayanan