Subject: Scaling hierarchical LDP/PW stitching

From: [email-masked]
Date: Wed, 21 Jul 2004 08:18:49 +0100

To address LDP scaling (and VPLS packet replication) a hierarchical LDP model is proposed (in draft-ietf-l2vpn-signaling-01.txt) where hubs (PErs routers) are connected via a full mesh of LDP sessions, but spokes (PEr/MTUs routers) only establish LDP sessions with the hub(s) that they connect to. This reduces the number of LDP sessions required and also reduces the number of packet replications that hub (PErs) routers must perform.

However, as yet I don't believe there are any drafts that discuss exactly how the hub and spoke PWs are signalled and stitched/bound together. Likewise for VPWS PW paths that are established by stitching multiple PWs together between endpoints. I believe this is an important area as one can imagine large service providers having hundreds of PErs routers connected via a full mesh of LDP sessions with tens of thousands of MTUs/PEr devices hanging off them. To support hundreds of thousands of PWs between tens of thousands of MTUs/PEr routers obviously requires a highly scalable provisioning solution.

The example below shows an example of a hierarchical VPLS/VPWS network with 4 PErs devices each connected to a single MTUs device:

MTUs4----PErs4--PErs3-----MTUs3
           |  \ /  |
           |   X   |
           |  / \  |
MTUs1----PErs1--PErs2-----MTUs2 
 
I assume that to provision a stitched VPWS PW path from MTUs1 to MTUs2, the individual PWs that form the path need to be established individually and then stitched together. Although the individual PWs can be established using single sided provisioning, the signalling for each PW must be initiated at one side for each PW either through manual configuration or via OSS. OSS (or some manual method) must also be used to keep track of and bind the different PWs together. To do this OSS needs to learn or be configured with information about which MTUs devices sit behind which PErs devices.

For VPLS, the point-to-multipoint PWs between the hubs (PErs devices) can be provisioned using singled sided provisioning, but the spoke PWs need to be signalled and associated with the relevant VPLS separately. Again, the signalling needs to be initiated at one side and the spokes need to be associated with the correct VPLS via manual configuration or via OSS.

The drawback with the above approach is that intermediate routers along the stitched PW path (i.e. PErs routers) must each be configured via OSS or via a manual method, and OSS must be also be used to bind and keep track of the individual PWs. The ideal solution would be to use a dynamic protocol(s) to establish the end-to-end PW path using single sided provisioning initiated from one end of the path. 

However, to establish and end-to-end path across multiple hops requires a path to be calculated and selected (if multiple paths exist) before the path can be signalled. Stitching PWs together to from a path between two endpoints is effectively creating a new layer network. Although targeted LDP has been proposed as the signalling mechanism to signal individual PWs, there is no routing at the PW layer to calculate/select an end-to-end path.

Route/path calculation/selection is fairly simple for the hierarchical LDP model using a centralised OSS approach, each MTUs is connected to one PErs (except where dual homing is used), and each PErs has a targeted LDP session with each other. Therefore, if the OSS knows that MTUsX sits behind PErsX, and MTUsY sits behind PErsY, then to establish a PW path between MTUsX and MTUsY requires stitching/signalling of 3 PWs, i.e. MTUsX<->PErsX, PErsX<->PErsY, and PErsY<->MTUsY. Obtaining information about which MTUs devices sit behind which PErs devices is straightforward in the OSS approach because the information is gathered, maintained, and processed using a centralised mechanism. Using a distributed approach however using dynamic protocols requires reachability information to be distributed, at least between the PErs devices, i.e. PErs devices need to advertise to each other which MTUs devices they are connected to. As yet a mechanism has not been defined to distribute this information, or to select primary/secondary paths where MTUs devices are multihomed.

Du Ke has submitted a draft recently (<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-control-protocol-ext-00.txt>) to address LDP scaling that proposes a solution with two different modes:

i) A mode called 'simple trunk' that reflects LDP messages to clients similar to the way in which BGP route reflectors distribute routes to clients. This potentially could be used to reduce the number of LDP sessions in an existing full mesh.

ii) A mode called 'next hop self' that describes a solution for PW stitching but does not scale as it relies on broadcasting labels to every PErs in the full mesh for every VPWS/VPLS (the route of the problem being that there is no routing functionality at the 'PW layer' which the draft does not address).

I would like to establish consensus as to whether a) a dynamic protocol based end-to-end single sided provisioning solution is required to aid scaling for stitched PWs, or if b) PW stitching is best handled via OSS.

Any comments/thoughts are welcomed.

Thanks,
Richard

---

From: Luca Martini <[email-masked]>
Date: Wed, 21 Jul 2004 09:58:17 -0600

Richard,

First I would like to point out that we are looking at solving a problem 
that does not really exist !
LDP should scale very well to many thousands of sessions. TCP/IP has 
been shown to scale to millions of sessions, and LDP uses TCP/IP as a 
transport, with a light session initialization message protocol on top.
We have to keep in mind that this is not BGP , with hundred of thousands 
routes , each requiring complex policy filtering procedures.

So you are bundling several issues together:
-VPLS packet replication
- LDP scaling
- VPLS Autodiscovery

So I believe that LDP scaling is not an issue, but VPLS autodiscovery , 
and packet replications certainly are.

VPLS autodiscovery can be done in several ways , by using BGP ( 
draft-ietf-l2vpn-signaling-01.txt ), LDP ( defunct draft from Olen 
Stokes ), or radius ( something like  
draft-ietf-l2vpn-radius-pe-discovery-00.txt ).

I believe that single sided signaling of PW is needed to help scaling of 
autodiscovery. ( this way only the VPLS instance ID is needed , not 
information about every link ). This seems to be a  common requirement 
of all the autodiscovery solutions.

As far as PW stitching , or as I call it "switching", it is addressed in 
draft-martini-pwe3-pw-switching-00.txt.
Note that in this draft we switch PDUs from one PW to another PW, 
running separate LDP control planes.
This can be used to reduce the number of LDP sessions, but that only 
makes LDP MD5 key management a little easier. It does not improve 
scaling. The other application is to use a PW switching point PE as a 
peering point between different administrative domains. ( note that this 
does not require a new protocol, which makes peering easier )

Luca

---

From: [email-masked]
Date: Wed, 21 Jul 2004 18:15:19 +0100

Luca,

> that does not really exist !

I disagree, I think that scaling stitched PW provisioning to hundreds of thousands of stitched PW paths between tens of thousands of edge devices is a problem that exists today.

> routes , each requiring complex policy filtering procedures.

I didn't state that scaling LDP sessions was the problem. I said that the one of the reasons hierarchical VPLS was introduced was to reduce the number of LDP sessions in the core.

> So you are bundling several issues together:

No, I did not bundle several issues together, I asked specifically about stitched PW provisioning scaling. I didn't mention autodiscovery and only mentioned VPLS packet replication and LDP scaling to explain why the hierarchical VPLS model was introduced.

>From your PW stitching or "switching" draft "The pseudowire switching methodology described in this document assumes manual configuration of the switching point" and "the PW switching point is configured to switch a SH-PW from a specific peer to another SH-PW destined for a different peer.

This does not address the problem I raised, quoting from my original email:

"The drawback with the above (existing) approach is that intermediate routers along the stitched PW path (i.e. PErs routers) must each be configured via OSS or via a manual method, and OSS must be also be used to bind and keep track of the individual PWs. The ideal solution would be to use a dynamic protocol(s) to establish the end-to-end PW path using single sided provisioning initiated from one end of the path."

Just to be clear, the question I asked was "I would like to establish consensus as to whether a) a dynamic protocol-based end-to-end single sided provisioning solution is required to aid scaling for stitched PWs, or if b) PW stitching is best handled via OSS."

Richard

> >To address LDP scaling (and VPLS packet replication) a 
> hierarchical LDP model is proposed (in 
> draft-ietf-l2vpn-signaling-01.txt) where hubs (PErs routers) 
> are connected via a full mesh of LDP sessions, but spokes 
> (PEr/MTUs routers) only establish LDP sessions with the 
> hub(s) that they connect to. This reduces the number of LDP 
> sessions required and also reduces the number of packet 
> replications that hub (PErs) routers must perform.
> >
> >However, as yet I don't believe there are any drafts that 
> discuss exactly how the hub and spoke PWs are signalled and 
> stitched/bound together. Likewise for VPWS PW paths that are 
> established by stitching multiple PWs together between 
> endpoints. I believe this is an important area as one can 
> imagine large service providers having hundreds of PErs 
> routers connected via a full mesh of LDP sessions with tens 
> of thousands of MTUs/PEr devices hanging off them. To support 
> hundreds of thousands of PWs between tens of thousands of 
> MTUs/PEr routers obviously requires a highly scalable 
> provisioning solution.
> >
> >The example below shows an example of a hierarchical 
> VPLS/VPWS network with 4 PErs devices each connected to a 
> single MTUs device:
> >
> >I assume that to provision a stitched VPWS PW path from 
> MTUs1 to MTUs2, the individual PWs that form the path need to 
> be established individually and then stitched together. 
> Although the individual PWs can be established using single 
> sided provisioning, the signalling for each PW must be 
> initiated at one side for each PW either through manual 
> configuration or via OSS. OSS (or some manual method) must 
> also be used to keep track of and bind the different PWs 
> together. To do this OSS needs to learn or be configured with 
> information about which MTUs devices sit behind which PErs devices.
> >
> >For VPLS, the point-to-multipoint PWs between the hubs (PErs 
> devices) can be provisioned using singled sided provisioning, 
> but the spoke PWs need to be signalled and associated with 
> the relevant VPLS separately. Again, the signalling needs to 
> be initiated at one side and the spokes need to be associated 
> with the correct VPLS via manual configuration or via OSS.
> >
> >The drawback with the above approach is that intermediate 
> routers along the stitched PW path (i.e. PErs routers) must 
> each be configured via OSS or via a manual method, and OSS 
> must be also be used to bind and keep track of the individual 
> PWs. The ideal solution would be to use a dynamic protocol(s) 
> to establish the end-to-end PW path using single sided 
> provisioning initiated from one end of the path. 
> >
> >However, to establish and end-to-end path across multiple 
> hops requires a path to be calculated and selected (if 
> multiple paths exist) before the path can be signalled. 
> Stitching PWs together to from a path between two endpoints 
> is effectively creating a new layer network. Although 
> targeted LDP has been proposed as the signalling mechanism to 
> signal individual PWs, there is no routing at the PW layer to 
> calculate/select an end-to-end path.
> >
> >Route/path calculation/selection is fairly simple for the 
> hierarchical LDP model using a centralised OSS approach, each 
> MTUs is connected to one PErs (except where dual homing is 
> used), and each PErs has a targeted LDP session with each 
> other. Therefore, if the OSS knows that MTUsX sits behind 
> PErsX, and MTUsY sits behind PErsY, then to establish a PW 
> path between MTUsX and MTUsY requires stitching/signalling of 
> 3 PWs, i.e. MTUsX<->PErsX, PErsX<->PErsY, and PErsY<->MTUsY. 
> Obtaining information about which MTUs devices sit behind 
> which PErs devices is straightforward in the OSS approach 
> because the information is gathered, maintained, and 
> processed using a centralised mechanism. Using a distributed 
> approach however using dynamic protocols requires 
> reachability information to be distributed, at least between 
> the PErs devices, i.e. PErs devices need to advertise to each 
> other which MTUs devices they are connected to. As yet a 
> mechanism has not been defined to distribute this 
> information, or to select primary/secondary paths where MTUs 
> devices are multihomed.
> >
> >Du Ke has submitted a draft recently

---

From: "Vach Kompella" <[email-masked]>
Date: Wed, 21 Jul 2004 10:19:48 -0700

I agree that scaling the number of LDP sessions is very similar to the
problem of how many concurrent TCP sessions you can handle.  It's not
equivalent to the issue in BGP where the route reflector was useful
because you basically flood BGP NLRIs to all neighbors (classical i-BGP
mesh topology).  LDP in the context of PWE3/VPLS has a very constrained
distribution.

In some **implementations**, there may be scaling limits, but those are
implementation issues.

Getting back to the question: PW stitching via discovery would be great,
but that is the second step after Luca's draft which provides the
mechanism for creating stitched PWs.

As far as auto-discovery goes, I have a proposal, which I didn't get in
before the deadline, which I will present, given there's time.  The
basic idea is:

 - send a "register" message for a FEC to a centralized discovery server
 - receive a "membership list" message containing members of the FEC
back
 - set up targeted LDP sessions to members in the membership list
 - exchange FECs over sessions
 - discovery server also sends new membership list to all current
members on the list

The rest of the draft deals with packet formats and redundant discovery
servers.  If I get time between now and SD, I will post a URL to the
list.

-Vach 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Luca Martini
> Sent: Wednesday, July 21, 2004 8:58 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: [PWE3] Re: Scaling hierarchical LDP/PW stitching
> 
> 
> Richard,
> 
> transport, with a light session initialization message 
> protocol on top. We have to keep in mind that this is not BGP 
> , with hundred of thousands 
> hierarchical LDP 
> >model is proposed (in draft-ietf-l2vpn-signaling-01.txt) where hubs 
> >(PErs routers) are connected via a full mesh of LDP sessions, but 
> >spokes (PEr/MTUs routers) only establish LDP sessions with 
> the hub(s) 
> >that they connect to. This reduces the number of LDP 
> sessions required 
> >and also reduces the number of packet replications that hub (PErs) 
> >routers must perform.
> >
> >However, as yet I don't believe there are any drafts that discuss 
> >exactly how the hub and spoke PWs are signalled and stitched/bound 
> >together. Likewise for VPWS PW paths that are established by 
> stitching 
> >multiple PWs together between endpoints. I believe this is 
> an important 
> >area as one can imagine large service providers having 
> hundreds of PErs 
> >routers connected via a full mesh of LDP sessions with tens of 
> >thousands of MTUs/PEr devices hanging off them. To support 
> hundreds of 
> >thousands of PWs between tens of thousands of MTUs/PEr routers 
> >obviously requires a highly scalable provisioning solution.
> >
> VPLS/VPWS network 
> >with 4 PErs devices each connected to a single MTUs device:
> >
> MTUs1 to MTUs2, 
> >the individual PWs that form the path need to be established 
> >individually and then stitched together. Although the individual PWs 
> >can be established using single sided provisioning, the 
> signalling for 
> >each PW must be initiated at one side for each PW either 
> through manual 
> also be used 
> >to keep track of and bind the different PWs together. To do this OSS 
> >needs to learn or be configured with information about which MTUs 
> >devices sit behind which PErs devices.
> >
> devices) 
> >can be provisioned using singled sided provisioning, but the 
> spoke PWs 
> >need to be signalled and associated with the relevant VPLS 
> separately. 
> >Again, the signalling needs to be initiated at one side and 
> the spokes 
> >need to be associated with the correct VPLS via manual 
> configuration or 
> >via OSS.
> >
> routers along 
> >the stitched PW path (i.e. PErs routers) must each be 
> configured via OSS or via a manual method, and OSS must be 
> also be used to bind and keep track of the individual PWs. 
> The ideal solution would be to use a dynamic protocol(s) to 
> establish the end-to-end PW path using single sided 
> hops requires 
> >a path to be calculated and selected (if multiple paths 
> exist) before 
> >the path can be signalled. Stitching PWs together to from a path 
> >between two endpoints is effectively creating a new layer network. 
> >Although targeted LDP has been proposed as the signalling 
> mechanism to 
> hierarchical 
> >LDP model using a centralised OSS approach, each MTUs is 
> connected to 
> >one PErs (except where dual homing is used), and each PErs has a 
> >targeted LDP session with each other. Therefore, if the OSS 
> knows that 
> >MTUsX sits behind PErsX, and MTUsY sits behind PErsY, then 
> to establish 
> >a PW path between MTUsX and MTUsY requires stitching/signalling of 3 
> >PWs, i.e. MTUsX<->PErsX, PErsX<->PErsY, and PErsY<->MTUsY. Obtaining 
> >information about which MTUs devices sit behind which PErs 
> devices is 
> >straightforward in the OSS approach because the information is 
> >gathered, maintained, and processed using a centralised mechanism. 
> >Using a distributed approach however using dynamic protocols 
> requires 
> the PErs 
> >devices, i.e. PErs devices need to advertise to each other 
> which MTUs 
> >devices they are connected to. As yet a mechanism has not 
> been defined 
> >to distribute this information, or to select primary/secondary paths 
> >where MTUs devices are multihomed.
> >
> >(<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-contr
ol-protoco
>l-ext-00.txt>) to address LDP scaling that proposes a solution with two

>different modes:
>
>i) A mode called 'simple trunk' that reflects LDP messages to clients 
>similar to the way in which BGP route reflectors distribute routes to 
>clients. This potentially could be used to reduce the number of LDP 
>sessions in an existing full mesh.
>
>ii) A mode called 'next hop self' that describes a solution for PW 
>stitching but does not scale as it relies on broadcasting labels to 
>every PErs in the full mesh for every VPWS/VPLS (the route of the 
>problem being that there is no routing functionality at the 'PW layer' 
>which the draft does not address).
>
>I would like to establish consensus as to whether a) a dynamic protocol

>based end-to-end single sided provisioning solution is required to aid 
>scaling for stitched PWs, or if b) PW stitching is best handled via 
>OSS.
>

_______________________________________________
pwe3 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pwe3

---

From: Eric Rosen <[email-masked]>
Date: Wed, 21 Jul 2004 15:02:28 -0400

Vach, how does your auto-discovery  proposal differ from a scheme which uses
BGP for auto-discovery  and then sets up targeted LDP  sessions based on the
results of the auto-discovery?

---

From: Mustapha Aissaoui <[email-masked]>
Date: Wed, 21 Jul 2004 15:36:52 -0400

Rajesh,
in this case, there is no stitching of any sort. All what you need
to do is to treat the incoming PW as a Attachement Circuit (AC)
terminating on a L3 entity (VRF). Not very different from
terminating a FR VC or ATM VC on a VRF. The major difference with a
traditional PW is that it is provisioned or signaled from the access
network side and does not terminate on a specific AC endpoint in the
PE.

Regards,
Mustapha.
----------
> RAJESH SALUJA wrote:
> 
> I have a similar question about hybrid networks where from MTU, a
> PW starts and terminates at a PE router which uses L3VPN (2547)
> mechanism to reach remote PE. In this case also, what is the best
> interoperable way to stitch a PW to L3VPN endpoint. Is there any
> document which describes this mechanism?
> 
> I believe that for your case also, a dynamic protocol is required
> otherwise provisioning will be nightmare.
> 
> Thanks,
> Rajesh
> 
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: Wednesday, July 21, 2004 3:19 AM
> To: [email-masked]; [email-masked]
> Subject: Scaling hierarchical LDP/PW stitching
> 
> draft-ietf-l2vpn-signaling-01.txt) where hubs (PErs routers) are
> connected via a full mesh of LDP sessions, but spokes (PEr/MTUs
> routers) only establish LDP sessions with the hub(s) that they
> connect to. This reduces the number of LDP sessions required and
> also reduces the number of packet replications that hub (PErs)
> routers must perform.
> 
> stitching multiple PWs together between endpoints. I believe this
> is an important area as one can imagine large service providers
> having hundreds of PErs routers connected via a full mesh of LDP
> sessions with tens of thousands of MTUs/PEr devices hanging off
> them. To support hundreds of thousands of PWs between tens of
> thousands of MTUs/PEr routers obviously requires a highly scalable
> provisioning solution.
> 
> The example below shows an example of a hierarchical VPLS/VPWS
> network with 4 PErs devices each connected to a single MTUs
> device:
> 
> I assume that to provision a stitched VPWS PW path from MTUs1 to
> MTUs2, the individual PWs that form the path need to be
> established individually and then stitched together. Although the
> individual PWs can be established using single sided provisioning,
> the signalling for each PW must be initiated at one side for each
> PW either through manual configuration or via OSS. OSS (or some
> manual method) must also be used to keep track of and bind the
> different PWs together. To do this OSS needs to learn or be
> configured with information about which MTUs devices sit behind
> which PErs devices.
> 
> devices) can be provisioned using singled sided provisioning, but
> the spoke PWs need to be signalled and associated with the
> relevant VPLS separately. Again, the signalling needs to be
> initiated at one side and the spokes need to be associated with
> the correct VPLS via manual configuration or via OSS.
> 
> The drawback with the above approach is that intermediate routers
> along the stitched PW path (i.e. PErs routers) must each be
> configured via OSS or via a manual method, and OSS must be also be
> used to bind and keep track of the individual PWs. The ideal
> solution would be to use a dynamic protocol(s) to establish the
> end-to-end PW path using single sided provisioning initiated from
> one end of the path.
> 
> However, to establish and end-to-end path across multiple hops
> requires a path to be calculated and selected (if multiple paths
> exist) before the path can be signalled. Stitching PWs together to
> from a path between two endpoints is effectively creating a new
> layer network. Although targeted LDP has been proposed as the
> signalling mechanism to signal individual PWs, there is no routing
> at the PW layer to calculate/select an end-to-end path.
> 
> hierarchical LDP model using a centralised OSS approach, each MTUs
> is connected to one PErs (except where dual homing is used), and
> each PErs has a targeted LDP session with each other. Therefore,
> if the OSS knows that MTUsX sits behind PErsX, and MTUsY sits
> behind PErsY, then to establish a PW path between MTUsX and MTUsY
> requires stitching/signalling of 3 PWs, i.e. MTUsX<->PErsX,
> PErsX<->PErsY, and PErsY<->MTUsY. Obtaining information about
> which MTUs devices sit behind which PErs devices is
> requires reachability information to be distributed, at least
> between the PErs devices, i.e. PErs devices need to advertise to
> each other which MTUs devices they are connected to. As yet a
> mechanism has not been defined to distribute this information, or
> to select primary/secondary paths where MTUs devices are
> multihomed.
> 
> (<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-control-protocol-ext-00.txt>)
> to address LDP scaling that proposes a solution with two different
> modes:
> 
> i) A mode called 'simple trunk' that reflects LDP messages to
> clients similar to the way in which BGP route reflectors
> distribute routes to clients. This potentially could be used to
> reduce the number of LDP sessions in an existing full mesh.
> 
> stitching but does not scale as it relies on broadcasting labels
> to every PErs in the full mesh for every VPWS/VPLS (the route of
> the problem being that there is no routing functionality at the
> 'PW layer' which the draft does not address).
> 
> I would like to establish consensus as to whether a) a dynamic
> protocol based end-to-end single sided provisioning solution is
> required to aid scaling for stitched PWs, or if b) PW stitching is
> best handled via OSS.
>

---

From: "Vach Kompella" <[email-masked]>
Date: Wed, 21 Jul 2004 12:43:11 -0700

It uses LDP for the discovery.

-Vach 

> From: Eric Rosen [mailto:[email-masked]] 
> Sent: Wednesday, July 21, 2004 12:02 PM
> To: [email-masked]
> Cc: 'Luca Martini'; [email-masked]; [email-masked]; 
> [email-masked]
> Subject: Re: [PWE3] Re: Scaling hierarchical LDP/PW stitching 
> 
> 
> Vach, how does your auto-discovery  proposal differ from a 
> scheme which uses BGP for auto-discovery  and then sets up 
> targeted LDP  sessions based on the results of the auto-discovery?
>

---

From: Eric Rosen <[email-masked]>
Date: Wed, 21 Jul 2004 21:42:22 -0400

I did realize  that your proposal differs from a  BGP-based solution in that
it doesn't  use BGP, the question  is whether there are  any other technical
differences  of  note that  might  make  it more  or  less  preferable to  a
BGP-based solution.

---

From: "RAJESH SALUJA" <[email-masked]>
Date: Wed, 21 Jul 2004 10:33:18 -0400

I have a similar question about hybrid networks where from MTU, a PW starts
and terminates at a PE router which uses L3VPN (2547) mechanism to reach
remote PE. In this case also, what is the best interoperable way to stitch a
PW to L3VPN endpoint. Is there any document which describes this mechanism?
I believe that for your case also, a dynamic protocol is required otherwise
provisioning will be nightmare.

Thanks,
Rajesh

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Wednesday, July 21, 2004 3:19 AM
To: [email-masked]; [email-masked]
Subject: Scaling hierarchical LDP/PW stitching

To address LDP scaling (and VPLS packet replication) a hierarchical LDP
model is proposed (in draft-ietf-l2vpn-signaling-01.txt) where hubs (PErs
routers) are connected via a full mesh of LDP sessions, but spokes (PEr/MTUs
routers) only establish LDP sessions with the hub(s) that they connect to.
This reduces the number of LDP sessions required and also reduces the number
of packet replications that hub (PErs) routers must perform.

However, as yet I don't believe there are any drafts that discuss exactly
how the hub and spoke PWs are signalled and stitched/bound together.
Likewise for VPWS PW paths that are established by stitching multiple PWs
together between endpoints. I believe this is an important area as one can
imagine large service providers having hundreds of PErs routers connected
via a full mesh of LDP sessions with tens of thousands of MTUs/PEr devices
hanging off them. To support hundreds of thousands of PWs between tens of
thousands of MTUs/PEr routers obviously requires a highly scalable
provisioning solution.

The example below shows an example of a hierarchical VPLS/VPWS network with
4 PErs devices each connected to a single MTUs device:

I assume that to provision a stitched VPWS PW path from MTUs1 to MTUs2, the
individual PWs that form the path need to be established individually and
then stitched together. Although the individual PWs can be established using
single sided provisioning, the signalling for each PW must be initiated at
one side for each PW either through manual configuration or via OSS. OSS (or
some manual method) must also be used to keep track of and bind the
different PWs together. To do this OSS needs to learn or be configured with
information about which MTUs devices sit behind which PErs devices.

For VPLS, the point-to-multipoint PWs between the hubs (PErs devices) can be
provisioned using singled sided provisioning, but the spoke PWs need to be
signalled and associated with the relevant VPLS separately. Again, the
signalling needs to be initiated at one side and the spokes need to be
associated with the correct VPLS via manual configuration or via OSS.

The drawback with the above approach is that intermediate routers along the
stitched PW path (i.e. PErs routers) must each be configured via OSS or via
a manual method, and OSS must be also be used to bind and keep track of the
individual PWs. The ideal solution would be to use a dynamic protocol(s) to
establish the end-to-end PW path using single sided provisioning initiated
from one end of the path. 

However, to establish and end-to-end path across multiple hops requires a
path to be calculated and selected (if multiple paths exist) before the path
can be signalled. Stitching PWs together to from a path between two
endpoints is effectively creating a new layer network. Although targeted LDP
has been proposed as the signalling mechanism to signal individual PWs,
there is no routing at the PW layer to calculate/select an end-to-end path.

Route/path calculation/selection is fairly simple for the hierarchical LDP
model using a centralised OSS approach, each MTUs is connected to one PErs
(except where dual homing is used), and each PErs has a targeted LDP session
with each other. Therefore, if the OSS knows that MTUsX sits behind PErsX,
and MTUsY sits behind PErsY, then to establish a PW path between MTUsX and
MTUsY requires stitching/signalling of 3 PWs, i.e. MTUsX<->PErsX,
PErsX<->PErsY, and PErsY<->MTUsY. Obtaining information about which MTUs
devices sit behind which PErs devices is straightforward in the OSS approach
because the information is gathered, maintained, and processed using a
centralised mechanism. Using a distributed approach however using dynamic
protocols requires reachability information to be distributed, at least
between the PErs devices, i.e. PErs devices need to advertise to each other
which MTUs devices they are connected to. As yet a mechanism has not been
defined to distribute this information, or to select primary/secondary paths
where MTUs devices are multihomed.

Du Ke has submitted a draft recently
(<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-control-protocol-ext
-00.txt>) to address LDP scaling that proposes a solution with two different
modes:

i) A mode called 'simple trunk' that reflects LDP messages to clients
similar to the way in which BGP route reflectors distribute routes to
clients. This potentially could be used to reduce the number of LDP sessions
in an existing full mesh.

ii) A mode called 'next hop self' that describes a solution for PW stitching
but does not scale as it relies on broadcasting labels to every PErs in the
full mesh for every VPWS/VPLS (the route of the problem being that there is
no routing functionality at the 'PW layer' which the draft does not
address).

I would like to establish consensus as to whether a) a dynamic protocol
based end-to-end single sided provisioning solution is required to aid
scaling for stitched PWs, or if b) PW stitching is best handled via OSS.

Thanks,
Richard

---

From: [email-masked]
Date: Fri, 23 Jul 2004 14:43:36 +0800

Luca,

At first, as a mewcomer on this forum, I must mean my warmest regards for
you, Eric Rosen, Vach Kompella, and other active people in the pwe3 WG &
l2vpn WG. I admire you for your creative work.

But I don't consider it nonsensical to solve the problem of LDP scaling.

Given a network, in which there are 20 or more PW PEs. If the provider has
a alternate solution, no one is willing to maintain full mesh LDP session
among all PEs. The more session, The more expensive and difficult for
previder to maintain them.
So we suggest that, a reflector can be intriduced in, every PE established
session to the reflector, and avoid to establish session with each other.

In other word, the problem of LDP scaling is issued not only by how many
session LDP can support, but also by total LDP session in the network the
previder must maintain.

as far as PW stitching, or as you call it "switching", (see
<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-control-protocol-ext-00.txt>

&
<http://www.ietf.org/internet-drafts/draft-martini-pwe3-pw-switching-00.txt>,

both published in Jul 14, 2004), I'm glad to see that there are many common
ground between them.

first, both two draft address Multi-hop PW(as you call it). i.e, two or
more contiguous PW segments (SH-PW) that behave and function as a single
point-to-point PW. although, the motives of the two draft's author are
somedeal different.
you want Multi-hop PW to come into play in two(three,this maybe a clerical
error, Luca?) general cases:
 -i. When it is not possible, desirable or feasible to establish a PW
control channel between the ultimate source and destination PEs.
 -ii. PWE3 signaling and encapsulation protocols are different.
 -iii. Where MPLS is deployed in the access network and User Facing
Provider Devices (UFPE) do not have capability to maintain multiple End to
End LDP sessions.
and I, want to solve the problem of "full mesh PSN tunnel", I think it is a
scaling-problem.
in any case, they are all reasonable excuse to introduce Multi-hop PW in.

On the other hand, these solution do not conflict at all. they can be
deployed in one network. The only thing to be done is a consensus to be
established, that the optional SPE-LSRID & TPE-LSRID TLV can be carrried by
GID FEC Element and VCID FEC Element, then the provider can choose a Static
or dynamical Multi-hop PW solution freely.

After all, you will agree that a solution to dynamically configure
Multi-hop PW should be found, your draft has implied such a solution may be
discussed,
you wrote:
"Multi-hop PW (MH-PW). Static or dynamically configured set of two or more
contiguous PW segments (SH-PW) that behave and function as a single
point-to-point PW."
although you hasn't discussed such a solution detailly, as you wrote:
"The pseudowire switching methodology described in this document assumes
manual configuration of the switching point at PE2. "

we are working on such a solution, I'm expecting you and other friends on
this forum to give me some advise, thanks.

regards,
Du Ke

---

From: [email-masked]
Date: Fri, 23 Jul 2004 16:45:19 +0100

Du Ke,

Although routing is the function missing from the 'PW layer', in solving the problem it will also be important to overcome the consequence of access points (i.e. router loopback IP addresses) in the PW layer network sharing the same address space as access points in the server/tunnel layer network.

To illustrate why this is an issue, please consider the example below:

UPE1----SPE1----P1----P2----SPE2----UPE2

To set up a PW from UPE1 to UPE2 using single sided provisioning:

1. UPE1 needs to know the address of UPE2 (which it can learn via discovery).
2. UPE1 needs to know what the next hop to reach UPE2 is. As UPE1 is only connected to SPE1, the IGP will tell UPE1 that the next hop to reach UPE2 is SPE1. This is fine as SPE1 is the next hop in the MH-PW to reach UPE2. 
3. UPE1 establishes a targeted LDP session with SPE1 (if it doesn't already have one) and sends a label mapping message to SPE1.
4. SPE1 receives the label mapping message from UPE1 and needs to know what the next hop to reach UPE2 is. This is where the problem lies, the IGP next hop to reach UPE2 will be P1, but the MH-PW next hop should be SPE2. 

The P routers do not belong to the PW layer and therefore they should be transparent to the PW layer network, i.e. the topology of the network at the PW layer should be:

UPE1----SPE1----SPE2----UPE2

One way to solve the problem might be to use IGP extensions (or a separate mechanism) to distinguish between MH-PW capable routers (UPEs and SPEs) and non MH-PW capable routers (P routers), in order to populate a separate PW layer routing table (e.g. PWRIB) with routing information that is only pertinent to the PW layer network. An alternative solution that supports layer independence/separation might be to use a second loopback address (a MH-PW address) on each MH-PW capable router and a separate IGP instance to distribute PW layer reachability information just to MH-PW capable routers and populate the PWRIB.

Whilst traditional IP networks are flat from a transport (or data plane) perspective, the introduction of PW switching/stitching makes IP/MPLS a multilayer network like SDH/SONET. Therefore, in solving the problem I would encourage you to take a look at ITU-T recommendations G.805 and G.809 which describe the functional architecture of connection-orientated and connectionless networks respectively. Although the terminology takes some getting used to, once you understand the basic concepts/components these will prove to be invaluable in developing an understanding the functional components and inter-layer relationships of layer networks, as well being extremely important for network management/inventory. 

Regards,
Richard

> From: [email-masked] [mailto:[email-masked]]
> Sent: 23 July 2004 07:44
> To: Luca Martini
> Cc: [email-masked]; [email-masked]; [email-masked];
> Spencer,R,Richard,XGH5 R

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Sun, 25 Jul 2004 15:53:41 -0400

Richard,

I am glad you took the initiative to discuss the architectural aspects of PW stitching. With several recent drafts that address this issue, it is the right time to look at the fundamentals.

I think we need to look at this problem from a slightly different perspective. One of the fundamental misconceptions is that there is anything special about nodes the form the intermediate hops in a multi-hop PW. Stitching PWs is nothing more than PW label switching and every LSR is capable of doing that. The question is therefore not: what routing protocols do we need to find multi-hop capable routers?; but: what are the policies that drive an LSR to bundle multiple PW-LSPs into a single transport LSP?

It is important to realize than any LSP could be considered a MH-PW. In one of your emails you show the following scenario:

Suppose an MH-PW is established that consists of five segments: UPE1-SPE1, SPE1 - P1, P1 - P2, etc. Since on each link the PW segment is established between adjacent nodes, there is no need to add an additional label on top of the PW label (see e.g. section 2 of the control draft). Hence, the MH-PW is just a regular LSP between UPE1 and UPE2. From a scaleability perspective it may make sense to tunnel all PWs that pass through SPE1 and SPE2 through a Transport LSP that connects those two nodes, but  whether to do that or not should be based on operator policies. 

The problem with this argument is that it may be true from a data plane perspective, but that it doesn't work from a control plane perspective. After all, in the data plane, a MH-capable router does nothing more than PW label switching, but in the control plane it has to tie together two targeted LSP sessions, or does it?

IMO, the appropriate conclusion is that Targeted LDP is really not the right tool to set up MH PWs. Draft-raggarwa-rsvpte-pw-00.txt describes how RSVP-TE can be used to set up MH-PWs. It is totally straightforward. The only thing different from regular RSVP is a TLV that is used by the PW end-points to identify the PW. (The current draft only shows PW-id, but a subsequent draft will include GID as well.)

I realize that the introduction of RSVP-TE for PW set up is a departure from earlier decisions to use LDP. However, if you compare it with the alternatives, I think it is clear that this the way to go.

draft-yudong-pwe3-control-protocol-ext-00.txt recognizes that intermediate hops need the LSP ids of the two PW end points. It also recognizes that in some cases there may be a need to carry the address of an intermediate hop. This contribution essentially recreates CR-LDP, but with a new and incomplete form of Explicit Route Object. CR-LDP would obviously be equivalent to using RSVP-TE for MH-PW setup, but since the decision has been made to abondon CR-LDP, RSVP-TE is the better choice at this point in time. 

draft-martini-pwe3-pw-switching-00.txt remains closer to the control draft. It sticks with targeted LDP. The problem is, as you wrote in an earlier email, that this mechanism seems to rely on manual provisioning by the operator. Another problem, IMO, is that it leads to proliferation of PW-specific identifiers (AGI and AII) throughput the network. Therefore, it turns the PW switching Points in specialized nodes that need to deal with non-routable identifiers. Architecturally, that is not the way to go.

Just to be clear: draft-martini-pwe3-pw-switching-00.txt contains information that is useful and that should become part of PW switching standards. E.g. if a provider would want to mix and match L2TP-based PWs with MPLS-based PWs, there is a need for the procedures defined in the draft. However, I believe that there is a need for only a few of such L2TP-MPLS interworking nodes. We should not accept a definition MH-PWs that assumes that this is functionality that is required in each and every PW Switching point. And we should not accept a signaling protocol for MH-PW setup that uses non-routable identifiers.

Peter

> From: [email-masked] [mailto:[email-masked]]
> Subject: [PWE3] Scaling hierarchical LDP/PW stitching
> 
> 
> I assume that to provision a stitched VPWS PW path from MTUs1 
> to MTUs2, the individual PWs that form the path need to be 
> established individually and then stitched together. Although 
> the individual PWs can be established using single sided 
> provisioning, the signalling for each PW must be initiated at 
> one side for each PW either through manual configuration or 
> via OSS. OSS (or some manual method) must also be used to 
> keep track of and bind the different PWs together. To do this 
> OSS needs to learn or be configured with information about 
> which MTUs devices sit behind which PErs devices.
> 
> (<http://www.ietf.org/internet-drafts/draft-yudong-pwe3-contro
l-protocol-ext-00.txt>) to address LDP scaling that proposes a solution with two different modes:

Thanks,
Richard

---

From: [email-masked]
Date: Mon, 26 Jul 2004 17:37:54 +0800

Peter,

I have studied your contribution, "draft-raggarwa-rsvpte-pw-00.txt". It's
really good production.

I'm glad we all work on the signaling protocol for MH-PW setup, and we have
many similar idea.
First, we all belive that, MH-PW is useful, and a MH-PW can be
automatically established by signaling.
Second, we all belive that, the signaling protocol for MH-PW setup should
use routable identifiers.
third, we all belive that, intermediate hops need the LSP ids of the two PW
end points, and, in some cases there may be a need to carry the address of
an intermediate hop.

Although, you think RSVP-TE protocol is more better for carrying PW
setup/maintenance information, and my work sticks with targeted LDP. But I
don't think there is much difference between our idea, after all, both our
fundamental starts are to automatically establish MH-PW launched by
UPE(ultimate PE), and without SPE(switching/stitching PE) being specially
configurated as to PW, except any control policy.

I think you have a misconception about my contribution
(draft-yudong-pwe3-control-protocol-ext-00.txt).
No "Explicit Route" will appeare in the PW label mapping message. UPE
needn't know path information about intermediate hops, for example, in the
following scenario:
UPE1----SPE1----P1----P2----SPE2----UPE2
LDP targeted hello(es) are established between UPE1 and SPE1, SPE1 and
SPE2, SPE2 and UPE2.
UPE1 send PW mapping to SPE1, SPE1 knows to forward mapping to SPE2 from
PWRIB, i,e. the routing information of the overlay network. so does SPE2,
then the mapping comes to the target, UPE2.
In the procedure mentioned above, if SPE1 or/and SPE2 take(s) the "next hop
self" mode as a result of a local policy, it addes or modifies a/the
"next-hop LSR-ID TLV"(the address of an intermediate hop), and will work as
a stitching point. otherwise, it takes the "simple trunk" mode, it forward
the mapping simplely without any change of it, and it will not work as a
stitching point, i,e. PSN tunnel needs to be established between UPE1 and
UPE2(or SPE2).

This will be one of the slight differeces among us, that we think whether a
signal intermediate hop work as a stitching point is a result of local
policy of itself, and you think it is a decision of upstream UPE, but it
isn't a essential discrepancy. after all the policy and configuration are
decided by operator.

An other of the slight differeces among us is that, your solution adopt the
technology of "explicit routing", and my solution work in the style of
"hop-to-hop", the next signal intermediate hop(i,e. reflector, may or may
not be a stitching point) is dicided by Source UPE and SPE locally (So, I
disagree that this contribution essentially recreates CR-LDP).

besides working in the MH-PW scenario, the PW reflector can be used to
solve the scaling problem of LDP session.

I'm glad to see any solutions on signaling protocol for MH-PW setup. all
members cooperate on the forum, and we shall adress all possible
solution(s) to the problem in discussing, then we can decide which is the
better, or which is the better in which scenario.
I think RSVP-TE will work in the scenario where Qos of MH-PW is required.
I think RSVP-TE will work in the scenario where Qos of MH-PW is required.
But, maybe, the complexity and non-compatibility will restrict it from wide
application.

Regard,
Du Ke

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Mon, 26 Jul 2004 15:03:49 -0400

Hi Du Ke,

I agree that the two solutions overlap significantly. It might be best to first agree on the fundamental architectural concepts and decide on the appropriate protocol later.

Regarding your solution, you may not have the intention to recreate CR-LDP, but you end up doing something that looks like it:

- you require ordered label distribution, as in CR-LDP

- you use certain TLVs (in your case the Extended FEC TLV) from which intermediate nodes infer that this is a special LSP, as in CR-LDP

- you define an optional TLV to identify intermediate nodes, as in CR-LDP

Add to this the QoS TLV proposed in draft-shah-pwe3-pw-qos-signaling-00.txt, and you have all the elements of CR-LDP.

Personally, I don't have anything against CR-LDP, but since the IETF made a firm decision to suspend work on CR-LDP and use RSVP-TE instead, I don't think we should create something that resembles CR-LDP in all aspects except for its name.

One additional comment. You write that the RSVP-TE solution requires explicit routing. That is actually not true. An ingress needs to know the address of the egress PE. It may or may not know the addresses of intermediate hops. If the ingress PE's policies do not prescribe PWs to be routed through specific intermediate hops, the ingress PE will simply send the RSVP-TE message to the next node on the path to the egress PE. And so forth. Therefore, it can be as much "hop-to-hop" as the solution that you propose.

Peter

> Sent: Monday, July 26, 2004 5:38 AM
> To: Busschbach, Peter B (Peter)
> [email-masked]'
> Subject: RE: [PWE3] Scaling hierarchical LDP/PW stitching
> 
> 
> 
> Peter,
>

---

From: John Drake <[email-masked]>
Date: Mon, 26 Jul 2004 13:09:40 -0700

Peter,

A very sensible analysis.

Thanks,

John

> From: Busschbach, Peter B (Peter) [mailto:[email-masked]]
> Sent: Sunday, July 25, 2004 12:54 PM
> To: [email-masked]'; [email-masked]; [email-masked]
> I am glad you took the initiative to discuss the architectural aspects of
> PW stitching. With several recent drafts that address this issue, it is
> the right time to look at the fundamentals.
> 
> I think we need to look at this problem from a slightly different
> perspective. One of the fundamental misconceptions is that there is
> anything special about nodes the form the intermediate hops in a multi-hop
> PW. Stitching PWs is nothing more than PW label switching and every LSR is
> capable of doing that. The question is therefore not: what routing
> protocols do we need to find multi-hop capable routers?; but: what are the
> policies that drive an LSR to bundle multiple PW-LSPs into a single
> transport LSP?
> 
> It is important to realize than any LSP could be considered a MH-PW. In
> one of your emails you show the following scenario:
> 
> Suppose an MH-PW is established that consists of five segments: UPE1-SPE1,
> SPE1 - P1, P1 - P2, etc. Since on each link the PW segment is established
> between adjacent nodes, there is no need to add an additional label on top
> of the PW label (see e.g. section 2 of the control draft). Hence, the MH-
> PW is just a regular LSP between UPE1 and UPE2. From a scaleability
> perspective it may make sense to tunnel all PWs that pass through SPE1 and
> SPE2 through a Transport LSP that connects those two nodes, but  whether
> to do that or not should be based on operator policies.
> 
> The problem with this argument is that it may be true from a data plane
> perspective, but that it doesn't work from a control plane perspective.
> After all, in the data plane, a MH-capable router does nothing more than
> PW label switching, but in the control plane it has to tie together two
> targeted LSP sessions, or does it?
> 
> IMO, the appropriate conclusion is that Targeted LDP is really not the
> right tool to set up MH PWs. Draft-raggarwa-rsvpte-pw-00.txt describes how
> RSVP-TE can be used to set up MH-PWs. It is totally straightforward. The
> only thing different from regular RSVP is a TLV that is used by the PW
> end-points to identify the PW. (The current draft only shows PW-id, but a
> subsequent draft will include GID as well.)
> 
> I realize that the introduction of RSVP-TE for PW set up is a departure
> from earlier decisions to use LDP. However, if you compare it with the
> alternatives, I think it is clear that this the way to go.
> 
> draft-yudong-pwe3-control-protocol-ext-00.txt recognizes that intermediate
> hops need the LSP ids of the two PW end points. It also recognizes that in
> some cases there may be a need to carry the address of an intermediate
> hop. This contribution essentially recreates CR-LDP, but with a new and
> incomplete form of Explicit Route Object. CR-LDP would obviously be
> equivalent to using RSVP-TE for MH-PW setup, but since the decision has
> been made to abondon CR-LDP, RSVP-TE is the better choice at this point in
> time.
> 
> draft-martini-pwe3-pw-switching-00.txt remains closer to the control
> draft. It sticks with targeted LDP. The problem is, as you wrote in an
> earlier email, that this mechanism seems to rely on manual provisioning by
> the operator. Another problem, IMO, is that it leads to proliferation of
> PW-specific identifiers (AGI and AII) throughput the network. Therefore,
> it turns the PW switching Points in specialized nodes that need to deal
> with non-routable identifiers. Architecturally, that is not the way to go.
> 
> Just to be clear: draft-martini-pwe3-pw-switching-00.txt contains
> information that is useful and that should become part of PW switching
> standards. E.g. if a provider would want to mix and match L2TP-based PWs
> with MPLS-based PWs, there is a need for the procedures defined in the
> draft. However, I believe that there is a need for only a few of such
> L2TP-MPLS interworking nodes. We should not accept a definition MH-PWs
> that assumes that this is functionality that is required in each and every
> PW Switching point. And we should not accept a signaling protocol for MH-
> PW setup that uses non-routable identifiers.
> 
> Peter
> 
> 
> 
> 
> stitching but does not scale as it relies on broadcasting labels to every
> PErs in the full mesh for every VPWS/VPLS (the route of the problem being
> that there is no routing functionality at the 'PW layer' which the draft
> does not address).
>

---

From: Mustapha Aissaoui <[email-masked]>
Date: Mon, 26 Jul 2004 17:20:14 -0400

Peter,
in networks where end-to-end traffic engineering of PWs is required,
or where any intermediate LSR could act as a stitching node, the use
of RSVP-TE for PW setup makes a lot sense. However, in many cases
this will not be a requirement. There will be a few stiching nodes
which are known a priori.

For instance, one may want to stitch PWs over a set of intra-domain
tunnel LSPs which may or may not have been separately traffic
engineered inside their own domain. The stitching nodes will be a
few ASBRs. Thus, having a solution based on Targeted LDP which
removes pre-configuration at the stitching node is desirable. One
possible solution is to use the "Self-Mode" described in
draft-yudong-pwe3-control-protocol-ext-00.txt where the boundary
routers act as the stitching nodes.

So, the targetted application one wants to address should be
examined to understand which solution is best suited.

Regards,
Mustapha. 
-----------

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 27 Jul 2004 13:52:25 -0400

Hi Mustapha,

I agree with your last point, i.e. that we need to understand the requirements before we decide which solution is best suited. However, IMO your email  obscures some important issues.

Peter

> From: Mustapha Aissaoui [mailto:[email-masked]]
> Sent: Monday, July 26, 2004 5:20 PM
> Cc: [email-masked]'; [email-masked]; [email-masked];
> [email-masked]; [email-masked]'
> Subject: Re: [PWE3] Scaling hierarchical LDP/PW stitching
> 
> 
> Peter,
> of RSVP-TE for PW setup makes a lot sense.

One of the things I was trying to get across in earlier emails is that every LSR _can_ function as a stitching node. A MH-PW is nothing more than a pt-to-pt LSP between two PEs and the stitching points are nothing more than LSRs that switch PW labels.

RSVP-TE is the appropriate tool for the setup of such pt-to-pt LSPs. CR-LDP is an alternative.

I don't think it is correct to say that RSVP-TE only makes sense if you need to do traffic engineering. RSVP-TE makes sense anyway. The fact that it can be used for traffic engineering is an added benefit.

> However, in many cases

I am not sure what your point is. You are saying that in many cases there is no need for a scaleable solution. But in other cases there is. Should we just ignore the latter?

The other problem I have with this statement is your suggestion that stitching nodes are known a priori and that (therefore?) we can get away with simple solutions. As far as I can see, every solution that uses a hierarchy of LSPs requires certain a-priori policies.  But that is not the point. The crux of the discussion between Du Ke and Richard Spencer was that you need to have mechanisms to find an end-to-end path between two PEs. Therefore, even if a PE would have the choice between only two stitching nodes, it needs to select one of the two. And if the primary selection fails, it has to be able to switch over to the secondary node.

draft-martini-pwe3-pw-switching-00.txt assumes that the stitching point is manually configured. That could create problems for protection switching. draft-yudong-pwe3-control-protocol-ext-00.txt adopts a solution that is at least capable of routing around failures, but it assumes that the operator assigns an absolute priority to the stitching nodes. That would nto work well if the operator is interested in load balancing.

I am sure that LDP-based proposals can be enhanced to deal with these issues, but why not use an existing protocol for which these issues have already been investigated and solved? And not only that. It already has solutions for traffic engineering, fast reroute and graceful restart as well.

> few ASBRs.

I consider inter-AS scenarios (and especially, inter-provider scenarios) as a special case and I can see that we adopt a different solution for inter-AS cases than for intra-AS cases. 

I agree that we need to uderstand the target applications. I would just be careful in making general statement based on an inter-AS example. It can easily lead you in the wrong direction.

> Thus, having a solution based on Targeted LDP which
[snipped]

---

From: Mustapha Aissaoui <[email-masked]>
Date: Tue, 27 Jul 2004 15:26:11 -0400

Peter,
I added a few comments below.

Regards,
Mustapha.
---------
"Busschbach, Peter B (Peter)" wrote:
> One of the things I was trying to get across in earlier emails is that >every LSR _can_ function as a stitching node. A MH-PW is nothing more >than a pt-to-pt LSP between two PEs and the stitching points are nothing >more than LSRs that switch PW labels.

MA> If every LSR could function as a stitching node, which implies
that it would keep state for each PW and know how to forward it, why
bother carrying PWs over a tunnel LSP? The reachability information
required by the PW will be exactly the same as that of the tunnel
LSP.
In my view, we cannot start with the assumption that every node
could be a stichting node as this has the consequence of
degenerating into an all PW routable network.
 
> I don't think it is correct to say that RSVP-TE only makes sense if you >need to do traffic engineering. RSVP-TE makes sense anyway. The fact that >it can be used for traffic engineering is an added benefit.

MA> I said RSVP-TE made sense in this case. I do not think it is the
only case. Have you thought of other cases?
 
> I am not sure what your point is. You are saying that in many cases there is no need for a scaleable solution. But in other cases there is. Should we just ignore the latter?

MA> My starting point is different from yours. I am looking into a
network where most of the label switching is performed at the tunnel
LSP level and where a few nodes act as stitching points. So the
forwarding information is mostly based on the tunnel LSP routing
information. I need to use a policy to decide how a stitching node
should move a PW from one tunnel LSP to another one. I do not need a
routable PW label to perform a local forwarding operation in a few
stiching nodes.  

> The other problem I have with this statement is your suggestion that >stitching nodes are known a priori and that (therefore?) we can get away >with simple solutions. As far as I can see, every solution that uses a >hierarchy of LSPs requires certain a-priori policies.  But that is not >the point. The crux of the discussion between Du Ke and Richard Spencer >was that you need to have mechanisms to find an end-to-end path between >two PEs. Therefore, even if a PE would have the choice between only two >stitching nodes, it needs to select one of the two. And if the primary >selection fails, it has to be able to switch over to the secondary node.

MA> How is this different from a node dual-homed to a network? A
local policy in the node states which is the primary node and if
load-balancing is allowed over both links. This node does not need
to participate in the exchange of reachability information with the
rest of the network. 

> draft-martini-pwe3-pw-switching-00.txt assumes that the stitching point >is manually configured. That could create problems for protection >switching. draft-yudong-pwe3-control-protocol-ext-00.txt adopts a >solution that is at least capable of routing around failures, but it >assumes that the operator assigns an absolute priority to the stitching >nodes. That would nto work well if the operator is interested in load >balancing.

MA> I am not sure why this should not work for load balancing. How
is this different from a node which performs load-balancing over
dual-homed links?
Unless you are thinking of load balancing PWs end-to-end over a set
of tunnel LSPs. In this case you want to traffic engineer your PWs.
 
> I am sure that LDP-based proposals can be enhanced to deal with these >issues, but why not use an existing protocol for which these issues have >already been investigated and solved? And not only that. It already has >solutions for traffic engineering, fast reroute and graceful restart as >well.
 
MA> Again, you can take advantage of all this but you really did not
need the tunnel LSPs and just let the PWs be routable across the
network.

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 27 Jul 2004 16:49:43 -0400

Hi Mustapha,

Every LSR *can* be a PW stitching node. It is not an assumption. It is a simple observation based on the recognition that a MH-PW is really nothing more than an LSP.

That does not mean that every LSR *should* be used as a stitching node. That would eventually not result in a scaleable solution. Operators will use Transport LSPs to aggregate PWs. They will identify specific nodes that will function as stitching nodes and they will set policies that prompt those nodes to establish the segments of a MH-PW through the Transport LSPs.

I think that thus far we agree. We seem to disagree on how you actually determine the end-to-end route.

You write: "I need to use a policy to decide how a stitching node should move a PW from one tunnel LSP to another one. I do not need a routable PW label to perform a local forwarding operation in a few stiching nodes."

By saying that there are only a few nodes, you imply that there is no need for a true routing solution. I disagree. The question is: when the first stitching node receives a PW setup request from a PE, how does it know what the next hop on the path is? Or, in your words, the PW has to be moved to the next tunnel, but which tunnel? You say: there are policies. What kind of policies? Are you suggesting that the operator manually provisions a routing table? Something like: for AGI=X and AII=Y, the next hop is Z?

I thought that the email exchange between Richard Spencer and Du Ke illustrated this problem very well and after a few emails up and down, Du Ke acknowledged that there is a need for some topology discovery mechanism.

My contention is: we already have a topology discovery mechanism. If you identify a MH-PW with the IP addresses of its ingress and egress PE and you use the principles of draft-ietf-mpls-lsp-hierarchy-08.txt, you have a solution.

Peter

> Sent: Tuesday, July 27, 2004 3:26 PM
> Cc: [email-masked]'; [email-masked]; [email-masked];
> [email-masked]'
> > I agree with your last point, i.e. that we need to 
> understand the requirements before we decide which solution 
> is best suited. However, IMO your email  obscures some 
> important issues.
> > 
> > Peter
> > 
> > One of the things I was trying to get across in earlier 
> emails is that >every LSR _can_ function as a stitching node. 
> A MH-PW is nothing more >than a pt-to-pt LSP between two PEs 
> and the stitching points are nothing >more than LSRs that 
> switch PW labels.
> 
> > I don't think it is correct to say that RSVP-TE only makes 
> sense if you >need to do traffic engineering. RSVP-TE makes 
> sense anyway. The fact that >it can be used for traffic 
> engineering is an added benefit.
> 
> > I am not sure what your point is. You are saying that in 
> many cases there is no need for a scaleable solution. But in 
> other cases there is. Should we just ignore the latter?
> 
> > The other problem I have with this statement is your 
> suggestion that >stitching nodes are known a priori and that 
> (therefore?) we can get away >with simple solutions. As far 
> as I can see, every solution that uses a >hierarchy of LSPs 
> requires certain a-priori policies.  But that is not >the 
> point. The crux of the discussion between Du Ke and Richard 
> Spencer >was that you need to have mechanisms to find an 
> end-to-end path between >two PEs. Therefore, even if a PE 
> would have the choice between only two >stitching nodes, it 
> needs to select one of the two. And if the primary >selection 
> fails, it has to be able to switch over to the secondary node.
> 
> > draft-martini-pwe3-pw-switching-00.txt assumes that the 
> stitching point >is manually configured. That could create 
> problems for protection >switching. 
> draft-yudong-pwe3-control-protocol-ext-00.txt adopts a 
> >solution that is at least capable of routing around 
> failures, but it >assumes that the operator assigns an 
> absolute priority to the stitching >nodes. That would nto 
> work well if the operator is interested in load >balancing.
> 
> > I am sure that LDP-based proposals can be enhanced to deal 
> with these >issues, but why not use an existing protocol for 
> which these issues have >already been investigated and 
> solved? And not only that. It already has >solutions for 
> traffic engineering, fast reroute and graceful restart as >well.
>  
> > I consider inter-AS scenarios (and especially, 
> inter-provider scenarios) as a special case and I can see 
> that we adopt a different solution for inter-AS cases than 
> for intra-AS cases.
> > 
> > I agree that we need to uderstand the target applications. 
> I would just be careful in making general statement based on 
> an inter-AS example. It can easily lead you in the wrong direction.
> >

---

From: Chris Metz <[email-masked]>
Date: Fri, 30 Jul 2004 10:12:16 -0700

At 04:49 PM 7/27/2004 -0400, Busschbach, Peter B (Peter) wrote:

<snip>

>My contention is: we already have a topology discovery mechanism. If you 
>identify a MH-PW with the IP addresses of its ingress and egress PE and 
>you use the principles of draft-ietf-mpls-lsp-hierarchy-08.txt, you have a 
>solution.

Hey Peter-
I do find the use of LSP hierarchy intriguing ... but it seems to me that 
if the ultimate source PE of the MH-PW in one AS must "learn" the IP 
address of the ultimate target
PE in another AS, then we are at risk of exposing one provider's internal 
addressing structure to another. I note that these addresses are used to 
establish the PW control channels ... be they RSVP or LDP or whatever. They 
are separate from whatever PW ID scheme (i.e. TAII) that might be used.

I find some merit in the idea that PW control channels are confined to 
intra-AS PE neighbors and ASBR peering points. Is this the case with LSP 
hierarchy?

Cheers ...

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Fri, 30 Jul 2004 14:36:52 -0400

Hi Chris,

You ask an important question. I don't have the answer. I don't know if there are documents that describe requirements for inter-provider services. If not, it would be good if service providers got together to produce one.

As I wrote in my email to Mustapha, I believe that inter-provider services form a special case. We should be careful not to mix a general discussion of multi-hop PWs with the special requirements that apply when you cross provider boundaries.

Two comments on your email:

1) draft-ayyangar-ccamp-inter-domain-rsvp-te-00.txt discusses extensions to RSVP-TE to make it suitable for inter-domain connection setup. The draft acknowledges that at the provider boundary there may be a need to stitch tunnels rather than to create a contiguous end-to-end tunnel. Therefore, applying LSP Hierarchy does not prevent a provider from confining the PW control channnel to inter-AS neighbors and ASBRs. 

2) I don't know how much providers care about the advertisements of specific IP addresses. After all, through BGP they advertise all addresses. Furthermore, in the IP VPN inter-AS scenarios described in 2547bis I don't see any attempt to hide the PE's IP address when crossing the AS boundary. 

Peter

> From: Chris Metz [mailto:[email-masked]]
> Sent: Friday, July 30, 2004 1:12 PM
> Cc: 'Mustapha Aissaoui'; [email-masked]; [email-masked];
> [email-masked]'
> >you use the principles of 
> draft-ietf-mpls-lsp-hierarchy-08.txt, you have a 
> >solution.
> 
> Hey Peter-