Subject: RE: Comments on draft-nagarajan-ccamp-mpls-packet-protection-00.t xt

From: "Nagarajan, Ramesh (Ramesh)" <[email-masked]>
Date: Tue, 19 Mar 2002 16:04:47 -0500

Hi,

I have attached some viewgraphs which we plan to present to MPLS WG. It
highlights features of our proposal and also compares to other recovery
schemes like you have mentioned. Please take a look and we can discuss more.
Fyi, the proposed approach is in field trail with a major service-provider.

ramesh.

 <<ietf_0302.pdf>> 

> -----Original Message-----
> From:	Jean Philippe Vasseur [SMTP:[email-masked]]
> Sent:	Tuesday, March 19, 2002 12:00 PM
> To:	Nagarajan, Ramesh (Ramesh)
> Cc:	[email-masked]
> Subject:	Comments on
> draft-nagarajan-ccamp-mpls-packet-protection-00.txt
> 
> Hi,
> 
> A few comments about your proposal:
> 
> I personally do not see the clear advantages compared to local protection 
> (FRR) especially in term of efficiency (local => <50ms convergence time ; 
> protection => control on the QOS for the rerouted LSPs) while allowing 
> protection bandwidth sharing.
> 
> while
> 
> I clearly see serious disadvantages/showstopper:
> - bandwidth consumption as you bridge the traffic onto the two LSPs. You 
> just double the bandwidth consumption for every protected LSP.
> - requires Hardware modification on every ingress and egress LSR,
> - ...
> 
> Let's have the comments from SPs ...
> 
> JP.

---

From: Francois Le Faucheur <[email-masked]>
Date: Tue, 19 Mar 2002 16:43:50 +0100

Nagarajan,

If I understand correctly, this approach fundamentally does not achieve 
protection at the MPLS layer (ie it does not rely on MPLS forwarding to 
achieve protection) but rather achieves protection through a "thin" 
additional protection layer on top of MPLS. In other words, the LSPs are 
really used as pure pipes and you rely on an additional layer (aka sequence 
number in terms of protocol + some module in the data path to do packet 
selection).
Is this not right?

I sort of look at that as re-building a kind of SDH-lite on top of MPLS 
(which seems a little odd to me since one objective of MPLS protection is 
to be able to take advantage of MPLS forwarding so as not to have to rely 
on underlying SDH protection).

You may argue that this additional "SDH-lite" protection layer is really 
really "thin" and should not be considered as an additional layer (it only 
needs a sequence number after all), but I have the impression that it is 
affecting the data path so significantly that MPLS hardware could not cope 
with that (ie before doing forwarding of any packet I need to select the 
copy of the packet - which BTW appears particularly interesting on egress 
LSRs with distributed architecture when the two LSPs terminate on different 
input cards).

So, am I wrong in assuming that this requires specific hardware support? or 
are you proposing that we adopt as MPLS protection an appraoch which cannot 
be achieved with existing MPLS hardware?

If the latter, that would be an interesting element to add to your 
comparison between your approach and pure MPLS protection schemes.

Thanks

Francois

---

From: "Metz, Eduard" <[email-masked]>
Date: Wed, 20 Mar 2002 11:27:10 +0100

Which applications / traffic types require this type of protection? (and why
would existing methods not be sufficient? -IGP re-route, MPLS FRR-) 

How do you ensure that in case of a link / node failure the affected LSP it
taken 'out-of-service'? Otherwise, it is very likely that both LSPs will end
up sharing resources on at least a part of the path between ingress and
egress LSR. Which is a waste of resources, but also could lead to congestion
of the available path between ingress and egress. So potentially the
protection mechanism could cause degradation of service (for those
situations it should solve some problems).

As the solution cannot cope with packet reordering between ingress and
egress, qos/scheduling on the path between ingress and egress LSR could
result in packet drop at the egress due to window-check failure.

Side note: In general I feel more comfortable with solutions that stick to
forwarding packets just once.

> From: Nagarajan, Ramesh (Ramesh) [mailto:[email-masked]]
> Sent: Tuesday, 19 March, 2002 22:05
> To: 'Jean Philippe Vasseur'
> Cc: [email-masked]; Qureshi, Muhammad A (Akber); Wang, Yung-Terng (Y T)
> Subject: RE: Comments on
> draft-nagarajan-ccamp-mpls-packet-protection-00.t xt
> 
> 
> Hi,
>

---

From: "Nagarajan, Ramesh (Ramesh)" <[email-masked]>
Date: Wed, 20 Mar 2002 12:11:39 -0500

> would existing methods not be sufficient? -IGP re-route, MPLS FRR-) 
[Ramesh] Signaling for optical transport networks is one application.
However, we believe that many
emerging real-time, high reliability applications on data networks will find
this attractive.

> situations it should solve some problems).
[Ramesh] The proposal requires the LSPs to be setup on diverse paths and
bandwidth reserved for 
each LSP (this is what the end customer will pay for when he/she subscribes
to the service). The above should, hence, not be an issue.

> result in packet drop at the egress due to window-check failure.
[Ramesh] The implementation shown in the proposal is an *example*
implementation. It is possible to have other implementations that will deal
with reordering. In general, we do not expect MPLS switches to reorder
traffic which was one of the motivations of the example implementation but
it is possible to deal with.

---

From: "Nagarajan, Ramesh (Ramesh)" <[email-masked]>
Date: Wed, 20 Mar 2002 14:22:03 -0500

First, Thanks for all your comments (including some other e-mails i
have received in this thread). I think we have some good discussions going. 

> on underlying SDH protection).
	[Ramesh] I think the view of a thin layer on top of MPLS is one way
of looking at it and a reasonable one.
      However, all MPLS protection approaches that i am aware of require
support outside of the MPLS *forwarding*             layer. This typically
includes functions such as failure detection, failure notification and LSP
switching (none                         of which we need for our proposal).
What  we are requiring is a different kind of thin function  outside of
basic MPLS forwarding. .

> LSRs with distributed architecture when the two LSPs terminate on
	[Ramesh] Selecting is done wherever the above thin layer is
terminated. This may 
	imply that the duplicated packets are carried to a different line
card than where 
	the LSPs are terminated. By the way, this provides additional
protection (in the box) and is more 
	end-to-end from a service standpoint. Also, even for a single LSP
one needs to carry  
	packets to a different line card if the service the LSP is carrying
does not originate/terminate 
	at the card the LSP terminates. 

> be achieved with existing MPLS hardware?
      [Ramesh] Hardware support is probably needed to implement the
selection function 
      at high speeds. But it can also be done in software if speed is not an
issue (also 
      someone may very well figure out/choose to do this simple/thin
function extremely fast 
      in software).

---

From: "Qureshi, Muhammad A (Akber)" <[email-masked]>
Date: Thu, 21 Mar 2002 11:15:43 -0500

Hi,

Packet 1+1 is providing fast end-to-end protection. I think end-to-end
protection has better failure coverage than any fast restoration scheme
based on local reroute. Reason is the dependence of fast local reroute
scheme on physical layer detection which may not be available all the time.
Also there can be failures at the above layers which can affect the LSP
forwarding but cannot be detected at the physical layer. So packet 1+1 fills
a gap that cannot be covered by either IGP-reroute (which is slow) or MPLS
FRR (which does not provide end-to-end protection).

1+1 protection in the transport world does duplicate traffic on two paths.
It exists because it is simple to manage and provides fast end-to-end
protection. Similarly, proposed packet 1+1 is also very simple to operate
and provides fast packet 1+1 protection.

Akber.

---

From: "Metz, Eduard" <[email-masked]>
Date: Thu, 21 Mar 2002 18:41:20 +0100

I'm not so sure about some of the statements below.

Why does MPLS FRR not provide end-to-end protection? It has the same scope
as 1+1 packet protection, which is the MPLS cloud. (btw also IGP reroute can
be fast (enough)).

I doubt that packet 1+1 is as 'simple' as SONET/SDH 1+1. It is simple in
transport networks because these (and the equipment) are built for it. Many
packet networks (and routers) are not like that. For one, transport networks
do not have dynamic routing which is one of the main ingredients of packet
networks. And also one of the strenghts, e.g. it allows recovery of 2+
failures.

Consider the following:
- LSP-A is setup with bandwidth constraint X.
- Now LSP-B should be the mated one of LSP-A. In an automated setup probably
LSP-B will be setup with bandwidth constraint X, and the additional
constraint to avoid SRLG used by LSP-A.

If a failure occurs on the path of LSP-A it will be re-routed. Assume it now
shares part of the path of LSP-B. Since the SRLG of LSP-A has changed (and
overlaps with LSP-B), LSP-B will be re-routed as well. Such a sequence of
events would probably result in packet loss on the egress, and minimise the
potential gain of this solution. Unless you nail down the LSP statically
-which has other downsides-, how would you avoid the above? 

cheers,
	Eduard

ps how do you plan to handle traffic like traceroute? as the intermediate
LSR are not aware of the special status of the traffic they carry, probably
one would get interesting reponses (of course this could be considered a
feature).

> From: Qureshi, Muhammad A (Akber) [mailto:[email-masked]]
> Sent: Thursday, 21 March, 2002 17:16
> To: 'Metz, Eduard'; Nagarajan, Ramesh (Ramesh); 'Jean 
> Philippe Vasseur'

---

From: "Qureshi, Muhammad A (Akber)" <[email-masked]>
Date: Thu, 21 Mar 2002 13:58:00 -0500

I suppose MPLS cloud cosnists of many hops. In this case end-to-end
path protection requires detection+notification which would be hard to
achieve in 10s of millliseconds unless network provides stringent QoS
requirements.
I don't think IGP reroute can achieve 10s of milliseconds recovery in a
reasonable size network. (We are targetting for restoration times in order
of
10s of milliseconds)

Below in your example you are assuming a specific policy of what
happens after failure of an LSP. No where in the draft we are advocating
that
policy. When computing the failed route one can always put a constraint to
avoid SRLGs being used in the working path. It can be done easily through
configuration as done in the transport network as well as signalling (if one
wants to use it).

Akber.

-----Original Message-----
From: Metz, Eduard [mailto:[email-masked]]
Sent: Thursday, March 21, 2002 12:41 PM
To: 'Qureshi, Muhammad A (Akber)'; Metz, Eduard; Nagarajan, Ramesh
(Ramesh); 'Jean Philippe Vasseur'
Cc: [email-masked]; Wang, Yung-Terng (Y T)
Subject: RE: Comments on
draft-nagarajan-ccamp-mpls-packet-protection-00.t xt

Consider the following:

cheers,
	Eduard

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 21 Mar 2002 22:27:43 +0100

Ramesh,

>What  we are requiring is a different kind of thin function  outside of

Yes, the other MPLS protection approaches make use of "mechanisms" in 
addition to MPLS forwarding, but all these mechanisms are in the control 
plane (e.g. failure detection, LFIB update). In terms of data path, the 
other protection approaches rely purely on existing forwarding mechanisms 
(e.g. label stacking).

A protection approach which cannot be supported by MPLS hardware strikes me 
as a pretty unattractive MPLS protection approach.

Also, this approach is very bandwidth wasteful when compared to the other 
approaches. First, the backup capacity is entirely wasted in the absence of 
failure (while other approaches allow complete reuse of it for traffic 
which can tolerate being squeezed out during failure). Secondly, the backup 
capacity is used on a path which should be SRLG disjoint end-to-end and 
thus may be much "longer" than back up paths used by local protection. 
You'd really have to be assuming an environment where bandwidth is of no 
concern at all.

You have similar inefficiencies in the terminating box in the cases 
discussed below where packets have to go to/fro inside the terminating LSR 
(BTW, I can't see the "protection in the box" argument since you still have 
a single point of failure in the card which actually does the packet selection)

The only thing this approach buys you is a slightly improved recovery time 
which is equal to the local failure detection time (note that failure 
notification is not required with bypass approaches).

I pesonnally can't see that such a small improvement (aka in the 10s of 
millisec) justifies a complete change in MPLS forwarding and MPLS hardware 
as well as the other drawbacks. Let's hear other opinions.

Francois

> > > > Let's have the comments from SPs ...

_________________________________________________________
Francois Le Faucheur
Development Engineer, IOS Layer 3 Services
Cisco Systems
Office Phone:          +33 4 97 23 26 19
Mobile :               +33 6 19 98 50 90
Fax:                   +33 4 97 23 26 26
Email:                 [email-masked]
_________________________________________________________
Cisco Systems
Domaine Green Side
400, Avenue de Roumanille
06 410  Biot - Sophia Antipolis
FRANCE
_________________________________________________________

---

From: Jean Philippe Vasseur <[email-masked]>
Date: Thu, 21 Mar 2002 23:19:46 +0100

Hi,

>FRR (which does not provide end-to-end protection).

I won't reiterate the same arguments as before (avoiding duplication ;-) )
Coming back to your statement "FRR does not provide end to end protection": 
every section along the path is being protected. Related to failure 
detection, various failure detection schemes may be used for different 
failure types: layer 2, RSVP hellos, ...

JP.

---

From: "Qureshi, Muhammad A (Akber)" <[email-masked]>
Date: Thu, 21 Mar 2002 17:56:21 -0500

I assume that by proposing other detection mechanisms you are trying to
approach
the failure coverage of an end-to-end path protection scheme. Use of other
detection mechanisms at higher layers will negatively affect (increase) the 
recovery times. This will also increase complexity in terms of processing,
etc.

Again, packet 1+1 provides maximum possible failure coverage with
instantaneous
recovery. It is extremely simple to operate. And only end nodes have to be
service aware.

Akber.

-----Original Message-----
From: Jean Philippe Vasseur [mailto:[email-masked]]
Sent: Thursday, March 21, 2002 5:20 PM
To: Qureshi, Muhammad A (Akber)
Cc: 'Metz, Eduard'; Nagarajan, Ramesh (Ramesh); [email-masked]; Qureshi,
Muhammad A (Akber); Wang, Yung-Terng (Y T)
Subject: RE: Comments on
draft-nagarajan-ccamp-mpls-packet-protection-00.t xt

Hi,

JP.

---

From: "Qureshi, Muhammad A (Akber)" <[email-masked]>
Date: Thu, 21 Mar 2002 17:58:53 -0500

Packet 1+1 not only provides a faster recovery (actually instantaneous) but
also
provides better failure coverage than any local reroute scheme. Its an
end-to-end
path protection scheme. Fast local bypass schemes rely on failure indication
from the physical layer which may not always be passed to the LSR by
facility provider or 
the underlying transport system.

For unidirectional failures, any scheme other than a 1+1 scheme requires
notification
to the upstream node. 

Akber.

-----Original Message-----
From: Francois Le Faucheur [mailto:[email-masked]]
Sent: Thursday, March 21, 2002 4:28 PM
To: Nagarajan, Ramesh (Ramesh)
Cc: Nagarajan, Ramesh (Ramesh); 'Francois Le Faucheur'; 'Jean Philippe
Vasseur'; [email-masked]; Qureshi, Muhammad A (Akber); Wang, Yung-Terng (Y
T)
Subject: RE: Comments on
draft-nagarajan-ccamp-mpls-packet-protection-00.t xt

Ramesh,

You have similar inefficiencies in the terminating box in the cases 
discussed below where packets have to go to/fro inside the terminating LSR 
(BTW, I can't see the "protection in the box" argument since you still have 
a single point of failure in the card which actually does the packet
selection)

Francois

---

From: "Nagarajan, Ramesh (Ramesh)" <[email-masked]>
Date: Thu, 21 Mar 2002 19:02:38 -0500

Hi,

To add to akber's comments, certain soft failures are not only hard to
detect but also local bypass schemes may exhibit some undesirable behaviours
when (and if) these types of failures are detected. Let us consider the case
of a) MPLS forwarding tables are corrupted for a given LSP and packets are
being incorrectly forwarded or b) a LSR is dropping packets of a LSP for
reasons other than congestion. Note that our proposal provides protection 
against these soft failures as long as the soft failure effects only one
(single failure case) of the two LSPs. 
Also, we do not require any explicit detection of such a failure since
packets would be arriving fine from the working LSP. 

Now, let us consider, how one would detect such a soft failure. This would
be needed to activate the bypass in a path or a local reroute scheme. The
only means of detecting such a failure is by insertion of some kind of 
periodic keep alive in the data path of the LSP (Note that keep alives such
as OSPF hello, RSVP refresh etc., do not use the data forwarding path of the
LSP) and monitoring the presence of these keep alives at downstream nodes.
All this needs extra processing as akber has noted. Further, it takes time
since one has to damp immediate reaction to lack of keep alives (they may be
missing due to congestion rather than failure). Finally, failures are
detected at multiple downstream nodes. Note that in a sequence of LSRs
A->B->C->D with a soft failure (such as above) at B, both C and D would
detect a failure due to missing keep alives. In fact, in general, all nodes
downstream of the soft failure will detect the failure and will try to
signal to the next-next upstream hop (C->A and D->B in above example). So,
there is a lot of notification traffic as well as many upstream nodes trying
to switch. Eventually of course, the A->C (around B) switch will dominate
but all this signaling and switching is not very desirable. So, probably it
is best to not detect such failures in local bypass schemes because of
complexity and cost.

To put all of above in proper perspective, what we are proposing is a scheme
with the broadest failure coverage (so very high service
availability/reliability) and instantaneous recovery as some of the key
features. There is a price tag of course due to the bandwidth consumption on
both paths. That is why we believe it is complimentary to existing
approaches rather than replacing any particular approach. Together with
existing excellent approaches such as IGP reroute and MPLS FRR, we believe
our proposal provides service-providers and their end customers a richer set
of choices to better match their application needs.

ramesh.

----------------------------------------------------------------------------
--------------------------------------------------
Room 3M-335, Bell labs
Tel: 732-949-2761
101 Crawfords Corner Road
Fax: 732-834-5906
Holmdel, NJ 07733.
e-mail: [email-masked].
----------------------------------------------------------------------------
---------------------------------------------------

> ----------
> From: 	Qureshi, Muhammad A (Akber)
> Sent: 	Thursday, March 21, 2002 5:56 PM
> To: 	'Jean Philippe Vasseur'; Qureshi, Muhammad A (Akber)