Subject: draft-meyer-mpls-soft-preemption-00.txt

From: Matthew Meyer <[email-masked]>
Date: Mon, 17 Feb 2003 17:14:33 -0700

Folks,

There have been some off line conversations showing interest 
regarding this draft.  Would anyone be willing to comment on-
list?

Matthew

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 18 Feb 2003 14:28:13 -0500

> regarding this draft.  Would anyone be willing to comment on-

As one of the offliners I'd like to say that this is much needed.
Every ISP that we talk to regards preemption as a hole in the MPLS
protocols that if utilized renders make-before-break and fast-reroute
ineffective.  Soft preemption fixes that.

I also support using the RRO as described in the draft.  It is the
best way to accomplish soft preemption.  It is saying exactly what is
happenning, namely that the LSP is still up but it is about to go
down so do something top avoid having bits dropped.

The one thing I would add is some object or bit somewhere that
indicates that the ingress is capable of understanding and acting on
the "Preemption pending" bit.  If the midpoint where the preemption
occurs knows that the ingress is capable of dealing with the
"Preemption pending" bit, it can do a soft preemption.  If not, then
the midpoint might as well do a hard preemption, because the older
ingress does not understand the "Preemption pending" bit and will
ignore it.

Non-broken midpoint LSRs already have to pass on changes to the "Local
protection available" and "Local protection in use" bits back to the
ingress.  Non-broken ingress LSRs already have to (or should) act upon
changes to the "Local protection available" and "Local protection in
use" bits.  Adding a "Preemption pending" bit to the same bitmask
poses no additional scaling issues for existing non-broken routers.

There might be some partially broken implementations that currently
don't look at "RRO IPv4/IPv6 Sub-Object Flags" and don't pass along
changes to it or act on it but they have to be fixed to support
existing fast-reroute related capabilities.

Curtis

---

From: Ina Minei <[email-masked]>
Date: Tue, 18 Feb 2003 11:40:13 -0800 (PST)

Please find comments inline below.

			Ina

> ingress does not understand the "Preemption pending" bit and will

The HE LSR is the one requesting soft-preemption by setting the
flag in the session attribute object. A transit will only send
a Resv message with the RRO 'Preemption Pending' bit set if it
previously received a path message requesting soft-preemption, meaning
that the HE always knows how to handle it.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 18 Feb 2003 15:33:23 -0500

> that the HE always knows how to handle it.

I missed the "Soft preempted desired" bit in the SESSION-ATTRIBUTE
object, sect 4.1.  My fault.

Thanks.

Curtis

---

From: "Matthew R. Meyer" <[email-masked]>
Date: Wed, 19 Feb 2003 01:21:29 -0700

Curtis,

Thanks for the response, comments below.

Thus spake Curtis Villamizar ([email-masked]):

 |
 |In message <[email-masked]>, Matthew Meyer writes:
 |> Folks,
 |> 
 |> There have been some off line conversations showing interest 
 |> regarding this draft.  Would anyone be willing to comment on-
 |> list?
 |> 
 |> Matthew
 |
 |
 |
 |As one of the offliners I'd like to say that this is much needed.
 |Every ISP that we talk to regards preemption as a hole in the MPLS
 |protocols that if utilized renders make-before-break and fast-reroute
 |ineffective.  Soft preemption fixes that.

Frequent re-sizing of LSPs in a multiple setup/hold priority level 
network will become basically non-intrusive as well.

 |I also support using the RRO as described in the draft.  It is the
 |best way to accomplish soft preemption.  It is saying exactly what is
 |happening, namely that the LSP is still up but it is about to go
 |down so do something top avoid having bits dropped.
 |
 |The one thing I would add is some object or bit somewhere that
 |indicates that the ingress is capable of understanding and acting on
 |the "Preemption pending" bit.  If the midpoint where the preemption
 |occurs knows that the ingress is capable of dealing with the
 |"Preemption pending" bit, it can do a soft preemption.  If not, then
 |the midpoint might as well do a hard preemption, because the older
 |ingress does not understand the "Preemption pending" bit and will
 |ignore it.

Section 7 misled you:

> - Section 7 - Interoperability
> "Backward compatibility is assured since any HE LSR not compliant with
> this draft that receives a Resv message with the RRO 'Preemption
> Pending' bit set will simply ignore the flag and treat the
> Resv message as a regular Resv refresh message."
 
This was in the document before the session object was added and
includes the now obsolete assumption that a correctly functioning
midpoint would make the mistake of sending an RESV+Flagged-RRO to a 
non-supporting HE.  Ina pointed this out off-line a few days ago
and it will be corrected for version 01.

[message clipped]

Matthew

---

From: Jim Boyle <[email-masked]>
Date: Wed, 19 Feb 2003 14:46:46 -0500 (EST)

First, great draft - I like this concept.

While a flag might be just the right thing for this particular
application, I wonder if it might be good to try to broaden 
soft preemption technique to include other polite preemptions.

These might include the following indicators

o) your LSP is being prempted on the indicated link (draft covers)
o) your LSP should be rerouted, this node is shutting down
o) your LSP should be rerouted, this link is being taken out of
   service
o) your LSP should be rerouted away from this node (administrative/CLI)
o) your LSP should be rerouted away from this link (administrative/CLI)

You can somewhat do some of these by changing metrics and waiting, but
just was wondering if something more general might be useful.

Also, with Diffserv TE, not sure that one can assume that a preemption
necessarily "implies exhausted bandwidth at the affected priority
level *and greater*" (section 5), but local implementations can do
as they please I suppose.

regards,

Jim

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 19 Feb 2003 16:39:27 -0500

In message <[email-masked]>, Jim Boyle 
> just was wondering if something more general might be useful.

Why couldn't you just use the same bit and allow soft preemption on
link shut, reload, CLI adminstrative without encoding the reason?
Since these are all CLI initiated, it wouldn't hurt to delay.
Alternately, the soft preempt on anything using a specific resource
could be done.

Link bundle lost a member might also qualify as a reason for soft
preemption for implementations of link bundling that can trasnparently
more flows to other members of the bundle.

> as they please I suppose.

Agreed.  The "and greater" should be dropped.

> Jim

Thanks,

Curtis

---

From: Jim Boyle <[email-masked]>
Date: Wed, 19 Feb 2003 18:17:22 -0500 (EST)

> Alternately, the soft preempt on anything using a specific resource

you could, but the path head could create more meaningful log message if 
the reason were known,  Just floating an idea here.  The flag is 
definitely "lighter" protocol wise,

---

From: "Matthew R. Meyer" <[email-masked]>
Date: Wed, 19 Feb 2003 17:17:50 -0700

Thus spake Jim Boyle ([email-masked]):

 |
 |First, great draft - I like this concept.

Thanks for your comments Jim.

 |While a flag might be just the right thing for this particular
 |application, I wonder if it might be good to try to broaden 
 |soft preemption technique to include other polite preemptions.
 |
 |These might include the following indicators
 |
 |o) your LSP is being prempted on the indicated link (draft covers)
 |o) your LSP should be rerouted, this node is shutting down
 |o) your LSP should be rerouted, this link is being taken out of
 |   service
 |o) your LSP should be rerouted away from this node (administrative/CLI)
 |o) your LSP should be rerouted away from this link (administrative/CLI)

All very interesting ideas. 

We could add as well:
  o) Your LSP should be rerouted away from this SRLG (administrative/CLI)
     (just a flag)

I don't see a problem with adding some more flags to convey 
administrative events though I do have healthy fear of 
proto-bloat.  I welcome more comments on this subject.

 |You can somewhat do some of these by changing metrics and waiting, but
 |just was wondering if something more general might be useful.

I think there are cases like zero-BW LSPs that still need to be 
handled where flooding a new zeroed IGP TLVs wouldn't help. 

 |Also, with Diffserv TE, not sure that one can assume that a preemption
 |necessarily "implies exhausted bandwidth at the affected priority
 |level *and greater*" (section 5), but local implementations can do
 |as they please I suppose.

Yes, I guess we need to be more specific there. This document assumed
independent Diffserv & TE deployments as opposed to Diffserv-TE.

Thanks,
Matthew

---

From: "Matthew R. Meyer" <[email-masked]>
Date: Wed, 19 Feb 2003 20:10:18 -0700

Comments inline...

 |
 |In message <[email-masked]>, Jim Boyle 
 |writes:
 |> 
 |> First, great draft - I like this concept.
 |> 
 |> While a flag might be just the right thing for this particular
 |> application, I wonder if it might be good to try to broaden 
 |> soft preemption technique to include other polite preemptions.
 |> 
 |> These might include the following indicators
 |> 
 |> o) your LSP is being prempted on the indicated link (draft covers)
 |> o) your LSP should be rerouted, this node is shutting down
 |> o) your LSP should be rerouted, this link is being taken out of
 |>    service
 |> o) your LSP should be rerouted away from this node (administrative/CLI)
 |> o) your LSP should be rerouted away from this link (administrative/CLI)
 |> 
 |> You can somewhat do some of these by changing metrics and waiting, but
 |> just was wondering if something more general might be useful.
 |
 |Why couldn't you just use the same bit and allow soft preemption on
 |link shut, reload, 

Or OL bit set...

 |CLI administrative without encoding the reason?
 |Since these are all CLI initiated, it wouldn't hurt to delay.
 |Alternately, the soft preempt on anything using a specific resource
 |could be done.
 |Link bundle lost a member might also qualify as a reason for soft
 |preemption for implementations of link bundling that can transparently
 |more flows to other members of the bundle.
 
I think there is value in conveying to the head-end the population 
category of the soft preemption (Single LSP|Link|Bundle Member|SRLG|
Node). Doing so provides prompt, more accurate info for the head-end
to use when trying to quickly find the new valid path for the first 
of (say) 30 arriving soft preemptions. 

    .---,A     .---,B
    |   |---1--|   |----- 
    |   |---2--|   |-----
    `---'      `---'
     HE        Maint

fe. Imagine we cause all LSPs transiting a node B to be soft preempted 
in prep for maintenance. HE A's LSPs all transited a certain circuit (1)
to the maintenance node B, however there is a parallel circuit as well(2).
Without the "node" flag, we might amend the HE TE-DB zeroing (1)s 
reservable BW (the circuit for which we were just soft preempted)
when we should have zeroed out (2) as well. The result could be our
soft preemption make before break sets up (or tries to) through the 
maintenance node despite being recently soft preempted off it..

Technically a HE need only receive the first LSP population oriented 
flag (Node|SRLG|etc) soft preemption to discover 1) what subset of 
locally originating LSPs need to be rerouted and 2) what links/nodes/
members need to be avoided. There is still a need for one-at-a-time
straight soft preemption of course. 

 |> Also, with Diffserv TE, not sure that one can assume that a preemption
 |> necessarily "implies exhausted bandwidth at the affected priority
 |> level *and greater*" (section 5), but local implementations can do
 |> as they please I suppose.
 |
 |Agreed.  The "and greater" should be dropped.

okay. Agreed.

Thanks 
Matthew

 |
 |> regards,
 |> 
 |> Jim
 |
 |Thanks,
 |
 |Curtis

---

From: Markus Jork <[email-masked]>
Date: Thu, 20 Feb 2003 14:57:44 -0500

> of (say) 30 arriving soft preemptions. 

I like the original soft preemption idea in the draft as it stands now.
But I have my doubts about the extensions discussed in this e-mail thread.
When planning an admin link or node shutdown, it is probably a better
idea to indicate that via the IGP (through a metric change or announcing
the available bandwidth as 0).
If you do it via RSVP signaling as discussed here, the HE routers
getting the notification now have two pieces of information: the TE-DB
tells them the link/node is still available and a good choice for
a path while RSVP tells them don't go there. So now they've got to
trust what RSVP tells them (for how long???) and reroute.
Other nodes in the network that didn't route any lsp through the
node that goes down now see that there's plenty of bandwidth
through it. They weren't notified by RSVP it's going down. So they
might "optimize" their lsp's and use that node now.
This seems like a somewhat messy situation.

To avoid problems you would have to give indication of link/node
shutdown via RSVP and the IGP at approximately the same time.
But why do that when you can get the job done via IGP alone and
without extra protocol extensions?

In the "higher priority lsp preempts lower priority lsp" scenario
of the draft, soft preemption via RSVP works better because you
will get link bandwidth usage updates via the IGP relatively quickly
at the same time.

Markus

---

From: Jim Boyle <[email-masked]>
Date: Thu, 20 Feb 2003 20:13:06 -0500 (EST)

Yeah, maybe lean and clean is better here.

---

From: Matthew Meyer <[email-masked]>
Date: Thu, 20 Feb 2003 20:08:57 -0700

Comments in line.

Thus spake Markus Jork ([email-masked]):

 |I like the original soft preemption idea in the draft as it stands now.

I think there have been some strong arguments to leave it as is.

 |But I have my doubts about the extensions discussed in this e-mail thread.
 |When planning an admin link or node shutdown, it is probably a better
 |idea to indicate that via the IGP (through a metric change or announcing
 |the available bandwidth as 0).

While I agree with you that it is perhaps more appropriately 
accomplished in the IGP, I don't believe the IGP has all 
the needed mechanisms. 

Cranking the metric & setting/flooding available reserveable BW 
at 0 is not enough.  Even if you soft preempt as well, you still
have 0BW LSPs or explicitly routed LSPs or disparate backups that
need to be forced to forget this link.  Currently SPs are forced
to remove configuration parameters if we want to drive some types
of tunnels off a link.  

At minimum what is missing is a (for lack of a better term) link 
'OL bit', and some handy almost crontab-able maintenance commands 
from the vendors.  I wonder if there are any providers on this 
list that agree.
 
 |If you do it via RSVP signaling as discussed here, the HE routers
 |getting the notification now have two pieces of information: the TE-DB
 |tells them the link/node is still available and a good choice for
 |a path while RSVP tells them don't go there. So now they've got to
 |trust what RSVP tells them (for how long???) and reroute.
 |Other nodes in the network that didn't route any lsp through the
 |node that goes down now see that there's plenty of bandwidth
 |through it. They weren't notified by RSVP it's going down. So they
 |might "optimize" their lsp's and use that node now.
 |This seems like a somewhat messy situation.

I imagine it could be maneuvered cleanly if the head ends set a timer.
The next IGP update (which would have been delayed since it as 
cli triggered) would overwrite the local tweaked working copy.

Really, I am kind of on the fence on these extra flags and where
they might belong. I think the discussion is beneficial though.

 |To avoid problems you would have to give indication of link/node
 |shutdown via RSVP and the IGP at approximately the same time.

I don't agree, see above timer comment.

 |But why do that when you can get the job done via IGP alone and
 |without extra protocol extensions?

I think the IGP can only do half the job right now, still, maybe 
fixing the IGP would be a better approach even if the IP native 
crowd ends up with a redundant feature from their perspective. 
(They can set metric high for a nearly instant impact on all 
traffic flows).

 |In the "higher priority lsp preempts lower priority lsp" scenario
 |of the draft, soft preemption via RSVP works better because you
 |will get link bandwidth usage updates via the IGP relatively quickly
 |at the same time.

Even if you don't get your IGP update quickly, you can make 
assumptions about the preempting link about how much capacity
for that preempted priority is left and fudge your working 
copy until your next TE TLV arrives to overwrite it with a
more accurate accounting.

Thank you for your comments.

Matthew

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 21 Feb 2003 13:31:33 -0500

> to remove configuration parameters if we want to drive some types

Keeping a admin-color (aka admin-group) and naming it "maintenance"
and always configuring exclude-admin-color maintenance to your LSPs
does solve this problem easily.  Just advertise the link with
"admin-color maintenance" and the LSPs should go away rather quickly.

This is just a good operations trick/practice to have handy.
Increasing the IGP cost keeps the non-MPLS packets away.

Curtis

---

From: "Matthew R. Meyer" <[email-masked]>
Date: Fri, 21 Feb 2003 16:33:49 -0700

|> While I agree with you that it is perhaps more appropriately 
 |> accomplished in the IGP, I don't believe the IGP has all 
 |> the needed mechanisms. 
 |> 
 |> Cranking the metric & setting/flooding available reserveable BW 
 |> at 0 is not enough.  Even if you soft preempt as well, you still
 |> have 0BW LSPs or explicitly routed LSPs or disparate backups that
 |> need to be forced to forget this link.  Currently SPs are forced
 |> to remove configuration parameters if we want to drive some types
 |> of tunnels off a link.  
 |
 |Keeping a admin-color (aka admin-group) and naming it "maintenance"
 |and always configuring exclude-admin-color maintenance to your LSPs
 |does solve this problem easily.  Just advertise the link with
 |"admin-color maintenance" and the LSPs should go away rather quickly.
 |
 |This is just a good operations trick/practice to have handy.
 |Increasing the IGP cost keeps the non-MPLS packets away.
 |
 |Curtis

This handles it in two steps and is not as clean as leaving the
config in place and setting a maintenance oriented command. It is 
somewhat annoying that this solution requires alteration of 
configured metric, a process which is typically tightly controlled
in many large operations organizations as, at very least, a matter 
of principal.  

A number of providers choose to fail the IGP adjacency by mis-config
of the link authentication though this may not act consistent between 
vendors and in relation to various tunnel types (I know, talk to the 
vendors, matt) - anyway metrics don't need to be messed with this
way.  Using this method provides no warning time though. IMO link 
OL bit would be more precisely what was needed though I willing
to take this sub-thread off line.

Matthew

---

From: Jean Philippe Vasseur <[email-masked]>
Date: Mon, 24 Feb 2003 11:51:17 -0500

Hi Curtis,

>down so do something top avoid having bits dropped.

Thanks for your feed-back

>ingress does not understand the "Preemption pending" bit and will

Right, but see, we added the "soft preempted desired" that solves this issue:

4.1. SESSION-ATTRIBUTES Flags

To explicitly signal the desire for a TE LSP to benefit from the soft
preemption mechanism (and so not to be 'hard' preempted), the following
new flag of the SESSION-ATTRIBUTE object (for both the C-Type 1 and 7)
is defined:

Soft preempted desired:  0x40

This allows to limit the overbooking ratio since the mid-point preempting 
node will only soft preempt the TE LSP having this bit set.

This way, soft preempted TE LSPs will be limited to TE LSPs originated by 
HE which are compliant with this draft and that do explicitly require soft 
preemption.

>poses no additional scaling issues for existing non-broken routers.

Right

>existing fast-reroute related capabilities.

Correct.

Thanks.

JP.

---

From: Jean Philippe Vasseur <[email-masked]>
Date: Mon, 24 Feb 2003 12:24:29 -0500

Hi Jim,

>o) your LSP should be rerouted away from this link (administrative/CLI)

Well one can generate exactly the same sequence of event, setting up the 
RRO Preemption pending flag (of course, that does not provide the 
preemption root cause to the HE).

By the way, there are a bunch of existing alternatives to handle those 
scenarios:

(1) Just flood an IGP LSA/LSP update:
         - with the link down for link maintenance,
         - with MAXAGE LSA (OSPF) or OL bit set (ISIS) for node maintenance

(2) increase the TE metric to MAX

(3) use affinities

and make sure the HE LSR treat those events in a non disruptive fashion 
with a make before break.

>as they please I suppose.

Right.

JP.