Subject: Some suggestions for LSP Ping

From: "Cheng-Yin Lee" <[email-masked]>
Date: 3 Jan 2003 17:41:12 -0500

Hello,
Some suggestions that may be helpful for LSP Ping.
It seems one could precede an MPLS Ping with the LSP "payload" header to
allow the test packet to be (deterministically) exercised by data path
and use Router Alert Option to allow the egress to intercept the LSP
Ping message :

1) For an LSP with IP traffic, an MPLS Ping may contain e.g.:
a) Payload header 
- IP dst = 192.168.1.1
- Protocol ID = IP or alternatively MPLS_PING_PROT
- Option = Router Alert (or alternatively a new Option, e.g. class
"debugging & measurement") OR alternatively set TTL=1
b) IP header
IP dst = e.g. 127.0.0.1/224.0.0.1 (a specific IP address allowing a
local IP/UDP
protocol stack to process the message)
c) UDP header
d) MPLS Ping message
[If Protocol ID is MPLS_PING_PROT, the inner header is the MPLS Ping
message, no IP & UDP header, but may need checksum in MPLS Ping message]

(b)-(d) are the same as in LSP Ping draft, unless stated otherwise
above.
The Router Alert Option is intended for the egress LSR to intercept the
MPLS Ping
message (existing implementations may vary). Using TTL=1 instead may
have issues as raised in previous email thread "Last call on LSP Ping"

If hashing includes Protocol ID, the above require changes such that
Protocol ID is set to the actual LSP "payload" Protocol ID, and another
field e.g. Router Alert Value or new Option class indicates/implies how
to decode rest of message.
The above will not work as is if hashing on port is required.

2) For an LSP with inner label, an MPLS Ping may contain eg: (I think
a-b may be similar to what Curtis  have proposed for labels) 
a) Payload header
- VC label = L
- This is suggested by Curtis/Shahram?
S' bit not bottom of stack (as a few people pointed out, this requires
data path change) 
OR alternatively TTL=1 (Using TTL may have issues as raised in past
email thread "Last call on LSP Ping" and may be trickier, lower layer
TTL setting etc)
b) IPv4 Explicit NULL label ("indicates that the label stack must be
popped, and the forwarding of the packet must then be based on the IPv4
header")
c) IP header
IP dst = eg. 127.0.0.1/224.0.0.1 
d) UDP header
e) MPLS Ping message

(c)-(e) are as specified in LSP Ping draft, with the exception of the
field(s) listed above.
If the inner label contains IP traffic, (2b-2e) may be replaced by
(1a-1d), 
(2a) 'S' bit is bottom of stack or TTL=1.

Some useful functions when including the "payload" hdr and using Router
Alert :
- test packet can be exercised on the same path (including ECMP) as the
data traffic (by data path)
- not necessary for MPLS ping to assume how ECMP is done in LSRs
- not necessary to use 127/8 address range
- avoiding some TTL issues 
- not necessary to have intermediate LSRs provide ECMP related
information

There will be some limitations, e.g. can exercise only one path with one
LSP Ping.

Other opinions and issues? Anything I missed?
Hope some of these are helpful.

Happy New Year.

Regards,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 04 Jan 2003 12:34:05 -0500

> message, no IP & UDP header, but may need checksum in MPLS Ping message]

IP and UDP were picked.  With that choice no MPLS Ping checksum is
needed.

> have issues as raised in previous email thread "Last call on LSP Ping"

Destination is in 127/8 and TTL=1 so it can't be forwarded unless the
action taken by the egress LSR on a VPN or PW label is to forward to a
specific interface unconditionally.  Then router alert in the MPLS
payload won't help either.

> The above will not work as is if hashing on port is required.

If source address, protocol ID and ports are fixed, then a set of
destination addresses can be found to exercise a specific path.

> IP dst = eg. 127.0.0.1/224.0.0.1 

The S bit is always at the bottom of the stack.  The VPN label is no
longer on the bottom, but an Explicit NULL label follows.  What the
egress LSR should do with this is so far undefined.  So far no one has
indicated that their router could not either 1) disable any
unconditional forwarding based on the VPN label and check the next
label or 2) process the packet on the egress LSR based on TTL expire
of TTL in the VPN shim.

> (2a) 'S' bit is bottom of stack or TTL=1.

No one has ever suggested that we use the multicast range (224.0.0.1).

> There will be some limitations, e.g. can exercise only one path with one

Router alert would eliminate the need to use a destination within
127/8 but has no other advantage.  The same procedure is needed to
exercise multipath only using a range of real destination addresses
may be much worse.  See below.

> Hope some of these are helpful.

Lets try to make progress and *not* propose arbitrary changes for no
good reason.

Proposing to use router alert to eliminate the need to use 127/8 is of
use only if people regard using 127/8 as an immense sacrilege.  Since
the source address must be an address of the router sending MPLS ping,
there is still a need to pick from a range to exercise all paths.  It
is probably a lot better to pick from a range of 127/8 than some other
range where a non-MPLS-ping capble router at the egress would happily
deliver the MPLS-ping traffic to the final IP destination.

There would be a small advantage to ignoring both the IP source and
destination address in that for a report of "I can't get there from
here" the ISP can put "here" in the source and "there" in the
destination, add router alert, and put the MPLS-ping return address in
the payload.  This is a minor advantage but may be offset if there is
a possibility that the test traffic actually gets delivered to the
destination and the destination sends ICMP back at the source, either
of which could interpret this as a DoS attack and start filtering.
That would compund the original problem and make it harder overall to
solve.

> Cheng-Yin

Thanks for sharing your ideas.

Regards,

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 6 Jan 2003 21:24:29 -0500

Curtis,
Thanks for your analysis of the suggestions.
Some comments at the end and a quick question.

> That would compund the original problem and make it harder overall to
I think if the egress is not MPLS Ping capable, there is always a chance
of the MPLS Ping leaking to the customer, unless a provider filter
egress traffic (MPLS Ping traffic). For e.g. in (2) above, where the
inner label may be hashed, if we can't use 'S' bit to indicate whether
there is another label underneath the inner label because egress does
not check the 'S' bit, using LSP TTT=1 (in the hope of intercepting the
LSP Ping at egress) would be a problem as well if the LSP mode is such
that the egress copies the TTL of the outer label to the inner label.
Since the TTL does not expire, the LSP Ping message would be forwarded
to customer. 

In the long term, forwarding should probably be improved such that
transport layer and control information are appropriately processed
[e.g. no "residue" header or control frames should be forwarded without
prior processing/examination by PE]. Meanwhile (especially for MPLS
incapable egress), is it feasible to somehow filter traffic that is not
supposed to go to customers?

Regarding ECMP, for e.g. to test connectivity between Src A and Dst B,
the appropriate path would be verified if the IP src and dst of the test
packet are set to Src A and Dst B, respectively (hashing on SA & DA).
[Add Router Alert Option to "catch" the test packet at egress]. If "the
destination IP address is a (randomly chosen) address from 127/8"
instead,  how is the appropriate path for Src A & Dst B verified? I
think I am missing something here. Could you kindly elaborate a little
on the procedure to accomplish this or provide some pointers, pls? 

Thanks,
Cheng-Yin
p.s I have intermittent email access this week.

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 06 Jan 2003 21:41:02 -0500

In message <[email-masked]>, "Cheng-Yin Lee" writes:
<trimmed for brevity>
> Since the TTL does not expire, the LSP Ping message would be forwarded

This is true for VPN tunnels.  There is always the change that a
non-MPLS ping capable router will deliver the packet to the next
router.  That is different from delivering to the endpoint.

If the destination is a real address it goes all the way to the final
destination.  If the destination is 127/8 it goes no further than one
router into the customer LAN (which may be one too many).

This problem only exists for MPLS-ping over VPN.  There is no reason
to make the problem worse by using a destination address not in 127/8.

> supposed to go to customers?

If the CE router is not MPLS capable it will dump all non IP including
a packet with MPLS explicit null label.  Otherwise it can just filter
out any MPLS traffic on its IP VPN.  If sending MPLS over the VPN.

If the traffic is destined to 127/8 and comes from the provider that
could also be filtered.  If the CE router is MPLS-ping incapable it
will just toss the packet anyway and try to send an ICMP.

> on the procedure to accomplish this or provide some pointers, pls? 

You test all paths.  We went through that in great detail last month.
Check the mailing list archives for "suggested added text for
multipath", "suggested clarification to intro parts", "suggested
clarification to section 4".

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 13 Jan 2003 12:57:17 -0500

Curtis,

> router into the customer LAN (which may be one too many).
I see your point, but filtering LSP Ping messages at egress would also
eliminate this problem, right? I would like to understand the tradeoffs
of this vs some of the mechanisms (not all are clear to me yet) that are
required if 127/8 is used as destination.

> > instead,  how is the appropriate path for Src A & Dst B verified? 
> 
To verify the connectivity between Src A & Dst B, one would not need to
test all paths if the payload & Router Alert Option/new option is
included in the payload message, and the path data traffic would take,
would be verified.

I have some questions regarding "suggested clarification to section 4"
+ 4.8 Intended Use of Traceroute and Ping Mode
As I understand, to verify connectivity between Src A & Dst B:
a) do a traceroute to find out if multiple paths are available
b) intermediate LSRs return the multipath destination addresses
c) ping using the destination address returned by intermediate LSRs
d) if multipath handling changes, additionally ping using random
addresses (does the latter mean the pings in (c) do not
deterministically rule out there is no connectivity?)
e) if the pings in (d) fail, do traceroute (what destination addresses
should be used?)

After doing (a)-(e), can one be sure there is no connectivity, or if the
test packet actually traverse the path data traffic take? This seems to
involve quite a bit of work on the network to test connectivity. Perhaps
you have other thoughts on this?

Thanks,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 13 Jan 2003 14:13:15 -0500

> required if 127/8 is used as destination.

Filters are generally not applied after the VPN label is encountered.
This is the only case where we have a problem with delivery of MPLS
ping into the VPN.

> e) if the pings in (d) fail, do traceroute (what destination addresses

I would do:

  traceroute to figure out paths.

  ping all paths.

  traceroute to verfy paths have not changed.

If all three pass, then you have verified connectivity.  Then if VPN
or PW need to be verified:

  ping with each VPN or PW label to verify the FEC mapping.

Periodically repeat the traceroute to insure the paths haven't
changed.  The shelf life provides a good hint regarding how often this
needs to be done.

> you have other thoughts on this?

Multipath is quite common today.

Today ping and traceroute are used primarily for a few purposes.

  1.  The NOC has received a report that "you can't get there from
      here" and the NOC needs to follow up.

  2.  SLAs have to be verified.

  3.  As a form of proactive monitoring (in addition to checking all
      sorts of counters for sanity).

What we seem most concerned about right now is proactive monitoring.
If checking core connectivity, there are at most a few hundred
(usually less than 100) LSPs to check.  Pings with periodic traceroute
to pick up multipath changes seem the best way to monitor the core and
does not differ much from current "ping matrix" data gathered by ISPs.

For proactive monitoring and SLA verification of VPN or PW the same
ping with periodic traceroute will be needed but the number of paths
to monitor may be a lot higher.  It may help to verify all possible
paths from PE to PE and then just ping to verfiy the individual FEC
mappings at the egress, thereby doing no traceroutes on a per VPN or
PW label basis.

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 13 Jan 2003 19:07:52 -0500

Curtis,
> changed.  The shelf life provides a good hint regarding how often this
** As I understand, to verify the connectivity, one would not need to
test all paths if the "payload" header (which includes fields required
for hashing) & Router Alert Option/new option is included in the LSP
Ping message. The above can be replaced by a single LSP Ping then. 
Even if multipath handling changes, the next LSP Ping will be forwarded
in the path data traffic would take. This approach seems to require less
work in implementation and network then doing the 3 steps above, but I
am very opened to be convinced otherwise.

> mappings at the egress, thereby doing no traceroutes on a per VPN or
If one do ** above then, then I think only a ping to verify individual
FEC is required.

Also is it feasible to verify all possible paths from PE to PE?
I still don't quite understand (d) & (e), is the result of the ping
deterministic, is it actually verifying the path data traffic take?

thanks
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 13 Jan 2003 23:17:12 -0500

> am very opened to be convinced otherwise.

One ping packet per path.

You need to test the forwarding plane with ping, not the control
plane.  Thats the whole point.  You can use MIBs to check the control
plane and still have no traffic flowing.

> deterministic, is it actually verifying the path data traffic take?

The traceroute takes the same path as the data.  At each N+1 hop, the
prior N hops use the forwarding to get there and TTL is the only thing
that stops the normal forwarding from occuring.

Ping just rechecks the forwarding without verifying that the path has
not changed or the mapping of traffic onto a multipath has not
changed.  A periodic traceroute is used to make sure that the path has
not changes such that the current set of pings is not providing
complete test coverage of the forwarding path.

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 14 Jan 2003 11:03:35 -0500

Curtis,

> plane and still have no traffic flowing.
Agree.

I would like to know if you agree that the ** (payload header & Router
Alert Option) approach could be used instead of the 3 steps above
(noting there is a chance that LSP Ping may leak to customer).

> complete test coverage of the forwarding path.
Understood and I appreciate your clarification. 

Actually the questions I have are:
1) if "random addresses" are used as in (d), it sounds like a ping may
not exercise the intended path. Is that right? 
2) If the pings in (d) fail, what destination addresses should be used
in traceroute in (e) ?

Thanks,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 14 Jan 2003 12:22:19 -0500

> (noting there is a chance that LSP Ping may leak to customer).

If IP router alert on a VPN IP payload stops the packet from being
delivered where adding a explicit null label would not, then fine.  I
see no reason to think it would.

> not exercise the intended path. Is that right? 

I didn't say use random addresses.  I said use addresses determined by
the prior traceroute.

> 2) If the pings in (d) fail, what destination addresses should be used

If the (d) and (e) you are referring to is in the document and
recommending random addresses as a means to exercise multipath, then
it should be removed or it should be stated that this method does not
deterministically exercise all paths.  If you are just referring to
*your* prior email, then yes using random addresses to exercise
multipath is not certain to work, but it is not what I'm suggesting.

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 14 Jan 2003 17:22:34 -0500

Curtis,

> see no reason to think it would.
Using the payload header and Router Alert option approach, you would
only need to send one LSP ping (similar to IP Ping), instead of
preceding the (LSP) ping with (LSP) traceroute and follow by (LSP)
traceroute (in the 3 step approach described in your previous email). I
think the former is simpler, the latter involves more work in
implementation and in the network (additional traceroutes), but I'm
still opened to be convinced otherwise.
A minor point is _if_ hashing involves the IP header as well, then I
think the explicit null label cannot be used.

BTW, would you know how many 127/8 addresses should be configured in the
egress LSR?

> multipath is not certain to work, but it is not what I'm suggesting.

I am referring to Section 4.8 from "suggested clarification to section 4
of draft-ietf-mpls-lsp-ping-01":
+   In the event
+   that multipath handling changes, randomly selected addresses should
+   be used in addition to destination addresses determined using
+   traceroute, rather than solely using those determined by
+   traceroute.

thanks,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 14 Jan 2003 20:31:14 -0500

> BTW, would you know how many 127/8 addresses should be configured in the

It is easy to configure an entire /8 subnet.  That would be one entry.
Some routers may require modification to the loopback device driver
but that is easily accomplished.

> +   traceroute, rather than solely using those determined by

All that one sentence adds is that providing some randomly selected
addresses **in addition** to destination addresses determined using
traceroute adds some random testing to the deterministic testing.

There is no (d) and (e) in this entire section.

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 15 Jan 2003 11:08:43 -0500

Curtis,

> traceroute adds some random testing to the deterministic testing.

From your previous email:
1)traceroute to figure out paths.

2)ping all paths.

3)traceroute to verfy paths have not changed.

4)If all three pass, then you have verified connectivity.  

5)Then if VPN or PW need to be verified ping with each VPN or PW label
to verify the FEC mapping.

6)Periodically repeat the traceroute to insure the paths haven't
changed.  The shelf life provides a good hint regarding how often this 
needs to be done.

May I know where would you add random testing to the above steps? 

Thanks,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 15 Jan 2003 11:36:20 -0500

> May I know where would you add random testing to the above steps? 

If you are doing continuous monitoring you would initialize monitoring
as you have outlined above.

After initialization, pings would be used to verify connectivity and
give a first order check on SLA to suppliment MIB based or other
statistics.  This is where random addresses can be used in addition to
the more deterministic addresses obtained from the traceroute.

Traceroute would then have to be repeated if one of two events might
have occurred, 1) an LSP reroute, 2) a change in the way load is
distributed at a multipath branch point.  Lets consider these two
separately.

Testing is initiated by the ingress.  For a RSVP/TE LSP the ingress
knows if the LSP has been rerouted, because the ingress must initiate
that action.  For LDP, a reroute would occur only after a topology
change which could be noted.

If the "shelf life" returned by the LSRs may indicate one of a number
of conditions, 1) the multipath distribution never changes, 2) the
shelf life duration is known but the start time is unavailable, 3) the
remaining shelf life is known.  The first case is easy.  If there is a
topology change, then the traceroute has to be repeated.  The third
case is also easy, the remaining shelf life allows a traceroute to be
scheduled.  The second case, where only shelf life duration is known,
requires periodic traceroutes initially.  Once a transition occurs,
the approximate start time is available and a traceroute can be
scheduled rather than performed periodically.

Between scheduled traceroutes (if any are needed) ping can be
performed using both the deterministic addresses and some random
addresses.

This sort of recommendation can be added to the document (although it
would start to get sort of long).  I would leave that decision along
with the decision to accept any of the prior suggestions I made up to
the authors.

Curtis

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 15 Jan 2003 12:28:47 -0500

Curtis,

> with the decision to accept any of the prior suggestions I made up to
If your suggestions are accepted, some recommendations as the above in
for e.g. the appendix would be useful.

My understanding from your clarification above is - this method does not
necessarily verify the test packet actually traverse the path data
traffic would take. Have you come to the same conclusion? 
[Note: If you use the payload header & Router Alert option method, that
method exercises the test packet on the path data traffic would take]

thanks,
Cheng-Yin

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 15 Jan 2003 12:38:37 -0500

> traffic would take. Have you come to the same conclusion? 

Your understanding is incorrect.  No I have not come to that
conclusion.

There is very weak basis for some of your prior statements and this
one so I may just discontinue this discussion.  I am very close to
coming to the conclusion that you are trying to waste my time.

To more directly address and provide more detail to your latest
assertion and question - For the case where the load balance changes,
there may be a period in which it is not certain which path some of
the prior pings took, but by specifying the shelf life, that period
can be made small while minimizing the overhead of repeating
traceroutes.

Curtis

---

From: George Swallow <[email-masked]>
Date: Wed, 15 Jan 2003 14:05:22 -0500

Cheng-Yin

> method exercises the test packet on the path data traffic would take]

If what you are trying to accomplish is to diagnose one single VPN
flow it might work if:

1. you set the source address to the address of the flow.
   (this raises some issues on getting the Ping Reply back to where it
    is needed)
2. the hash includes no other fields than source and dest
3. the hash has not been reseeded or changed by some topological event
   since the problem occurred.

I believe that once an operator discovered a problem they would want
to  explore all the paths to determine the health of their network.

The use of RA also raises three further issues.  

1.  If a 127/24 addressed packet were to escape from a provider, it
would be eaten by the first router it encouters.  If you send a ping
with RA and it escapses, if the customer's routers don't recognize
LSP-Ping, the message will be delivered all the way to the host.

2.  If an LSP-Ping were to escape from a broken LSP at a node that did
not recognize LSP-Ping, it could end up being successfully forwarded
to the destination node causing you to *not* detect the break.

3.  If you eliminate use of the 127/24 range, you don't have a set of
addresses that you can employ to explore the EMCP paths of an LSP.

So I think there is little benefit and many drawbacks to your proposal.

...George

==================================================================
George Swallow       Cisco Systems                  (978) 497-8143
                     250 Apollo Drive
                     Chelmsford, Ma 01824

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 15 Jan 2003 15:39:34 -0500

George,
> 1. you set the source address to the address of the flow.
This has been noted in this thread.

>    (this raises some issues on getting the Ping Reply back to where it
Good point. Perhaps the return address could be specified in the LSP
Ping message.

> 2. the hash includes no other fields than source and dest
Agree, I noted this in the first email I sent on this thread, and
suggested some additional features.

>    since the problem occurred.
If you do another ping after this event, then the test packet should
traverse the path data take. 

> LSP-Ping, the message will be delivered all the way to the host.
Agree, this has been noted in the thread.

> to the destination node causing you to *not* detect the break.
Wouldn't the destination node/customer node drop it if it is not LSP
Ping aware?

> addresses that you can employ to explore the EMCP paths of an LSP.
If we look at the problem this way, yes.

> So I think there is little benefit and many drawbacks to your proposal.
I think we should not ignore the advantages of this approach or the
disadvantages of the other proposal in drawing the above conclusion.

In any case, these suggestions are meant to be helpful and the authors
did ask for suggestions. If the consensus is they are not useful, then I
am fine with that.

Thanks,
Cheng-Yin

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 15 Jan 2003 16:03:26 -0500

Curtis,

Curtis Villamizar wrote:
> the prior pings took, 
I think we are in agreement here, my understanding is the test packet
may not traverse the path data traffic would take  (if a data packet is
sent out at the same time)
> but by specifying the shelf life, that period
agree.

I agree it is not useful to continue discussion at this point but I hope
you appreciate that you have given good clarification to the LSP Ping
draft and that documenting functions including scope & limitations is
useful.

Thanks
Cheng-Yin