Subject: RE: Last call on LSP Ping

From: [email-masked]
Date: Thu, 5 Dec 2002 10:17:52 -0000

Initial set of commments:

The 1st sentence of the Introduction says:
"This document describes a simple and efficient mechanism that can be used
to detect data-plane failures in MPLS LSPs."

Let me examine this single statement:

'Simple' - This is simple?  This is quite complex IMO.  I have read it
several times now, and I have to apologise and say I still don't fully
understand all aspects of it (also please refer to the specific protocol
observations/questions from Shahram Davari and Dave Allan here).  So I am
not sure that a claim that is it 'simple' is appropriate.....however, other
operators may disagree with me on this, and if so please say so.

'Efficient' - I am missing understanding where the efficiency arises, so can
you please explain what you mean by this?  It seems as though it's a
solution largely aimed at addressing the fault-management complexity created
by LDP, eg merging and use of proprietary ECMP.  For p2p LSPs the solution
described in Y.1711 is signficantly 'simpler and more efficient' (by a very
large margin IMO), so I don't think its valid to assume this is the right
solution for all types of LSP.  Do you disagree with this?
Note - Here is a suggestion/thought.  Since all instances of mp2p LSP *must*
run a p2p LSP instance above them to remove the merging....directly so in
Martini et al, and indirectly so in L3/rfc2547 VPNs where resolution to the
VPN (inner LSP) then requires assumption of IP to properly demerge......then
a more 'efficient' solution might be to use Y.1711 on the top-level
essentially p2p LSPs and only invoke ad hoc tools (like LSP-Ping) on the
lower mp2p LSPs when the top-level LSP indicates a problem.  Seems
reasonably simple to do for PWE3 cases of XoverMPLS, but not so sure its
that straightfoward for the L3 VPNs.  As I say, just a suggestion/thought.

'Detect data-plane failures' - I can't seem to find anywhere that these
data-plane failures are specified (eg entry criteria, exit criteria and
consequent actions).  Can you please point me to the appropriate document
where these failures are specified so that I can judge if this claim is
true?

thanks and regards, Neil

---

From: [email-masked]
Date: Fri, 6 Dec 2002 00:23:41 -0000

Comments on section 4 and 5:

14	Section 4.1.....I don't understand why there is no explanation given
as to why the source of an echo request contains an IP pkt with a 127/8
destination address, chosen apparently at random.   I assume it must be for
attempting to exercise any ECMP load-balancing paths.  If so, why does the
draft not make this explicitly clear instead of expecting the reader to
guess?

I am not an expert on these addresses, but my understanding of them is that
are internal loopback addresses that are only supposed to stay within
end-systems and not venture out into networks.  So is this allowable....it
looks potentially not-allowable to me?   On assumption we can do this (and
I'd like to see this confirmed or otherwise) it hardly gives any
deterministic behaviour....esp if the odd pkt gets lost (either
direction).....do you then re-send under same conditions or what (indeed can
you replicate them)?  Would the intention be to sample 1 of 16 million
addresses and ping-away hoping for the best?  I also guess its really only
applicable to the ping-mode and not the trace-mode....or is it?  If trace
mode allowed here then behaviour seems highly unpredictable and not easily
repeatable, ie as TTL does step-increase destination IP address is random at
each step.

BTW - this seems to be straying now far, far away from any notion of
(deterministic) defect detection and well into the realms of 'guessing
diagnostics'.

In summary...a far better explanation of what is going here is definitely
required IMO.

15	In the same section......I note that the same sequence number can be
repeated.  When taken into consideration with the above comments I am really
struggling to get a feel for what kind of (help to) determinacy this is
going to produce.

16	In section 4.2....it talks about " ....and the Subcode set to the
appropriate value".  But back in section 3 it says "The return Subcode is
unused at present and should be set to zero"  Please clarify.
BTW - also talks about filling in the subcode in section 4.3, but no
guidance given...despite what it says (that it is in the previous section).

17	In section 4.3...it says "An MPLS echo reply is a UDP pkt".  How
does this stack with a request wanting an RSVP response? 

18	In section 5...it effectively talks about the critical need for a
reliable return path.  Well yes, as described this is obvious. But the fact
of the matter is that the basic defect detection/handling problem is
architecturally a uni-directional issue.  There are simply things that have
to be done here on a unidirectional basis, ie consequent actions require
attention at a trail sink point, not its source point.  Trying to force this
'bi-directional behaviour' is IMO not very sensible and leads to
complexities.  Just an observation.

Timed out on this draft.....so far.

regards, Neil

---

From: Shahram Davari <[email-masked]>
Date: Fri, 6 Dec 2002 07:31:37 -0800

> > - The originator  of the echo request  decides to test  a 
> FEC or a  stack of
> >   FECs. 
> 
> I think you're really only testing one FEC.  The stack gives you a way
> of noting how you set off the get to that FEC (assuming you had
> multiple options - like two TE tunnels).

How you set off to get to that FEC is by pushing correct labels not
by testing multiple FECs belonging to multiple LSP levels using the
same packet. My suggestion is to remove the notion of FEC stack and make
it a single FEC.

> 
> > - The originator  determines the next hop  for that FEC, or 
>  for the topmost
> >   FEC in the stack. 
> > 
> > - The originator labels  the echo request with the label  
> stack that is used
> >   for sending packets with that FEC via that next hop. 
> > 
> > - The  originator  sends the  labeled  echo request  to  
> that  next hop.  (If
> >   reaching the  next hop  requires going through  an LSP 
> tunnel,  this might
> >   causes more labels to be pushed on the stack.) 
> 
> Yes.  I think we need to rework that section and be more explicit
> about how it works.  As far as I'm concerned the only place you might
> have an unlabelled Ping is after a PHP.  I was also assuming that you
> only Ping one FEC at a time.  I think we should also say that in the
> text.  If others think we need to be able to do more than one at a
> time then we need a good deal of text to describe that.

I think you need to be able to test more than one FEC belonging to the
same LSP (not different LSPs at different levels). 

> > 
> > The suggestion  to randomly choose  IP destination 
> addresses from  the 127/8
> > block is,  presumably, the way to  ensure that the various  
> paths created by
> > equal cost load splitting get exercised.  The question is, 
> how many pings do
> > you need to send in order to get good coverage?  
> 
> Obviously this depends on the number of ECMPs that exist.  Which is
> both a function of the topology and the number of hops between the
> source and destination.  If you have better suggestions on how to do
> this I'm quite eager to hear them.  This is the best we've been able
> to come up with so far.

This has nothing to do with the number of ECMPs. In order to
exercise all the ECMP paths (even off a single LSR), you need to produce
IP addresses that hash to all those paths. But since the ECMP hash is
proprietary, I have no idea how you could select the 127/8 addresses
to cover all those ECMP paths. 

> 
> > If the request  has a stack of n  FECs, and the responder 
> is  the egress for
> > the topmost  m FECs but not  for the bottommost  n-m, it 
> might be  useful to
> > have a way to indicate that in the response.
> 
> Or presumably you would just ping the outer FECs - which seems to me
> to be sufficient.

My suggestion is to change the FEC stack to single FEC.

> 
> > Some minor observations: 
> > 
> > - The notion of "time  of day" is not well-defined.  I have 
>  no idea what is
> >   supposed to  be put in the time  of day fields.  This  
> doesn't really seem
> >   that useful anyway unless there is synchronized time. 
> 
> Kireeti - could you comment here and also on Sharam's question on why
> you deviated from ICMP Ping?
> 
> > - I'm not sure that the sequence  number really needs to be 
> a separate field
> >   from the sender's handle.  It just  seems like an 
> extension of the handle,
> >   whose semantics are private to the sender. 
> 
> Some implementors would like to put a PID in the sender handle so the
> separation seems useful.

PID for what? Are you thinking of encapsulating other packets/protocols
in the LSP-ping packet? or is it going to verify what type of traffic
the LSP is carrying? If the latter, then where is the error code for it?

-Shahram

---

From: Shahram Davari <[email-masked]>
Date: Fri, 6 Dec 2002 08:18:21 -0800

> Neil> Ping  test will  detect simple  breaks (either  
> direction....this then
> Neil> needs  the   direction  resolving  of  course).    Ping 
>  won't  detect
> Neil> misrouting/mismerging. 
> 
> This  statement does  not  appear to  me to  be  correct.  In 
>  the event  of
> misrouting, the Echo Request packet will be processed by an 
> LSP egress which
> isn't the  proper egress for  the indicated FEC.   This 
> results in  an error
> code 4  response indicating that  misrouting has occurred.  
> It  doesn't tell
> you where the fault lies, but it does tell you that a fault exists. 

What if the Echo Request packet is misrouted to an egress that happens to have that FEC in its routing table, but is not the intended egress?

-Shahram

---

From: Shahram Davari <[email-masked]>
Date: Fri, 6 Dec 2002 08:27:27 -0800

Never mind, I figured it out.

> What if the Echo Request packet is misrouted to an egress 
> that happens to have that FEC in its routing table, but is 
> not the intended egress?
> 
> -Shahram
>

---

From: [email-masked]
Date: Fri, 6 Dec 2002 17:18:58 -0000

Thanks Eric, please see below.  regards, Neil

>Eric R=> This  statement does  not  appear to  me to  be  correct.  In 
>  the event  of
NH=> If IP only return forwarding is allowed *and* the node where this ends
up is able to respond this seems true.  However in case of core MPLS only
network, then any return with 'router alert' set will have no choice but to
use LSP return....so an IP return, which is being asked for, can't be done.
Similarly in case of respond via RSVP, there will almost certainly be no
return RSVP-LSP to return over (from where the pkt ends up).  At least,
that's how I read/understand it.....and perhaps someone could please clarify
if I have got this wrong?

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 06 Dec 2002 14:43:05 -0500

>I understand the intent, but these RFCs explicitly prohibit sending
>127/8 addresses out of any box. It is for a node's internal use only.

         The addresses are not sent to be included in a  routeable
packet on the wire.  The processing behavior is well defined for
how to process the LSP Ping payload, and I agree with Curtis
that the address range use is perfectly valid.

         --Tom

>-Shahram
>
> > -----Original Message-----
> > From: George Swallow [mailto:[email-masked]]
> > Sent: Friday, December 06, 2002 12:23 PM
> > To: Shahram Davari
> > Cc: [email-masked]; [email-masked]; [email-masked]
> > Subject: Re: Last call on LSP Ping
> >
> >
> > > RFC 3330 says:
> > >
> > >
> > >    127.0.0.0/8 - This block is assigned for use as the Internet host
> > >    loopback address.  A datagram sent by a higher level
> > protocol to an
> > >    address anywhere within this block should loop back
> > inside the host.
> > >    This is ordinarily implemented using only 127.0.0.1/32
> > for loopback,
> > >    but no addresses within this block should ever appear on
> > any network
> > >    anywhere [RFC1700, page 5].
> > >
> > > And RFC1700 says:
> > >
> > >    g)   {127, <any>}
> > >
> > >          Internal host loopback address.  Should never
> > appear outside
> > >          a host.
> > >
> > >
> > > Doesn't this draft violate these RFCs?
> >
> > This it falls within the intent.  If the label stack gets completely
> > poped off you don't want it to go any further.  That's the whole point
> > of using that address range.  I really don't see a need to find a new
> > range.
> >
> > (Note that I'm counting the implicit null as still being a label stack
> > - so the packet gets as far as the stack will take it and no further.)
> >
> > ...George
> >
> > ==================================================================
> > George Swallow       Cisco Systems                  (978) 497-8143
> >                      250 Apollo Drive
> >                      Chelmsford, Ma 01824
> >

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 06 Dec 2002 15:08:19 -0500

>More proof:

         Of what?

>RFC 1812 (Requirements for IP Version 4 Routers):
>
>
>    A router SHOULD NOT forward any packet that has an invalid IP
>    destination address or a destination address on network 0.  A router
>    SHOULD NOT forward, except over a loopback interface, any packet that
>    has a destination address on network 127.  A router MAY have a switch
>    that allows the network manager to disable these checks.  If such a
>    switch is provided, it MUST default to performing the checks...
>
>    A router SHOULD provide a mechanism to filter out obviously invalid
>    routes (such as those for net 127).

         Isn't this why the 127 addresses were used (i.e.: to prevent mis-
routing if the MPLS header was removed)?

         --Tom

> > From: Thomas D. Nadeau [mailto:[email-masked]]
> > Sent: Friday, December 06, 2002 2:43 PM
> > To: Shahram Davari
> > Cc: [email-masked]
> > Subject: RE: Last call on LSP Ping
> >
> >
> >

---

From: Shahram Davari <[email-masked]>
Date: Fri, 6 Dec 2002 12:23:35 -0800

> routing if the MPLS header was removed)?

RFC1812 was wrotten in 1995. I don't believe there was any MPLS at that time!

-Shahram

---

From: "Gray, Eric" <[email-masked]>
Date: Fri, 6 Dec 2002 15:34:32 -0500

Tom,

	I would assume proof of the writer's position - which I 
think is obvious.  :-)

	Also, you must realize that the requirement that a "router 
SHOULD provide a mechanism to filter out obviously invalid routes 
(such as those for net 127)" may technically be satisfied by a
simple destination address filter.  That does not mean that every
router would by default be configured with such a filter, given
that such a filter must consume some resources and such packets
SHOULD never appear in the network in the first place.

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

> Sent: Friday, December 06, 2002 3:08 PM
> To: Shahram Davari
> Cc: [email-masked]

---

From: "Gray, Eric" <[email-masked]>
Date: Fri, 6 Dec 2002 15:44:00 -0500

Eric (Rosen),

	Use of SHOULD NOT just means it is _not_ expressly
forbidden.  It does not in any way imply that one can do
whatever one SHOULD NOT do whenever one feels the urge.

	:-)

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

> From: Eric Rosen [mailto:[email-masked]]
> Sent: Friday, December 06, 2002 3:37 PM
> To: Shahram Davari
> Cc: 'Thomas D. Nadeau'; [email-masked]
> Thanks for finding this passage.  I  was worried that the IESG might balk
> at
> the proposed use of  127/8, but the above quote, by its  use of "SHOULD
> NOT"
> rather than  "MUST NOT", shows pretty  clearly that the proposed  use is
> not
> prohibited.
> 
>

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 06 Dec 2002 15:58:56 -0500

>SHOULD never appear in the network in the first place.

         Are there really any routers out there today that
anyone would spend money on that would behave
in this (incorrect) way?   Are we really splitting semantic
hairs now?

         --Tom

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Mon, 09 Dec 2002 12:01:02 -0500

[snip]

>         I object to the use of this address range for the
>purpose that has been stated thus far.  The reason why I
>object to it, is that I see it as pointless and possibly
>problematic.  I have no objection to MPLS-Ping itself.
>
>         From a posting by George Swallow (roughly 7:00
>P.M. on 12/05/02 - see excerpt below), I got the clear
>impression that the use of this address range was
>intended to allow a sender to "tickle" all of the paths
>that might be used by an ECMP load sharing set of LSPs.
>If that is the case, this is hopelessly useless for that
>intent, since the last byte in the IP destination address
>is NOT going to be used for ECMP load sharing of LSPs in
>many reasonable LSR ECMP implementations.  It is not even
>guaranteed to be used in IP forwarding.  There are, for
>example, perfectly reasonable hashing algorithms that use
>the first two bytes in both the source and destination IP
>addresses.  There are other ECMP hash algorithms that use
>SIPA exclusively.  What is the point in using the 127/8
>address range if any of these are in use?
>
>         If there is another reason for using this address
>range, it either has not been stated, or I missed it.  If
>there is a good reason to do it, I would not object to it.
>But there is not and - IMHO - there is a non-zero risk in
>using it.  In other words, cost is possibly higher than
>we know and benefit is definitely lower than we seem to
>be led to believe.
>
>         So I object to the use of the address range, not to
>the usefulness of MPLS-Ping.

         What addresses other than the 127.* addresses
would you put into the UDP packet contained in the
LSP ping message?  This satisfies the requirement that
LSP ping packets are not forwarded if the header is popped,
and also the requirement that the ping messages are forwarded
the same way as a data packet. This works whether hashing is
done whether or not you look past the MPLS header.
Quite a nifty and effective (albeit a bit hacky) solution if you
ask me.

         --Tom

>Eric W. Gray
>Systems Architect
>Celox Networks, Inc.
>[email-masked]
>508 305 7214
>
> > ----- Excerpt from a message from George Swallow -------
> > Sent: Thursday, December 05, 2002 7:01 PM
> > To: [email-masked]
> > Cc: George Swallow; [email-masked]; [email-masked]
>[...]
> >> The suggestion to randomly choose IP destination addresses from the 127/8
> >> block is, presumably, the way to ensure that the various paths created by
> >> equal cost load splitting get exercised.  The question is, how many pings
>
> >> do you need to send in order to get good coverage?
> >
>[...]
> > ----- Original Message [from Curtis Villamizar] -----
> > From: Curtis Villamizar [mailto:[email-masked]]
> > Sent: Saturday, December 07, 2002 6:52 PM
> > To: [email-masked]
> > Cc: [email-masked]; [email-masked]; [email-masked]
> > In message <0536FC9B908BEC4597EE721BE6A35389014CFCB5@i2km07-
> > ukbr.domain1.system
> > host.net>, [email-masked] writes:
> > > > > George....I think the point Shahram was making it is not
> > > > even allowed
> > > > > outside the host/source box according to the text quoted
> > > > from the RFcs
> > > > > below.  Wrapping it in some other package (any, not just
> > > > MPLS) is still
> > > > > violating this requirement...and note that it *must* be
> > > > placed in some
> > > > > package to get across any tranmission medium....IP pkts
> > > > don't go raw on
> > > > > wires.
> > > >
> > > > As far as I'm concerned it is within the spirit of the RFCs.  If you
> > > > want to insist on the letter of the law, perhaps you should look into
> > > > a career as a solicitor.
> > > >
> > > > ...George
> > >
> > > George, I am asking a serious question here and I would have appreciated
> > a
> > > serious answer please.  I'd also like to see some responses to the other
> > > issues that have been raised.  My experience to date suggests one only
> > gets
> > > responses 'when it suits'.  At least others (esp Erics R and G and
> > Curtis)
> > > have attempted to provide reasoned answers.
> > >
> > > This stuff is going to get delivered whatever right?....so I want to be
> > sure
> > > that its as fit for purpose as possible (and that's more than this
> > > particular 127/8 point).
> >
> > A lot of packets from my site (here at home) go through the Internet,
> > all destined to a subnet of 10/8 (in the office).  This (10/8) is also
> > not a routeable address covered in the same sections as 127/8 in
> > RFC1700 and RFC1812.  Its OK to deliver these packets since they are
> > encapsulated in something else.  It is not considered a violation of
> > RFC1700 or RFC1812 to forward these across the Internet where 10/8 and
> > 127/8 are considered to be "martian packets".
> >
> > Going way back to 1992, the mbone tunnels were originally built out of
> > source routed tunnels with a multicast destination.  They later
> > switched to a tunnelled encapsulation due to performance problems of
> > certain routers when handling large volumes of source routed packets.
> > This was not considered a violation of RFC1700 or RFC1812.
> >
> > There are plenty of precedences for using private addresses inside
> > something else.
> >
> > In many circumstances, packets with private addresses are tunneled.
> > You have to separately look at what happens at the end of the tunnel
> > after the encapsulation is stripped off.
> >
> > > Here's my understanding of the current position on this specific point:
> > >
> > > There are (at least) 2 possible cases:
> > > 1   127/8 addressed pkts should never leave a box
> > > 2   127/8 can always leave a box.....since I can put 127/8 addressed
> > > pkts in ATM or FR (or ethernet?) directly.....I see no significant
> > technical
> > > difference here to the MPLS case that is argued as being justified.
> > > [Aside - one could generalise the above case 2 to *any* IP addressed
> > packet,
> > > or we could make 127/8 a special case]
> >
> > 127/8 cannot be forwarded based on the 127/8 destination.  Just as
> > traffic from 10/8 can leave a subnet, croos the big-I Internet, and
> > reenter a subnet of 10/8, so can a packet be formed with a 127/8
> > destination and be encapsulated in something and sent.
> >
> > What happens when the outside encapsulation is stripped off also has
> > to be examined.
> >
> > When the labels are gone, the 127/8 remains and the packet is sent to
> > the loopback address.  That is the one exception allowed by RFC1812
> > and delivery to the loopback is the explicit intended use of 127/8.
> >
> > > Now I just want a ruling that 2 is fine (perhaps as a special case?),
> > and as
> > > long as its agreed now then that's OK....I don't want this raised by
> > someone
> > > X months hence after this stuff is being shipped and operators have
> > deployed
> > > it.
> >
> > If you had a legitimate reason to deliver traffic to the loopback
> > device on the other end of a FR or ATM link, then putting 127.0.0.1 in
> > the destination and a valid UDP payload with a port number for that
> > application, then it would be fine.
> >
> > There is a legitimate reason for selecting a 127/8 address for this
> > application.  Because it is used for a diagnostic we DON"T want it to
> > be routeable.  RFC1700 and RFC1812 guarentee that it won't be
> > routeable.  RFC1700 and RFC1812 explicitly set this address space
> > aside for use with the loopback device.
> >
> > > regards, Neil
> >
> > Once again, I think the MPLS-ping objectors are simply grasping at
> > straws.
> >
> > Curtis

---

From: Shahram Davari <[email-masked]>
Date: Tue, 10 Dec 2002 07:32:42 -0800

Since MPLS label has no protocol ID field, either an IP
header (for IP carrying LSPs) or a reserved label (for
non-IP carrying LSPs) is required to identify the protocol.

-Shahram

> From: Abhishek Saxena [mailto:[email-masked]]
> Sent: Tuesday, December 10, 2002 5:59 AM
> To: George Swallow
> Cc: [email-masked]
> Hi,
> I was wondering why an IP header is required in the LSP ping 
> packet. The 
> information that we need to get the packet across to the 
> egress LSR is 
> simply the mpls label. The source IP address of this LSR is needed by 
> the egress LSR to return the packet back. This is already 
> there in the 
> TLV. We can therefore have a UDPoMPLS kind of a setup in this 
> case, with 
> no IP header in the packet therefore preventing any IP routing at all 
> for this packet.
> 
> Does this make sense?
> 
> Regards,
> - Abhishek
> 
> George Swallow wrote:
> > This message begins a workgroup last call on:
> > 
> >     Detecting MPLS Data Plane Liveness
> > 
> >      draft-ietf-mpls-lsp-ping-01.txt
> > 
> > The last call ends on Dec 16, 2002 2400H GMT.
> > 
> > George & Loa
> > 
> > 
> ======================================================================

---

From: "Gray, Eric" <[email-masked]>
Date: Wed, 11 Dec 2002 10:27:16 -0500

Kireeti,

	The main issue is that this approach is actually
unlikely to result in full coverage of all ECMP paths.
I would need to send PING packets to more than half of 
the entire address range in order to be certain of full
coverage in the simplest case in which there are only
two alternative paths.  And that would only work if I
know for certain that the destination IP address of the
PING packets is being used in the ECMP hash function.
In fact, given the objections I have stated already in 
other mail (which you will likely see as you catch up 
on this thread), this "door-to-door PING" approach is
not very likely to provide coverage for more than one 
path for most ECMP/MPLS implementations and LSPs.  

	However, the mere fact that it is included may
suggest to some that it will work in some cases.  Since 
this is certainly not true, it is misleading as well as 
not particularly useful.

	I frankly see little point in including something 
in a specification that is so very likely to require a
later "considered dangerous" RFC.  I actually think that
Curtis is on the right track with his proposal.

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

> From: Kireeti Kompella [mailto:[email-masked]]
> Sent: Wednesday, December 11, 2002 6:07 AM
> To: Gray, Eric
> Cc: [email-masked]
> Hi Eric,
> > 	I am suggesting that attempting to use any address range
> > as a mechanism for testing ECMP routes is a mistake.
> 
> I don't follow.  If I send ping packets with 127.0.0.1 and 127.31.41.59,
> the worst that can happen is that they end up on the same path.  But
> I may get lucky and hit two paths.  If I only use 127.0.0.1 (or the
> ALL-ROUTERS.MCAST.NET address or whatever), I am guaranteed not to
> test more than one path.
> 
> Kireeti.

---

From: "Gray, Eric" <[email-masked]>
Date: Wed, 11 Dec 2002 10:31:47 -0500

> Sent: Wednesday, December 11, 2002 7:31 AM
> To: Eric Rosen
> Cc: George Swallow; [email-masked]
> ...
> > The suggestion to randomly choose IP destination addresses from 
> > the 127/8 block is, presumably, the way to ensure that the 
> > various paths created by equal cost load splitting get exercised.  
> > The question is, how many pings do you need to send in order to 
> > get good coverage?
> 
> We've beaten this one to death.  Suggestions are most welcome.

Actually, I wish we had beaten this one to death - it surely
needs to die...  :-)

For suggestions, see Curtis Villamizar's proposal.

> > ...

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:22:13 -0800

Thanks Kireeti. Understood and accepted.

-Shahram

> Sent: Wednesday, December 11, 2002 5:31 AM
> To: Shahram Davari
> Cc: [email-masked]
> Boy, I decide to take a break from reading the mpls list for a few
> days and I miss all the fireworks!
> 
> On Fri, 6 Dec 2002, Shahram Davari wrote:
> 
> > More proof:
> >
> Let me start here, as this one is easy:
> 
> As several people have pointed out, the ping packet with destination
> 127.x.y.z is *not* to be forwarded.
> 
> Clarification: this talks about filtering *routes*, not packets.
> So, if I were to send you a BGP route to 127/8, you're supposed to
> ignore it, and not install it in your forwarding table.
> 
> Kireeti.
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:19:43 -0800

Hi Kireeti,

Thanks for (finally :)) replying to my comments. Some follow ups:

> Sent: Wednesday, December 11, 2002 7:06 AM
> To: Shahram Davari
> Cc: [email-masked]
> Hi Shahram,
> > Although I have no hope of getting any response, but here 
> is a repeat of my ignored comments raised in March, 2002 and 
> some new ones:
> 
> Does that mean I don't have to reply? :-)
> 
> > 1) Time stamp: Why is the time stamp in both seconds and 
> microseconds? The microsecond could count up to 71 minutes. 
> Isn't that enough? Why isn't the
> > same timestamp used in ICMP not used here which says:
> >
> >         The Timestamp is a right-justified, 32-bit timestamp in
> >         milliseconds since midnight UT.
> 
> Is there a problem with having both seconds and microseconds?

It seems redundant and not needed.

> 
> > 2) Section 3: Why the echo reply time stamp is not similar 
> to ICMP echo-reply time-stamp in which a received time-stamp 
> and sent-time stamp is set by the receiver? Which gives you 
> the processing time at the receiver too.
> 
> Is there a problem if the time stamp is different?

Since people are familiar with ICMP, and there is no specific reason
for introducing a new notion for time stamping, it seems logical to
stick to the same format. It is like inventing a different CRC-32 for each
new protocol. If the purpose is the same, then let's use what we already have.

> 
> > 3) Section 3.1: If I understand it correctly a Target FEC TLV could
> > consist of a number of FEC stack sub-TLVs. Therefore it 
> seems that the
> > number of these sub-TLVs could be determined by looking at 
> the Target
> > FEC TLV length field and subtracting the sub-TLV lengths 
> from it. But
> > the text says it could be determined by looking at the 
> sub-TLV length
> > field. Needs clarification.
> 
> You just said it.

Thanks. Appreciate if you could make it more clear in the text.

> 
> > 4) Section 3.1: How can you ping multiple FECs 
> simultaneously (i.e., piggy back multiple pings), while each 
> FEC belongs to a different level of the label stack? This 
> requires that all those hierarchical labels (LSPs) be 
> initiated and terminated at the same LSRs. Also how do you 
> intend to generate error response if the result of some of 
> those FEC pings are positive and some are negative?
> 
> Send multiple pings.

But the text says you could send a ping with multiple FEC (aka FEC stack)!
My comment was that sending multiple FECs (belonging to different LSP
levels) in a single ping applies only to cases where those FECs
originate and terminate at the same LSR (e.g., L2VPN and L3VPN).
Also what happens if a few of the FECs are matched but others are
not, what is the error code?

Another point is regarding the example given in the draft that Eric Rosen
also agrees is not accurate. By that I mean in L2VPN and L3VPN it is
better not to push the PW/VPN label, but only push PSN label. However,
to test the binding between the PW/VPN label and the VPN route I suggest
adding the PW/PSN label to the VPN IPV4 and IPV6 prefix as following:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      Route Distinguisher                      |
      |                          (8 octets)                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         IPv4 prefix                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | Prefix Length |                 Must Be Zero                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                  PW/PSN  Label                |  Must be Zero |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      Route Distinguisher                      |
      |                          (8 octets)                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         IPv6 prefix                           |
      |                                                               |
      |                                                               |
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | Prefix Length |                 Must Be Zero                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                  PW/PSN  Label                | Must be Zero  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> > 5) Section 3 under return code. There is no return code for 
> the case that one of the FECs fail while others pass, or when 
> one of the label mappings don't pass, while others pass.
> 
> Can you give me an example of such a case?

In the example of section 3.1, where it says:

"  Alternatively, X can send
   a FEC Stack TLV with two FECs, the first of type LDP IPv4 with a
   prefix of 192.168.1.1/32 and the second of type of IP VPN with a
   prefix 10/8 in VPN foo. "

What happens if the 192.168.1.1/32 is the correct prefix for that egress
but prefix 10/8 is not recognized or does not correspond to the "inner label" ?

> 
> > 6) Why doesn't the MPLS ping permit pinging multiple FECs 
> that belong to the same LSP level? It seems that if 1000s of 
> FECs are all mapped to the same LSP, then 1000's of 
> MPLS-pings are required.
> 
> [There is a delicious _attitude_ in the way you pose your questions.]
> 
> One could do that.

Thanks. My suggestion is to let each FEC element in the FEC stack be an
FEC stack itself, in which each FEC in that stack corresponds to one of the FECs bound to that LSP.

> 
> > 7) Why is the downstream label field in the downstream 
> mapping TLV 24 bits? Is it for ATM? How is the 20 bit label 
> mapped to this 24 bit?
> 
> Good question.  20 bits right justified.  Will clarify.

Thanks. To support ATM (28-bit VPI/VCI) and FR (10-bit or 23-bit DLCI)
may I suggest changing the size of the Downstream label to 28 bits and 
right justifying any ATM/FR/MPLS label in this field as following:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                   Downstream IPv4 Router ID                   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               MTU             | Address Type  |  Reserved     |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                   Interface Address                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               Downstream Label                        |Protoco|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      .                                                               .
      .                                                               .
      .                                                               .
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               Downstream Label                        |Protoco|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> > 8) Section 3.2: When X receives a packet with TTL of n>1, 
> shouldn't it compute the downstream LSRs that could receive 
> the packet with TTL=n-1 (rather than TTL=n+1 as indicated)?
> 
> It's obvious I need to clarify this section.

Thanks.

> 
> > 9) Section 3.2 last paragraph: Does this apply to 
> trace-route mode only?
> 
> See last para of section 4.1.

May be it is best if you could add a single sentence to 3.1 to say that
this applies to trace-route only.

> 
> > 10) Section 4.3: Does the downstream mapping TLV of the 
> echo-reply replace the downstream mapping TLV of the 
> echo-request or it is appended to it?
> 
> As the section makes clear the fields that are copied from the echo
> request to the reply, it should be obvious that the downstream mapping
> that the receiver computes is the only one in the echo reply (i.e.,
> replace, not append).  But it doesn't hurt to say that explicitly.

Thanks.

> 
> > 11) Section 4.5: If the egress for an FEC stack being 
> pinged doesn't support MPLS ping, isn't an ICMP TTL expire 
> message generated for the IP TTL=1 received (instead of no 
> reply as mentioned in the draft)?
> 
> No LSP echo reply will be sent.

I think I answered my question. If 127/8 is used then no ICMP
TTL expire will be generated.

> 
> > 12) Section 5 2nd paragraph last sentence I think should be 
> changed to: "Note that this may not work if some transit LSR 
> does not support IP options" (not 'MPLS ping' as mentioned in 
> the draft)
> 
> No, it is correct as is.  A router that doesn't support MPLS 
> ping will not
> know how to deal with the MPLS echo reply.  It would get this
> strange-looking packet and probably discard it.

OK.

> 
> > 13) Section 5: Is RSVP-TE return path used only in "ping" 
> mode or also in trace-route mode?
> 
> Both.

In both cases what happens if the packet is misrouted? the
RSVP RESV will be returned to a different ingress !

> 
> > 14) Section 5.3 last paragraph: I think you need to delete 
> ICMP here. It has no meaning here.
> 
> Okay.
> 
> > 15) Section 5.5: I think you need to mention that if 
> intermediate LSRs don't support refresh reduction, then it 
> could take many minutes before the echo-reply is received at 
> the source.
> 
> Explain.

LSP-ping using the RSVP return path should have refresh reduction
enabled: meaning a refresh should be generated immediately in all intermediate
nodes.

> 
> > 16) Section 3.1.7 and 3.1.8: What is Encapsulation Type and 
> how is it encoded?
> 
> I'll add a reference.

Thanks.

> 
> > 17) Section 3.2: Why are there multiple downstream labels 
> in the downstream mapping? do they belong to different LSP 
> levels? Or the figure is simply showing many downstream mapping TLVs?
> 
> That LSR may swap and push.

Thanks, could you please add this clarification. Also please indicate
the correspondence between order of labels in the TLV and the label stack.

> 
> > 18) Section 3.3: what is the purpose of Pad TLV?
> 
> To pad the ping packet, e.g., to test large packets.

Thanks, could you please write down this example.

> 
> > 19) Section 4.3: Should an Echo reply always have a 
> downstream mapping TLV even if no downstream mapping TLV is 
> in the Echo request?
> 
> No.

Please add this to the text explicitly. 

> 
> > 20) Section 4.4: Seems like back to back echo-request can't 
> be sent before receiving the echo-reply of previous 
> echo-request. Why this limitation?
> 
> Where did you read this?

Text says:

   "Once an
   Echo Reply is received for a given Sequence Number (for a given UDP
   port and Handle), the Sequence Number for subsequent Echo Requests
   for that UDP port and Handle SHOULD be incremented."

If the echo requests could be sent without needing to receive the 
echo response of previous echo request, then there should not be any relationship between seq # of echo request and echo reply. Right?

> 
> > 21) Section 5.1: How is the downstream mapping TLV conveyed 
> in the echo-reply sent over RSVP-TE? it seems no downstream 
> mapping TLV is defined for RSVP-TE.
> 
> Do you think it should be added?

Sure, if RSVP-TE return path is a valid return path, then you need
to somehow convey this TLV. right?

-Shahram

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:39:24 -0800

Hi Kireeti,

> Sent: Wednesday, December 11, 2002 7:23 AM
> To: Shahram Davari
> Cc: [email-masked]
> On Thu, 5 Dec 2002, Shahram Davari wrote:
> 
> > 22) I can't understand section 3.2 at all. Since TTL=n+1 
> makes no sense, for the sake of argument I'll assume you 
> meant TTL=n-1. Now let's assume the following topology:
> 
> It's quite simple: read "initial TTL=n>1", "initial TTL=n+1", etc.

OK, I think what you meant is:

   If a packet with outermost label L and initial TTL n>1 arrived at X
   on interface I (with TTL=1), X must be able to compute which LSRs could
   receive the packet if the initial TTL=n+1, and what label they would see.  (It
   is outside the scope of this document to specify how this computation
   may be done.)  The set of these LSRs are the downstream routers (and
   their corresponding labels) for X with respect to L.

Correct?

> 
> >    L1       L2       L3
> > -------R1-------R2-------R3
> >                   \   L4
> >                    \-----R4
> >
> > In which R1, R2, R3, and R4 are LSRs and L1, L2, L3 and L4 
> are each a
> > set of labels that are received on those LSRs.
> 
> Correction: L1, ... L4 are all label mappings for a given FEC.

So am I correct to conclude that the format of the downstream Mapping is actually
as follows and multiple of them can exist in a single echo reply? Also the downstream routers reported in the echo reply are in fact representing the
ECMP paths?

       0                   1                   2                   3
      |               Downstream Label                |    Protocol   |
      |               Downstream Label                |    Protocol   |

  Also,
> R3 and R4 are equidistant from R2.  To make this more exciting, add an
> R5 with label L5 such that R5's metric from R2 is bigger than R3's.

Will R5 also be included in the echo reply?

> 
> > It seems from the text that the following sequential 
> behavior is expected: (DM means Downstream Mapping TLV)
> >
> > Ingress => Echo Request: DM (R1, L1)
> > R1      => Echo Reply  : DM (R2, L2)
> > Ingress => Echo Request: DM (R2, L2)
> > R2      => Echo Reply  : DM (R3, L3 and R4, L4)
> > Ingress => Echo Request: DM (R3, L3 and R4, L4)
> > etc.
> >
> > Is this the behavior that is intended? And does it apply 
> only to trace-route?
> 
> Yes (with the changes above), and yes.
> 
> > What is the purpose of this function? it seems that DM is 
> being used to extract the
> > whole switching table of each interface, is that correct? 
> If so, then it is neither ping, nor trace-route?
> 
> No, just the label bindings for a given FEC.

Thanks. Could you please clarify all that you said in the text.

> 
> > 23) section 4.1: In order to exercise all the ECMP paths, 
> you need to
> > scan through all 127/8 addresses (16 million of them), 
> don't you think
> > this is not feasible?
> 
> "The difficult, we do immediately.  The impossible takes a 
> little longer."

I would say it is impossible. I concur to Curtis's suggestion
of returning ECMP identifiers from split points.

> 
> > 24) Section 4.1: Why should an LSR send multiple echo 
> requests with the same seq#? What function is in mind that 
> can't be achieved by incrementing seq #?
> 
> To increase the probability of receipt of a ping with a given sequence
> number.

But by incrementing the echo requests, not only you could achieve the same goal, but also detect packet loss.

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:45:45 -0800

Kireeti,

Isn't it better to use two different receiver's CE-ID, one for
each receiver CE interface, rather than introducing FEC stack?

Thanks,
-Shahram

> Sent: Wednesday, December 11, 2002 7:45 AM
> To: George Swallow
> Cc: [email-masked]
> Hi George,
> Sometimes you're testing just the one FEC, sometimes you're really
> testing the stack.  Say you're testing prefix 10/8 in VPN foo, and
> it so happens that you go over LSP bar.  You don't really care, you
> create a ping packet with target FEC "10/8 in VPN foo", slap a label
> stack on it, and off it goes.
> 
> However, suppose you learned 10/8 in VPN foo from two PEs (dual-homed
> CE), with LSPs bar1 and bar2.  You may want to test each 
> independently.
> So, in one case you'd have a target FEC stack of "LSP bar1; 
> 10/8 in VPN
> foo" and in the other case, "LSP bar2; 10/8 in VPN foo".
> 
> Does that make sense?
> 
> Kireeti.
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:50:29 -0800

Kireeti,

I agree that random addresses chosen from a range is better
than a single address. But we don't want to rely on pure luck
either to detect breaks. I think Curtis's suggestion has
good merits in solving this problem.

Yours,
-Shahram

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:47:49 -0800

Hi Kireeti,

Saying that LSRs that receive a packet with 127/8 address should not discard
the packet, rather they should send the packet to the control-plane for processing.

Thanks,
-Shahram

> Sent: Wednesday, December 11, 2002 5:40 AM
> To: Curtis Villamizar
> Cc: [email-masked]
> Hi Curtis,
> On Fri, 6 Dec 2002, Curtis Villamizar wrote:
> 
> > Maybe one of the authors can join (do they really want to?)
> 
> Not really :-)
> 
> > whether they would see fit to add a sentence or deem it too obvious.
> 
> Saying what?  "Setting the destination IP address to 
> something from 127/8
> causes some to break out in a rash, but is perfectly valid"?
> 
> Kireeti.
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 11 Dec 2002 08:51:15 -0800

Agree and accepted.

-Shahram

> Sent: Wednesday, December 11, 2002 6:13 AM
> To: Curtis Villamizar
> Cc: [email-masked]
> On Mon, 9 Dec 2002, Curtis Villamizar wrote:
> 
> > > First of all I did propose using the egress IP address 
> and setting the
> > > TTL=1 in the IP header. This makes sure that the LSP-ping 
> packet will
> > > not get forwarded after the MPLS labels are poped.
> >
> > Glad to see this change to a technical discussion.  Took a while.
> 
> :-)
> 
> > This proposal has two flaws, 1) when the label gets POPed and the
> > egress isn't the router you thought it should be the response is an
> > ICMP TTL expired, 2) no way to handle ECMP is proposed.
> 
> Big flaw: there ain't no such thing as *the* egress router.  Some
> router "at the other end of the network" announces a label mapping
> for 9/8, and this mapping makes its way to me.  How do I know who
> the egress router is?  Moreover, there could be many routers 
> announcing
> 9/8 ...
> 
> Kireeti.
> 
> PS: IP TTL = 1 is already in the draft.
>

---

From: "David Allan" <[email-masked]>
Date: Thu, 12 Dec 2002 13:47:46 -0500

Hi Kireeti:

<snip>
> > I have a concern with the draft. Embedded in the protocol 
> are a number of
> > features designed to test proprietary behaviors (load 
> balancing). How these
> > would be used IMHO is insufficiently well documented to ensure
> > interoperability. On that basis I do not think the document 
> is ready.
> 
> I think this has been done sufficiently to death.  I'll hunt down and
> read Curtis's proposed text and comment.

OK
> 
> > One specific example that come to mind is the description 
> of "downstream
> > router". I consider to be poorly explained.
> >
> > Now the text:
> > "a packet with outermost label L and TTL n>1 arrives at X"..... OK
> >
> > "X must be able to compute which LSRs could receive the 
> packet with TTL=n+1"
> > whoa, where the packet came from? I thought TTL 
> decremented. I presume the
> > correct text should be "what LSRs would receive the packet 
> if forwarded from
> > X with TTL=1".
> >
> > "the set of LSRs are the downstream routers....for X with 
> respect to L",
> >
> > So what are we discussing here, the LSRs upstream of X, 
> downstream of X, the
> > specific NHLFE that load balancing resolves to given the 
> specific details of
> > the packet (src address and 127./8 and whatever else the propretary
> > implementation considers) or the set that the FTN/ILM maps 
> to prior to any
> > actual load balancing computations (is it "a" packet or 
> "any" packet mapping
> > to the FEC associated with the LSP).
> 
> Explained in reply to Shahram -- does that work for you?

I think when Curtis is finishes his proposal, we'll have somthing I'm happy
with.

<snip>
> > Given the description, and the ambiguities in the text, it 
> is not clear to
> > me what I could build and successfully interoperate with 
> the LSR next
> > door....
> 
> MPLS to the kerb (curb)?
You're having too much fun ;-)

> 
> > BTW it would be useful to explicitly state in 3.2 that the 
> downstream
> > mapping TLV is only valid in "traceroute mode", and that it 
> is optional in
> > echo requests, and mandatory in echo replies.
> 
> See reply to Shahram.

OK
> 
> > Other minor comments:
> >
> > - won't claim to be an expert, would TTL exhaust on an 
> internal loopback
> > address generate an ICMP message back to the ping 
> originator (which it
> > should be ignoring)?

At the moment there is no mention of what the ingress should do with the
extra ICMP TTL exhaust messages it will receive when running traceroute
mode.

> >
> > - downstream mapping permits multiple labels per interface 
> to be described
> > in an echo request, but the reply can only indicate that 
> "it arrived on one
> > of them" rather than which specific one (which may matter).
> 
> The "multiple" labels are a stack (see reply to Shahram).

Couldn't find anything relevant, but noting it is a stack helps and explains
a lot. Presumably when an LSR returning a downstream mapping TLV has
knowledge to return a stack, it does so in response to what is in the FEC
TLV.

3.2 would be better worded if the statement was that the length of the TLV
was "12+4*N octets where N is the depth of the stack returned". The "number
of downstream labels" is somewhat misleading.

As an aside, it would be most useful if implicit null was how PHP appeared
in the downstream mapping TLV. If you agree some text to explicitly mention
this should be there.

> 
> > IMHO it would be useful to have the sequence number 
> explicitly correspond to
> > the LSB of the 127./8 used as I would presume otherwise the 
> echo request
> > originating LSR would need to remember such details 
> associated with the
> > sequence number (as the reply would not contain the 
> address). (I'm assuming
> > that the sequence number is used uniquely with the LSP 
> under test, and the
> > handle exists as the indexing mechanism for replies 
> independent of the
> > specific LSP under consideration). T'would also facilitate 
> picking out gaps
> > due to black holes in load balancing constructs esp. in 
> don't reply mode.
> 
> I don't see the utility of this.  The LSB of the address is in the
> packet.
Not in a reply, although I suppose the originating LSR could maintain
associated with sequence number the actual address used. I was looking to
minimize the state maintained at the origin.

> 
> > - not clear how the example of testing stacked FECs would 
> work in CSC
> > scenarios, perhaps an author could comment?
> 
> You'll have to explain, starting with expanding "CSC".
Carrier supporting carrier. Although traceroute with downstream mapping
containing a stack would appear to address some of this.

cheers
Dave

---

From: "Gray, Eric" <[email-masked]>
Date: Fri, 13 Dec 2002 15:00:43 -0500

Curtis,

	You're applying the argument in the reverse of the
way I intended.  :-)

	In the same way that it is possible to argue that a
PLR knows where a back up path goes, so does an ECMP 
splitter.

	On a slightly different but related topic, note that 
- for the trace-route capability - the Downstream Mapping 
object makes it quite straight forward for the ECMP splitter 
to indicate to the head-end that there are multiple labels
and next-hops it might use for a specific FEC.  Too bad, is
it not, that this is not as true for the Ping capability.

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

> Sent: Friday, December 13, 2002 2:54 PM
> To: Gray, Eric
> Cc: 'George Swallow'; [email-masked]; Shahram Davari; [email-masked]
> In message <1117F7D44159934FB116E36F4ABF221B0267EDA8@celox-ma1-
> ems1.celoxnetwor
> ks.com>, "Gray, Eric" writes:
> > And so, naturally, you could make this same argument for
> > how an ECMP splitter would test out all of the equal cost
> > paths...
> 
> 
> The ECMP split affects the primary path.  The traffic for a branch has
> to be injected upstream of the branch or you are not adequately
> testing it.  There is no way to send traffic from upstream of a detour
> to exercise a detour that is not in use.
> 
> Curtis
> 
> 
> > Eric W. Gray
> > Systems Architect
> > Celox Networks, Inc.
> > [email-masked]
> > 508 305 7214
> >
> >
> > > Sent: Friday, December 13, 2002 2:22 PM
> > > To: [email-masked]
> > > Cc: Shahram Davari; [email-masked]; [email-masked]
> > > > Currently you'd have to do that from each PLR.  We don't provide a
> > > > means to tell the ingress of the existance of a detour (except the
> > > > local-protect-available bit in the signaling itself which one major
> > > > implementation wasn't using last I checked).  We also don't provide
> a
> > > > means to tell a specific PLR to exercise its detour and report back.
> > > >
> > > > Let's wait and hear from Kireeti or other authors whether they think
> > > > anything should be added or whether the intent is to require testing
> > > > initiated at each PLR.
> > >
> > > The PLR is responsible for the bypass.  It knows what sender-template
> > > it used, so it can form the proper FEC for the ping.  It can send a
> > > lsp-ping with the proper label stack.  So it is the natural place to
> > > do this.  If it fails, it should
> > >
> > > a) report to network management
> > > b) reset the backup in place bit
> > > c) look for another way of establishing a backup
> > > d) if successful on c) set the backup in place bit.
> > >
> > > You may want to put a hold-down on b) to avoid reporting to the
> > > head-end on a transient situation.
> > >
> > > I don't see what advantage you would gain by moving all this to the
> > > head-end.  I do see a lot of complications...
> > >
> > > ...George
> > >
> > >

---

From: Shahram Davari <[email-masked]>
Date: Fri, 13 Dec 2002 12:20:50 -0800

Kireeti,

Since the PLRs are in charge of deciding when to use the
backup/detour, it seem that there is no other choice, but to
generate MPLS-pings at PLRs (as you said it). 

Yours,
-Shahram

> Sent: Friday, December 13, 2002 3:00 PM
> To: Shahram Davari
> Cc: [email-masked]
> On Fri, 13 Dec 2002, Shahram Davari wrote:
> 
> > Are you saying that now all LSRs (even the intermediate ones)
> > need to be able to generate and process MPLS-ping and keep
> > MPLS-ping states?
> 
> To be more precise, PLRs.
> 
> Do you have an alternative suggestion?
> 
> Note that PLRs testing detours can do so at a more leisurely pace.
> 
> Kireeti.
>

---

From: "Gray, Eric" <[email-masked]>
Date: Fri, 13 Dec 2002 16:06:09 -0500

George,

	Here are my last call comments on the LSP Ping draft (nits
intermixed with technical comments):

1) In general, there are several minor indications that the draft may
   not have been quite ready for last call.  The draft includes many
   partially described features and at least one incomplete section.
   Some features seem not quite fully baked, even if they are fairly
   adequately described. TLV definitions are not described in a way
   that is conducive to compatibility with future versions, and there 
   are other such indications as are reflected in comments below.
   However, once the issues are addressed, this specification will be
   very useful piece of work. 

2) The "Return Subcode" field in the format "picture" (page 5) should 
   be "Reserved" instead; no values are defined for it and it is
   supposed to be set to zero.  The text on page 6 describing this 
   field should also be changed to reflect the field name change.
   Other text should be changed to omit references to this field.
   The current field name presupposes a use for this field not yet
   (and therefore, possibly, never to be) defined.

3) At the end of the first paragraph on page 6, the parenthesized 
   text should be changed to read "(this is currently only defined
   for control planes that uses RSVP-TE)."
                    =               ===

4) For the Sender's handle, the text "There are no semantics associated 
   with this handle," (page 6) should be changed to replace "associated 
   with" with "defined for".  The intention should be that the receiver
   is not allowed to infer meaning from the bits in this field, not that
   there is no meaning.  With this change, the text "although the sender
   may find this for ..." could be removed.

5) The sentence at the top of page 7 should be changed to read - "TLVs 
   (Type-Length-Value tuples) and sub-TLVs have the following format"
   (there is no separate description of sub-TLV formats).

6) The first paragraph in section 3.1 should be changed to read - 
=======================================================================
   A Target FEC Stack is a list of sub-TLVs.  The number of elements is
   determined as a result of the combination of TLV length, and sub-TLV 
   length, fields.
=======================================================================

7) Immediately after the table in section 3.1 (page 7), the phrase 
   "Other FEC Types will be defined as needed" is not reflected in 
   IANA  considerations section - which is currently empty (the draft
   should not have been sent for WG last call with this sections
   empty).

8) First paragraph in section 3.1.3 should read - "... [RSVP-TE], 
   sections 4.6.1.1 and 4.6.2.1" - (corrected reference and moved
   close bracket).  Similarly, the first paragraph in section 3.1.4 
   should read "... [RSVP-TE], sections 4.6.1.2 and 4.6.2.2".

9) In the formats specified in sections 3.1.3 and 3.1.4 (RSVP IPv4 
   and IPv6 Session TLVs, an entire word can be eliminated from the
   TLV by simply replacing the first "Must be zero" field with the
   LSP ID field in the last word (eliminating the last 32-bit word).

10) The document editor should use nroff blocking with format diagrams
   (example - format diagrams in 3.1.3, 3.1.8 and elsewhere).

11) Sub-TLV lengths are inconsistent with format diagrams in sections
   3.1.5, 3.1.6, 3.1.7 and 3.1.8. If the authors intend that sub-TLVs
   should be contiguous, the "Must be zero" fields at the end of these
   diagrams should simply be omitted.  If the authors intended these
   fields to indicate padding to nearest word boundary, then sub-TLV
   lengths should be "rounded up" to the nearest multiple of 4.  Based
   on the definitions for LDP IPv4 and IPv6 Prefix sub-TLVs, it looks
   as if the intent was not to pad (there is no format diagram, nor a
   statement as to the alignment - therefore, the "Must be zero" fields
   would actually contain sub-TLV Type and Length information.  Based 
   on the supposition that the definition of TLV is meant to apply as
   well to sub-TLVs (see #4 above), valid length values for TLV value
   fields should be an even multiple of 4.

12) In section 3.2 the format diagram contains the fields Interface 
   Address and Downstream Label which are not described in subsequent 
   text.  In addition, the description for Address Type is buried in a
   paragraph describing MTU.  These fields need to be described briefly
   in bullets immediately following the diagram. If the descriptions
   would not make sense without the text that follows the diagram, then
   that text should precede the diagram.

13) The first sentence in the paragraph at the very bottom of page 11
   should be changed to read "The notion of 'downstream router' is 
   explained as follows:" (otherwise, it is not clear that authors
   are not leaving this as an exercise for the reader).  This should
   be made clearer, because it is hard to be sure that the notion has
   been explained - i.e. - some return codes indicate that the receiver 
   is (or is not) a "downstream router" (presumably from its own point
   of view) while the definition appears to be from the sender's point
   of view.

14) The delimited (===) text below is hard to understand.
=======================================================================
                                                       ... Consider an 
   LSR X.  If a packet with outermost label L and TTL n>1 arrived at X
   on interface I, X must be able to compute which LSRs could receive
   the packet with TTL=n+1, and what label they would see.  (It is
   outside the scope of this document to specify how this computation
   may be done.)  The set of these LSRs are the downstream routers (and
   their corresponding labels) for X with respect to L.

   The case where X is the LSR originating the echo request is a special
   case.  X needs to figure out what LSRs would receive a labelled
   packet with TTL=1 when X tries to send a packet to the FEC Stack that
   is being pinged.
=========================================================================

   What the first paragraph seems to say is that X determines the set
   of downstream LSRs as those to which it might forward an unexpired
   labeled packet for the FEC(s) in question.  If that is what it is 
   trying to say, it could be said in an easier to understand way.

   I do not understand in what way X - as the originator of the Ping
   - is a special case.  Presumably it is the only LSR that would do
   this, therefore, it is "special" only in the sense that it is the
   only LSR to which this applies.

15) The use of the Pad TLV (defined in section 3.3) is not described 
   in the text other than to state whether or not the Pad TLV is to
   be returned to the sender.  Exactly how would the Pad TLV be used
   and why is it needed?

16) The Error code TLV should not be included in this specification
   if it is not going to be defined in this specification.  Instead,
   the IANA considerations section should include requirements for
   management of the TLV type space associated with this specification.

17) In section 4.1, the point in randomly selecting an address from   
   the address range 127/8 is inadequately specified in the draft and
   has not been resolved in mailing list discussions. The common belief
   at this point is that this is provided in order to provide some hope
   of being able to test multiple ECMP paths.  The specification should 
   either specifically address behavior required to support Ping and
   Trace-Route of ECMP equal cost paths, or it should not address it at
   all.  Several suggestions have been made to date as to how the
   specification could handle ECMP paths; either one of them should be
   documented in this specification, or the specific wording should be
   changed to indicate that the destination address in the UDP packet
   should be chosen from the address range 127/8 using a mechanism not
   defined by this specification.

18) First sentence in section 4.3 should read "When the Reply Mode in
   an MPLS echo request is 2 or 3, an MPLS echo reply is constructed 
   as a UDP packet."  Alternatively, this should be a subsection, or
   the section heading should distinguish this from a control plane
   echo reply.  The latter approach is probably better, since all of
   the paragraphs in this section apply only to the UDP reply case. I
   would suggest something like -

   "4.3 Sending an MPLS Echo Reply via UDP"

19) The second sentence, first paragraph of section 4.4 should be
   changed to read - 

=======================================================================
                               "...  Thus, on receipt of an MPLS Echo
   Reply, X should parse the packet to assure that it is well-formed,
   then attempt to match up the Echo Reply with an Echo Request that it
   had previously sent, using (for example) the destination UDP port 
   and the Sender's Handle."
=======================================================================

   The sender should be free to determine on its own what means it
   will use to match a request with a reply and can do so as long 
   as the receiver has followed the rules.  Several approaches may
   be used: for example, a sender may have only one outstanding 
   echo request at a time or it may use the Sequence Number (and, 
   possibly, Time Stamp Sent).
   
20) For the RSVP-TE extension, the Time Stamp Sent should be copied to
   the Time Stamp field in the LSP ECHO object - or there should be a
   field for both Send and Receive Time Stamps. Because of differences 
   in local times likely (especially in microseconds) at two different 
   devices in the network, the time sent is likely to be of more use 
   than the time received.  For example, the time sent may be used to 
   determine which of several Pings sent is being responded to in the 
   event that multiple Pings were sent with the same sequence number.  
   Also, the Time Stamp sent can be used as a reliable way to detect
   changes in round-trip time for the Ping request/reply messages.

21) I do not see why [RSVP] and [RSVP-REFRESH] are listed as Normative
   references.  In the [RSVP] case, it does not much matter because it
   is necessarily the case that this specification has a Normative
   references to [RSVP-TE] which it self has normative references to 
   [RSVP] - though the explicit reference is redundant, especially
   given that this specification makes no apparent direct references
   to [RSVP].  However, there is no reason to include [RSVP-REFRESH]
   as a reference at all.

22) RFC 768 [UDP] should be listed as a Normative reference.

23) The last paragraph in section 8 (appropriately named Security
   Considerations) is wrong, in particular with respect to the 
   Downstream Mapping TLV.  The contents of this TLV are of a
   particularly useful nature to anyone wishing to deny access
   to the service being "Ping-ed" and should be secured against
   the possibility of un-trusted viewing.

Eric W. Gray
Systems Architect
Celox Networks, Inc.
[email-masked]
508 305 7214

> Sent: Tuesday, December 03, 2002 5:52 PM
> To: [email-masked]
> Subject: Last call on LSP Ping
>

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 13 Dec 2002 15:03:51 -0500

>need to be able to generate and process MPLS-ping and keep

         I will let George speak for himself, but to me it seems clear that
since the PLR is responsible for setting up the bypass that if it
is also interested in testing the bypass path's liveliness, that it
should be capable of generating and processing MPLS ping messages
too.  The only state being kept is by PLRs that wish to verify the liveliness
of their backup paths.  Those that do not wish to provide this sort
of functionality need not keep any state or provide such functionality.

         --Tom

> > > implementation wasn't using last I checked).  We also don't
> > provide a

"The difficult, we do immediately.  The impossible takes a little longer." 
-- Air Force Motto

---

From: [email-masked]
Date: Mon, 16 Dec 2002 08:50:13 -0000

Cheng-Yin....simple answer from me is 'I don't know'....others may be able
to provide a more concrete answer.  However, the question misses the point
IMO.

What we have here is a simple client/server relationship between 2 layer
networks:
-	we have a PSN server layer technology, eg MPLS say;
-	we have some client layer network technology X.

In architectural terms there is a client layer trail of X (between wherever
its source/sink points are defined) and a server layer layer trail which is
acting as a (ie one) link connection of the client layer trail between the
PE devices.  That is, the server layer trail has a trail termination source
at one PE device, and a trail termination sink at tbe other PE device.  Now,
the specification and behaviour (of the traffic-bearing data-plane) of the
server layer trail (all aspects, including OAM) should be independent of
which client layer signal is being carried.....of course, this assumes that
the server layer trail is actually capable of delivering the
availability/QoS performance metrics/objectives that the client layer would
expect (and this may not be possible for arbitrary client/server layer
relationships, ie some make more sense than others).  Further, it should
also be independent of how the server layer trail was instantiated.

These are the fundamental requirements for any client/server
relationship....so this is not peculiar to PWE3, as it is a general
architectural requirement.

Putting this another way, what I am saying here is that the server layer
should be able to 'look after itself' as a stand-alone entity.

The same considerations apply to the client layer of course....noting that
client/server relationships recurse, and can indeed be arbitrary (within the
bounds of what are/are-not sensible client/server relationships).  And this
is why layer independence is such a fundamantal requirement.

There is of course a need for communication *between* layer networks under
failures, as this is vital to constrain alarms to their layer of orgin so
that we don't have Ops people all all levels running around trying to work
out what is happening.
Note - in the general case there could be (i) many recursive client/server
relationships here, and this communication has to reach the top level layer
networks, (ii) the various layer networks may not be in the same operator's
management domain, and (iii) the various client layer networks (and
specifically their trail termination points) may be in geographically
disperse locations, even different countries.

The way this is normally catered for is that on server layer failure a FDI
(Forward Defect Indication) is conveyed to the client layer network(s) that
the affected server layer trail supported when it was working normally.
Since defect detection/handling is a uni-directional function, FDI is
required to be generated at the server layer trail sink termination point
and (from G.805) adapted into the appropriate FDI of the affected client
layer (trails).  This is not something novel, it is a fundamental operator
requirement, and it is how those layer networks that operators have had a
large hand in defining are specified, eg PDH, SDH, OTN, ATM.

Now if both the client layer and server layer are self-sufficient as I have
described above then this directly answers your question, ie there is no
need for anything else.

The problems arise when layer networks aren't self-sufficient......be this
from poor initial design, or the fact that are being used in a role that
they were never designed for (ethernet is a good example of this latter
point).

And it is these cases that lead to architectural violations.  Whether such
violations are acceptable or not is for the potential user of them to
decide.  Some are clearly unacceptable....for example there was a
requirement in a particular ATMoverMPLS ID a while back that said
(paraphrased) "on near-end ATM network partition failure (ie CE->PE) inject
Sonet/SDH AIS (=FDI) in the far-end ATM network partition (ie PE->CE)".  I
hope its obvious why this would be unacceptable behaviour for an operator
(but please ask me to explain if not clear).

In the case of mp2p MPLS LSPs, as I have pointed out previously these are
bizarre architectural contructs.  There is no concept of merging in cnls IP
networks (this is clearly pkt multiplexing), nor does the concept of merging
appear in the more usual p2p (or p2mp) co network topology constructs.  In
the case of mp2p LSPs therefore, one cannot appeal to 'normal'
specifications....fault-management is obvious, but it also applies to
performance specifications for availability and QoS metrics/objcetives (a
moment's thought will illustrate its not obvious how one could define
these).

And it is for these mp2p constructs that LSP Ping is being defined as the
fault management tool.  In the case of p2p constructs, and specifically in
the PWE3 case of XoverMPLS, then one can use the simple yet powerful
techniques given in Y.1711.  This is of particular significance/importance
IMO for operators who may be considering MPLS as a convergence solution for
all co pkt-sw technologies, eg FR/ATM/MPLS relationships.

regards, Neil

> From: Cheng-Yin Lee [mailto:[email-masked]]
> Sent: 16 December 2002 03:44
> To: Harrison,N,Neil,IKL2 R
> Cc: [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]
> Neil,
> > Now whilst the above does at least agree that the 
> control-plane must not
> > proxy for missing data-plane functionality (and its taken a 
> while to even
> > get this bit recognised as required), it still fudges the 
> client/server
> > data-plane layer network relationships.  This is not a good 
> idea for PWE3
> > applications say where the MPLS client can vary.  Ideally 
> one wants the
> > *same* solution/behaviour for all client (or server) layer 
> cases (and all
> > control-plane cases...inc case of 'none', ie static trails).
> Is there a requirement to test the data plane liveliness from ideally
> the input
> interface of the ingress PE to the output interface of the 
> egress PE for
> PWE3 applications?
> 
> thanks
> cheng-yin
> 
> [email-masked] wrote:
> > 
> > Hi Curtis...please see below.  regards, Neil
> > 
> > Curtis Villamizar wrote 10 December 2002 15:52
> > <snipped>
> > > > NH=> Agreed, this was what I was driving at in my 2nd set
> > > of questions that
> > > > I posed 5 Dec, and specifically Q6 therein.  I am still
> > > waiting a response
> > > > on all my postings.....to repeat that specific question:
> > > > "6  In the same section......why is there an option to
> > > reply by the RSVP
> > > > control-plane?  What scenario is this addressing that the
> > > above (NH=> ie
> > > > using IP only) doesn't?  Further, some guidance on when to
> > > select which
> > > > option would be useful."
> > >
> > > I wouldn't mind getting rid of this.  C&W (for example) has talked
> > > about running a no-IP routing core with MPLS over it.  I 
> think this is
> > > a very bad idea.  It is speculative on my part but I 
> think someone is
> > > either directly driving this or it was added to head off any
> > > objections that might arise.  I thought you were one that 
> insisted on
> > > a means to decouple diagnostics from an IP control plane. 
>  Maybe I'm
> > > mistaken.
> > NH=> Yes, but this is an obvious and general requirement.  
> Let me clarify
> > again what the requirement is here:  When traffic bearing 
> data-plane layer
> > networks (there are also control/management protocol 
> bearing data-plane
> > networks, often disjoint from the traffic's data-plane network) are
> > specified/built correctly they should have self-contained 
> fault handling
> > functionality that is not dependent on:
> > -       what client layer network (or applications) they carry;
> > -       what sever layer network they are supported on 
> (remember client
> > layer link connections = server layer trails......please 
> see G.805 if you
> > don't understand this);
> > -       what signalling (or routing) protocols are used to 
> instantiate the
> > traffic bearing data-plane trails.
> > 
> > This is obvious IMO (not just to me but to loads of people, 
> though many of
> > these don't even tune-in to this list), for without it you 
> cannot have
> > independent layer network evolution or 
> control/management-plane (protocol)
> > evolution.  Further, without adherering to this principle 
> one cannot use
> > layer networks in a flexible manner (ie any client/server
> > combinations....think PWE3 say in the case of MPLS, or even 
> SDH, or even
> > OTN).
> > 
> > The need to decouple data-plane fault handling behaviour 
> from specific
> > client, server or control-plane cases is given in Y.1710 
> (which is based on
> > agreed operator requirements).  Some of these requirements 
> are also given in
> > draft-nadeau-ip-basedtool-requirements-00.txt, where 
> requirement 'r' therein
> > says:
> > "r) Separation of Data and Control Plane OAM. The inherent 
> separation of
> > control and data planes in MPLS lends itself to being 
> sometimes implemented
> > independently within an MPLS LSR.  For example, in a 
> multi-slot LSR, one
> > slot may run a control process responsible for running MPLS control
> > protocols such as LDP and RSVP, and then programming line 
> cards residing in
> > other slots to forward traffic accordingly. In doing so, the switch
> > separates out the data plane from the control plane in such 
> as way that it
> > is possible for the line card to be mis-programmed whilst 
> the control card
> > is unaware. This leads to a potential for the line card to 
> black hole data
> > plane traffic.  This is one example of why it is important 
> for LSRs to
> > possess functions that allow an operator to detect data 
> plane liveliness.
> > Data Plane liveliness MUST use the exact same path as data."
> > 
> > Now whilst the above does at least agree that the 
> control-plane must not
> > proxy for missing data-plane functionality (and its taken a 
> while to even
> > get this bit recognised as required), it still fudges the 
> client/server
> > data-plane layer network relationships.  This is not a good 
> idea for PWE3
> > applications say where the MPLS client can vary.  Ideally 
> one wants the
> > *same* solution/behaviour for all client (or server) layer 
> cases (and all
> > control-plane cases...inc case of 'none', ie static trails).
> > 
> > However, making an an appeal to an IP layer return route for LDP
> > instantiated mp2p LSPs is not in the least surprising here 
> to try and
> > mitigate against the fault-handling complexity that mp2p 
> constructs create.
> > So breaking layering rules here seems unavoidable, and 
> whilst I don't like
> > it, it is perhaps the only way forward (I have suggested 
> running Y.1711 CV
> > on the p2p LSPs that must always exist above any mp2p LSP 
> (needed to start
> > to get rid of the merging) for initial proxied defect 
> detection of lower
> > layers, and then use trace-route as an ad hoc tool for the 
> mp2p constructs,
> > as a possible alternative approach.  I know Dave Allan has 
> recognised this
> > as one solution (to mp2p) a long time ago, but I don't 
> think anyone else is
> > considering it).
> > 
> > The RSVP return path case does not work as I tried to point 
> out previously
> > in the event of misdirected traffic, ie I think its 
> reasonable to assume
> > there is a probability close to 1 that there will be no 
> valid RSVP return
> > LSP from where any misdirected traffic will end up.  Hence, 
> why I said why
> > not just use an IP return mode in all cases?.......but also wanted a
> > clarification of why there is also a plain IP + 'router 
> alert' IP option
> > here (the latter of which I assume is to try and avoid 
> picking up a server
> > layer LSP return path?).
> > <snipped>
> > 
> > > Implementation is greatly simplified by putting most of the fault
> > > management smarts at the application layer.
> > NH=> I would not put it like this.  Basic fault handling 
> behaviour (I am not
> > talking about more complex ad hoc diagnostics tools here) 
> should be an
> > inherent design of the data-plane in a layer network.  If 
> it is not, one
> > will not get the proper behaviour.  Check out SDH and 
> GMPLS....the general
> > principles are the same (except one is not allowed mp2p 
> constructs here, for
> > obvious reasons).
> > >
> > > Lets just agree to disagree about what is a nifty hack.
> > NH=> As one my (well known/respected) engineering colleagues who is
> > responsible for defining our future network strategy here 
> in BT remarked to
> > me on seeing these words originally:
> > "Yes, and "nifty hacks" are the bane of carrier class 
> operations!! Tells us
> > everything we need to know I think!"
> > Not my words please note.
> > <snipped to end>
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 18 Dec 2002 14:38:45 -0800

Curtis,

> > > 4) What happens if in an FEC stack ping, some of the FECs 
> are confirmed
> > > and some are not? Do we need more error codes? How many more?
> > 
> > Good question.  Let me answer that one separately in 
> another message.
> > It is an ommision in the document.
> 
> There are a couple of legitimate situations where this will happen.
> With varying degrees of "legitimate".  Some have to do with TTL
> processing with hierarchy.  Another is situation is an LSR that
> doesn't decrement TTL (hopefully not on an LDP LSP that loops) or one
> that doesn't understand lsp-ping.
> 
> What you'd see at the ingress is an error.  If you resent the Echo
> Request without any of the prior mapping TLVs, then you'd see an Echo
> Reply that confirmed the FEC and gave a downstream mapping TLV.  You
> could then continue the traceroute.  Having reached the egress you
> have a portion of the trace for which you aren't certain how the
> packets go.  If the pings work and the area of uncertainty is outside
> your administrative domain, you have to accept that delivery happens
> by some undisclosed magic until delivery fails to happen.  If it is
> within your administrative domain, you can MIB query until such time
> as you can resolve the problem to get complete traces.

What you just said is true and acceptable for when some of the
labels in a Downstream mapping TLV (DM TLV) are confirmed and some are not.
In that case, sending an echo-request without DM TLV will be successful and
will give you the downstream mapping. My suggestion is changing the description
of error code 7 to:

Replying router is one of the "Downstream Routers", but its mapping for this FEC are the given label stack."

My  concern was more on pinging Traget FEC stack. Let's say one of the
FECs in the stack is confirmed but other ones are not confirmed. What is the error reporting procedure? My suggestion is changing the description of error code 4 to:

" Replying router has no mapping for one or more FECs in the Target FEC stack"

> 
> > > 5) How are the ATM/FR labels supported in Label-mapping TLV?
> <snip>
> 
> Lets try again on this one.
> 
> AFAIK, an ATM or FR LSR doesn't decrement TTL.  Therefore this would
> appear to to be a transparent couple of hops just like FSC, LSC, TDM.

Agree.

-Shahram

> 
> This might be one of those situations in point 4 above where there is
> a discontinuity in the sanity of the traceroute.  The downstream
> mapping gives the next ATM LSR but incrementing TTL does to the end of
> the VC if TTL isn't decremented.
> 
> Curtis
> 
> ps - IMHO - PHP is a thing of technical elegance and beauty compared
> to MPLS over ATM (or anything over ATM).  :-)
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 18 Dec 2002 14:56:21 -0800

A minor correction:

Change this:
 
> Replying router is one of the "Downstream Routers", but its 
> mapping for this FEC are the given label stack."
> 

to:

"Replying router is one of the Downstream Routers, but its mapping
 for this FEC are not the given label stack."

-Shahram

---

From: "David Allan" <[email-masked]>
Date: Fri, 20 Dec 2002 14:09:04 -0500

Hi Curtis:

<snipped>

> Dave,
> 
> You seem to be assuming ommision of the VPN label.

????????
 
> The problem does not occur for:
> 
>   1.  Uniform mode.  (Outer label TTL expires).
> 
>   2.  Pipe mode with PHP.  (Outer label removed, VPN TTL expires).
> 
>   3.  Pipe mode without PHP for routers that support the ability to
>       look for a label below the VPN label (newly specified).
> 
> The problem does occur when
> 
>   Pipe mode is used AND PHP is not used AND the router does not
>   support the ability to look under the VPN label.
> 
> Since we are talking about a practical problem, specifying a change in
> forwarding, lets look at a significant chunk of the deplyed base of
> core routers:
> 
>   Cisco - does PHP as egress.
>   Juniper - does PHP as egress.
>   Avici - does not request PHP but can support the ability to look
>     under the VPN label without hardware change.  Even if not, PHP
>     can be enabled by configuration.
> 
> Someone from Cisco or Juniper can object if this won't work with their
> routers.
> 
> If there are any other deployed routers that can't request PHP via
> implicit null and process it as an egress, and cannot modify
> forwarding to look under the VPN label if another label exists, then
> someone needs to speak up.  Otherwise it is a hypothetical problem for
> which there is no known instance and we can move forward.
> 
> Curtis

If I understand the problem statement correctly, it was how do you test the
inner label in a PW or VPN application. Currently as written in the document
it is via testing the outer label with an FEC stack (proxy test) on the
assumption that a probe under the inner label MAY NOT be visible to the
egress PE. I can also try to force visibility by setting the inner label TTL
to 1 which assumes pipe mode at the PHP LSR. Uniform model at the egress and
all bets are off as the VPN label TTL will be overwritten by the PSN label
TTL value. 

TTL exhaustion for ping mode is potentially undesirable as theoretically
ICMP TTL exhaust messages are a byproduct. I got the impression that testing
the inner label was primarily via testing the outer label with a FEC stack.
Looks simpler than some of the other implications (what is the address of
the inner label ping origin..?)

Dave

---

From: Shahram Davari <[email-masked]>
Date: Fri, 20 Dec 2002 12:53:24 -0800

Curtis,

> You seem to be assuming ommision of the VPN label.

Seems you are assuming only trace-route

>   1.  Uniform mode.  (Outer label TTL expires).

True for trace-route, not true for ping.

>   2.  Pipe mode with PHP.  (Outer label removed, VPN TTL expires).

No it doesn't. The packet arrives with VPN label, which has TTL=1 to the egress
LSR. The egress LSR should accept this packet and forward it if possible. Remember
RFC1812 says:

"
   A router MUST NOT discard a datagram just because it was received
   with TTL equal to zero or one; if it is to the router and otherwise
   valid, the router MUST attempt to receive it. "

So the packet received would be accepted and a lookup of the VPN label
results forwarding the payload to the Customer.

>       look for a label below the VPN label (newly specified).

I think you mean for those that can check the S-bit and make decision based
on S-bit. But I think you are fundamentally changing the standard MPLS behavior.
Now the forwarding table is based on not only on MPLS label (VPN label) but also
on S-bit. While the standard MPLS forwarding does not S-bit for forwarding decision, rather only as a PID for encapsulated packet.

Yours,
-Shahram

> > > From: Alia Atlas [mailto:[email-masked]]
> > > Sent: Friday, December 20, 2002 9:36 AM
> > > Cc: Shahram Davari; [email-masked]'; 'Kireeti Kompella';
> > > [email-masked]'
> > > At 04:04 PM 12/19/2002 -0500, Curtis Villamizar wrote:
> > > > > 1) Use explicit null label even though the traffic is IP, 
> > > and detect 
> > > > LSP-ping
> > > > > packets from S=0 in the VPN label. This may require HW 
> > > change, because 
> > > > curren
> > > > > t implementations don't look at S-bit.
> > > > >
> > > > > 2) Only push the PSN label, and encode the VPN  label in 
> > > the echo-request.
> > > > >
> > > > > 3) In echo-request set the TTL=0 for VPN label, and 
> > > detect the LSP-ping by
> > > > > sending the TTL=0 packets to the OAM module. This may 
> > > also require HW 
> > > > change
> > > > > as I am not sure current implementations check the TTL of 
> > > the VPN label.
> > > > >
> > > > > My suggestion was the 2nd one. What method do you suggest?
> > > > >
> > > > > Thanks,
> > > > > -Shahram
> > > >
> > > >
> > > >I was suggesting you set TTL=1 on the VPN label and include an
> > > >explicit null label, also with TTL=1.  Others can speak up if the
> > > >change will be a problem.
> > > 
> > > Currently the incoming TTL, which I believe is that checked 
> > > for the TTL 
> > > expiring, is based upon the top incoming label.  I.e., if the 
> > > PSN label is 
> > > still on the packet when it reaches the egress, then its TTL 
> > > would be used, 
> > > rather than the TTL of the VPN label underneath.  Of course, 
> > > if the PSN 
> > > label is not there, because of PHP, then the incoming TTL 
> > > would be that of 
> > > the VPN label, and this could work - but that requires PHP :-(.
> > > 
> > > Alia
> > > 
> > > 
> > 
>