Subject: Re: [mpls] MPLS-RT review of draft-villamizar-mpls-multipath-use

From: David Allan I <[email-masked]>
Date: Thu, 17 Jan 2013 14:45:06 +0000

Hi:

I was asked to review this document as part of the review team; the following are my comments.

I think such a document in general has merit and addresses a gap. I believe the issues that exist with this particular version of such a document are in the incomplete discussion of a solution in section 3. I'd want to see that cleaned up such that it read as a technically viable solution....(see below)....

Nit:

Section 1 para 2: Refers to MPLS-TP packet ordering as the constraint. I'm a bit confused by this, any aggregate that shares an ordering constraint does not get spread across parallel links while a set of traffic COULD be in theory load spread across a set of MPLS-TP paths so long as individual ordering constraints were respected. Is this text document really discussing midbox spreading out of load that respects ordering constraints but would mean MPLS-TP OAM no longer fate shared?  Should be reworded if so. Clarified otherwise.

Main issue:

Section 3: I think this needs a bit of work, it starts to discuss tractable solutions but IMO is technically incomplete. IF an entropy label and ELI (where all traffic associated with the MPLS_TP LSP shares a common randomly selected entropy label value, which is not stated) are the only mechanisms of ensuring proper treatment (with all transit nodes implementing entropy and ELI processing such that seeking sources of entropy is capped for such LSPs), and the ingress node (which by some means SHOULD be able to know it is a TP LSP as it has layer visibility) and the egress node can strip entropy and ELI then a solution exists for proper fate sharing and ordering of a TP LSP over MPLS. It is IMO not quiet eluciated completely.

If I'm unclear on any of the above, am happy to discuss...

Cheers
Dave

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 22 Jan 2013 21:38:57 -0500

In message <[email-masked]>
David Allan I writes:
> I think such a document in general has merit and addresses a gap. I
> believe the issues that exist with this particular version of such a
> document are in the incomplete discussion of a solution in section
> 3. I'd want to see that cleaned up such that it read as a technically
> viable solution....(see below)....

Dave,

If it is unclear to you, then the document is not sufficiently clear
and needs fixing.  See below.

> Nit:
>  
> Section 1 para 2: Refers to MPLS-TP packet ordering as the
> constraint. I'm a bit confused by this, any aggregate that shares an
> ordering constraint does not get spread across parallel links while a
> set of traffic COULD be in theory load spread across a set of MPLS-TP
> paths so long as individual ordering constraints were respected. Is
> this text document really discussing midbox spreading out of load that
> respects ordering constraints but would mean MPLS-TP OAM no longer
> fate shared?  Should be reworded if so. Clarified otherwise.

Section 1 paragraph 2 contains:

   RFC 5654 requirement 33 requires the capability to carry a client
   MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer
   [RFC5654].  This is possible in all cases with one exception.  When
   an MPLS LSP exceeds the capacity of any single component link it
   may be carried by a network using multipath techniques, but may not
   be carried by an MPLS-TP LSP due to the inherent MPLS-TP capacity
   limitation imposed by MPLS-TP OAM packet ordering constraints.

The sentence in question is:

   When an MPLS LSP exceeds the capacity of any single component link
   it may be carried by a network using multipath techniques, but may
   not be carried by an MPLS-TP LSP due to the inherent MPLS-TP
   capacity limitation imposed by MPLS-TP OAM packet ordering
   constraints.

Perhaps I could clarify this by changing "carried by an MPLS-TP LSP"
to "carried by a single MPLS-TP LSP".

The "MPLS-TP OAM packet ordering constraints" is the "fate sharing"
which is the reason that MPLS-TP prohibits using ECMP (aka multipath,
though ECMP is only one form of multipath) with MPLS-TP.  This is why
an MPLS LSP (client layer) with capacity greater than a component link
cannot be carried over a *single* MPLS-TP LSP.

Since this is explained in later sections I could make the
substitution suggested above or I could drop the last sentence and
mention that limitation later.

> Main issue:
>  
> Section 3: I think this needs a bit of work, it starts to discuss
> tractable solutions but IMO is technically incomplete. IF an entropy
> label and ELI (where all traffic associated with the MPLS_TP LSP
> shares a common randomly selected entropy label value, which is not
> stated) are the only mechanisms of ensuring proper treatment (with all
> transit nodes implementing entropy and ELI processing such that
> seeking sources of entropy is capped for such LSPs), and the ingress
> node (which by some means SHOULD be able to know it is a TP LSP as it
> has layer visibility) and the egress node can strip entropy and ELI
> then a solution exists for proper fate sharing and ordering of a TP
> LSP over MPLS. It is IMO not quiet eluciated completely.

Your paragraph above describes the solution, but then says "It is IMO
not quiet eluciated completely."

So lets parse your paragraph and see what the draft is missing:

> Section 3: I think this needs a bit of work, it starts to discuss
> tractable solutions but IMO is technically incomplete.

    OK ...

> IF an entropy label and ELI (where all traffic associated with the
> MPLS_TP LSP shares a common randomly selected entropy label value,
> which is not stated)

    You mention "IF an entropy label and ELI (where all traffic
    associated with the MPLS_TP LSP shares a common randomly selected
    entropy label value, which is not stated)".  Regarding the "which
    is not stated" part of that phrase I can change the following
    paragraph:

      OLD:

        MPLS-TP LSP can be carried as client LSP within an MPLS server
	LSP if an Entropy Label Indicator (ELI) and entropy label (EL)
	is added after the server layer LSP label(s) in the label
	stack, just above the MPLS-TP LSP label entry [RFC6790].
	[...]

      NEW:

        MPLS-TP LSP can be carried as client LSP within an MPLS server
        LSP if an Entropy Label Indicator (ELI) and Entropy Label (EL)
        is added after the server layer LSP label(s) in the label
        stack, just above the MPLS-TP LSP label entry [RFC6790].  The
        value of EL can be randomly selected at LSP setup time and the
        same EL value used for all packets of the MPLS-TP LSP.  [...]

    One sentence is added to the end.  (and Entropy Label is capitalized).

> are the only mechanisms of ensuring proper treatment

    Actually link-bundling is mentioned as another mechanism.

> (with all transit nodes implementing entropy and ELI processing such
> that seeking sources of entropy is capped for such LSPs),

    The requirement to terminate search for additional entropy below
    the EL is a requirement of RFC 6790.  The intent is to say that
    non-compliant midpoint LSR would in this case cause packet order
    problems where in MPLS only (no TP carried) use of RFC 6790, those
    non-compliant midpoint LSR are tolerable as long as they can get
    enough entropy from the traffic to adequately load split.

> and the ingress node (which by some means SHOULD be able to know it
> is a TP LSP as it has layer visibility)

    That is discussed in the following paragraph:

       There is currently no signaling mechanism defined to support
       requirement MP#1.  In the absense of a signaling extension,
       MPLS-TP can be identified through some form of configuration,
       such as configuration which provides an MPLS-TP compatible
       server layer to all LSP arriving on a specific interface or
       originating from a specific set of ingress LSR.  Alternately an
       MPLS-TP LSP can be created with and Entropy Label Indicator
       (ELI) and entropy label (EL) below the MPLS-TP label [RFC6790].

    Obviously the EL can be added at the MPLS-TP ingress LSR.  This
    paragraph is considering the case where a set of MPLS-TP LSR are
    not EL capable but also do not have multipath enabled, but
    multipath is enabled elsewhere (ie: the core where lots of traffic
    needs to get aggregated into a smaller number of LSP).  The lack
    of a signaling extension is noted as a (fixable) limitation.  The
    fix comes in a later draft (draft-villamizar-mpls-multipath-extn).

> and the egress node can strip entropy and ELI

    That is a hard requirement of RFC 6790 for an LSR claiming to
    support RFC 6790.

> then a solution exists for proper fate sharing and ordering of a TP
> LSP over MPLS.

     OK ... agreed.

> It is IMO not quiet eluciated completely.

     You seem to have figured out all of the details.

If there are additional clarifications that are needed besides the
ones suggested above, please point out what details of the solution
are still not clear in the existing text.

> If I'm unclear on any of the above, am happy to discuss...

Thanks.  Please let me know if the above clarifications are sufficient
or whether additional clarifications are needed.

> Dave

Cheers,
Curtis

btw - s/I-D.ietf-mpls-entropy-label/RFC6790/g

I didn't think it was worth reving the doc for that one change but I
do know about it.

---

From: David Allan I <[email-masked]>
Date: Wed, 23 Jan 2013 18:43:15 +0000

HI Curtis: 

Some snipping to help readability and replies prefixed by [Dave]

<snipped>

If it is unclear to you, then the document is not sufficiently clear and needs fixing.  See below.

[Dave] Great....

> Nit:
>  
> Section 1 para 2: Refers to MPLS-TP packet ordering as the constraint. 
> I'm a bit confused by this, any aggregate that shares an ordering 
> constraint does not get spread across parallel links while a set of 
> traffic COULD be in theory load spread across a set of MPLS-TP paths 
> so long as individual ordering constraints were respected. Is this 
> text document really discussing midbox spreading out of load that 

The "MPLS-TP OAM packet ordering constraints" is the "fate sharing"
which is the reason that MPLS-TP prohibits using ECMP (aka multipath, though ECMP is only one form of multipath) with MPLS-TP.  This is why an MPLS LSP (client layer) with capacity greater than a component link cannot be carried over a *single* MPLS-TP LSP.

Since this is explained in later sections I could make the substitution suggested above or I could drop the last sentence and mention that limitation later.

[Dave] the sentence I had a problem with was "but may not be carried by an MPLS-TP LSP due to the inherent MPLS-TP capacity limitation imposed by MPLS-TP OAM packet ordering constraints."
It is not an ordering constraint, it is a fate sharing constraint. It read fairly strangely to me as to what the problem was you were trying to expose. IMO the TP requirements are focused on both determinism of commitment of network resources for network planning & SLA purposes, and ability to fully and reliably instrument the connectivity. IMO multipath violates the latter, whether it violates the former would be a topic of debate unresolvable in the reality of LAG. LAG simply scoping the domain of uncertainty for OAM purposes to a link, and requiring extra link OAM and inheritance...

[Dave] To make the discussion short, replacing "OAM packet ordering constraints" with "OAM fate sharing constraints" should be sufficient.

> LSP over MPLS. It is IMO not quiet eluciated completely.

Your paragraph above describes the solution, but then says "It is IMO not quiet eluciated completely."

[Dave] Well I know you and I know the solution, but someone newer to the subject would not necessarily be able to infer it from the document in its current state.

    OK ...

      OLD:

      NEW:

[Dave] That edit works for me.

[Dave] Given the relative newness of 6790, I'm assuming some statement to the effect that this solution is not necessarily backwards compatible with existing deployments needs to be said.

> and the ingress node (which by some means SHOULD be able to know it is 
> a TP LSP as it has layer visibility)

     OK ... agreed.

If there are additional clarifications that are needed besides the ones suggested above, please point out what details of the solution are still not clear in the existing text.

> If I'm unclear on any of the above, am happy to discuss...

Thanks.  Please let me know if the above clarifications are sufficient or whether additional clarifications are needed.

[Dave] I think with the change of ordering constraint to fate sharing constraint in section 1, the EL "fixed value" addition and some disclaimer about 6790 support in the MPLS network, I would be happy. I'm clearer on what you were saying.

Best
Dave

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 23 Jan 2013 17:18:50 -0500

In message <[email-masked]>
David Allan I writes:
> [Dave] the sentence I had a problem with was "but may not be carried
> by an MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> imposed by MPLS-TP OAM packet ordering constraints."

The phrase you are objecting to is "packet ordering constraints".

Carlos suggested that I reference or quote RFC 5960 which would clear
up exactly what the dataplane requirements are.

If direct LM OAM is supported, then there is a strict packet
reordering requirement (can't reorder the LM OAM packet relative to
payload packets or accuracy suffers) so maybe I need to cite RFC 6374
Section 2.9.4 "Equal Cost Multipath" and Section 4.2.10 "Message Loss
and Packet Misorder Conditions".

> It is not an ordering constraint, it is a fate sharing constraint. It
> read fairly strangely to me as to what the problem was you were trying
> to expose. IMO the TP requirements are focused on both determinism of
> commitment of network resources for network planning & SLA purposes,
> and ability to fully and reliably instrument the connectivity. IMO
> multipath violates the latter, whether it violates the former would be
> a topic of debate unresolvable in the reality of LAG. LAG simply
> scoping the domain of uncertainty for OAM purposes to a link, and
> requiring extra link OAM and inheritance...

This is an aside, but since you brought this up I'll respond.

Begin aside.

You made two implied assertions.  1) MPLS-TP is more deterministic, 2)
MPLS-TP is necessary to reliably instrument the connectivity.

A circuit swtiched (or MPLS-TP) network is more deterministic with
respect to how much capacity is allocated.  When the capacity in
question is a large aggregate of traffic, the packet switched (or
MPLS) network is more deterministic with respect to individual
customer experience.  For example, if many LSP travel across a core
and if one city pair is exceeding its expected capactiy but others are
slidghtly underutilizing their expected capactiy, then in the circuit
switched or MPLS-TP network individual flows for that city pair
experience congestion, whereas in a packet switched or MPLS network,
capacity is shared and therefore no congestion is experienced.

In MPLS core networks the SLA traffic is a fraction of traffic and
does fine with little more than TC marking and diffserv treatment.

The determinism you describe may be an issue in metro networks, such
as where mobile backhaul is mixed with pedestrian Internet traffic and
capacity is scarse.

Kireeti Kompella and other wrote a paper about two years ago on
exactly this topic and it might help if I dug up the reference and
included in the document.

The latter point regarding reliably instrumenting the connectivity is
valid.  But if the benefit of instrumenting is for a minority of the
traffic (SLA traffic) and the cost is a reduction in network
efficiency for the vast majority of traffic (plain old Internet), then
there is a substantial increase in cost to provide the instrumented
service and then it may not be worth it.  This is why MPLS-TP carried
over MPLS is of interest.

End aside.

> [Dave] To make the discussion short, replacing "OAM packet ordering
> constraints" with "OAM fate sharing constraints" should be
> sufficient.

In practice the "OAM fate sharing constraints" and "OAM packet
ordering constraints" are almost one in the same where LAG or ECMP are
the cause of the misordering.  LM OAM requires strict packet ordering,
not just fate sharing.

I'll add the references to RFC 5960 and RFC 6374 and let those RFCs
state the requirements.

> [Dave] Well I know you and I know the solution, but someone newer to
> the subject would not necessarily be able to infer it from the
> document in its current state.

OK.  Fair enough.

> [Dave] That edit works for me.

Its in the xml source and will be in the next iteration.

> [Dave] Given the relative newness of 6790, I'm assuming some statement
> to the effect that this solution is not necessarily backwards
> compatible with existing deployments needs to be said.

OK.  Most of the draft-villamizar-mpls-multipath-extn is about fixing
the limitation that MPLS-TP LSP need to be identified in signaling and
that the ingress needs to know which links can't support MPLS-TP
requirements.

This text addresses the need to know which links can't support
MPLS-TP.

   MP#4  Where an RSVP-TE control plane is used, it MUST be possible
   	 for an ingress LSR which is setting up an MPLS-TP or MPLS LSP
   	 to determine at CSPF time whether a link or MPLS PSC LSP
   	 within the topology can support the MPLS-TP requirements of
   	 the LSP.

I'll explicitly mention that legacy LAG and legacy ECMP that do not
support ELI and EL are among the "link or MPLS PSC LSP within the
topology" that cannot "can support the MPLS-TP requirements of the
LSP."  Without protocol extensions, the ingress can still know this if
color (administrative attributes in RFC 3209) is used to identify
these links.  Its worth mentioning this, so thanks for bringing it up.

So far all I say is:

   Requirement MP#4 can be supported using administrative attributes.
   Administrative attributes are defined in [RFC3209].  Some
   configuration is required to support this.

I'll preceed that with:

   Links or MPLS LSP which traverse LAG or ECMP with adjacent LSR that
   do not support [RFC6790] cannot support MPLS-TP requirements.  This
   is the reason for including requirement MP#4.  The ingress of an
   MPLS-TP LSP must be able to avoid these links or MPLS LSP.

I think this addresses your concern.  [btw- sometimes it is better to
state what is obvious to some readers than confuse the others
readers, so I agree that this is an improvement.]

> [Dave] I think with the change of ordering constraint to fate sharing
> constraint in section 1, the EL "fixed value" addition and some
> disclaimer about 6790 support in the MPLS network, I would be
> happy. I'm clearer on what you were saying.

If the changes above don't adequately address your concerns then
please say so.

Cheers,

Curtis

---

From: David Allan I <[email-masked]>
Date: Wed, 23 Jan 2013 22:43:13 +0000

Hi Curtis

Thanks for the reply, and yes LM has a strict ordering constraint, while FM wants fate sharing, I was brain-farting on the PM/LM component of OAM when reading it and thinking FM which is the usual discussion on this list....

So now I understand the motivation for the way you had phrased the text. Not spreading an individual TP LSP out across multiple links actually satifies both so IMO there is a dual motivation for the requirement. If referring to the OAM ordering requirement, I'd suggest calling it "the strict packet ordering requirement that TP Performance monitoring requires", so it cannot be confused with simply in order delivery of OAM PDUs in isolation; the OAM PDU's relative position with respect to the interleaved real traffic actually matters.

I also have no issue with the digression on capacity allocation determinism and SLA monitoring vs. actual behavior. Other that that I would observe actual mileage observed by the different modes of operation depends on the actual traffic profile, e.g. large number of small flows vs. small number of large flows....

Cheers
D

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Wednesday, January 23, 2013 2:19 PM
To: David Allan I
Cc: [email-masked]; [email-masked]; Martin Vigoureux; [email-masked]; Loa Andersson
Subject: Re: MPLS-RT review of draft-villamizar-mpls-multipath-use

Carlos suggested that I reference or quote RFC 5960 which would clear up exactly what the dataplane requirements are.

If direct LM OAM is supported, then there is a strict packet reordering requirement (can't reorder the LM OAM packet relative to payload packets or accuracy suffers) so maybe I need to cite RFC 6374 Section 2.9.4 "Equal Cost Multipath" and Section 4.2.10 "Message Loss and Packet Misorder Conditions".

Begin aside.

You made two implied assertions.  1) MPLS-TP is more deterministic, 2) MPLS-TP is necessary to reliably instrument the connectivity.

A circuit swtiched (or MPLS-TP) network is more deterministic with respect to how much capacity is allocated.  When the capacity in question is a large aggregate of traffic, the packet switched (or
MPLS) network is more deterministic with respect to individual customer experience.  For example, if many LSP travel across a core and if one city pair is exceeding its expected capactiy but others are slidghtly underutilizing their expected capactiy, then in the circuit switched or MPLS-TP network individual flows for that city pair experience congestion, whereas in a packet switched or MPLS network, capacity is shared and therefore no congestion is experienced.

In MPLS core networks the SLA traffic is a fraction of traffic and does fine with little more than TC marking and diffserv treatment.

The determinism you describe may be an issue in metro networks, such as where mobile backhaul is mixed with pedestrian Internet traffic and capacity is scarse.

Kireeti Kompella and other wrote a paper about two years ago on exactly this topic and it might help if I dug up the reference and included in the document.

The latter point regarding reliably instrumenting the connectivity is valid.  But if the benefit of instrumenting is for a minority of the traffic (SLA traffic) and the cost is a reduction in network efficiency for the vast majority of traffic (plain old Internet), then there is a substantial increase in cost to provide the instrumented service and then it may not be worth it.  This is why MPLS-TP carried over MPLS is of interest.

End aside.

> constraints" with "OAM fate sharing constraints" should be sufficient.

In practice the "OAM fate sharing constraints" and "OAM packet ordering constraints" are almost one in the same where LAG or ECMP are the cause of the misordering.  LM OAM requires strict packet ordering, not just fate sharing.

I'll add the references to RFC 5960 and RFC 6374 and let those RFCs state the requirements.

> > stated) are the only mechanisms of ensuring proper treatment (with 
> > all transit nodes implementing entropy and ELI processing such that 
> > node (which by some means SHOULD be able to know it is a TP LSP as 
> > it has layer visibility) and the egress node can strip entropy and 
> > ELI then a solution exists for proper fate sharing and ordering of a 
> > TP LSP over MPLS. It is IMO not quiet eluciated completely.
>  

OK.  Fair enough.

> compatible with existing deployments needs to be said.

OK.  Most of the draft-villamizar-mpls-multipath-extn is about fixing the limitation that MPLS-TP LSP need to be identified in signaling and that the ingress needs to know which links can't support MPLS-TP requirements.

This text addresses the need to know which links can't support MPLS-TP.

I'll explicitly mention that legacy LAG and legacy ECMP that do not support ELI and EL are among the "link or MPLS PSC LSP within the topology" that cannot "can support the MPLS-TP requirements of the LSP."  Without protocol extensions, the ingress can still know this if color (administrative attributes in RFC 3209) is used to identify these links.  Its worth mentioning this, so thanks for bringing it up.

So far all I say is:

I'll preceed that with:

I think this addresses your concern.  [btw- sometimes it is better to state what is obvious to some readers than confuse the others readers, so I agree that this is an improvement.]

> disclaimer about 6790 support in the MPLS network, I would be happy. 
> I'm clearer on what you were saying.

If the changes above don't adequately address your concerns then please say so.

Cheers,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 24 Jan 2013 19:24:45 -0500

Dave,

I changed the wording on one of the changes.  See below.

Regards,

Curtis

In message <[email-masked]>
> In message <[email-masked]>

[ ... huge snip ... ]

> readers, so I agree that this is an improvement.]

I replaced the suggested new paragraph above with:

   An MPLS-TP LSP may not traverse multipath links on the path where
   MPLS-TP forwarding requirements cannot be met.  Such links include
   any using pre-RFC6790 Ethernet Link Aggregation, pre-RFC6790 Link
   Bundling using the all-ones component link, or other form of
   multipath not supporting termination of the entropy search at the
   EL label as called for in [RFC6790].  An MPLS-TP LSP must not
   traverse a server layer MPLS LSP which traverses any form of
   multipath not supporting termination of the entropy search at the
   EL label.  For this to occur, the MPLS-TP ingress LSR must be aware
   of these links.  This is the reason for requirement MP#4.

This is a bit verbose, but "LAG and ECMP" in the original suggested
wording isn't quite right and there was no mention of avoiding the FA
for server layer LSP that traversed these "bad links".

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 24 Jan 2013 19:38:32 -0500

Dave,

When explicitly referencing the requirements, there was enough to
create a small subsection (3.1).  So now section 3 is:

   3.  MPLS as a Server Layer for MPLS-TP
     3.1.  MPLS-TP Forwarding and Server Layer Requirements
     3.2.  Methods of Supporting MPLS-TP client LSPs over MPLS

In section 3.2 I now use the phrase "MPLS-TP forwarding requirements"
and refer back to section 3.1 rather than using either the phrase
"MPLS-TP OAM fate-sharing requirements" or the phrase "MPLS-TP packet
ordering requirements" in section 3.2.

I did some rewording and additions to improve clarity and be more
specific in section 3, as requested in general comments by reviewers.
I'll send annotated diffs in a little while so that you and other
MPLS-RT expert reviewers can make sure all comments (so far) are
addressed and the changes are to your liking.

Regards,

Cuertis

In message <[email-masked]>
David Allan I writes:

Hi Curtis

Cheers
D

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Wednesday, January 23, 2013 2:19 PM
To: David Allan I
Cc: [email-masked]; [email-masked]; Martin Vigoureux; [email-masked]; Loa Andersson
Subject: Re: MPLS-RT review of draft-villamizar-mpls-multipath-use

Begin aside.

End aside.

OK.  Fair enough.

So far all I say is:

I'll preceed that with:

Cheers,

Curtis

---

From: David Allan I <[email-masked]>
Date: Fri, 25 Jan 2013 00:47:35 +0000

The set of changes you describe in the last couple of emails addresses all my concerns...

Thanks
Dave 

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Thursday, January 24, 2013 4:39 PM
To: David Allan I
Cc: [email-masked]; [email-masked]
Subject: Re: MPLS-RT review of draft-villamizar-mpls-multipath-use

Dave,

When explicitly referencing the requirements, there was enough to create a small subsection (3.1).  So now section 3 is:

In section 3.2 I now use the phrase "MPLS-TP forwarding requirements"
and refer back to section 3.1 rather than using either the phrase "MPLS-TP OAM fate-sharing requirements" or the phrase "MPLS-TP packet ordering requirements" in section 3.2.

I did some rewording and additions to improve clarity and be more specific in section 3, as requested in general comments by reviewers.
I'll send annotated diffs in a little while so that you and other MPLS-RT expert reviewers can make sure all comments (so far) are addressed and the changes are to your liking.

Regards,

Cuertis

Hi Curtis

Cheers
D

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]]
Sent: Wednesday, January 23, 2013 2:19 PM
To: David Allan I
Cc: [email-masked]; [email-masked]; Martin Vigoureux; [email-masked]; Loa Andersson
Subject: Re: MPLS-RT review of draft-villamizar-mpls-multipath-use

Begin aside.

End aside.

OK.  Fair enough.

So far all I say is:

I'll preceed that with:

Cheers,

Curtis

---

From: Markus Jork <[email-masked]>
Date: Fri, 25 Jan 2013 04:11:38 +0000

I have been asked to review draft-villamizar-mpls-multipath-use.

I think this is a useful document. It focuses in on an area that deserves attention and has not otherwise been covered. The initial terminology definition section provides a good basis for discussion and gives a good overview of the technology space.

The current version of the draft would benefit from fixing some typos and grammar to make it more readable.
As other reviewers have pointed out already, section 3 could be improved over time by adding some more details and clarifications (which Curtis is already providing now). Nevertheless, I believe even as is, the document can be considered for WG adoption now.

-Markus