Subject: OPS-Dir review of draft-ietf-mpls-multipath-use-03

From: "Black, David" <[email-masked]>
Date: Sun, 19 Jan 2014 21:05:17 -0500

I have reviewed this document as part of the operations directorate's
ongoing effort to review all IETF documents being processed by the
IESG.  These comments were written primarily for the benefit of the
operations area directors.  Document editors and WG chairs should treat
these comments just like any other last call comments.

Summary:
This draft is on the right track but has open issues, described in the review.

I apologize for this review showing up a couple of days after the end of
IETF Last Call.

This draft discusses considerations for what are effectively MPLS-in-MPLS
tunnels for multipathing - an LSP carries another LSP, with the inner
(client) LSPs being distributed across multiple outer (server) LSPs that
use different paths.  This draft is motivated by considerations specific
to MPLS-TP that complicate multipathing when one of the two MPLS instances
is MPLS-TP.

I consider the following to be open issues that need attention (each
letter is used to tag the issue in the text that follows - [B] occurs
multiple times, as there are multiple aspect to that issue):

[A] Summary text in Introduction appears to be wrong.
[B] Section 3.2 needs a summary.
[C] Entropy label "sandwich" seems peculiar and at least needs an explanation.
[D] Discussion of multipath impacts on MPLS OAM should be added to Section 4.

--- Comments on draft text

[A] Section 1 (Introduction) 2nd paragraph:

   RFC 5654 requirement 33 requires the capability to carry a client
   MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654].
   This is possible in all cases with one exception.  When an MPLS LSP
   exceeds the capacity of any single component link it MAY be carried
   by a network using multipath techniques, but MAY NOT be carried by a
   single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
   imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
   fate sharing constraints and MPLS-TP LM OAM packet ordering
   constraints (see Section 3.1).

Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above
exception appears to concern the reverse, and Section 4 on carrying MPLS
over MPLS-TP allows for MPLS LSPs that exceed component link capacity.
Was the exception intended to be for an MPLS-TP LSP that exceeds the
capacity of any single component link?

Also, "MAY NOT" is not an RFC 2119 term (for good reason - it's equivalent
to "MAY OR MAY NOT").  I think "cannot" was intended her, but "MUST NOT"
is another possibility.

[B] Section 3.2 contains a lot of useful details, but it could really use
a summary at the end on the options and recommendations for meeting the
four requirements (this summary could be a new Section 3.3).  It's a bit
difficult to discern what's going on, as I had to read this section more
than once in order to discern the following:

[C] In Section 3.2, use of an MPLS entropy label to meet requirement MP#1
requires that it be "below the MPLS-TP label" whereas, to meet requirement
MP#2, the MPLS entropy label needs to be "just above the MPLS-TP LSP label".
This combination appears to suggest "sandwiching" the MPLS-TP label between
two entropy labels that effectively carry the same information (albeit,
applied and removed at different places in the network) - was that intended?
This is important, as there seems to be no reasonable alternative
to the entropy label for meeting requirement MP#2, and the draft's
alternative to the entropy label for meeting requirement MP#1 is "some form
of configuration" - the entropy label would seem to be preferred to that
based on my reading of this draft and the comment in RFC 5706 section 2.2
that "Anything that can be configured can be misconfigured."

[B] A summary of requirements for implementation and deployment at the end
of Section 3.2 would have made this much more obvious.  That should be
provided in addition to addressing the above question about duplicate use
of the MPLS entropy label.

--- Relevant Q&A based on RFC 5706 Appendix A

I'm omitting management concerns, as both MPLS and MPLS-TP have extensive
management infrastructure, and this draft's notion of stacking LSPs so
that one instance of MPLS (or MPLS-TP) can be a server to (i.e., carry)
another is not novel to this draft.

A.1.1.  Has deployment been discussed?

[B] Yes, although Section 3.2 is weak on deployment requirements (some of
which are implementation requirements) and could use a summary.

A.1.2.  Has installation and initial setup been discussed?

No, although because this draft reuses existing functionality in different
configurations, this really reduces to the A.1.9 configuration question
below.  In practice, quite a bit of operator knowledge and insight is
required to get initial setup right, e.g., as traffic engineering LSPs to
avoid congestion problems (needed to meet requirement MP#3) requires
significant knowledge of network structure and expected traffic flows.
This should be obvious to anyone who works w/MPLS, but it's probably
worth stating for completeness.

A.1.6.  Have suggestions for verifying correct operation been discussed?

Yes and no.  Section 3.2 discusses OAM, and contains this important
requirement for OAM traffic:

   An Entropy Label must be used to insure that all of the
   MPLS-TP payload and OAM traffic are carried on the same component,
   except during very infrequent transitions due to load balancing.

[D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in
Section 4; that should be added.

A.1.9.  Is configuration discussed?

[B] Not much. Configuration is required to meet requirements MP#3 and MP#4,
and is one of the options for MP#1.  A summary of what has to be configured
as part of the summary at the end of Section 3.2 would be a good idea.

A.2.* [skipped, see above]

A.3 Documentation

There's quite a bit of operational material contained in this draft; a
separate section on operations and management considerations would not
be appropriate.

There is a useful implementation status section which appears to imply
that MPLS-TP over MPLS is not currently deployable due to absence of
entropy label support, and because deployed MPLS equipment does not
generally support multipathing.  Nonetheless, data center experience
indicates significant value and widespread use of multipathing based
on link aggregation and ECMP; corresponding benefits can be expected
for use of multipathing in the Internet beyond data centers.

--- Editorial comments/nits:

- Section 3.1, after RFC 5960 quotes:

   [RFC5960] paragraph 3 requires that packets within a specific ordered

Insert "Section 3.1.1., before "paragraph 3" and likewise for "paragraph 6"
later in the same paragraph in the draft.

- Section 3.2, requirement MP#3:

   MP#3  It SHOULD be possible to insure that MPLS-TP LSPs will not be

"insure" -> "ensure" or "assure".  Surely this draft does not envision
operators taking out insurance policies on MPLS TP LSP behavior ;-).

- Section 3.2, last paragraph on p.8:

   An MPLS-TP LSP may not traverse multipath links on the path where
   MPLS-TP forwarding requirements cannot be met.

"may not" is meaningless - I believe "MUST NOT" was intended here.

- Section 4, middle of 3rd para:

Editorial suggestions for clarity:

OLD
            For those LSP that are larger than component link
    capacity, their capacity are not integer multiples of convenient
    capacity increments such as 10Gb/s.
NEW
            For those LSPs that are larger than a component link
   capacity, the LSP capacities need not be (and often are not)
   integer multiples of convenient capacity increments such as 10Gb/s.

In particular, "are not" seems wrong, as the "integer multiples"
case is possible, so I changed "are not" to "need not be (and often
are not)" in the suggested new text.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 20 Jan 2014 10:41:51 -0000

Thanks David,
Any review is better than no review :-)

Authors, this document is on Thursday's telechat and I would like to continue
with that plan.
Are you able to respond to David's review in double-quick time?

Thanks,
Adrian

> -----Original Message-----
> From: ietf [mailto:[email-masked]] On Behalf Of Black, David
> Sent: 20 January 2014 02:05
> To: [email-masked]; [email-masked]
> Cc: [email-masked]; Black, David
> Subject: OPS-Dir review of draft-ietf-mpls-multipath-use-03
>

---

From: "Black, David" <[email-masked]>
Date: Tue, 21 Jan 2014 19:02:33 -0500

Curtis,

> Thanks for the review and for pointing out lack of clarity due to
> underlying assumptions about the use of RFC 4206 (and perhaps use of
> RFC 4201 as well for large MPLS LSP over MPLS-TP).
>
> Sometimes it is too easy to forget that not all readers are going to
> be as steeped in MPLS "as deployed today" details as a relatively
> small number of long time MPLS participants are.
>
> Adding a few paragraphs to the introduction and a few concluding
> paragraphs may significantly improve clarity for people who have not
> been following MPLS closely for a decade or more without requiring
> them to read the references and figure out usage details themselves.
>
> Please let me know how you feel about the suggested additions to the
> document.

Most of them look good - details below.  Many thanks for the prompt and
comprehensive response.  I have a few more suggestions in what follows
- anything that I don't comment on below is fine with me.

-- Introduction

> There is an underlying assumption in the draft that the reader
> understands that MPLS-in-MPLS exists today, most commonly but not
> exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> to make use of the management plane, using CLI or NETCONF or the MPLS
> cross connect MIB to program a multiple label MPLS push operation.
>
> If a very experienced reviewer could miss that, then the introduction
> could be improved by better stating assumptions.  It is also the case
> that many participants in MPLS, particularly those primarily
> interested in MPLS-TP, began participation long after RFC 4206 was
> written and would benefit by better stated assumptions in the intro.
>
> I suggest that the following paragraphs be added to the introduction.

The suggested new paragraphs will definitely improve the introduction.

-- [A] Summary text in Introduction appears to be wrong.

> > capacity of any single component link?

[... snip ...]

> To answer your question directly: the exception is a very large MPLS
> LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> this is occurring.

It looks like  my confusion stemmed from the original text only
discussing only what won't work and referencing a section that
concerns a different encapsulation order than the one that causes
the problem.  Including the "MAY NOT" -> "SHOULD NOT" change, I suggest:

OLD
   When an MPLS LSP
   exceeds the capacity of any single component link it MAY be carried
   by a network using multipath techniques, but MAY NOT be carried by a
   single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
   imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
   fate sharing constraints and MPLS-TP LM OAM packet ordering
   constraints (see Section 3.1).
NEW
   When an MPLS LSP
   exceeds the capacity of any single component link it MAY be carried
   by a network using multipath techniques, but SHOULD NOT be carried by a
   single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
   imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
   fate sharing constraints and MPLS-TP LM OAM packet ordering
   constraints (see Section 3.1).  Instead, multiple MPLS-TP LSPs SHOULD
   be used to carry a large MPLS LSP (see Section 4).

-- [B] Section 3.2 needs a summary.

The proposed summary looks good, and helps with [C], but the phrase
"access and edge" needs qualification in the following sentence:

   However access and edge is the least likely to support
   RFC 6790 in the near term.

"access and edge" what? e.g., areas of the network? LSRs?

-- [C] Entropy label "sandwich" seems peculiar and at least needs an explanation.

> The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> paragraph.
>
>    Alternately, the need for requirement MP#1 can be eliminated if
>    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
>    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
>    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
>    in a deployment support Entropy Label, which may render it
>    impractical in many deployments.
>
> Please not the "Alternately" at the beginning and "impractical in many
> deployments" at the end.
>
> The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> above that but assumes knowledge of RFC 6790.  I think the following
> change within this paragraph would clear things up:

That change is helpful, but my "sandwich" concern was actually raised
by text two paragraphs further down:

   MPLS-TP LSPs can be carried as client LSPs within an MPLS server LSP
   if an Entropy Label Indicator (ELI) and Entropy Label (EL) is pushed
   below the server layer LSP label(s) in the label stack, just above
   the MPLS-TP LSP label entry [RFC6790].

Read after the "Alternately" paragraph, this text appeared to suggest
a "sandwich" that would have an ELI/EL pair both above and below the
MPLS-TP label.  That was clearly not intended, so I suggest inserting
"When Entropy Label Indicators (ELIs) and Entropy Labels (ELs) are not
applied by MPLS-TP ingresses," before the start of the first sentence
above, so that the result is:

   When Entropy Label Indicator (ELIs) and Entropy Labels (ELs) are
   not applied by MPLS-TP ingresses, MPLS-TP LSPs can be carried as
   client LSPs within an MPLS server LSP if an ELI and EL are pushed
   below the server layer LSP label(s) in the label stack, just above
   the MPLS-TP LSP label entry [RFC6790].

-- [D] Discussion of multipath impacts on MPLS OAM

> > Section 3.2 discusses OAM, and contains this important
> There is no impact on MPLS OAM traffic.  There are three cases to
> consider.
>
>   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
>      one link.  MPLS OAM is unaffected and need not use multipath
>      support in LSP Ping.
>
>   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
>      LSP is carried over one link thanks to the fixed EL value.
>      MPLS-TP OAM is unaffected.
>
>   3.  Where MPLS is carried over MPLS (an existing case) or over
>       multiple MPLS-TP, the multipath support in LSP Ping (as
>       imperfect as it may already be) is used and LSP Ping (RFC 4379)
>       operation is unaffected.
>
> If needed something can be added to state the above (maybe with less
> informal wording).  What is obvious to some people may not be obvious
> to everyone.  Briefly stating the almost obvious is usually not a bad
> thing.

Points 1-3 are useful explanation as complement to the Entropy label
requirement to correctly handle OAM traffic for the other encapsulation.

I like your suggestion to add this text as part of a new concluding
Section 5, and the rest of the proposed conclusion section text looks
good.

Thanks,
--David

> From: Curtis Villamizar [mailto:[email-masked]]
> Sent: Tuesday, January 21, 2014 2:23 PM
> To: Black, David
> Cc: [email-masked]; [email-masked]; draft-ietf-mpls-multipath-
> [email-masked]
> Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03
>
>
> In message <[email-masked]>
> "Black, David" writes:
> >
> Skipping to the discussion quite a ways down, there seems to be some
> confusion about what is being proposed here.  While MPLS-in-MPLS is
> being used, no new form of MPLS-in-MPLS is being proposed.
>
> There is an underlying assumption in the draft that the reader
> understands that MPLS-in-MPLS exists today, most commonly but not
> exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> to make use of the management plane, using CLI or NETCONF or the MPLS
> cross connect MIB to program a multiple label MPLS push operation.
>
> If a very experienced reviewer could miss that, then the introduction
> could be improved by better stating assumptions.  It is also the case
> that many participants in MPLS, particularly those primarily
> interested in MPLS-TP, began participation long after RFC 4206 was
> written and would benefit by better stated assumptions in the intro.
>
> I suggest that the following paragraphs be added to the introduction.
>
>    MPLS LSPs today are able to function as a server layer and carry
>    client MPLS LSPs.  When control plane signaling is used, forwarding
>    adjacency (FA) advertisements are used to inform the set of LSR of
>    Packet Switching Capable (PSC) LSP within the MPLS topology
>    [RFC4206].  Client MPLS LSP at a higher layer (lower PSC number)
>    may signal their intention to use PSC LSP as hops in the RSVP-TE
>    Explicit Route Object (ERO).  LSR with no explicit support for RFC
>    4206 see the PSC LSP as ordinary links and therefore use them.
>
>    An MPLS LSP that has been set up using RSVP-TE appears to its
>    ingress LSR as a viable IP next hop to a distant LSR.  If LDP is
>    used and bidirectional RSVP-TE LSP connectivity is available, then
>    LDP signaling can be set up among the RSVP-TE LSP endpoints and LDP
>    can make use of the RSVP-TE LSP as an LDP hop.  This is another
>    form of existing MPLS-in-MPLS use.
>
>    MPLS LSPs may also make use of hierarchy that is configured through
>    the management plane rather than signaled using RSVP-TE.
>
>    These existing forms of MPLS-in-MPLS may traverse multipath hops
>    such as Ethernet LAG [IEEE802.1AX] or MPLS Link Bundling [RFC4201].
>    MPLS-TP brings with it a new set of requirements not considered in
>    past deployments of the various forms of MPLS-in-MPLS where
>    multipath was in use.  This document merely discusses use of
>    existing forwarding and protocol mechanisms that can support the
>    case where either the client layer LSPs or the server layer LSPs
>    are MPLS-TP and where multipath is used.
>
> If there are no objections I will add these paragraphs.
>
> Now on to the specific comments.
>
> There is no problem today with MPLS LSP carried by MPLS-TP LSP except
> the existing case where MPLS LSP capacity is too big.  For example, a
> 500 Gb/s LSP is possible today carried over N x 100 Gb/s or N x 10
> Gb/s multipath links but could not be carried over a single MPLS-TP
> LSP which would be limited to a single 100 Gb/s or 10 Gb/s link.  So
> the exception is where the MPLS LSP is too big to fit in any possible
> single MPLS-TP LSP.
>
> To answer your question directly: the exception is a very large MPLS
> LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> this is occurring.
>
> > to "MAY OR MAY NOT").  I think "cannot" was intended here, but "MUST NOT"
> > is another possibility.
>
> Good point.  I suggest we replace the MAY NOT with SHOULD NOT.  Many
> people would insist that it should be MUST NOT, but in reality
> networks SHOULD NOT be congested but sometimes they are.
>
> OK.  A Summary paragraph can be added.  See below.  Perhaps conclusion
> section can be added.  See far below.
>
> The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> paragraph.
>
>    Alternately, the need for requirement MP#1 can be eliminated if
>    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
>    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
>    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
>    in a deployment support Entropy Label, which may render it
>    impractical in many deployments.
>
> Please not the "Alternately" at the beginning and "impractical in many
> deployments" at the end.
>
> The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> above that but assumes knowledge of RFC 6790.  I think the following
> change within this paragraph would clear things up:
>
>  OLD
>
>    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
>    and Entropy Label (EL) are added to the label stack [RFC6790].  For
>    those client LSP that are MPLS-TP LSP, a single EL value must be
>    chosen.  For those client LSP that are MPLS LSP, per packet entropy
>    below the top label must, for practical reasons, be used to
>    determine the entropy label value.
>
>  NEW
>
>    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
>    and Entropy Label (EL) are added to the label stack by the midpoint
>    LSR for the client MPLS-TP LSP, at the ingress of the MPLS LSP
>    [RFC6790].  For those client LSP that are MPLS-TP LSP, a single EL
>    value must be chosen.  For those client LSP that are MPLS LSP, per
>    packet entropy below the top label must, for practical reasons, be
>    used to determine the entropy label value.  The resulting label
>    stack contains the server MPLS LSP label, ELI, EL and the client
>    LSP label.
>
> The above rewording is simply more explicit about the label stack
> order and which LSR puts the ELI and EL in the label stack.
>
> The following summary can be added to the end of Section 3.2:
>
>    Two methods of adding an Entropy Label are described above.  The
>    MPLS-TP ingress must have a means to determine which links can
>    support MPLS-TP in selecting a path (MP#4).  Administrative
>    attributes can satisfy that requirement.  If the MPLS-TP LSR is
>    capable of adding ELI/EL to the label stack, this method is
>    preferred.  However access and edge is the least likely to support
>    RFC 6790 in the near term.  For portions of the topology where an
>    MPLS-TP is carried within a server layer MPLS LSP the ingress of
>    the server layer MPLS LSP can add ELI/EL using a fixed EL value per
>    client LSP, except those known not to require MPLS-TP treatment.
>    There are numerous ways to determine which client LSP are MPLS-TP
>    LSP and which are not.  While this determination is out of scope
>    and will vary among deployments, configuration or the presence of
>    specific attribute affinities in RSVP-TE signaling are among the
>    likely means to do so.
>
> If there are no objections I will add the above paragraph.
>
> The use of RFC 4206 is assumed throughout the discussion of signaling
> in section 3.2.  This assumption is made more clear by suggested
> additions to the introduction.
>
> FA advertisements defined in RFC 4206 make it possible for ingress to
> signal viable paths.  Either the use of Ethernet Link Aggregation or
> MPLS Link bundling using the all-ones component link (RFC 4201) allows
> the total capacity of a multipath to be advertised.  Meeting the
> requirement to balance traffic over the component links in a multipath
> (MP#3) is then a local matter and requires no further signaling
> extensions.
>
> There is no impact on MPLS OAM traffic.  There are three cases to
> consider.
>
>   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
>      one link.  MPLS OAM is unaffected and need not use multipath
>      support in LSP Ping.
>
>   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
>      LSP is carried over one link thanks to the fixed EL value.
>      MPLS-TP OAM is unaffected.
>
>   3.  Where MPLS is carried over MPLS (an existing case) or over
>       multiple MPLS-TP, the multipath support in LSP Ping (as
>       imperfect as it may already be) is used and LSP Ping (RFC 4379)
>       operation is unaffected.
>
> If needed something can be added to state the above (maybe with less
> informal wording).  What is obvious to some people may not be obvious
> to everyone.  Briefly stating the almost obvious is usually not a bad
> thing.  Please tell me if a small subsection on "Imopact on MPLS and
> MPLS-TP OAM" is needed.
>
> Note that the limitations of LSP Ping are currently being discussed in
> the MPLS WG in the context of extensions to LSP Ping to better support
> Entropy Label.  A few limitations not related to EL may get fixed in
> the process.
>
> Perhaps a worked example appendix could be added but I don't think
> this is common and may not be necessary with some of the
> clarifications suggested so far.
>
> If you would like I can add a conclusion section as Section 5 with
> these paragraphs:
>
>    MPLS equipment deployed in the core currently supports multipath.
>    For large service providers, core LSR must support some form of
>    multipath to be deployable.  Deployed MPLS access and edge
>    equipment is often oblivious to the use of multipath in the core.
>    It is expected that at least first generation MPLS-TP equipment
>    will be oblivious to the use of multipath in the core.  This first
>    generation MPLS-TP equipment is deployable in a core using
>    multipath, with no adverse impact to RSVP-TE signaling, if the edge
>    equipment can support administrative attributes (RFC 3209) and the
>    core equipment can support ELI/EL and put a per-LSP fixed EL value
>    on any LSP that indicates a particular attribute affinity or can
>    identify client MPLS-TP LSP through some other means.
>
>    There are no issues carrying MPLS over MPLS-TP, except when the
>    MPLS LSP is too big to be carried by a single MPLS-TP LSP.  Most
>    MPLS core equipment and some edge equipment can configure an MPLS
>    Link Bundle [RFC4201] over multiple component links where the
>    component links are themselves MPLS LSP.  This existing capability
>    can be used to carry large MPLS LSP and overcome the limited
>    capacity of any single server layer MPLS-TP LSP.
>
> If this conclusion section is added, then the three bullets above
> regarding impact on OAM can be put there.  The three bullets can be
> added as follows:
>
>    MPLS OAM and MPLS-TP OAM are unaffected in the following cases
>    proposed in this document:
>
>      1. Where MPLS is carried over a single MPLS-TP all traffic flows
>         on one link.  MPLS OAM is unaffected and need not use
>         multipath support in LSP Ping.
>
>      2. Where MPLS-TP is carried over MPLS, all traffic for that
>         MPLS-TP LSP is carried over one link thanks to the fixed EL
>         value.  MPLS-TP OAM is unaffected.
>
>      3.  Where MPLS is carried over MPLS (an existing case) or over
>          multiple MPLS-TP, the multipath support in LSP Ping is used
>          and LSP Ping (RFC 4379) operation is unaffected.
>
> I think adding the above as a conclusion section as Section 5 would
> improve clarity and therefore is worth doing.
>
> > Insert "Section 3.1.1.", before "paragraph 3" and likewise for "paragraph 6"
> GenArt review caught this.  Done.
>
> Good catch.  I think "ensure" is the right word.
>
> I suggested SHOULD NOT above.
>
> Good suggestion.  GenArt review suggested adding "generally" making
> the statement read "their capacity are generally not integer
> multiples".  Your suggested rewording may be more clear.
>
> Also "10Gb/s" should be "10 Gb/s".
>
> Thanks for the review and for pointing out lack of clarity due to
> underlying assumptions about the use of RFC 4206 (and perhaps use of
> RFC 4201 as well for large MPLS LSP over MPLS-TP).
>
> Sometimes it is too easy to forget that not all readers are going to
> be as steeped in MPLS "as deployed today" details as a relatively
> small number of long time MPLS participants are.
>
> Adding a few paragraphs to the introduction and a few concluding
> paragraphs may significantly improve clarity for people who have not
> been following MPLS closely for a decade or more without requiring
> them to read the references and figure out usage details themselves.
>
> Please let me know how you feel about the suggested additions to the
> document.
>
> Curtis

---

From: "Black, David" <[email-masked]>
Date: Tue, 21 Jan 2014 21:23:24 -0500

> I think we are in agreement and I will edit the document to reflect
> your comments shortly.  I think Adrian is AD on this and Loa the
> shepard, so I'll wait for a go-ahead from either of them before
> submiting the edits.

Yes, we're in agreement, and indeed, "sandwich" is a term I coined in my
review - it seemed descriptive of an MPLS-TP label in a stack between
two ELI/EL pairs.

Thanks,
--David

> Sent: Tuesday, January 21, 2014 8:08 PM
> To: Black, David
> Cc: [email-masked]; [email-masked]; [email-masked]; draft-ietf-mpls-
> [email-masked]
> In message <[email-masked]>
> "Black, David" writes:
> >
> > Curtis,
> >
> Thank you for the quick turnaround on this as well.
>
> > -- Introduction
> >
> OK.  Good addition.
> Oh.  Good point.
> Core, edge, aggregation, and access are terms commonly used for the
> tiers in provider networks.  Core is central, high traffic.  Access is
> furthest out with lower traffic (think of subscriber connections for
> consumer Internet).
>
> Perhaps I can just change the sentence to:
>
>   However equipment furthest from a provider's network core is the
>   least likely to support RFC 6790 in the near term.
>
> I think we all know "core" at this point so furthest from core might
> be easier to understand.
>
> That was your term.  I didn't propose adding Entropy label "sandwich"
> to the text.  I think this is just an excerpt from your prior email as
> a marker in following up on it.  Please correct me if I am wrong.
>
> That should have said "Please note".
>
> OK.  I will make that replacement.
>
> Thanks again.
> I think we are in agreement and I will edit the document to reflect
> your comments shortly.  I think Adrian is AD on this and Loa the
> shepard, so I'll wait for a go-ahead from either of them before
> submiting the edits.
>
> Curtis
>
>
> > > > [D] Discussion of multipath impacts on MPLS OAM should be added to
> Section
> > > 4.
> > > >

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 21 Jan 2014 14:23:27 -0500

OLD

 NEW

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 21 Jan 2014 20:07:47 -0500

OK.  Good addition.

Oh.  Good point.

Thanks again.

Curtis