Subject: [mpls] AD review of draft-ietf-mpls-forwarding

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 26 Jan 2014 18:02:08 -0000

Hi,

Thank you for a really thorough and clear document.

I have done my usual AD review upon receiving the publication request 
for this document. The purpose of the review is to catch any issues that
might otherwise show up in IETF last call and IESG evaluation. In this
case it is also an extra pair of eyes on what is a very detailed 
document.

I have a number of comments below. A few are editorial nits (sorry) and
some are questions for clarification of the text. There is very little 
where I come even remotely close to saying "wrong" or missing".

It looks to me that a new revision would be useful just to mop up these
comments, so I have put the document into "Revised I-D Needed" state and
will wait to see the next revision before starting the IETF last call.
Please debate any issues where you disagree with me or think that no
document change is needed.

Thanks for the work,
Adrian

===

idnits notes that...

  == Outdated reference: draft-ietf-pwe3-vccv-impl-survey-results has been
     published as RFC 7079

---

Andy may want to update his coordinates.

---

Your acronym list is commendably thorough, but a little enthusiastic.

AC is only used once and then only at the point of expansion.
CE doesn't appear to be used at all
FEC only seems to be used in this document in the context of 
  Forwarding Equivalence Classes (LDP)

The following are all well-known acronyms according to
  http://www.rfc-editor.org/rfc-style-guide/abbrev.expansion.txt
  so they don't need to be included
BGP
CPU
DDoS
DoS
GMPLS
IANA
IP
IPv4
IPv6
LDP
MPLS
NTP
QoS
RTP
TCP
UDP
WG

I think Curtis may have heard this before :-)
The "preferred" (by the RFC editor) expansion of ECMP is
"Equal-Cost Multipath"

Several acronyms are qualified with (LDP) (CE, FEC, P, PE). I haven't
checked the usage in this document, but it seems to me that the terms
might have wider applicability. Indeed, a quick search revealed 
"RSVP-TE PE".

The third expansion of PSC could have a reference to RFC 6378. I think
that to avoid confusion, you need to expand this in the text when you 
use it. You do this well everywhere except in Section 4.7 T#32, and in
this acronym list itself at E-LSP and L-LSP.

---

Section 1.3 bullet 5

   5.  The implementer and system designer MUST support pseudowire
       control word (CW) if MPLS-TP is supported or if ACH [RFC5586] is
       being used on a pseudowire.

The wording is a bit odd. "The implementation and system design..."?

Ditto bullets 6 and 7

---

Section 1.3

While there is not wrong with the statements made in the bullets, some
of the later ones refer to recent additions to the MPLS suite. Yet the
list is presented as "there were some misconceptions." Clearly the
early silicon did not have misconceptions about the inclusion of entropy
labels. 

Just tweak the words at the top of the list?

---

Section 2.1

   Tunneling encapsulations which may carry MPLS, such as MPLS in GRE,
   L2TP, or LDP, are out of scope.

I think s/LDP/UDP/

---

2.1.1

Maybe the first paragraph should clarify "special purpose labels at 
the top of the label stack"

---

2.1.1

I think that this section should note that labels 0-15 were commonly
referred to as "reserved labels" and are only renamed to "special
purpose labels" by [I-D.ietf-mpls-special-purpose-labels].

---

2.1.4

   MPLS hierarchy as described in [RFC4206] can in principle add up to
   four additional labels.  MPLS hierarchy is discussed in
   Section 2.1.6.

Similar text in 2.1.6

I wonder whether the choice of "four" here is because RFC 4203 defines
      1     Packet-Switch Capable-1 (PSC-1)
      2     Packet-Switch Capable-2 (PSC-2)
      3     Packet-Switch Capable-3 (PSC-3)
      4     Packet-Switch Capable-4 (PSC-4)
If that is the case, we should note that RFC 7074 deprecates PSC-2 
through 4 recognising that different switching levels within the PSC
world were not applied and that LSP nesting was arbitrary and not 
limited to four levels.

On the other hand, if "four" comes from somewhere else, perhaps you 
could give a clue in the text.

---

While per platform label space is mentioned in 2.1.7 I wonder whether
more information on per platform and per interface label spaces is 
needed. I recall early implementations that got very confused when
parallel interfaces used the same label for different purposes.

I guess the point there is that you cannot assume that your neighbor
is or is not using the per platform label space.

Upstream label allocation may also come into this.

---

2.1.8.1

   1.  The most common case is where reordering occurs is rare,

One too many instances of "is"

---

2.1.8.1

   3.  If the edge is not using pseudowire control word (CW) and the
       core is using multipath, reordering will be far more common.  If
       this is occurring, the best solution is to use CW on the edge,
       rather than try to fix the reordering using resequencing.

Completely agree, but isn't the sequence number contained in a control
word meaning that the resequencing could, in any case, not be done 
without using a control word?

---

2.1.8.1

   4.  Another avoidable case is where some core equipment has multipath
       and for some reason insists on periodically installing a new
       random number as the multipath hash seed.  If supporting MPLS-TP,
       equipment MUST provide a means to disable periodic hash reseeding
       and deployments MUST disable periodic hash reseeding.  Even if
       not supporting MPLS-TP, equipment should provide a means to
       disable periodic hash reseeding and deployments should disable
       periodic hash reseeding.

Are those two "should" really "SHOULD"?

---

Should 2.2 distinguish the order of magnitude of replication at branch
nodes? This impacts the replication method used (some devices make a
copy and cycle around, some devices can do multiple copies at once). On
the whole is no different from IP multicast processing except (as you
note) that each outgoing packet may be different by its label value.

---

2.4

So obvious you didn't say it?

   In order to support an adequately balanced load distribution across
   multiple links, IP header information must be used.  Common practice
   today is to reinspect the IP headers at each LSR and use the label
   stack and IP header information in a hash performed at each LSR.
   Further details are provided in Section 2.4.5.

Missing is the statement that a single "flow" must not be distributed 
across multiple paths because of the implication for potentially
significant packet misordering. And feeding that is a common requirement
that such packet misordering must not occur because applications and
transport protocol implementations cannot survive such misordering.

---

2.4.2 uses "composite link" and "component link". I suggest picking just
one term.

---

2.4.5.1 notes that special purpose and extended special purpose labels 
need to be excluded from the hash. Good.
But it seems that some special purpose labels will indicate that the 
next label stack entry contains a label with special meaning. (ELI is
an example that we specifically don't have to worry about.)
How do we handle that?
Should we be dividing up the extended special purpose label space to
have one set of code points meaning "just this label is special" and
another set meaning "this label is special and the next label stack
entry is magic"?

---

An issue that arises from the multipath support (2.4.5.1) is that
hardware assumes that after a label stack entry with the S-bit set,
there are only three possible next bytes...
- a control word (indicated by b0000 or b0001)
- an IPv4 header
- an IPv6 header
This is the case regardless of how the LSP was set up, and the next
bytes cannot ever be further MPLS stack entries.

While this comes up 2.4.5.1 it may merit further discussion in an
earlier section of the document.

I note that discussion of support of PWs without the CW drives you
to say that hashing beyond the S-bit should be a configurable option
which would (of course) support any payload including MPLS in MPLS
with repeated bottom of stack. However, you might want to specifically
preclude that.

---

Q#7 introduces the terms "short-pipe" and "uniform model". It provides
a pointer to 2.1.6, but that section does not mention these terms (and
doesn't refer to RFC 3270).

---

Section 7 could usefully point back at Section 2.6.1

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 26 Jan 2014 23:58:50 -0500

In message <[email-masked]>
> document change is needed.

Adrian,

Thanks for the review.  Better to get the changes in now than later.

There are a few dicussion points below before coming out with a new
revision of document.

>   == Outdated reference: draft-ietf-pwe3-vccv-impl-survey-results has been

Got it.

> Andy may want to update his coordinates.

Approximate coordinates updated as per Andy's most recent request.

>   Forwarding Equivalence Classes (LDP)

If I provide a section with a list of acronyms, do I still have to
expand on first use.  If so, AC, NSP, OAM, and a few others appear
before that section.

AC is used in "Specific pseudowire AC and NSP are out of scope."

I included CE since PE and P are included.

>   so they don't need to be included

Given that this takes up only 17 lines in a 50 page document I'd
rather be thorough.  The audience here is assumed to include but not
be limited to people who are not quite as familiar with MPLS, and are
somehow involved in chips, systems, or eval, and need guidance
regarding MPLS forwarding (though it might be a deep dive into the
guide book and I pitty the MPLS newbie starting here or anywhere).

> The "preferred" (by the RFC editor) expansion of ECMP is

The form without the hyphen is more common, even among recent
documents.  I prefer to keep it without the hyphen.

> might have wider applicability. Indeed, a quick search revealed 

I will change this to "(LDP, RSVP-TE, other protocols)" in P, PE, CE.

> this acronym list itself at E-LSP and L-LSP.

Reference to RFC 6378 added.

In T#32, the context MPLS-TP AIS/RDI and PSC should make the choice of
which PSC is meant sufficiently obvious.  I'll expand it anyway and in
the one other case where not expanded on first use in a section.

It should be obvious in E-LSP and L-LSP from the context plus the
EXP-Inferred-PSC is the official expansion of E-LSP.  If not obvious,
the reader can look at RFC3270 since that is in the same line of text.

> The wording is a bit odd. "The implementation and system design..."?

Target audience is explained in Section 1.4.  If you like I can flip
Section 1.3 and 1.4 so target audience is first, then use of the roles
called for in the target audience section won't seem quite so odd.

> Just tweak the words at the top of the list?

I'd like to keep that as is and split into two lists.  The second list
would have the last two items (fat-pw and EL).  The first list would
end with "implement CW" (sic).

 OLD

   6.  The implementer and system designer SHOULD support adding a
       pseudowire Flow Label [RFC6391].  Deployments MAY enable this
       feature for appropriate pseudowire types.  See Section 2.4.3.

   7.  The implementer and system designer SHOULD support adding an MPLS
       entropy label [RFC6790].  Deployments MAY enable this feature.
       See Section 2.4.4.

 NEW

   The following statements provide clarification regarding more
   recent requirements that are often missed.

   1.  The implementer and system designer SHOULD support adding a
       pseudowire Flow Label [RFC6391].  Deployments MAY enable this
       feature for appropriate pseudowire types.  See Section 2.4.3.

   2.  The implementer and system designer SHOULD support adding an MPLS
       entropy label [RFC6790].  Deployments MAY enable this feature.
       See Section 2.4.4.

I've made this change.  Let me know if this is not OK.

>    L2TP, or LDP, are out of scope.

Yes.  Thanks.

> the top of the label stack"

That phrase doesn't appear anywhere in the document.  Exactly what am
I clarifying?  What to do with unknown special purpose labels is in
the last paragraph of this subsection.

> purpose labels" by [I-D.ietf-mpls-special-purpose-labels].

 OLD

   [RFC3032] specifies that label values 0-15 are special purpose
   labels with special meanings.

 NEW

   [RFC3032] specifies that label values 0-15 are special purpose
   labels with special meanings.
+   [I-D.ietf-mpls-special-purpose-labels] renamed these from the term
+   "reserved labels" used in [RFC3032] "special purpose labels".

BTW - draft-ietf-mpls-special-purpose-labels is also waiting for WG
chair go-ahead.  Hopefully since it is a short document (and I think
non-controversial) it will go through smoothly and quickly so we can
change this to an RFC reference.

> could give a clue in the text.

OK.  Missed this in CCAMP otherwise I would have objected to making
this change without changing the SCSI for the PSC type to includes a
level of hierarchy.  Though perhaps a 4 byte drop in MTU is a hint.

I'll change this from "add up to four additional labels" to "at least
one additional label" and cite RFC 7074.

The PSC-2 and up were needed if you wanted more than one level of
nesting and needed to know whether your PSC-1 LSPs could make use of
other PSC FA.  Can be kludged with admin attributes though.

> Upstream label allocation may also come into this.

The only mention of label allocation is that MPLS FRR bypass method
(more formally known as facilitles backup) uses platform label space.

The only reason platform label space is of any significance in a
document about forwarding is "The use of platform label space impacts
the size of the LSR ILM for LSR with a very large number of
interfaces."

Label allocation, per platform or per interface and upstream or
downstream, is not a forwarding issue.  It is a software issue
and a matter of getting the protocol bits right.  Therefore I think
expanding any further on label allocation should be out of scope.

But yes ... it is something that can trip up vendors.

> One too many instances of "is"

yeah or something like that.

 NEW

   1.  The most common case is where reordering is rare,

> without using a control word?

I suppose you can't fix reordering caused by not using CW without the
sequence number in the CW.  That is going to require fixing the text.

 OLD

   3.  If the edge is not using pseudowire control word (CW) and the
       core is using multipath, reordering will be far more common.
       If this is occurring, the best solution is to use CW on the
       edge, rather than try to fix the reordering using resequencing.

 NEW

   3.  If the edge is not using pseudowire control word (CW) and the
       core is using multipath, reordering will be far more common.
       If this is occurring, using CW on the edge will solve the
       problem.  Without CW, resequencing is not possible since the
       sequence number is contained in the CW.

That was a big oops on our part.

> Are those two "should" really "SHOULD"?

"No" at this point because of the way we stated we would use the RFC
2119 keywords in Section 1.2.

Since no existing RFC calls for this it has to be lower case.

I could (and did so far) change the last sentence:

 OLD

   Even if not supporting MPLS-TP, equipment should provide a means to
   disable periodic hash reseeding and deployments should disable
   periodic hash reseeding.

 NEW

   Operator experience dictates that even if not supporting MPLS-TP,
   equipment SHOULD provide a means to disable periodic hash reseeding
   and deployments SHOULD disable periodic hash reseeding.

This would be consistent with the use of RFC 2119 terms in this
document, explicitly calling out any requirement not already stated in
an existing RFC.

The prior MUSTs in the paragraph are upper case becausse you would
violate MPLS-TP requirements by not doing something.

> note) that each outgoing packet may be different by its label value.

Is it possible to quantify the fanout?  YMMV?

The only thing I could say is that an implementation may need to make
lots of copies in some roles (access routers for example).

Making a copy and cycling yields poor performance but for low
multicast traffic volumes might be OK.  But you are right - some
mostly low-end-ish chips to this.

I'm not sure I can describe how multicast with high fanout is done
without wading into implementation details of specific vendors.

Perhaps the best I can do is add this:

   Careful consideration should be given to the performance
   characteristics of high fanout multicast for equipment that is
   intended to be used in such a role.

I'll add this before the last paragraph in the section.

> transport protocol implementations cannot survive such misordering.

Yes.  That requirement was missed.  Add new second paragraph to this
subsection.

   The Differentiated Services requirements for good reasons dictate
   that packets within a common microflow SHOULD NOT be reordered
   [RFC2474].  Service providers generally impose stronger
   requirements, commonly requiring that packets within a microflow
   MUST NOT be reordered except in rare circumstances such as load
   balancing across multiple links or path change for load balancing
   or path change for other reason.

Another SP requirement is stated here and I'm quite sure this
requirement is well accepted.

> 2.4.2 uses "composite link" and "component link". I suggest picking just

They are two different things.  Two or more component links make up a
composite link.  Knowing that, give it another read please.

I'd rather not cite draft-ietf-rtgwg-cl-requirements as an
informational reference just for this one term.  In favor of citing
it, draft-ietf-rtgwg-cl-requirements is moving along.  Against citing
it is there is far less than a ground swell of providers calling for
the full set of things asked for in draft-ietf-rtgwg-cl-requirements.

> another set meaning "this label is special and the next label stack

I did list ELI (bullet 2) before the more general rule of not useing
special purpose labels.  The ELI is not used, just the EL, so the text
could be considered correct as-is.

So far the only special purpose label that is not just ignored and
skipped over is ELI.

Regarding this being magic -- All of this is somewhat programable
specialized silicon magic.  The silicon generally has some form of
very fast, very light weight parsing engine at the front of the
pipeline.  One thing it does is pick out fields for load balance.

The better silicon hashes as it goes rather that pick out a set of
fields and then hashes that set of fields when its done.  When it sees
13 it skips and hashes the next thing and stops hashing completely.
If it sees 0-12,14 it skips and continues.  If it sees 15 it skips two
labels and continues.  Its should be programable enough that if
someone defines a new ELI like label it is likely to be able to deal
with it.

The not so good silicon has this all so hard wired that it won't be
able to do ELI without at least a respin.

At most I could add "If a new special purpose label or extended
special purpose label is defined which requires special load balance
processing then, as is the case for the ELI label, a spacial action
may be needed rather than skipping the special purpose label or
extended special purpose label."  I really don't think this is needed.

> bytes cannot ever be further MPLS stack entries.

Right.  Note that in (5) is says that some SP will require IP headers
and some will require an ability to disable IP headers.

The rule is really look for 4, 6, or anything else in the first
nibble.  If 4 or 6 assume IP.  If anything else stop.

And yes if the payload is MPLS after a S-bit you have a screwed up
MPLS implementation to start with and you won't get load balance on
any set of MPLS labels after the first S-bit.  This is a fact of life
in the field and is as it should be.

> earlier section of the document.

This text is part of 2.4. ("MPLS Multipath Techniques").  The third
paragraph contains "Further details are provided in Section 2.4.5."
Section 2.4.5. is "Fields Used for Multipath Load Balance".

> with repeated bottom of stack. However, you might want to specifically

It says the same thing here in bullet 5 regarding being configurable.
The wording "ability to disable" is same as "configurable option".

At no point in this document do we imply that looking beyond the S-bit
means looking at anything beyond the S-bit other than looking for IP.
This is very clear in [RFC4385] and [RFC4928] which is cited in the
text about PW CW.

All it says in the places discusing PW is that without CW the traffic
might get reordered.

If you feel that we at any point imply that lack of PW CW allows
looking at anything past the S-bit rather than just looking for an IP
header please point to where and we will have to correct that.  I
looked at all occurances of CW and did not find anything.

Bullet 5 is very clear that a 4 or 6 has to be found in the first
nibble of payload.

> doesn't refer to RFC 3270).

It really should say "See Section 2.1.6 regarding MPLS hierarchy.  See
[RFC3443] regarding PHP, UHP, and pipe, short-pipe, and uniform models."

It does now.

> Section 7 could usefully point back at Section 2.6.1

I'll add:

   Some advice on hardware and other equipment hardenning against
   Denial-of-Service attack can be found in Section 2.6.1.

The security ADs will have fun with that.  They'll love the comments
about crypto auth being the last line of defense not the first and
only line of defense.  I'll plan on discussing this in greater detail
when SecDir review comes along.

I thik the security AD will want to reword the prior paragraph as well
as they seem to have some very precise preferred wording.

Curtis

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Mon, 27 Jan 2014 12:45:09 +0000

Hi Curtis,

Many thanks for these! Ack to all.

Reading through your proposed changes, please find one small request inline:

>>   L2TP, or LDP, are out of scope.

Could this be reworded as follows?

NEW:
   Tunneling encapsulations carrying MPLS, such as MPLS in IP [RFC 4023], 
   MPLS in GRE [RFC 4023], MPLS in L2TPv3 [RFC 4817], or MPLS in UDP
   [draft-ietf-mpls-in-udp], are out of scope.

And then for consistency below:

OLD:
   7.  Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
       L2TPv3, and IPSEC.  These provide a greater source of entropy
       which some provider networks carrying large amounts of tunneled
       traffic may need.  The use of tunneling header information is out
       of scope for this document.
NEW:
   7.  Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
       L2TPv3, and IPSEC.  These provide a greater source of entropy
       which some provider networks carrying large amounts of tunneled
       traffic may need (e.g., as used in [RFC 5640] for GRE and L2TPv3).
       The use of tunneling header information is out of scope for this document.

Thanks!

Carlos.

> _______________________________________________
> mpls mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mpls

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 27 Jan 2014 10:47:43 -0500

Carlos,

Since your apple mailer setting scrambled the plain text version a
bit, I'm going to just snip out the relevent parts and reply inline.

In message <[email-masked]>
"Carlos Pignataro (cpignata)" writes:
 
> Reading through your proposed changes, please find one small request > inline:
>  
> On Jan 26, 2014, at 11:58 PM, Curtis Villamizar <[email-masked]> > wrote:
>  
> > 

[... big snip ...]

>    [draft-ietf-mpls-in-udp], are out of scope.

I don't cite documents that define things that are declared out of
scope.  For example, I don't cite documents defining the various
flavors of PW AC that are declared out of scope.

In this case I listed a few examples so that the phrase "Tunneling
encapsulations carrying MPLS" was a bit more clear.

The references section is already quite large.

>        traffic may need (e.g., as used in [RFC 5640] for GRE and > L2TPv3).
>        The use of tunneling header information is out of scope for this > document.

Same reasoning.  If we declare something out of scope we shouldn't
then describe it further and provide citations.

At one point Andy floated the idea of a PWE3 forwarding draft in the
relevant WG.  If you want to do a GRE forwarding draft, have at it,
but it might be in another WG with MPLS being only one of the things
that GRE can carry that might need entropy.

> Carlos.

Good suggestions if we hadn't declared these things out of scope.  We
do have to limit the scope to avoid turning this into a complete "how
to build a router" document and I've tried to avoid scope creep into
realms outside of MPLS.

One place were we don't do similar citations but maybe we should is:

   Support for other protocols that share a common Layer-4 header such
   as RTP, UDP-lite, SCTP and DCCP SHOULD be provided, particularly
   for edge or access equipment where additional entropy may be
   needed.  Equipment SHOULD also use RTP, UDP-lite, SCTP and DCCP
   headers when creating an entropy label.

I didn't add citations to each of these protocols to avoid further
reference section bloat and because they are well known and easy to
find in the rfc-index file.  This is creeping to the edge of the
defined document scope, but the WG raging discussion of UDP checksums
indicates that we really need the SHOULD for at least UDP-Lite and
probably for the others as well.

Curtis

[... big snip ...]

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Mon, 27 Jan 2014 18:59:39 +0000

Hi Curtis,

> bit, I'm going to just snip out the relevent parts and reply inline.

Thanks :-)

>> Reading through your proposed changes, please find one small request =
>> inline:
>> 
>> On Jan 26, 2014, at 11:58 PM, Curtis Villamizar <[email-masked]> =
>> wrote:
>> 
>>> =20
>>>> =20

OK. Please note that there are a couple of (very small) changes besides the citations:
* Tunneling encapsulations which may carry MPLS -> Tunneling encapsulations carrying MPLS
* such as MPLS in GRE, L2TP, or LDP -> such as MPLS in IP, MPLS in GRE, MPLS in L2TPv3, or MPLS in UDP

> The references section is already quite large.

The document is not short :-)

Looking at it right now, the "Normative References" section is actually not large for this document.

The "Informative References" is large, but I do not see that as an issue. As you know, informative references only provide additional contextual non-normative information. Only readers that are interested in a specific tangent will go there. I see that adding these informative references has no draw-back, but can help readers that would like to understand more about "Tunneling encapsulations carrying MPLS". I;d really add them.

That said, if you believe that the document gets worst (e.g., too large and distractive) by adding these, or that it is inconsistent with other references, it's OK.

>>       traffic may need (e.g., as used in [RFC 5640] for GRE and =
>> L2TPv3).
>>       The use of tunneling header information is out of scope for this =
>> document.
> 

It definitely is out of scope for the document. But if a reader's scope is interested, it would help to be informational about it.

As per above, either way is fine. Thanks for considering it.

> that GRE can carry that might need entropy.

Ack.

> probably for the others as well.

:-)

To me, the line is drawn at areas that directly touch MPLS. For example, the UDP one is a good one to add, as we saw that discussion. Similarly, encapsulating MPLS in other tunnels is of interest to folks, even if out of scope for the doc (and hence, not Normative, as opposed to not "Citable/Referenceable").

Thanks,

-- Carlos.

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 27 Jan 2014 23:04:06 -0000

Hello Curtis,

These replies modulo the conversation with Carlos. I find I can't read that
conversation and back apply the results here :-(

[snip]

> expand on first use.  If so, AC, NSP, OAM, and a few others appear

Afraid so :-(

> rather be thorough.

Yup. OK. Go for it.

[snip]

> documents.  I prefer to keep it without the hyphen.

A discussion for a rainy day with the RFC Editor. 
Leave as is.

> called for in the target audience section won't seem quite so odd.

No issue with the section order.
Just puzzled by "the implementer MUST support" when I (pedantically) thought
that the implementer supporting something might not be the same as the
implementation supporting it.

Not a big deal.

> I've made this change.  Let me know if this is not OK.

That's fine.

> the last paragraph of this subsection.

WTF?
You have to wonder where these senile ADs get their ideas.

Complete brain fart. Sorry.

[snip]

> expanding any further on label allocation should be out of scope.

OK. I'm convinced.

> That was a big oops on our part.

Well, on the scale of IETF oopsies, I don't think you score too high. Maybe
Narten could run a weekly script?

[snip]

> I'll add this before the last paragraph in the section.

That works.

> requirement is well accepted.

Looks good.

> the full set of things asked for in draft-ietf-rtgwg-cl-requirements.

Yes. Sorry. It has been a looooooooong time since I had a pass on the CL
document. Atrophy.

> extended special purpose label."  I really don't think this is needed.

You're right, and my worry is more about the special-purpose draft and the
consequences of possibly adding other special purpose labels that have child
labels associated. We certainly don't want to have to retrain the silicon at
transit LSRs to specially know what to do for each new special-purpose label.
Currently we propose that you don't hash on a special purpose label, but you can
carry on hashing immediately after.

If I introduce the foo-label, your silicon will recognise it as special purpose,
but it I say the label after the foo-label is magic you won't know that.

A way to fix this is to have (just punting here) the top bit of the extended
special purpose label range set mean "magic label follows".

> Bullet 5 is very clear that a 4 or 6 has to be found in the first

OK. I misread bullet 5.

[snip]

Thanks.
Adrian

---

From: <[email-masked]>
Date: Mon, 27 Jan 2014 23:07:17 +0000

>> A discussion for a rainy day with the RFC Editor.

Basic English grammar. Hyphenate related adjectives.
http://www.grammar-monster.com/lessons/hyphens_in_compound_adjectives.htm

Lloyd Wood
http://about.me/lloydwood
________________________________________
From: mpls [[email-masked]] On Behalf Of Adrian Farrel [[email-masked]]
Sent: 27 January 2014 23:04
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls] AD review of draft-ietf-mpls-forwarding

Hello Curtis,

[snip]

Afraid so :-(

Yup. OK. Go for it.

[snip]

Not a big deal.

That's fine.

WTF?

[snip]

OK. I'm convinced.

[snip]

That works.

Looks good.

OK. I misread bullet 5.

[snip]

Thanks.
Adrian

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 27 Jan 2014 19:09:50 -0500

> http://about.me/lloydwood

Hi Lloyd,

Monster.com lists this as a "should".  They don't seem to be using RFC
2119 keywords since its lower case.  :-)

  In the UK, your readers will expect you to use hyphens in compound
  adjectives.

  Americans are more lenient. The US ruling is: Use a hyphen if it
  eliminates ambiguity or helps your reader, else don't bother. If
  you're unsure, use hyphens. You won't be marked down for using
  hyphens.

Good thing we use US English in IETF and don't have to stick with
those pesky British rules.  They don't even know how to pronounce
router over there.  :-)

There is no ambiguity caused by leaving out the hyphen.  My reasoning
is that The form without the hyphen is more common, even among recent
documents - and it looks better to a US English reader.  We have given
the matter due consideration and are going against the monster.com
"should".

Curtis

---

From: <[email-masked]>
Date: Tue, 28 Jan 2014 01:11:27 +0000

Curtis,

my mistake. I hadn't realised that RFCs and internet-drafts were written
solely for an American audience, and that the preferences of US readers takes
priority over the ease of reading of the other 6.5 billion+ people on the
planet.

Lloyd Wood
http://about.me/lloydwood
________________________________________
From: Curtis Villamizar [[email-masked]]
Sent: 28 January 2014 00:09
To: Wood L  Dr (Electronic Eng)
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [mpls] AD review of draft-ietf-mpls-forwarding

Hi Lloyd,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 27 Jan 2014 21:17:01 -0500

In message <[email-masked]>
[email-masked] writes:
 
> Curtis,
>  
> my mistake. I hadn't realised that RFCs and internet-drafts were
> written solely for an American audience, and that the preferences of
> US readers takes priority over the ease of reading of the other 6.5
> billion+ people on the planet.
>  
> Lloyd Wood

Well now you know.  :-)

See for example RFC-Editor's FAQ #40.

  40.  I used British English. Why did you change it to American English?

    If there is a mix of British and American English within the
    Internet-Draft, the document is updated to use American English
    for consistency. 

The default has always been American English.  Perhaps since a DARPA
funded project started the RFC series and Jon Postel was not a Brit.

Even though RFC 6949 says "The official language of the RFC Series is
English" and otherwise stays out of the fight.  :-)

Cherio old chap,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 27 Jan 2014 22:05:33 -0500

Carlos,

I gave this some thought and perhaps we should add citations.  I got
your change below.  I also took another look at the normative vs
informational references and promoted quite a few to normative.  The
bad news is we now have a dependency on a draft.

See inline ... but

I am going to summarize in my reply to Adrian so that we can maintain
one thread on this with Adrian's full set of comments.

So please reply on that thread if you need to.

Curtis

In message <[email-masked]>
"Carlos Pignataro (cpignata)" writes:
> 
> Hi Curtis,
>  
> On Jan 27, 2014, at 10:47 AM, Curtis Villamizar <[email-masked]> > wrote:
>  
> > 
> > Carlos,
> > 
> >> Reading through your proposed changes, please find one small request > =
> >> inline:
> >> 
> >> On Jan 26, 2014, at 11:58 PM, Curtis Villamizar > <[email-masked]> =
> >> wrote:
> >> 
> >>> =20
> >>>>  Tunneling encapsulations which may carry MPLS, such as MPLS in > GRE,
> >>   Tunneling encapsulations carrying MPLS, such as MPLS in IP [RFC > 4023],
> OK. Please note that there are a couple of (very small) changes besides > the citations:
> * Tunneling encapsulations which may carry MPLS -> Tunneling > encapsulations carrying MPLS
> * such as MPLS in GRE, L2TP, or LDP -> such as MPLS in IP, MPLS in GRE, > MPLS in L2TPv3, or MPLS in UDP

Got it.  With citations.

> Looking at it right now, the "Normative References" section is actually > not large for this document.
>  
> The "Informative References" is large, but I do not see that as an > issue. As you know, informative references only provide additional > contextual non-normative information. Only readers that are interested > in a specific tangent will go there. I see that adding these informative > references has no draw-back, but can help readers that would like to > understand more about "Tunneling encapsulations carrying MPLS". I;d > really add them.
>  
> That said, if you believe that the document gets worst (e.g., too large > and distractive) by adding these, or that it is inconsistent with other > references, it's OK.
>  
> > 
> >>   7.  Some IP encapsulations support tunneling, such as IP-in-IP, > GRE,
> >>       traffic may need.  The use of tunneling header information is > out
> >>   7.  Some IP encapsulations support tunneling, such as IP-in-IP, > GRE,
> >>       The use of tunneling header information is out of scope for > this =
> >> document.
> > 
> It definitely is out of scope for the document. But if a reader's scope > is interested, it would help to be informational about it.
>  

OK,  With citation as requested.

> > probably for the others as well.

Might as well add citations to RTP, UDP-lite, SCTP and DCCP.

> To me, the line is drawn at areas that directly touch MPLS. For example, > the UDP one is a good one to add, as we saw that discussion. Similarly, > encapsulating MPLS in other tunnels is of interest to folks, even if out > of scope for the doc (and hence, not Normative, as opposed to not > "Citable/Referenceable").
>  
> Thanks,
>  
> -- Carlos.
>  
> > Curtis
> > 
> > 
> > [... big snip ...]

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 27 Jan 2014 23:08:37 -0500

In message <[email-masked]>
> conversation and back apply the results here :-(

I'll summarize the outcome of that conversation with Carlo at the
bottom so we can have one thread.

> > expand on first use.  If so, AC, NSP, OAM, and a few others appear

OK.  Expanded except RFC-Editor listed well known abbreviations.

> A discussion for a rainy day with the RFC Editor. 

Cheers.

> implementation supporting it.

Good point.  Its changed.

> Yes. Sorry. It has been a looooooooong time since I had a pass on the CL

Its also in Gwiz.8080 or something like that.

> special purpose label range set mean "magic label follows".

I added this:

   4.  If a new special purpose label or extended special purpose
       label is defined which requires special load balance
       processing, then, as is the case for the ELI label, a spacial
       action may be needed rather than skipping the special purpose
       label or extended special purpose label.

This will be the new bullet 4, bumping down the old 4 and 5.

> > Bullet 5 is very clear that a 4 or 6 has to be found in the first

OK

> Adrian

Summary of conversation with Carlos:

>>   L2TP, or LDP, are out of scope.

Carlos noted some wording issues in this sentence and suggested adding
citations for each.  After some discussion:

 OLD (original)

 NEW

   Tunneling encapsulations carrying MPLS, such as MPLS in IP
   [RFC4023], MPLS in GRE [RFC4023], MPLS in L2TPv3 [RFC4817], or MPLS
   in UDP [I-D.ietf-mpls-in-udp], are out of scope.

He also suggested the following.

 OLD

   Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
   L2TPv3, and IPSEC.  These provide a greater source of entropy which
   some provider networks carrying large amounts of tunneled traffic
-   may need.
   The use of tunneling header information is out of scope for this
   document.

 NEW

   Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
   L2TPv3, and IPSEC.  These provide a greater source of entropy which
   some provider networks carrying large amounts of tunneled traffic
+   may need, for example as used in [RFC5640] for GRE and L2TPv3.  
   The use of tunneling header information is out of scope for this
   document.

As part of that conversation it was noted that the following also had
no citations.

 OLD

   Support for other protocols that share a common Layer-4 header such
-   as RTP, UDP-lite, SCTP and DCCP 
   SHOULD be provided, particularly for edge or access equipment where
   additional entropy may be needed.

 NEW

   Support for other protocols that share a common Layer-4 header such
+   as RTP [RFC3550], UDP-Lite [RFC3828], SCTP [RFC4960] and DCCP
+   [RFC4340]
   SHOULD be provided, particularly for edge or access equipment where
   additional entropy may be needed.

btw- this is wrt fields used in load balancing and is saying in effect
"use more than just port 6 and 17 (UDP and TCP)".

Other than that, I looked at the list of informational references and
found the following do affect MPLS forwarding and therefore should be
promoted to normative.

   [RFC4950]  Bonica, R., Gan, D., Tappan, D., and C. Pignataro, "ICMP
              Extensions for Multiprotocol Label Switching", RFC 4950,
              August 2007.

   [RFC5082]  Gill, V., Heasley, J., Meyer, D., Savola, P., and C.
              Pignataro, "The Generalized TTL Security Mechanism
              (GTSM)", RFC 5082, October 2007.

   [RFC5085]  Nadeau, T. and C. Pignataro, "Pseudowire Virtual Circuit
              Connectivity Verification (VCCV): A Control Channel for
              Pseudowires", RFC 5085, December 2007.

   [RFC5332]  Eckert, T., Rosen, E., Aggarwal, R., and Y. Rekhter, "MPLS
              Multicast Encapsulations", RFC 5332, August 2008.

   [RFC5880]  Katz, D. and D. Ward, "Bidirectional Forwarding Detection
              (BFD)", RFC 5880, June 2010.

   [RFC5884]  Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow,
              "Bidirectional Forwarding Detection (BFD) for MPLS Label
              Switched Paths (LSPs)", RFC 5884, June 2010.

   [RFC5885]  Nadeau, T. and C. Pignataro, "Bidirectional Forwarding
              Detection (BFD) for the Pseudowire Virtual Circuit
              Connectivity Verification (VCCV)", RFC 5885, June 2010.

   [RFC6374]  Frost, D. and S. Bryant, "Packet Loss and Delay
              Measurement for MPLS Networks", RFC 6374, September 2011.

   [RFC6375]  Frost, D. and S. Bryant, "A Packet Loss and Delay
              Measurement Profile for MPLS-Based Transport Networks",
              RFC 6375, September 2011.

   [RFC6378]  Weingarten, Y., Bryant, S., Osborne, E., Sprecher, N., and
              A. Fulignoli, "MPLS Transport Profile (MPLS-TP) Linear
              Protection", RFC 6378, October 2011.

   [RFC6427]  Swallow, G., Fulignoli, A., Vigoureux, M., Boutros, S.,
              and D. Ward, "MPLS Fault Management Operations,
              Administration, and Maintenance (OAM)", RFC 6427, November
              2011.

   [RFC6428]  Allan, D., Swallow Ed. , G., and J. Drake Ed. , "Proactive
              Connectivity Verification, Continuity Check, and Remote
              Defect Indication for the MPLS Transport Profile", RFC
              6428, November 2011.

   [RFC6720]  Pignataro, C. and R. Asati, "The Generalized TTL Security
              Mechanism (GTSM) for the Label Distribution Protocol
              (LDP)", RFC 6720, August 2012.

   [I-D.ietf-mpls-psc-updates]
              Osborne, E., "Updates to PSC", draft-ietf-mpls-psc-
              updates-00 (work in progress), October 2013.

Upgrades were mostly due to BFD and MPLS-TP OAM.  For example, direct
LM needs to be in hardware and most things related to protection.

Note that "Updates to PSC" affects MPLS-TP protection state machine
which needs hardware assist if not direct support in hardware in order
to be fast.  The race condition may be the most severe problem with
RFC 6378 state machine.

If you think any of the above should be put back into informative,
please let me know.

Curtis

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 28 Jan 2014 11:23:30 -0000

Curtis,
Thanks for all this.
I'm good with the changes recorded here.
Time to post AFAIK.

A

> -----Original Message-----
> From: Curtis Villamizar [mailto:[email-masked]]
> Sent: 28 January 2014 04:09
> To: [email-masked]
> Cc: [email-masked]; [email-masked];
> [email-masked]

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 28 Jan 2014 14:54:36 -0500

In message <[email-masked]>

Adrian,

I just checked the diffs after submitting 05 and I want to make two very
minor changes.

 OLD

   ... renamed these from the term "reserved labels" used in [RFC3032]
   "special purpose labels".

 NEW

   ... renamed these from the term "reserved labels" used in [RFC3032]
   to "special purpose labels".

Added the word "to" as in "rename from X to Y".

 OLD

   registry reachable at IANA's pages at [1].

 NEW

   registry reachable at IANA's pages at http://www.iana.org.

The above is the return of a xml2rfc eref bug.  Need to hand edit the
.txt file or fix the python source again.  Didn't catch this since
there was no change in the xml file.

Should I hold off on these or submit a 06 version just to fix this?

Curtis

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 28 Jan 2014 20:35:11 -0000

Please post the update.

Thanks for checking with me.
Adrian

> Sent: 28 January 2014 19:55
> To: [email-masked]
If
> > > > Currently we propose that you don't hash on a special purpose label, but
> you
> > > can

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 28 Jan 2014 16:58:40 -0500

In message <[email-masked]>

Adrian,

Done.

BTW- Also had to hand edit to get draft-ietf-mpls-in-udp-05 and
draft-ietf-mpls-psc-updates-01 since
http://xml.resource.org/public/rfc/bibxml3 is a few days behind (last
updated Jan 13, 2014) and this time I got the dates right in the hand
edit (Jan 2014).

xml2rfc is great stuff when it works.

Curtis

---

From: t.petch <[email-masked]>
Date: Thu, 30 Jan 2014 17:53:15 +0000

Curtis

MP2MP Multipoint to Point ?

I would like the first sentence of the Abstract to start the
Introduction - otherwise there is no intended audience (and I tend to
skip Abstracts when I know I am going to be interested in a document).

And is that second clause for Operators eg
a basis for operators to evaluate forwarding implementations.

(more to follow - the I-D is big and my reading is slow:-(

Tom Petch

----- Original Message -----
From: "Curtis Villamizar" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, January 27, 2014 4:58 AM

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 30 Jan 2014 16:06:55 -0500

> MP2MP Multipoint to Point ?

Multipoint to multipoint.  Ie: capable of supporting <*,G> though
possibly for a policy constrained value of "*".  Some chips can do it
but I'm not sure what routers can do (whether software supports it)
and if so if anyone deploys MP2MP.  At least some P2MP for
distribution of stuff like video, maybe a lot but I don't know.
Plenty of MP2P since LDP is inherently MP2P.

> skip Abstracts when I know I am going to be interested in a document).

The intended audience is in the intro later on in Section 1.4 ("Target
Audience").

> a basis for operators to evaluate forwarding implementations.

It is meant for both operators and system designers.  System designers
(ie: people working for router vendors that buy off the shelf silicon)
need to pick a forwarding chip long before operators are evaluating.
See Section 1.4.

In the case of forwarding the "implementor" is designing a chip.  The
"system designer" is using it in a product.  The operator is (we hope)
trying to find out what if any bad decisions were made in an
evaluation to determine what effect any shortcomings would have before
deploying and finding out the hard way.

> (more to follow - the I-D is big and my reading is slow:-(

No problem.  Thanks for taking a look.

Curtis