Subject: Re: [L2tpext] Propagating IP/ECN field when L2TP sits between IP as both L2 payload and PSN

From: Bob Briscoe <[email-masked]>
Date: Thu, 1 Jun 2017 14:09:05 +0100

Ignacio,

I realized that my previous email may have come across as critical of 
L2TP implementers that don't already comply with RFC6040. No criticism 
was intended - it is not at all easy to determine which specs to follow 
- one reason for this clarifying draft.

I've also corrected one sentence below...

On 01/06/17 05:02, Bob Briscoe wrote:
> Ignacio,
>
> On 01/06/17 00:55, Ignacio Goyret wrote:
>> At 15:11 5/31/2017, Bob Briscoe wrote:
>>
>>>> I do not understand however why "Update", or what that implies to
>>>> a compliant implementation. Many features of L2TP are specified in
>>>> various WGs, not "Updating" the base specs.
>>>>
>>> By my understanding of the word extension, if a implementation that
>>> is compliant with RFC3931 does not implement an extension, it is
>>> still compliant with RFC3931. Is this how the word "extension" is
>>> used in L2TP land?
>>>
>>> Whereas, if one IP header encapsulates another, if the ECN field
>>> were processed in any way other than RFC6040, it would be invalid
>>> (unless another RFC were written to update RFC6040). Therefore,
>>> whenever L2TP is tunnelling IP in IP, it is mandatory to comply
>>> with RFC6040. So I am asking the L2TPext WG for advice on how you
>>> guys would make a mandatory update to L2TP, rather than 'just' an
>>> optional extension.
>> I'm not sure you will get much traction for a "mandatory" update
>> because it would immediately invalidate any compliant deployed code
>> out there.
> I'm not asking for permission to make something mandatory. RFC6040 (or 
> its predecessors {Note 1}) is already mandatory.
> draft-ietf-tsvwg-rfc6040update-shim is just the "messenger", telling 
> L2TP implementers that RFC6040 is already mandatory.
>
> A mandatory update wouldn't invalidate any compliant deployed code, 
> because the only way that deployed code could be compliant would be to 
> already comply with RFC6040 or its predecessors. There is no other RFC 
> that says how to construct an ECN field during tunnelling. The 
> standards track predecessors of RFC6040 pre-dated L2TPv3 by 3.5 years. 
> And the experimental track predecessor of ECN tunnelling pre-dated 
> L2TPv2.
>
> The ECN field is in the main IP header (v4 and v6). It cannot be 
> omitted. When an L2TPv3 implementation creates an outer IP header 
> during encap, or forwards an IP header after decap, it has to set the 
> new ECN fields to some value. One would hope that an implementer would 
> try to find a spec to follow, rather than guessing this value. There 
> is only one RFC that says how to construct the ECN field during 
> tunnelling (RFC6040 or its predecessors). If an implementer has not 
> complied with one of these, she has not complied with the Internet 
> Protocol.
>
> draft-ietf-tsvwg-rfc6040update-shim is not changing the technical 
> effect of the ECN tunnelling spec [RFC6040]. All it is really doing is:
>
> a) Typing "Updates: 3931" at the top of a proposed RFC. This would 
> ideally have been typed at the top of the original ECN RFC [RFC3168]. 
> This flags to L2TP implementers that they need to take note of the 
> standards track update to IP tunnelling that happened in 2001. 
> Because, like you say, real life dictates there will be some L2TP 
> implementations that don't do it right.
>
> b) and helping to do it right, by providing the L2TP negotiation 
> machinery (that is also already mandatory for any tunnelling of IP in 
> IP).
>
>
> {Note 1}: The predecessors of RFC6040 are RFC3168 and RFC4301. They 
> all have the same technical effect - the deltas between them merely 
> altered trivia like handling combinations of ECN fields that wouldn't 
> normally be expected and other tinkering round the edges.
>
>> This goes back to the question of why negotiate it if it is mandatory.
>
> The standards action that is mandatory is compliance with RFC6040, 
> which made it mandatory to negotiate (or use configuration).
>
> Specifically, RFC6040 made it mandatory for a tunnel ingress to check 
> for (or be configured for) whether the egress supported ECN decap. In 
> the predecessors to RFC6040, negotiation was only specified for IPsec. 
> The only other type of tunnel considered was an RFC2003 tunnel, where 
> no negotiation of tunnel set up process had been defined, so 
> negotiation wasn't discussed for that case.
[BB adds:] RFC3168 did mention MPLS, GRE, L2TP and PPTP, but just said 
they "will be considered as the need arises"
>
>>
>> Real life says that there will be some L2TP deployments that will
>> support ECN and some that won't, thus, the negotiation is needed
>> to maintain interoperability.
>>
>> But if the negotiation is required, then this can't be an update
>> since it must allow for LCCEs that don't support ECN (which it must,
>> because some deployments may choose (or may have to) stay with
>> their non-ECN supporting code for a variety of reasons).
>
> As above, the update is to make negotiation mandatory.
>
> I hope this helps clarify.
Cheers

Bob
>
> Cheers
>
>
>
> Bob
>
>>
>> Yes, I know: it is sort of a circular argument but hopefully you
>> see what I mean.
>>
>> -Ignacio
>>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Thu, 1 Jun 2017 13:23:17 +0000

Hi, Bob,

> On May 31, 2017, at 6:11 PM, Bob Briscoe <[email-masked]> wrote:
> 
> Carlos,
> 
> On 31/05/17 17:44, Carlos Pignataro (cpignata) wrote:
>> Hi, Bob,
>> 
>> Adding on a couple of bits on top of Ignacio's response. 
>> 
>> L2TPv2, RFC 2661, transports only PPP (and multi protocols within it, including of course IP) and runs over UDP. In contrast, L2TPv3 transports directly a variety of protocols (Ethernet, HDLC, etc.) and runs directly over IP protocol 115 or over UDP. RFC 3931 is the base L2TPv3 specification but needs a protocol companion document. Some of those protocols are finished, e.g. RFC 4719 but others like PPP or "raw" IP are in Internet-Drafts. 
> Yup, got all that. Thanks. 
> 
> I just didn't understand the implication of Ignacio's email that updating the v2 protocol (that solely supports PPP) could be more complex than updating the v3 protocol that I thought supported PPP and many more protocols. It implied (to me) that maybe v3 doesn't support PPP in some way.

Admittedly, I have not read in detail RFC 6040 or draft-ietf-tsvwg-rfc6040update-shim — however, I do have some questions to better understand the applicability.

draft-ietf-tsvwg-rfc6040update-shim talks about “IP-in-IP Tunnels with Tightly Coupled Shim Headers”.

GRE can be thought of an IP-in-IP tunnel with a tightly coupled shim, when transporting IP directly (although GRE can transport multi-protocols).

[and a small note that you might need a reference to RFC 7676].

However, GRE can carry Something-in-IP, with “something" potentially carrying also IP. In that case, I do not believe that’s an IP-in-IP-with-shim.

For the L2TPv2 case, L2TP carries PPP. PPP can in turn carry IP, and/orsomething else, and usually a mixture. Is this really applicable to your draft?

For L2TPv3, L2TP carries a variety of protocols, one of which is IP directly. However, although there is running code, there’s no RFC for it (https://tools.ietf.org/html/draft-ietf-l2tpext-pwe3-ip-05.) Similarly, RFC 3931 seems outside scope.

[and another question ,what about MPLS, with RFC 5129?]

> 
>> 
>> By the way you can find example of capabilities in subsections of Section 5.1.1 of https://tools.ietf.org/html/draft-ietf-l2tpext-l2tp-ppp-08
>> 
>> I do not understand however why "Update", or what that implies to a compliant implementation. Many features of L2TP are specified in various WGs, not "Updating" the base specs. 
> By my understanding of the word extension, if a implementation that is compliant with RFC3931 does not implement an extension, it is still compliant with RFC3931. Is this how the word "extension" is used in L2TP land?
> 

That’s my understanding of the unqualified word “extension” beyond L2TP land. Is that different anywhere?

> Whereas, if one IP header encapsulates another, if the ECN field were processed in any way other than RFC6040, it would be invalid (unless another RFC were written to update RFC6040). Therefore, whenever L2TP is tunnelling IP in IP, it is mandatory to comply with RFC6040. So I am asking the L2TPext WG for advice on how you guys would make a mandatory update to L2TP, rather than 'just' an optional extension.
> 

I think first we probably need to understand the applicability because it is not an IP-in-IP encap, there’s a Layer 2 header in between, not tightly coupled but rather part of the transported payload.

Now, what you describe in terms of level of mandatory applies to RFC 6040. If an implementation does not comply with RFC 6040 it is, well, not compliant with RFC 6040.

Thanks!

Carlos.

> 
> Bob
> 
>> Thanks,
>> 
>> Sent from my iPad
>> 
>> On May 30, 2017, at 10:48 PM, Bob Briscoe <[email-masked]> wrote:
>> 
>>> Ignacio,
>>> 
>>> Thanks for your rapid response.
>>> 
>>> On 31/05/17 02:51, Ignacio Goyret wrote:
>>>> I am a little confused by the double negative in "non-optional".
>>>> Do you mean "mandatory"? If so, why not just say that?
>>> Yes, mandatory.
>>> That language was not in the draft; only in my email.
>>>> 
>>>> The lack of a capabilities AVP that you were looking for is mainly
>>>> because it hasn't been needed so far. You are welcome to submit a
>>>> draft with something like that. Generally speaking, adding a simple
>>>> binary AVP is easy and simpler to handle.
>>> Good.
>>>> 
>>>> I have no problem with the new suggested AVP, but can you explain
>>>> why is it important to know if the remote supports or not this
>>>> extension? Can't a tunnel end deduce this information by observing
>>>> the IP header received?
>>> I'm afraid not. It is critical for the ingress (in each direction) to check that the egress decap will correctly propagate the outer ECN field into the forwarded payload protocol. Otherwise explicit congestion signals added by a node within the tunnel could just get stripped at decap. With congestion feedback black-holed, the load would continue to increase until overload and tail drop.
>>> 
>>> BTW, the forwarded ECN field after decap has to be a combination of the ECN fields in the outer and inner IP headers (RFC6040 gives a 4x4 matrix of all the combinations of the two 2-bit ECN codepoints). So I am interested to know whether this might present problems for L2TP implementations (ECN propagation on decap has been implemented OK in Linux for similar structures like VXLAN).
>>> 
>>>> 
>>>> Regarding the extension vs update question: if it is an update,
>>>> then why negotiate support? Wouldn't ECN propagation be mandated
>>>> by this update?
>>> Unfortunately, a mandatory update still has to cope with the above incremental deployment problem.
>>>> 
>>>> There is no section for L2TPv2 (RFC2661) or any further suggestion
>>>> on how to handle it. I'm not saying that it is necessary, but since
>>>> it is mentioned earlier in the document, you may want to add a few
>>>> words. Note that things are a bit more complicated in v2 because
>>>> it transports PPP packets, which may include IP packets (or portions
>>>> of PPP packets if using Multilink). I have no issue punting on L2TPv2
>>>> details.
>>> * Given you say L2TPv2 is more complicated cos it transports PPP packets, that seems to imply that L2TPv3 does not transport PPP, which doesn't sound right?
>>> 
>>> I don't fully understand the relationship between L2TPv2 & v3 (I have come across L2TP a lot in an operational context, but never really studied it as a protocol until now).
>>> 
>>> * Is there still a lot of L2TPv2 around?
>>> 
>>> * Would it be believable that a L2TPv2 implementation might be updated for ECN support but not updated to L2TPv3?
>>> 
>>> Cheers
>>> 
>>> 
>>> Bob
>>> 
>>>> 
>>>> -Ignacio
>>>> 
>>>> 
>>>> At 15:49 5/30/2017, Bob Briscoe wrote:
>>>> 
>>>>> Hi l2tpext list,
>>>>> [repeating previous post, but with self-explanatory subject line]
>>>>> 
>>>>> This is a plea for help on a short subsection of a short draft that updates L2TP (amongst other tunnel protocols).
>>>>> 
>>>>> In the Transport Area WG (tsvwg) we have been working on defining propagation of the ECN field between IP headers separated by shim header(s):
>>>>>    draft-ietf-tsvwg-rfc6040update-shim
>>>>> 
>>>>> L2TP is one such protocol (when the payload of the L2 protocol is IP, and the outer PSN is IP). This is particularly relevant at the moment, because of the low latency (L4S) work based on ECN and because deployment of ECN has finally taken off {Note 1}.
>>>>> 
>>>>> I presented this tunnelling work in intarea just under a year ago. Since then tsvwg has adopted the draft and asked that we include specific update text for each affected tunnel protocol. I just posted my first attempt at L2TP update text:
>>>>>    <https://tools.ietf.org/html/draft-ietf-tsvwg-rfc6040update-shim-01>draft-ietf-tsvwg-rfc6040update-shim-01
>>>>> 
>>>>> Specific questions:
>>>>> 
>>>>> Prize for the best answer to all 3 questions: you get to become a co-author (if you want :)
>>>>> 
>>>>> Q1. L2TP is meant for all sorts of outers and inners, so it is not clear where to define behaviour for the special (but very common) case of IP as the L2 payload and IP and as the PSN. That is:
>>>>>     IP    :    [UDP]    :    L2TP    :    L2-specific-sublayer    :    L2    :    IP
>>>>> 
>>>>> I decided on updating <Specific%20questions:>S.4.5 of RFC3931, but suggestions for a better place are welcome.
>>>>> 
>>>>> Q2. L2TP is usually extended not updated. However, in this case, compliance with RFC6040 is non-optional, so I think update rather than extension is appropriate. Reason: Although RFC6040 is non-optional it includes a compatibility mode that defines what has to happen when a tunnel endpoint doesn't support ECN propagation.
>>>>> 
>>>>> Pls read the (short) draft and, if you disagree, suggest how an extension can be non-optional.
>>>>> 
>>>>> Q3. The tunnel initiator needs to check that the other end supports ECN propagation. I have proposed an Attribute Value Pair (AVP) for this that is effectively just a boolean choice from each end (Yes or silence). I expected to find an AVP with per-control-connection flags for tunnel endpoint capabilities that I could just extend with one more flag, but there doesn't seem to be such a thing already in L2TP. Did I miss it?
>>>>> 
>>>>> 
>>>>> Cheers
>>>>> 
>>>>> 
>>>>> Bob
>>>>> 
>>>>> {Note 1}: ~70% of Web servers, ~50% of Apple client devices on fixed or WiFi access, and now increasing router deployment of ECN is appearing.
>>> 
>>> -- 
>>> _______________________________________________
>>> L2tpext mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/l2tpext
> 
> --

---

From: Ignacio Goyret <[email-masked]>
Date: Thu, 1 Jun 2017 13:24:10 -0700

>I think first we probably need to understand the applicability because
>it is not an IP-in-IP encap, there's a Layer 2 header in between, not
>tightly coupled but rather part of the transported payload.

And an IP header (if present) may be down various layers and not easily
accessible.

So far, LCCEs do not need to decapsulate all the stacked layers trying
to look for a potential IP header. Adding that requirement as a MUST is
a tall order that may be too onerous and impossible to satisfy.

>Now, what you describe in terms of level of mandatory applies to RFC 6040.
>If an implementation does not comply with RFC 6040 it is, well, not
>compliant with RFC 6040.

Exactly.

And that's why I believe this should be a _recommended_ extension when
inner IP headers can be found easily but it cannot be a mandatory
requirement for the L2TP protocol as a whole.

-Ignacio

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 2 Jun 2017 14:23:59 +0100

Ignacio,

RFC6040 contains the following para:

    If an ingress claims compliance with this specification, it MUST NOT
    permanently disable ECN processing across the tunnel (i.e., only
    using compatibility mode).  It is true that such a tunnel ingress is
    at least safe with the ECN behaviour of any egress it may encounter,
    but it does not meet the central aim of this specification:
    introducing ECN support to tunnels.

To be safe, an L2TP ingress could permanently use compatibility mode. 
Compatibility mode simply involves the ingress clearing the 2-bit ECN 
field in the outer (setting to 00). This would not allow the L2TP 
endpoint to claim compliance with RFC6040, but at least it would be 
safe. I think it would be useful to add this as operational advice to 
draft-ietf-tsvwg-rfc6040shim-update.

I would imagine most implementations of L2TP could be configured to do 
this (do you agree?). I suspect some still allow config of the setting 
the whole 8-bit ToS bytes, in which the 2 lowest significant bits are 
the ECN field (distinct from the 6-bit Diffserv field).

Nonetheless, altho safety is important, the main purpose of this draft 
is to go beyond that - to say how an implementer would add ECN support 
to L2TP. See inline...

> a tall order that may be too onerous and impossible to satisfy.

As recently discussed on int-area, I am going to widen the applicability 
of draft-ietf-tsvwg-rfc6040shim-update to include cases where a L2 
header is being encapsulated. But this wider scope would not be as 
mandatory.

I definitely do not want to make this a MUST. I made this point myself, 
in response to Tom Herbert on the int-area list a couple of days ago, 
pasted here:

> On 31/05/17 02:45, Bob Briscoe wrote:
>> One way to include all these cases in the scope of the draft is to 
>> say the tunnel endpoint SHOULD check the L2 payload for an IP header, 
>> even when the L2 header is not being added / removed. But that is 
>> rather an onerous requirement, don't you think? Especially given this 
>> would be asking the implementer to violate layering, we can only ask 
>> that it is done when feasible; it cannot be a hard requirement (e.g. 
>> L2 payload could be encrypted). 

I think MAY is too weak, but SHOULD is too strong, so how about this 
form of wording:
"If feasible, the tunnel endpoint SHOULD check the L2 payload for an IP 
header"

And I intend to add something along the lines of: "Otherwise, if the 
outer is IP, it MUST zero the outer ECN field."

> requirement for the L2TP protocol as a whole.
As above, I strongly agree with you.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 2 Jun 2017 15:50:40 +0100

Carlos,

> Admittedly, I have not read in detail RFC 6040 or draft-ietf-tsvwg-rfc6040update-shim — however, I do have some questions to better understand the applicability.
You will find that section 4 of RFC 6040 contains all the normative text 
(4 pages).
> However, GRE can carry Something-in-IP, with “something" potentially carrying also IP. In that case, I do not believe that’s an IP-in-IP-with-shim.
See email to Ignacio just now (which included my response to a similar 
point from Tom Herbert on int-area)...

In summary, I intend to widen the scope of 
draft-ietf-tsvwg-rfc6040update-shim to include non-mandatory /guidance/ 
for when the shim encapsulates a L2 rather than L3 header (essentially, 
interpreting the guidance in draft-ietf-tsvwg-ecn-encap-guidance for the 
specific protocols in this draft like VXLAN, GRE, L2TP).
> For the L2TPv2 case, L2TP carries PPP. PPP can in turn carry IP, and/orsomething else, and usually a mixture. Is this really applicable to your draft?
Yes, very much so.

One of the main motivations is to be able to support ECN between an LNS 
and a LAC, as used in many broadband settings.

We have implemented a prototype of the ultra-low latency form of ECN 
(L4S-ECN) for the downstream queue of a BRAS.
For RTTs ranging from <5ms to >100ms, every packet has ultra-low queuing 
delay (in the hundreds of microseconds, whereas the best previous 
solution added 5-10ms of queuing and required thousands of per-microflow 
queues). ECN also gives zero congestion loss, of course. We have 
demonstrated interactive video processing in the cloud for virtual 
reality, remote presence, finger-gesture panning and zooming etc, along 
with many other delay sensitive apps - all running simultaneously over 
the same broadband line (all in the same queue on the BRAS). See video 
of demo here:
https://riteproject.eu/dctth/#1511dispatchwg

In the downstream queues of the BRAS, the IP packets have already been 
decapsulated from L2TP, so when they are ECN-marked they are only within 
their PPP encapsulation. However, there will be scenarios where the 
operator will want to deploy ECN on nodes within the L2TP tunnel. Then 
any outer ECN markings will need to be propagated to the IP header 
inside the PPP payload.

> For L2TPv3, L2TP carries a variety of protocols, one of which is IP directly. However, although there is running code, there’s no RFC for it (https://tools.ietf.org/html/draft-ietf-l2tpext-pwe3-ip-…
See above, re adding guidance for widened scope.

> [and another question ,what about MPLS, with RFC 5129?]
Do you mean, "Is L2TP over an MPLS PSN outer header in the scope of this 
draft?"
It wasn't initially.

After I add guidance for what to do if the inner is a L2 header, I could 
also mention that there might be other PSN outers too. However, this 
draft was originally intended to clarify the scope of RFC6040, which is 
solely about IP-in-IP. So, whereas IP-shim-L2-IP can be legitimately 
included in scope, I think MPLS as the PSN is too much mission creep.

>>> By the way you can find example of capabilities in subsections of Section 5.1.1 of https://tools.ietf.org/html/draft-ietf-l2tpext-l2tp-ppp-08
Thanks. I actually based my text on similar examples that I found in 
RFC2661 via the IANA registry of L2TP AVPs after Googling "L2TP 
capability AVP".

A question:
Do you think I should call this an LCCE Capability AVP to be more 
generic, then assign 1 bit for an ECN capability? (As opposed to an ECN 
Capability AVP that only uses 1 of 16 bits).

I doubt another LCCE capability will ever be added, since there have 
been none so far, and L2TP is already mature. However, would you agree 
that there is no harm in being generic like this?

Another question, I used 4-octet alignment for the AVP, cos other 
examples seemed to do that. Was I correct?

> That’s my understanding of the unqualified word “extension” beyond L2TP land. Is that different anywhere?
So, to be clear, here's further questions:

Is your advice (and Ignacio's) that I should write the L2TP-specific 
text in my draft as a extension to L2TP, rather than an update?

Is there a distinction in L2TP between an "extension" and a "recommended 
extension" (as mentioned earlier in this thread)?

Should the safety guidance still be an update to the base L2TP specs (v2 
& v3)? By safety guidance, I mean text about the ingress zeroing the ECN 
field in an IP PSN outer if it does not check within the L2 payload, or 
does not check the egress ECN capability.

> Now, what you describe in terms of level of mandatory applies to RFC 6040. If an implementation does not comply with RFC 6040 it is, well, not compliant with RFC 6040.
See my recent response to Ignacio on this.

Cheers

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 14 Jun 2017 12:20:30 +0100

Ignacio,

On 31/05/17 02:51, Ignacio Goyret wrote:
> Note that things are a bit more complicated in v2 because
While going back over all the emails in this thread (I am writing up the 
next rev of the draft), the above point raised a couple of questions in 
my mind:

Q1. When you say "portions" of IP packets, does L2TPv2 fragmented these 
as per IP fragmentation so that each 'portion' has an IP header, or are 
they carved up in a PPP-specific way without IP headers on each?

Q2. Does L2TPv3 support PPP in the same way?

Q3. Are there other L2 protocols currently supported by L2TPv3 that can 
carve up IP packets so that there will not be an IP header at the start 
of each payload?

If framing boundaries might be different, I will need to refer to 
section 5.6 of draft-ietf-tsvwg-ecn-encap-guidelines 
<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-encap-guidelines-08#section-5.6> 
"Reframing and Congestion Markings", but I don't want to add complexity 
if it is not necessary.

Cheers

Bob