Subject: Re: [Detnet] [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

From: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>
Date: Fri, 1 Apr 2022 10:12:04 +0000

Tony, What would be the counter proposal to Bruno’s draft we would check on HW feasibility?
This allows us to validate potential feasibility.

From: Tony Li <[email-masked]> on behalf of Tony Li <[email-masked]>
Date: Thursday, 31 March 2022 at 22:41
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>
Cc: Greg Mirsky <[email-masked]>, DetNet WG <[email-masked]>, mpls <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Wim,

WH> not all HW has this capability or is very constraint without another compromise. This is in my view an important consideration to select the proposal

Understood, but without clear evidence of these limitations, it’s somewhat hard to justify limiting our approach.  The commonality of the hardware in relevant locations is key.

If we find that Brand X has hardware limitations and there are a million deployed throughout service providers everywhere, that is worth noting.

If we find that Brand Y has hardware limitations, but is only deployed in 10 home offices world wide, it is inconsequential.

As always, we have to make tradeoffs using the data that we have at hand.

If that’s accurate, then we are a software upgrade away, regardless of which solution we select. The true hardware limitations will arise as we define more complex actions, which require more data that is not readily accessible, either because there is too much in-stack data or access to the post-stack data that cannot be done without a performance penalty, if at all.

WH> I don’t believe this is true with what we have on the table right now and this is why restate to start with Bruno’s proposal to start and go to a more advanced solution that can be supported with new HW capabiltiies.

Well, I would appreciate hard data.  I have had a hand in MPLS since day one and have seen many hardware designs along the way. There are many hardware platforms that have limitations on label stack depth. That is definitely a concern, but that is already an issue for the entropy label today.  I’m unaware of any hardware implementation that cannot support the use of a different bSPL.  I welcome education, but without clear contradictory evidence, not just opinion or unsupported assertion, the data that I have suggests that we effectively have a small Turing-equivalent processor operating on some significant chunk of the label stack.

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 1 Apr 2022 12:13:23 +0000

Wim,

I did a quick check of the current network action proposals and came up with the following list:

Resource partition [HBH, Forwarding]

E2E Loss Measurement [E2E, Network]

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

No Reroute [HBH, Forwarding]

Flow ID [HBH, Network]

5G Slice ID [E2E, Network]

Traffic Accounting [HBH, Network]

PTP/Bounded Delay [E2E, Forwarding]

L2 Frame Reassembly [E2E, Network]

As far as I can tell, the only network not requiring ancillary data is No Reroute.  Given that Bruno's proposal doesn't meaningfully support ancillary data, how much use is it?

Yours Irrespectively,

John

Juniper Business Use Only
From: Pals <[email-masked]> On Behalf Of Henderickx, Wim (Nokia - BE/Antwerp)
Sent: Thursday, March 31, 2022 2:22 PM
To: Greg Mirsky <[email-masked]>; Tony Li <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

[External Email. Be cautious of content]

This is my point and this is what I verified on some systems that don't care about the TTL and hence work as is, because the ambiguity in the RFC.

Now also we mix 2 discussions points in my view. One is backward compatibility and 2nd is leveraging current HW to support the extensions.
For me the 2nd is also very important as this is actually an important characteristic for the speed at which we can adopt solutions. If we can get extensions with the current HW this is a big pro of the proposals out there. This is why I am advocating to adopt Bruno's draft as it allows to leverage the existing HW assets as is. Of course we need to do a SW upgrade, but this is still faster than swapping HW is most cases.

From: Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Thursday, 31 March 2022 at 17:59
To: Tony Li <[email-masked]<mailto:[email-masked]>>
Cc: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>, detnet WG <[email-masked]<mailto:[email-masked]>>, mpls <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)
I agree that the wording in RFC 6790 is open to interpretation. It is quite possible that a more pedantic developer would put a check for the zero value of the EL TTL field "to ensure that it is not used inadvertently for forwarding". Is it possible to check all existing implementations that support ELI/EL? And I'm surprised that the authors of the draft claim precisely the opposite:
   Hence essentially the TTL field of the EL behaves as a reserved field
   which must be set to zero when sent and ignored when received.

Regards,
Greg

On Thu, Mar 31, 2022 at 8:43 AM Tony Li <[email-masked]<mailto:[email-masked]>> wrote:

Gentlebeings,

On Mar 31, 2022, at 8:29 AM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:

my interpretation of bullet 4 in Section 4.2 RFC 6790 "The TTL for the EL MUST be zero to ensure that it is not used inadvertently for forwarding" leads me to believe that any other than zero value in the EL TTL field is invalid per RFC 6790. Consequently, that packet MUST be dropped. If that is not breaking the existing network, please help me understand what is it.

Normally, we write clauses that describe such fields as "must be transmitted as zero and ignored upon receipt" just to avoid such ambiguity. It is unfortunate that RFC 6790 did not utilize this phrase. As it stands, it has certainly specified that the TTL field must be transmitted as zero. Yes, that implies that any other value is invalid. However, that does not guarantee that implementations will check.  In fact, the Law of Lethargy (people will do the least amount of work possible) suggests that most implementations will not check and will simply ignore the TTL field completely.

However, this is not a guarantee. Any design that attempts to reuse this TTL field does run a non-zero risk of being impacted by designs that do check and reject such entries.

IMHO, this by itself is not a serious risk, but risk evaluation is always subjective.

Designs should always acknowledge and articulate the risks that they undertake. It is then up to the collective wisdom of the group to weigh and evaluate the risks, benefits, and tradeoffs when making a decision.

Regards,
Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 16:06:44 +0000

Greg,

> From:  Greg Mirsky
> I agree that the wording in RFC 6790 is open to interpretation. It is quite possible that a more pedantic developer would put a check for the zero value of the EL TTL field

RFC 6790 says

< 4.1<https://datatracker.ietf.org/doc/html/rfc6790#section-4.1>.  Egress LSR >
[...]

"The EL's TTL MUST be ignored."
https://datatracker.ietf.org/doc/html/rfc6790#section-4.1

To me, that does not read like open to interpretation.

> And I'm surprised that the authors of the draft claim precisely the opposite

Are you now feeling better about the authors?

--Bruno

Orange Restricted
From: mpls <[email-masked]> On Behalf Of Greg Mirsky
Sent: Thursday, March 31, 2022 6:00 PM
To: Tony Li <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Regards,
Greg

Gentlebeings,

Regards,
Tony

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 16:12:01 +0000

Tony,

Please see 2 comments inline [Bruno]

Orange Restricted
From: mpls <[email-masked]> On Behalf Of Tony Li
Sent: Thursday, March 31, 2022 5:43 PM
To: Greg Mirsky <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Gentlebeings,

Normally, we write clauses that describe such fields as "must be transmitted as zero and ignored upon receipt" just to avoid such ambiguity. It is unfortunate that RFC 6790 did not utilize this phrase. As it stands, it has certainly specified that the TTL field must be transmitted as zero.
[Bruno] Agreed (*3)

Yes, that implies that any other value is invalid. However, that does not guarantee that implementations will check.  In fact, the Law of Lethargy (people will do the least amount of work possible) suggests that most implementations will not check and will simply ignore the TTL field completely.

[Bruno] RFC 6790 also says that on the Egress LSR "The EL's TTL MUST be ignored."
https://datatracker.ietf.org/doc/html/rfc6790#section-4.1

Thanks
Regards,
Bruno

Regards,
Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 1 Apr 2022 09:38:56 -0700

Bruno,

> [Bruno] RFC 6790 also says that on the Egress LSR “The EL's TTL MUST be ignored.”
> https://datatracker.ietf.org/doc/html/rfc6790#section-4.1 <https://datatracker.ietf.org/doc/html/rfc6790#section-4.1>

Thank you, that’s a good point.  However, that’s still not a guarantee (i.e. 100% mathematical certainty). There is still risk involved. Small, but non-zero.

T

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 16:51:36 +0000

Tony,

Please see inline [Bruno]

Orange Restricted

Wim,

Implicit in this is the assumption that a hardware upgrade is required. That is not at all clear. In fact, that runs counter to my understanding of what's in the field: almost everything out there has some level of microcode capability.

[Bruno] It's pretty clear that you have a better knowledge of hardware capability than I have. Partly because vendors do their best not to disclose (if not hide) their limitations.
However, my understanding is that all hardware have limitations and even by design as they are tradeoff involves. (again, you know better).
Yet from my perspective and from information I have:
- it seems clear that many if not all hardware have limitations with regards to the size of the stack they can impose and read. So pushing more bytes in the header is not for free. (e.g, pushing 20/32 bits for entropy and 20/32 bits for slice id is indeed more scalable but also more costly than pushing 20/32 bits for both slice ID and entropy.)
- somehow, I'm getting occasional feedbacks from vendors that my currently deployed platforms (including some still been sold so still up to date), can't be software upgraded to support the new dataplane feature (not qos related). (even though the same vendor is claiming that ias silicon is flexible, programmable, can adapt to new features, is futureproof, much better than the competition...). So probably, there are limitations or at least trade-offs. (and I don't think that I'm contradicting you as you said "some level of")

--Bruno

If that's accurate, then we are a software upgrade away, regardless of which solution we select. The true hardware limitations will arise as we define more complex actions, which require more data that is not readily accessible, either because there is too much in-stack data or access to the post-stack data that cannot be done without a performance penalty, if at all.

In short, Bruno's draft offers us limitations that we want to avoid, and advantages that seem dubious.

Regards,
Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 17:07:33 +0000

Tony

> From: Tony Li <[email-masked]> On Behalf Of Tony Li
> 
> 
> Bruno,
> 
> 
> However, this is not a guarantee. Any design that attempts to reuse this TTL field
> does run a non-zero risk of being impacted by designs that do check and reject
> such entries.
> Thank you, that’s a good point.
Thank you for acknowledging.

>  However, that’s still not a guarantee (i.e. 100%
> mathematical certainty). There is still risk involved. Small, but non-zero.

Indeed. Exactly like any other reserved field that an IETF WG had defined and then use to extend the protocol.
In the WGs that I'm following (mostly routing), I don't recall that extending (actually even defining) a reserved field had raised that amount of discussion about this mathematically non-zero probability. Discussions -if any- are mostly whether we should say MUST/MUST, SHOULD/SHOULD, MUST/SHOULD or SHOULD/MUST for the sender and receiver behaviors. And even with SHOULD, I don't recall that anyone express that non-zero probability.

Plus obviously, non-zero probability is everywhere. Including in much more live critical aspects. E.g. landing is mandatory 😉. And we all live with that. (or die occasionally)

--Bruno

Orange Restricted

---

From: Tony Li <[email-masked]>
Date: Fri, 1 Apr 2022 10:12:27 -0700

Hi Bruno,

> Implicit in this is the assumption that a hardware upgrade is required. That is not at all clear. In fact, that runs counter to my understanding of what’s in the field: almost everything out there has some level of microcode capability.
>  
> [Bruno] It’s pretty clear that you have a better knowledge of hardware capability than I have. Partly because vendors do their best not to disclose (if not hide) their limitations.

This is certainly true. However, this is wholly irrelevant to the selection of SPL to be used to indicate our sub-stack.

> - somehow, I’m getting occasional feedbacks from vendors that my currently deployed platforms (including some still been sold so still up to date), can’t be software upgraded to support the new dataplane feature (not qos related). (even though the same vendor is claiming that ias silicon is flexible, programmable, can adapt to new features, is futureproof, much better than the competition…). So probably, there are limitations or at least trade-offs. (and I don’t think that I’m contradicting you as you said “some level of”)

I cannot speak to whatever feedback that you’re getting. There are always trade-offs. As we introduce new functionality, processing it typically requires more code space, more data space, and more cycles. Again, this is true across all proposals. And as such, without further hard data, it’s very hard to see how this impacts our choice of SPL.

Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 17:14:42 +0000

Hi Greg,

Orange Restricted
From: Greg Mirsky <[email-masked]>
Sent: Thursday, March 31, 2022 5:30 PM
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>
Cc: John E Drake <[email-masked]>; DECRAENE Bruno INNOV/NET <[email-masked]>; mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Wim,
my interpretation of bullet 4 in Section 4.2 RFC 6790 "The TTL for the EL MUST be zero to ensure that it is not used inadvertently for forwarding" leads me to believe that any other than zero value in the EL TTL field is invalid per RFC 6790.
[Bruno] « MUST be zero when sent” is how we (IETF) typically define a reserved field. Are you really saying that extending a reserved field is invalid?

Consequently, that packet MUST be dropped. If that is not breaking the existing network, please help me understand what is it.
[Bruno] No you are incorrect. Quite the contrary RFC 6790 explicity says that on the Egress “The EL's TTL MUST be ignored.” https://datatracker.ietf.org/doc/html/rfc6790#section-4.1

--Bruno

Regards,
Greg

On Wed, Mar 30, 2022 at 9:11 PM Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>> wrote:
John,

We have 2 cases here in my view.

  *   A node that is not upgraded with the SW capabilities to support Bruno’s extension -> will still load-balance as before and might or might not take the bits into account that would be used for the slice ID

     *   I believe we are discussing this case when we discuss “we will break existing  ELI/EL Implementations”. Let’s say even if they take the slice id bit into account, they continue to hash/load-balance on the interface set assigned.
     *   Of course this node is not slice aware as the SW is not supporting this extension, but this has is not related to “breaking existing ELI/EL implementations”.
     *   Even when I did a quick test some HW does not even take into account these bits. So in my view we don’t break existing EL/ELI implementations.
     *   We should really validate the behavior of the existing systems to see the behavior if we make such claims.

  *   A node that is upgraded with the extension of Bruno’s draft will do the right thing. This can easily be resolved with an extension in the Control plane to say which nodes support and don’t support slicing.

From: John E Drake <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 30 March 2022 at 22:02
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Cc: mpls <[email-masked]<mailto:[email-masked]>>, detnet WG <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Subject: RE: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)
Wim,

I think I would term it a thought experiment.  An RFC 6790 compliant node will take the value in the EL label field and use it to select an outgoing interface.  If the value in the EL field is a slice ID, such an node will select an outgoing interface which is not necessarily part of the slice in question and that outgoing interface will be to a node which is not necessarily part of the slice in question.

Yours Irrespectively,

John

Juniper Business Use Only
From: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, March 30, 2022 3:21 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Cc: mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

John, do you have evidence of this or is this a theoretical claim ?

From: mpls <[email-masked]<mailto:[email-masked]>> on behalf of John E Drake <jdrake=[email-masked]<mailto:jdrake=[email-masked]>>
Date: Wednesday, 30 March 2022 at 19:13
To: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Cc: mpls <[email-masked]<mailto:[email-masked]>>, detnet WG <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)
Except that putting a slice ID in the Entropy Label field will break existing  ELI/EL Implementations because their hashing of the slice ID won’t necessarily place a packet on the correct outgoing I/F
Sent from my iPhone

On Mar 30, 2022, at 1:00 PM, [email-masked]<mailto:[email-masked]> wrote:

From: Tony Li <[email-masked]<mailto:[email-masked]>> On Behalf Of Tony Li
Sent: Wednesday, March 30, 2022 4:08 PM
> [Kireeti]: suggest attending talk by Tony on danger of reusing ELI before making any decision.
https://notes.ietf.org/notes-ietf-113-pals<https://urldefense.com/v3/__https:/notes.ietf.org/notes-ietf-113-pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcqrP3gOo$>

Done. The talk raised no “danger of reusing ELI” for draft draft-decraene-mpls-slid-encoded-entropy-label-id; quite the contrary.
I quote: “claims of backward compatibility apply to draft-decraene-mpls-slid-encoded-entropy-label-id-03”. With more details on slide 18
https://datatracker.ietf.org/meeting/113/materials/slides-113-mpls-05-policy-on-mpls-special-purpose-labels-reuse-00<https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/113/materials/slides-113-mpls-05-policy-on-mpls-special-purpose-labels-reuse-00__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcNEC7QKk$>

Yes, the issue with this proposal is that it has no space for in-stack data and not enough space for possible expansion of additional actions.

[Bruno] There are two steps:
- This proposal allows for carrying 8 Indicators and a slice ID while been backward compatible with egress LER hance providing faster deployment with incremental benefit.
- If more in-stack data is required the proposal is extensible (e.g. draft-jags-mpls-ext-hdr) but at the cost of losing the above benefits for the ASes & uses-cases requiring more than 8 Indicators per AS or In-Stack Data.
So we can have both worlds: simple first step and extensibility for those who need it.

Independently, we also/already have the post stack data option to carry ancillary data, which may limit the need for In-Stack data extension.

--Bruno

Tony

Orange Restricted

Thank you.
_______________________________________________
Pals mailing list
[email-masked]<mailto:[email-masked]>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$>
_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 17:27:44 +0000

Tony, all

I realize that your subsequent email was already covering my minutes ago reply. Sorry for the spam, I was replying by emails in order after a long day of work.

> Well, I would appreciate hard data.

So am I.
But clearly vendors are never disclaiming their hardware limitations. In my experience, a very limited set are disclosed privately when the hardware limitations become undefendable by the vendor, and under strict NDA.
I personally don't find credible that we would have public knowledge of such data.

This email is a public call to all vendors: if you are ready to privately fully disclose the limitation of your forwarding silicon, I'm ready to take the time for the discussions (and reading the written data). (Actually, I'm pretty sure that my employer would even pay me for that time)
Obviously, if you are also ready to share publicly, you are very welcome to do so by replying to this email.

Regards,
--Bruno

Orange Restricted
From: mpls <[email-masked]> On Behalf Of Tony Li
Sent: Thursday, March 31, 2022 10:41 PM
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>
Cc: detnet WG <[email-masked]>; [email-masked]; mpls <[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Wim,

Understood, but without clear evidence of these limitations, it's somewhat hard to justify limiting our approach.  The commonality of the hardware in relevant locations is key.

WH> I don't believe this is true with what we have on the table right now and this is why restate to start with Bruno's proposal to start and go to a more advanced solution that can be supported with new HW capabiltiies.

Well, I would appreciate hard data.  I have had a hand in MPLS since day one and have seen many hardware designs along the way. There are many hardware platforms that have limitations on label stack depth. That is definitely a concern, but that is already an issue for the entropy label today.  I'm unaware of any hardware implementation that cannot support the use of a different bSPL.  I welcome education, but without clear contradictory evidence, not just opinion or unsupported assertion, the data that I have suggests that we effectively have a small Turing-equivalent processor operating on some significant chunk of the label stack.

Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 17:33:39 +0000

John,

Orange Restricted
From: mpls <[email-masked]> On Behalf Of John E Drake
Sent: Friday, April 1, 2022 2:13 PM
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; Greg Mirsky <[email-masked]>; Tony Li <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Wim,

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

Flow ID [HBH, Network]

[Bruno]
- draft-decraene-mpls-slid-encoded-entropy-label-id has two other examples.
- from an architecture perspective, Indicators can live independently of ancillary data (location)
- again draft-decraene-mpls-slid-encoded-entropy-label-id can be extended to support in-stack data if needed (this has been expressed multiple times) i.e. if post-stack data is found not to be enough.

--Bruno

Yours Irrespectively,

John

Juniper Business Use Only
From: Pals <[email-masked]<mailto:[email-masked]>> On Behalf Of Henderickx, Wim (Nokia - BE/Antwerp)
Sent: Thursday, March 31, 2022 2:22 PM
To: Greg Mirsky <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>
Cc: mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Regards,
Greg

Gentlebeings,

Regards,
Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 1 Apr 2022 10:48:52 -0700

Bruno,

> - again draft-decraene-mpls-slid-encoded-entropy-label-id can be extended to support in-stack data if needed (this has been expressed multiple times) i.e. if post-stack data is found not to be enou…

Extending your proposal to support in-stack data effectively puts us at draft-jags-mpls-ext-hdr-00, with the backward compatibility issues that we’ve discussed. That is, IMHO, no longer in the realm of acceptable risk.

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 1 Apr 2022 17:53:04 +0000

The elephant in the room is that new network actions, whether encoded in the EL LSE TTL field and in post-stack ancillary data or in a new bSPL with a mix of in-stack and post-stack data, will require software and/or hardware upgrades.  This is true, regardless of your assertions to the contrary.  I.e., ELI/EL re-use is either a red herring or a blue whale.

Sent from my iPhone

On Apr 1, 2022, at 1:34 PM, [email-masked] wrote:

John,

Orange Restricted

Wim,

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

Flow ID [HBH, Network]

As far as I can tell, the only network not requiring ancillary data is No Reroute.  Given that Bruno’s proposal doesn’t meaningfully support ancillary data, how much use is it?

[Bruno]

--Bruno

Yours Irrespectively,

John

This is my point and this is what I verified on some systems that don’t care about the TTL and hence work as is, because the ambiguity in the RFC.

Now also we mix 2 discussions points in my view. One is backward compatibility and 2nd is leveraging current HW to support the extensions.
For me the 2nd is also very important as this is actually an important characteristic for the speed at which we can adopt solutions. If we can get extensions with the current HW this is a big pro of the proposals out there. This is why I am advocating to adopt Bruno’s draft as it allows to leverage the existing HW assets as is. Of course we need to do a SW upgrade, but this is still faster than swapping HW is most cases.

Regards,
Greg

Gentlebeings,

Normally, we write clauses that describe such fields as “must be transmitted as zero and ignored upon receipt” just to avoid such ambiguity. It is unfortunate that RFC 6790 did not utilize this phrase. As it stands, it has certainly specified that the TTL field must be transmitted as zero. Yes, that implies that any other value is invalid. However, that does not guarantee that implementations will check.  In fact, the Law of Lethargy (people will do the least amount of work possible) suggests that most implementations will not check and will simply ignore the TTL field completely.

Regards,
Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 17:56:15 +0000

Hi Gyan,

Thanks for the clarifications questions.
I’ll take the liberty to top post to re-organize in two distinct points.

  1.  RFC 6790 specification of the use the EL’s TTL field.

You have correctly highlighted the two relevant text:

On the receiver side, §4.1 says “The EL's TTL MUST be ignored.”

On the sender side, §4.2 says “The TTL for the EL MUST be zero”

“a” So do we agree that RFC 6790 says “MUST be sent as zero and MUST ignored on receipt” ?
“b” Do we agree that this how reserved field are defined at the IETF (e.g. https://datatracker.ietf.org/doc/html/rfc7176#section-2.1.1) ?
“c” Do we agree that the way this field is specified (cf “a”) has always allowed the IETF to further extend this field?

2)    Implicit null /PHP
In case of PHP, the transport label is removed by the PHP and the ultimate node (egress LER) receives a label stack with the ELI, EL as top two labels.
As per MPLS architecture, LER looks at the top label (ELI) and either:

- supports RFC 6790 and then applies “Y then recognizes the ELI and pops two labels: the ELI and the EL.” In no way the EL is exposed and used for forwarding.
- does not support RFC 6790 and hence will drop the packet as per RFC 3031 (§3.18). In no way the EL is exposed and used for forwarding.

So in summary, the EL is never exposed (as top label) and can never be used for forwarding.

Regards,
--Bruno

Orange Restricted
From: Gyan Mishra <[email-masked]>
Sent: Thursday, March 31, 2022 4:58 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; John E Drake <[email-masked]>; detnet WG <[email-masked]>; mpls <[email-masked]>; [email-masked]
Subject: Re: [Detnet] [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno

Please provide clarification on how existing implementations using Entropy Label ELI/EL RFC 6790 with your proposal to reuse the entropy label.

In section 2 you talk about the new entropy label control field in your proposal to reuse the TTL field as the entropy label control field.

I have some questions related below that are concerning with your proposal.

RFC 6790 section 4 talks about the TTL processing below excerpt.

   If an ingress LSR X chooses to impose an EL, then Y will receive a

   tunnel termination packet with label stack <TL, ELI, EL> <remaining

   packet header>.  Y recognizes TL as the label it distributed to its

   upstreams for the tunnel and pops it.  (Note that TL may be the

   implicit null label, in which case it doesn't appear in the label

   stack.)  Y then recognizes the ELI and pops two labels: the ELI and

   the EL.  Y then processes the remaining packet header as normal; this

   may require further processing of tunnel termination, perhaps with

   further ELI+EL pairs.  When processing the final tunnel termination,

   Y MAY enqueue the packet based on that tunnel TL's or ELI's TC value

   and MAY use the tunnel TL's or ELI's TTL to compute the TTL of the

   remaining packet header.  The EL's TTL MUST be ignored.

So the TL or ELI is used to compute the TTL of the remaining packet header.  States that EL’s TTL is ignored.

Section 4.2 mentions that the TTL for the EL MUST be set to 0 so it’s not used for forwarding.  The issue here is related to implicit null PHP case where the TL is popped and ELI,EL are exposed and to ensure that the EL is not used for forwarding the EL MUST be set to 0.

   4.  If, for the chosen tunnel, Y has not indicated that it can

       process ELs, push <TL> onto the packet.  If Y has indicated that

       it can process ELs for the tunnel, push <TL, ELI, EL> onto the

       packet.  X SHOULD put the same TTL and TC fields for the ELI as

       it does for TL.  X MAY choose different values for the TTL and TC

       fields if it is known that the ELI will not be exposed as the top

       label at any point along the LSP (as may happen in cases where

       PHP is used and the ELI and EL are not stripped at the

       penultimate hop (see Section 4.4<https://datatracker.ietf.org/doc/html/rfc6790#section-4.4>).  The BoS bit for the ELI MUST

       be zero (i.e., BoS is not set).  The TTL for the EL MUST be zero

       to ensure that it is not used inadvertently for forwarding.  The

       TC for the EL may be any value.  The BoS bit for the EL depends

       on whether or not there are more labels in the label stack.

The EL is not used for forwarding as long as the field is set to 0 which is a MUST.  However if you reuse the TTL field as the entropy label control field it will not be set to 0 and thus that could break implementations in the PHP case where the ELI/EL are exposed.

Also the TTL being set to 0 is different then the field being actually a Reserved or not applicable field.

I disagree with sentence below in section 2.

The TTL field as I stated MUST be set to 0 so it’s not used for forwarding.  So it’s not reserved and it’s read by the LSR looking for the field to be set to 0 so it’s not used for forwarding.  I can’t see how that won’t break existing implementations.

Kind Regards

Gyan

On Thu, Mar 31, 2022 at 12:30 AM Gyan Mishra <[email-masked]<mailto:[email-masked]>> wrote:

I like Bruno’s idea of reusing the entropy label as indicator of MEH in the label stack and is backwards compatibility for devices not supporting can continue to use for ECMP load balancing.

I think this is a solid interim solution to get the ball rolling with minimal software updates and being able to support ancillary data in the label stack and as other solutions are progressed that may take longer or implement and deploy at least in the near term we have a quick solution that could be promising for operators.

I think we do have to vett out the backwards compatibility and scenario I can think of is if you want to be able to use the entropy label for ECMP load balancing and simultaneously want to also use as ancillary data indicator I am guessing won’t work and that is something we would have to be cognizant of if deployed.

Kind Regards

Gyan

On Wed, Mar 30, 2022 at 4:04 PM John E Drake <jdrake=[email-masked]<mailto:[email-masked]>> wrote:
Wim,

Yours Irrespectively,

John

--Bruno

Tony

Orange Restricted

Thank you.
_______________________________________________
Pals mailing list
[email-masked]<mailto:[email-masked]>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$>
_______________________________________________
detnet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/detnet
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [email-masked]<mailto:[email-masked]>

M 301 502-1347

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [email-masked]<mailto:[email-masked]>

M 301 502-1347

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 18:02:10 +0000

Hi Greg,

Please see inline [Bruno2]

Orange Restricted
From: Greg Mirsky <[email-masked]>
Sent: Thursday, March 31, 2022 5:48 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: Tony Li <[email-masked]>; mpls <[email-masked]>; detnet WG <[email-masked]>; Andrew G. Malis <[email-masked]>; [email-masked]
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno,
thank you for the expedient response to my notes. Please find a follow-up in-lined below under the GIM>> tag.

Regards,
Greg

On Thu, Mar 31, 2022 at 12:59 AM <[email-masked]<mailto:[email-masked]>> wrote:
Hi Greg,

Please see inline [Bruno] tentative replies to your questions.

Orange Restricted
From: Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, March 30, 2022 9:48 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]<mailto:[email-masked]>>
Cc: Tony Li <[email-masked]<mailto:[email-masked]>>; mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; Andrew G. Malis <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno, et al.,
I have several questions and would greatly appreciate your thoughts:

  *   Do you see the need for extensions to the routing protocols that now advertise the support of RFC 6790 to avoid the scenario that John has pointed out?
[Bruno] No and so far I'm not even seeing any (negative) scenario in John's email.
GIM>>  I hope you can point me to the verbiage in RFC 6790 that supports the following statement in your draft:
   Hence essentially the TTL field of the EL behaves as a reserved field
   which must be set to zero when sent and ignored when received.
I can only find the following text in Section 4.2 RFC 6790:
       The TTL for the EL MUST be zero
       to ensure that it is not used inadvertently for forwarding.
which says nothing about ignoring the EL TTL field value. Quite the opposite, my understanding is that the value must be checked, and if it is non-zero, the packet must be discarded. Based on that, I don't see how the extended interpretation of ELI can be safely deployed without any extensions in the control plane.

[Bruno2] Your understanding is incorrect. And yes I can point you to the text in RFC 6790 saying that the value must not be checked.

"The EL's TTL MUST be ignored." https://datatracker.ietf.org/doc/html/rfc6790#section-4.1

  *   As I understand the draft, using EL and Slice ID in the same stack is mutually exclusive. If that is the case, how entropy can be created within the particular network slice?
[Bruno] Please reread the draft and more specifically sections 3 and 3.1. You are not correct: the use of EMCP entropy and slice ID is not mutually exclusive, in the stack but even in a single EL. In short, one part of the EL is used to encode the slide ID and the remaining part carry the entropy number. Legacy transit LSR may use the whole EL transparently as the whole EL _is_ an entropy value which may be used for load-balancing (how that EL value is constructed by the ingress is not the concern of the transit).
GIM>> Thank you for clarifying this use case for me. As that is the case, I believe that there's even more reason for extensions in the control plane to signal how the EL field is used.
[Bruno2] I don't share your believe but I don't think that this point is the key one so I'd propose to skip it.

  *   I think that you've agreed that the draft does not address the requirements for MIAD<https://datatracker.ietf.org/doc/draft-bocci-mpls-miad-adi-requirements/>. You've pointed out Jags' draft<https://datatracker.ietf.org/doc/draft-jags-mpls-ext-hdr/> as the possible solution that is based on the re-definition of the ELI and may address MIAD ADI requirements, To that, I can make two observations:

     *   re-using ELI is only one optional approach discussed in draft-jags-mpls-ext-hdr. I think that is not guaranteed that that would be the option used if the document is published.
     *   And if draft-jags-mpls-ext-hdr continues with the solution based on the re-definition of the ELI, it appears to me that all Tony's concerns are valid.
[Bruno] I'm not sure what anything new your are bringing in addition to Tony's analysis and email. So I'll just copy past the same reply:
There are two steps:
- This proposal allows for carrying 8 Indicators and a slice ID while been backward compatible with egress LER hence providing faster deployment with incremental benefit.
GIM>> As I don't agree with your assumption that EL TTL value must be ignored and thus can be re-used for ADIs. I find that there are only three flags that can be safely used with your proposal. And that is without any ISD support.
[Bruno2] Do you know agree that the EL TTL field must be ignored? If so your point is moot.

--Bruno

Thank you in advance for your consideration of my notes.
[Bruno] You are very welcome. Thank you in advance for considering my answers.

Regards,
Bruno
Regards,
Greg

On Wed, Mar 30, 2022 at 9:59 AM <[email-masked]<mailto:[email-masked]>> wrote:

> [Kireeti]: suggest attending talk by Tony on danger of reusing ELI before making any decision.
https://notes.ietf.org/notes-ietf-113-pals

Done. The talk raised no "danger of reusing ELI" for draft draft-decraene-mpls-slid-encoded-entropy-label-id; quite the contrary.
I quote: "claims of backward compatibility apply to draft-decraene-mpls-slid-encoded-entropy-label-id-03". With more details on slide 18
https://datatracker.ietf.org/meeting/113/materials/slides-113-mpls-05-policy-on-mpls-special-purpose-labels-reuse-00

--Bruno

Tony

Orange Restricted

Thank you.
_______________________________________________
Pals mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/pals

Thank you.

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 18:06:14 +0000

Tony,

Orange Restricted
From: Tony Li <[email-masked]> On Behalf Of Tony Li
Sent: Friday, April 1, 2022 7:49 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: John E Drake <jdrake=[email-masked]>; mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; Greg Mirsky <[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Bruno,

Extending your proposal to support in-stack data effectively puts us at draft-jags-mpls-ext-hdr-00, with the backward compatibility issues that we've discussed. That is, IMHO, no longer in the realm of acceptable risk.
[Bruno] Extending the proposal to add LSE requires signaling that the Egress support that capability. Solutions requiring a new SPL requires a similar signaling. In both cases, I don't think that this may be called a backward compatibility issue, if but so, I think the statement equally applies to other proposals defining a new SPL

--Bruno

Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 1 Apr 2022 11:14:11 -0700

Bruno,

> On Apr 1, 2022, at 11:06 AM, <[email-masked]> <[email-masked]> wrote:
> 
> [Bruno] Extending the proposal to add LSE requires signaling that the Egress support that capability. Solutions requiring a new SPL requires a similar signaling. In both cases, I don’t think that this may be called a backward compatibility issue, if but so, I think the statement equally applies to other proposals defining a new SPL

If you add LSE and use signaling to avoid incapable nodes, then signaling must identify every node that could potentially pop the EL sub-stack. If EL has been populated multiple times, that is potentially more than the Egress node. 

Yes, requiring a new SPL requires exactly the same signaling. This is the point that we’ve been making: there is no advantage to reusing ELI. It becomes the same. Other proposals are not claiming backward compatibility.

Tony

---

From: <[email-masked]>
Date: Fri, 1 Apr 2022 18:14:12 +0000

John,

I would invite you to (re) read the email thread as you have probably missed some text.

> So we can have both worlds: simple first step and extensibility for those who need it.

This does not match the words that you put in my mouth.

--Bruno

Orange Restricted
From: John E Drake <[email-masked]>
Sent: Friday, April 1, 2022 7:53 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; Greg Mirsky <[email-masked]>; Tony Li <[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

On Apr 1, 2022, at 1:34 PM, [email-masked]<mailto:[email-masked]> wrote:

John,

Orange Restricted
From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of John E Drake
Sent: Friday, April 1, 2022 2:13 PM
To: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>; Greg Mirsky <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>
Cc: mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Wim,

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

Flow ID [HBH, Network]

[Bruno]

--Bruno

Yours Irrespectively,

John

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: John E Drake <[email-masked]>
Date: Fri, 1 Apr 2022 18:27:37 +0000

Hi,

Comments inline

Sent from my iPhone

On Apr 1, 2022, at 2:14 PM, [email-masked] wrote:

John,

> - This proposal allows for carrying 8 Indicators and a slice ID while been backward compatible with egress LER hence providing faster deployment with incremental benefit.

And how do existing RFC 6790 implementations understand these 8 indicators without a software and/or hardware upgrade?

> So we can have both worlds: simple first step and extensibility for those who need it.

As has been documented, this draft wreaks havoc in a network containing RFC 6790 compliant nodes

--Bruno

Orange Restricted

John,

Orange Restricted

Wim,

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

Flow ID [HBH, Network]

[Bruno]

--Bruno

Yours Irrespectively,

John

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: Greg Mirsky <[email-masked]>
Date: Fri, 1 Apr 2022 13:37:32 -0700

Hi Bruno,
thank you for pointing me to the text in Section 4.1. I agree that for the
case described in RFC 6790, i.e., a single ELI, EL in the stack, that text
sets clear requirements for disposing of ELI,EL. What I wanted to bring to
the discussion is how a transit node does that in the SR-MPLS scenario (RFC
8662 <https://datatracker.ietf.org/doc/html/rfc8662>). I think that there's
no text in RFC 8662 that establishes equivalency between a transit node
disposing of ELI,EL if ELI is at the top of the stack and the LER per RFC
6790. That might cause different interpretations resulting in different
handling. How practically we can collect information about deployed
solutions?

Regards,
Greg

On Fri, Apr 1, 2022 at 9:06 AM <[email-masked]> wrote:

> Greg,
>
>
>
> > *From:*  Greg Mirsky
>
> > I agree that the wording in RFC 6790 is open to interpretation. It is
> quite possible that a more pedantic developer would put a check for the
> zero value of the EL TTL field
>
>
>
> RFC 6790 says
>
> « 4.1 <https://datatracker.ietf.org/doc/html/rfc6790#section-4.1>.  Egress LSR »
>
> […]
>
> “The EL's TTL MUST be ignored.”
>
> *From:* mpls <[email-masked]> *On Behalf Of *Greg Mirsky
> *Sent:* Thursday, March 31, 2022 6:00 PM
> *To:* Tony Li <[email-masked]>
> *Cc:* mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]
> *Subject:* Re: [mpls] [Pals]
> draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review
> the PALS/MPLS/DetNet Joint Session minutes)
>
>
>
> I agree that the wording in RFC 6790 is open to interpretation. It is
> quite possible that a more pedantic developer would put a check for the
> zero value of the EL TTL field "to ensure that it is not used inadvertently
> for forwarding". Is it possible to check all existing implementations that
> support ELI/EL? And I'm surprised that the authors of the draft claim
> precisely the opposite:
>
> On Thu, Mar 31, 2022 at 8:43 AM Tony Li <[email-masked]> wrote:
>
>
>
> Gentlebeings,
>
>
>
> On Mar 31, 2022, at 8:29 AM, Greg Mirsky <[email-masked]> wrote:
>
>
>
> my interpretation of bullet 4 in Section 4.2 RFC 6790 "The TTL for the EL
> MUST be zero to ensure that it is not used inadvertently for forwarding"
> leads me to believe that any other than zero value in the EL TTL field is
> invalid per RFC 6790. Consequently, that packet MUST be dropped. If that is
> not breaking the existing network, please help me understand what is it.
>
>
>
>
>
> Normally, we write clauses that describe such fields as “must be
> transmitted as zero and ignored upon receipt” just to avoid such ambiguity.
> It is unfortunate that RFC 6790 did not utilize this phrase. As it stands,
> it has certainly specified that the TTL field must be transmitted as zero.
> Yes, that implies that any other value is invalid. However, that does not
> guarantee that implementations will check.  In fact, the Law of Lethargy
> (people will do the least amount of work possible) suggests that most
> implementations will not check and will simply ignore the TTL field
> completely.
>
>
>
> However, this is not a guarantee. Any design that attempts to reuse this
> TTL field does run a non-zero risk of being impacted by designs that do
> check and reject such entries.
>
>
>
> Designs should always acknowledge and articulate the risks that they
> undertake. It is then up to the collective wisdom of the group to weigh and
> evaluate the risks, benefits, and tradeoffs when making a decision.
>
>
>
> Regards,
>
> Tony
>
>
>

---

From: John E Drake <[email-masked]>
Date: Fri, 1 Apr 2022 20:57:05 +0000

Greg,

I pointed this out on the list last June and Bruno and Sasha basically called me a twit.  However, the fact remains that RFC 8662 does not describe how to get rid of an ELI/EL that is about to rise to the top of stack.

The correct answer is that the node which will pop the forwarding label at the top of stack which will expose the ELI/EL must remove the ELI/EL from the stack.  Apparently, this is blindingly obvious to the informed reader.

Yours Irrespectively,

John

Juniper Business Use Only
From: Pals <[email-masked]> On Behalf Of Greg Mirsky
Sent: Friday, April 1, 2022 4:38 PM
To: Bruno Decraene <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; Tony Li <[email-masked]>; [email-masked]
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno,
thank you for pointing me to the text in Section 4.1. I agree that for the case described in RFC 6790, i.e., a single ELI, EL in the stack, that text sets clear requirements for disposing of ELI,EL. What I wanted to bring to the discussion is how a transit node does that in the SR-MPLS scenario (RFC 8662<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8662__;!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NA2fzyJ8$>). I think that there's no text in RFC 8662 that establishes equivalency between a transit node disposing of ELI,EL if ELI is at the top of the stack and the LER per RFC 6790. That might cause different interpretations resulting in different handling. How practically we can collect information about deployed solutions?

Regards,
Greg

On Fri, Apr 1, 2022 at 9:06 AM <[email-masked]<mailto:[email-masked]>> wrote:
Greg,

RFC 6790 says

< 4.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NUbPRgzM$>.  Egress LSR >
[...]

"The EL's TTL MUST be ignored."
https://datatracker.ietf.org/doc/html/rfc6790#section-4.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NUbPRgzM$>

--Bruno

Orange Restricted
From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Greg Mirsky

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: Greg Mirsky <[email-masked]>
Date: Fri, 1 Apr 2022 14:28:53 -0700

Hi John,
indeed, if ELI is at the top of the stack, then it must be removed along
with EL. In my reading of RFC 8662 I did not find a reference to how EL's
TTL is handled. I think that "a SPRING router: MUST be entropy label
capable and, as a consequence, MUST apply the data-plane procedures defined
in [RFC6790]" helps with the imposition of ELI,EL but seems lacks
sufficient clarity in case of the transit SPRING router disposing of
ELI,EL, particularly in regard of EL's TTL handling. WDYT?

Regards,
Greg

On Fri, Apr 1, 2022 at 1:57 PM John E Drake <[email-masked]> wrote:

> Greg,
>
>
>
> I pointed this out on the list last June and Bruno and Sasha basically
> called me a twit.  However, the fact remains that RFC 8662 does not
> describe how to get rid of an ELI/EL that is about to rise to the top of
> stack.
>
>
>
> The correct answer is that the node which will pop the forwarding label at
> the top of stack which will expose the ELI/EL must remove the ELI/EL from
> the stack.  Apparently, this is blindingly obvious to the informed reader.
>
>
>
> Yours Irrespectively,
>
>
>
> John
>
>
>
>
>
> *From:* Pals <[email-masked]> *On Behalf Of * Greg Mirsky
> *Sent:* Friday, April 1, 2022 4:38 PM
> *To:* Bruno Decraene <[email-masked]>
> *Cc:* mpls <[email-masked]>; detnet WG <[email-masked]>; Tony Li <
> [email-masked]>; [email-masked]
> *Subject:* Re: [Pals] [mpls]
> *[External Email. Be cautious of content]*
>
>
>
> Hi Bruno,
>
> 8662
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8662__;!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NA2fzyJ8$>).
> I think that there's no text in RFC 8662 that establishes equivalency
> between a transit node disposing of ELI,EL if ELI is at the top of the
> stack and the LER per RFC 6790. That might cause different interpretations
> resulting in different handling. How practically we can collect information
> about deployed solutions?
>
>
>
> Regards,
>
> Greg
>
>
>
> « 4.1 <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NUbPRgzM$>.  Egress LSR »
>
> […]
>
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NUbPRgzM$>
>
>
>
>
>

---

From: John E Drake <[email-masked]>
Date: Fri, 1 Apr 2022 21:33:30 +0000

RFC 8662 is completely under-specified, assertions to the contrary.

Sent from my iPhone

On Apr 1, 2022, at 5:29 PM, Greg Mirsky <[email-masked]> wrote:

Hi John,
indeed, if ELI is at the top of the stack, then it must be removed along with EL. In my reading of RFC 8662 I did not find a reference to how EL's TTL is handled. I think that "a SPRING router: MUST be entropy label capable and, as a consequence, MUST apply the data-plane procedures defined in [RFC6790]" helps with the imposition of ELI,EL but seems lacks sufficient clarity in case of the transit SPRING router disposing of ELI,EL, particularly in regard of EL's TTL handling. WDYT?

Regards,
Greg

On Fri, Apr 1, 2022 at 1:57 PM John E Drake <[email-masked]<mailto:[email-masked]>> wrote:
Greg,

Yours Irrespectively,

John

Juniper Business Use Only
From: Pals <[email-masked]<mailto:[email-masked]>> On Behalf Of Greg Mirsky
Sent: Friday, April 1, 2022 4:38 PM
To: Bruno Decraene <[email-masked]<mailto:[email-masked]>>
Cc: mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno,

Regards,
Greg

RFC 6790 says

« 4.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!TlHg2NIqz8tuFJm-BGu4GUaE6em2PA4SyD5ryXBVvckZ8ca_EDq4xk-NUbPRgzM$>.  Egress LSR »
[…]

--Bruno

Orange Restricted

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 2 Apr 2022 01:28:46 +0200

Hey Bruno,

This email is a public call to all vendors: if you are ready to privately
> fully disclose the limitation of your forwarding silicon, I’m ready to take
> the time for the discussions (and reading the written data). (Actually, I’m
> pretty sure that my employer would even pay me for that time)
>

As you must be aware it is not always a question of hardware capabilities.

Very often vendors make a business decision if enhancing existing and
deployed platforms makes sense in terms of revenue as well as new cost of
features development for older gear.

To boost revenue vendors live by selling hardware and now also services,
Revenue from software new features has always been marginal if at all
present. I have never seen an action "Oh there is a new RFC published -
let's implement it on all still alive platforms".

Best,
Robert

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 2 Apr 2022 15:32:58 -0400

Hi Bruno

Responses in-line

> “a” So do we agree that RFC 6790 says “MUST be sent as zero and MUST
    Gyan Yes…but ..RFC 6790 defines the value of the TTL, however RFC 3032
defines the rules for processing all the fields of the label stack
encoding.  RFC 3032 Section 2.4.2 describes the processing when TTL is set
to 0 which means the TTL has expired and the packet must not be forwarded.
RFC 6790 section 4.2 bullet 4 states “The TTL for the EL MUST be zero to ensure
that it is not used inadvertently for forwarding.”  My interpretation of
that is even though the specification states the label stack is {TL,ELI,EL}
and that at the PHP node when the TL is popped in the case of implicit null
packets is forwarded to the egress LER which pops both ELI and EL.
However, the sentence is stating the TTL must be 0 for EL so the packet
cannot be inadvertently used for forwarding which I am interpreting as
there is a possibility that the EL could be exposed at the top of the
stack.  If hypothetically the case where the EL were not exposed were not
possible then it would have been sufficient to state that the TTL is
ignored on receipt and not worry about TTL processing which is a concern it
seems as stated in RFC 6790.  I think we have to understand why RFC 6790 is
requiring the TTL of EL MUST be set to 0.  So based on implementations of
MPLS their maybe possibilities that that even though the ELI must proceed
the EL, maybe they are corner cases where the ELI is missing label
imposition on the LER and thus the EL is exposed and required the EL to
have its TTL set to 0.  I think for this we would have to dig into
implementations of MPLS label stack and see what implementations have done
and if they are setting the TTL to 0 or not.  If we find that most all
implementations of RFC 6790 are not setting the EL TTL to 0 then it’s
possible we can reuse the TTL field as entropy label control field.  We
would have to really understand the corner case with missing ELI and how
the EL can possibly be exposed and that possibility as reasons the authors
of RFC 6790 stated the verbiage that has drawn my attention.

> “b” Do we agree that this how reserved field are defined at the IETF (e.g.
> https://datatracker.ietf.org/doc/html/rfc7176#section-2.1.1) ?
>
>    Gyan> So this I am guessing in RFC 7176 is an example of how Reserved
field is encoded with TRILL.  I don’t see the TTL field being marked as
reserved in RFC 3032.  I am missing your point here.

“c” Do we agree that the way this field is specified (cf “a”) has always
> allowed the IETF to further extend this field?
>
>  Gyan> RFC 3032 defines the label stack and RFC 6790 defines an entropy
> label for load balancing two new 4 byte label shims ELI and EL which use
> the same label stack encoding and processing rules defined in RFC 3032
> MPLS-SHIM.  I don’t see anywhere in RFC 3031 or 3032 that states that the
> TTL field can be further extended or repurposed for other uses.  I am
> afraid that if we use the field repurposed for something else it may have
> dire consequences with load balancing.
>
> 2)    Implicit null /PHP
>
> In case of PHP, the transport label is removed by the PHP and the ultimate
> node (egress LER) receives a label stack with the ELI, EL as top two labels.
>
> - does not support RFC 6790 and hence will drop the packet as per RFC 3031
> (§3.18). In no way the EL is exposed and used for forwarding.
>
>
>
>  Gyan>  Please see my first long comment related to RFC 6790 section 4.2
> bullet 4 states “The TTL for the EL MUST be zero to ensure that it is not
> used inadvertently for forwarding.”
>

    Gyan> Section 3.5 of NS packet draft mentions how a IETF network slices
map to a slice flow aggregate.
“ the routers correlate markers

present in the packets that belong to the Slice-Flow Aggregate.

So is the idea that you would split the 20 bit EL label carve out

bits to be used for slice flow aggregate and remaining bits to be

preserved for LB function.  The common size that has been used for flow

identifier has been a 20 bit field which is the size of MPLS label used

as well for IPv6 flow label where the 5 tuple header hash keys are used

to generate the 20 bit flow label which is an input key to hashing

function for stateless uniform load balancing.  VXLAN source port entropy uses

a 5 tuple header hash to generate a 16 bit source port input key to
hashing function.

I think the tradeoff here is how many bits to use for slice aggregate without

diminishing the load balancing functionality.

Gyan> RFC 8662 SR entropy refers to RFC 6790 on the ELI, EL semantics
and does not

get into the details described in RFC 6790. So as far as what I mentioned above

related to discussion that TTL Must be ignored and be set to 0 is not
mentioned in RFC 8662.  So there

maybe a gray areas as to RFC 8662 implementations as to what was actually done.

>
> Regards,
>
> --Bruno
>
>
>
>
>
> Orange Restricted
>
> *From:* Gyan Mishra <[email-masked]>
>
> *Sent:* Thursday, March 31, 2022 4:58 PM
> *To:* DECRAENE Bruno INNOV/NET <[email-masked]>
> *Cc:* Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>;
> John E Drake <[email-masked]>; detnet WG <[email-masked]>; mpls <
> [email-masked]>; [email-masked]
> *Subject:* Re: [Detnet] [mpls] [Pals]
> Please provide clarification on how existing implementations using Entropy
> Label ELI/EL RFC 6790 with your proposal to reuse the entropy label.
>
>
>
> In section 2 you talk about the new entropy label control field in your
> proposal to reuse the TTL field as the entropy label control field.
>
>
>
> Section 4.2 mentions that the TTL for the EL MUST be set to 0 so it’s not
> used for forwarding.  The issue here is related to implicit null PHP case
> where the TL is popped and ELI,EL are exposed and to ensure that the EL is
> not used for forwarding the EL MUST be set to 0.
>
>
>
>        penultimate hop (see Section 4.4 <https://datatracker.ietf.org/doc/html/rfc6790#section-4.4>).  The BoS bit for the ELI MUST
>
> The EL is not used for forwarding as long as the field is set to 0 which
> is a MUST.  However if you reuse the TTL field as the entropy label control
> field it will not be set to 0 and thus that could break implementations in
> the PHP case where the ELI/EL are exposed.
>
>
>
> The TTL field as I stated MUST be set to 0 so it’s not used for
> forwarding.  So it’s not reserved and it’s read by the LSR looking for the
> field to be set to 0 so it’s not used for forwarding.  I can’t see how that
> won’t break existing implementations.
>
>
>
> Kind Regards
>
>
>
> Gyan
>
>
>
> On Thu, Mar 31, 2022 at 12:30 AM Gyan Mishra <[email-masked]>
> wrote:
>
>
>
> I like Bruno’s idea of reusing the entropy label as indicator of MEH in
> the label stack and is backwards compatibility for devices not supporting
> can continue to use for ECMP load balancing.
>
>
>
> I think this is a solid interim solution to get the ball rolling with
> minimal software updates and being able to support ancillary data in the
> label stack and as other solutions are progressed that may take longer or
> implement and deploy at least in the near term we have a quick solution
> that could be promising for operators.
>
>
>
> I think we do have to vett out the backwards compatibility and scenario I
> can think of is if you want to be able to use the entropy label for ECMP
> load balancing and simultaneously want to also use as ancillary data
> indicator I am guessing won’t work and that is something we would have to
> be cognizant of if deployed.
>
>
>
> Kind Regards
>
>
>
> Gyan
>
>
>
> On Wed, Mar 30, 2022 at 4:04 PM John E Drake <jdrake=
> [email-masked]> wrote:
>
> Wim,
>
>
>
> I think I would term it a thought experiment.  An RFC 6790 compliant node
> will take the value in the EL label field and use it to select an outgoing
> interface.  If the value in the EL field is a slice ID, such an node will
> select an outgoing interface which is not necessarily part of the slice in
> question and that outgoing interface will be to a node which is not
> necessarily part of the slice in question.
>
>
>
> Yours Irrespectively,
>
>
>
> John
>
>
>
>
>
> *From:* Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>
> *Sent:* Wednesday, March 30, 2022 3:21 PM
> *To:* John E Drake <[email-masked]>; [email-masked]
> *From: *mpls <[email-masked]> on behalf of John E Drake <
> jdrake=[email-masked]>
> *Date: *Wednesday, 30 March 2022 at 19:13
> *To: [email-masked] <[email-masked]>
> *Cc: *mpls <[email-masked]>, detnet WG <[email-masked]>, [email-masked] <
> [email-masked]>
> *Subject: *Re: [mpls] [Pals]
> Except that putting a slice ID in the Entropy Label field will break
> existing  ELI/EL Implementations because their hashing of the slice ID
> won’t necessarily place a packet on the correct outgoing I/F
>
> Sent from my iPhone
>
>
>
> On Mar 30, 2022, at 1:00 PM, [email-masked] wrote:
>
> 
>
> *From:* Tony Li <[email-masked]> *On Behalf Of *Tony Li
> *Sent:* Wednesday, March 30, 2022 4:08 PM
>
> <https://urldefense.com/v3/__https:/notes.ietf.org/notes-ietf-113-pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcqrP3gOo$>
>
>
>
> Done. The talk raised no “danger of reusing ELI” for draft
> draft-decraene-mpls-slid-encoded-entropy-label-id; quite the contrary.
>
> I quote: “claims of backward compatibility apply to
> draft-decraene-mpls-slid-encoded-entropy-label-id-03”. With more details on
> slide 18
>
>
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/113/materials/slides-113-mpls-05-policy-on-mpls-special-purpose-labels-reuse-00__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcNEC7QKk$>
>
>
>
>
>
> Yes, the issue with this proposal is that it has no space for in-stack
> data and not enough space for possible expansion of additional actions.
>
>
>
> - This proposal allows for carrying 8 Indicators and a slice ID while been
> backward compatible with egress LER hance providing faster deployment with
> incremental benefit.
>
> - If more in-stack data is required the proposal is extensible (e.g.
> draft-jags-mpls-ext-hdr) but at the cost of losing the above benefits for
> the ASes & uses-cases requiring more than 8 Indicators per AS or In-Stack
> Data.
>
> Independently, we also/already have the post stack data option to carry
> ancillary data, which may limit the need for In-Stack data extension.
>
>
>
> --Bruno
>
>
>
> Tony
>
>
>
>
>
>
>
> Orange Restricted
>
> [email-masked]
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/pals__;!!NEt6yMaO-gk!Sw9ofU9AyD7Z-JKwyAqMlHk5xhNLxZNMSu31Yt6-K7yh-6JehvlSPLDcSqI60Zo$>
>
> [email-masked]
> <http://www.verizon.com/>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
> *Email [email-masked] <[email-masked]>*
>
> *M 301 502-1347*
>
>
>
> --
>
> <http://www.verizon.com/>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
> *Email [email-masked] <[email-masked]>*
>
> *M 301 502-1347*
>
>
>

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 2 Apr 2022 17:23:09 -0400

Hi Bruno
On Sat, Apr 2, 2022 at 3:32 PM Gyan Mishra <[email-masked]> wrote:

>> “b” Do we agree that this how reserved field are defined at the IETF
>> (e.g. https://datatracker.ietf.org/doc/html/rfc7176#section-2.1.1) ?
>>
>> In case of PHP, the transport label is removed by the PHP and the
>> ultimate node (egress LER) receives a label stack with the ELI, EL as top
>> two labels.
>>
>> - does not support RFC 6790 and hence will drop the packet as per RFC
>> 3031 (§3.18). In no way the EL is exposed and used for forwarding.
>>
>>
>>
>> bullet 4 states “The TTL for the EL MUST be zero to ensure that it is
>> not used inadvertently for forwarding.”
>>
>
>     Gyan> Section 3.5 of NS packet draft mentions how a IETF network
> slices map to a slice flow aggregate.
> a 5 tuple header hash to generate a 16 bit source port input key to hashing function.
>
> Gyan> RFC 8662 SR entropy refers to RFC 6790 on the ELI, EL semantics and does not
>
> related to discussion that TTL Must be ignored and be set to 0 is not mentioned in RFC 8662.  So there
>

    Gyan> There could be a possible implementation specific recirculation
issue or the sort that causes the ELI and EL to get separated and I am
guessing that feedback from implementations prior to WGLC that maybe was
found and it was then required to make the EL set TTL to 0 and verbiage
added so it’s not used for forwarding.

>> Please provide clarification on how existing implementations using
>> Entropy Label ELI/EL RFC 6790 with your proposal to reuse the entropy label.
>>
>>
>>
>> - This proposal allows for carrying 8 Indicators and a slice ID while
>> been backward compatible with egress LER hance providing faster deployment
>> with incremental benefit.
>>

*Gyan Mishra*

*M 301 502-1347*

---

From: <[email-masked]>
Date: Mon, 4 Apr 2022 10:09:31 +0000

John,

Orange Restricted
From: John E Drake <[email-masked]>
Sent: Friday, April 1, 2022 10:57 PM
To: Greg Mirsky <[email-masked]>; DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; Tony Li <[email-masked]>; [email-masked]
Subject: RE: [Pals] [mpls] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Greg,

[Bruno] This is specified in RFC 6790.
>From what I read, you seem to read "Egress" as the egress of the last LSP/label of the label stack. This is indeed one point of view, especially from a VPN point of view. However, RFC 6790 (and RFC 3031) defines Egress as the Egress of a LSP. Cf https://datatracker.ietf.org/doc/html/rfc3031#section-3.15

The correct answer is that the node which will pop the forwarding label at the top of stack which will expose the ELI/EL must remove the ELI/EL from the stack.  Apparently, this is blindingly obvious to the informed reader.
[Bruno] No, this is not the correct answer when PHP is enabled. The correct answer is specified in RFC 6790: the Egress of the LSP must remove the ELI/EL from the stack. (with PHP enabled the PH "will pop the forwarding label at the top of stack which will expose the ELI/EL" but must not remove the ELI/EL from the stack. To begin with, this PH may not be 6790 capable)

--Bruno

Yours Irrespectively,

John

Hi Bruno,

Regards,
Greg

RFC 6790 says

--Bruno

Orange Restricted

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: <[email-masked]>
Date: Mon, 4 Apr 2022 10:15:45 +0000

Hi John,

Comments inline [Bruno2]

Orange Restricted
Sent: Friday, April 1, 2022 8:28 PM

Hi,

Comments inline
Sent from my iPhone

On Apr 1, 2022, at 2:14 PM, [email-masked]<mailto:[email-masked]> wrote:

John,

And how do existing RFC 6790 implementations understand these 8 indicators without a software and/or hardware upgrade?
[Bruno2] Obviously, a transit LSR requiring to read indicators for a new feature need the new software to support both the indicators and the new feature. However, if it does not support it, it will still forward the packet.
The main issue is on the Egress: if the Egress does not support the new SPL, it will drop the packet. Hence the use of the new feature absolutely requires the Egress to support the SPL. Hence the deployment benefit when the SPL is 10 years and widely deployed.

As has been documented, this draft wreaks havoc in a network containing RFC 6790 compliant nodes
[Bruno2] Absolutely not. And this has not been documented.
--Bruno

--Bruno

Orange Restricted
Cc: mpls <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>; Greg Mirsky <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>

John,

Orange Restricted

Wim,

E2E OAM (E2E, Network]

HBH OAM [HBH, Network]

Flow ID [HBH, Network]

[Bruno]

--Bruno

Yours Irrespectively,

John

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Mon, 4 Apr 2022 10:24:41 +0000

Hi Greg,

Orange Restricted
From: Greg Mirsky <[email-masked]>
Sent: Friday, April 1, 2022 10:38 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: mpls <[email-masked]>; detnet WG <[email-masked]>; [email-masked]; Tony Li <[email-masked]>
Subject: Re: [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno,
thank you for pointing me to the text in Section 4.1. I agree that for the case described in RFC 6790, i.e., a single ELI, EL in the stack, that text sets clear requirements for disposing of ELI,EL.
[Bruno] Good. Thank you.

What I wanted to bring to the discussion is how a transit node does that in the SR-MPLS scenario (RFC 8662<https://datatracker.ietf.org/doc/html/rfc8662>).
[Bruno] The SR-MPLS scenario is just a scenario of MPLS LSP hierarchy (with a different name). When we have a hierarchy of LSPs, the MPLS architecture defines the term Ingress, Transit, Egress on a per LSP basis https://datatracker.ietf.org/doc/html/rfc3031#section-3.15
So in you example there are two cases for LSP/segment level m ("m" refers to RFC3031 section 3.15):
- if the LSR is swapping (or PHP) the top label, this LSR is a transit LSR for that LSP.
- if the LSR is the end of the LSP (i.e. the originator of the FEC), this LSR is an Egress for that LSP.
In both cases, RFC 6790 applies on that LSP.

Regards,
--Bruno

I think that there's no text in RFC 8662 that establishes equivalency between a transit node disposing of ELI,EL if ELI is at the top of the stack and the LER per RFC 6790. That might cause different interpretations resulting in different handling. How practically we can collect information about deployed solutions?

Regards,
Greg

RFC 6790 says

--Bruno

Orange Restricted

Regards,
Greg

Gentlebeings,

Regards,
Tony

Thank you.

---

From: <[email-masked]>
Date: Mon, 4 Apr 2022 13:26:47 +0000

Hi Gyan,

Orange Restricted
Sent: Saturday, April 2, 2022 9:33 PM

Hi Bruno

Responses in-line

On Fri, Apr 1, 2022 at 1:56 PM <[email-masked]<mailto:[email-masked]>> wrote:
Hi Gyan,

“a” So do we agree that RFC 6790 says “MUST be sent as zero and MUST ignored on receipt” ?
    Gyan Yes…but ..RFC 6790 defines the value of the TTL, however RFC 3032 defines the rules for processing all the fields of the label stack encoding.  RFC 3032 Section 2.4.2 describes the processing when TTL is set to 0 which means the TTL has expired and the packet must not be forwarded.  RFC 6790 section 4.2 bullet 4 states “The TTL for the EL MUST be zero to ensure that it is not used inadvertently for forwarding.”  My interpretation of that is even though the specification states the label stack is {TL,ELI,EL} and that at the PHP node when the TL is popped in the case of implicit null packets is forwarded to the egress LER which pops both ELI and EL.
[Bruno] Agreed.

  However, the sentence is stating the TTL must be 0 for EL so the packet cannot be inadvertently used for forwarding which I am interpreting as there is a possibility that the EL could be exposed at the top of the stack.  If hypothetically the case where the EL were not exposed were not possible then it would have been sufficient to state that the TTL is ignored on receipt and not worry about TTL processing which is a concern it seems as stated in RFC 6790.
[Bruno] I agree that your interpretation may be seem as logical. IMHO, authors of 6790 have added the EL TTL set to zero as an additional  safe guard in case of buggy LSR, and because that this decision cost was zero. (not because it was technically needed for correct operation). On this, I guess that the reference would be Kireeti. That been said, let’s agree that so far nobody has found a text in a spec that would lead a compliant LSR to forward based on an exposed EL. Quite the contrary, we both agree that RFC 6790 is quite explicit on the behavior (pop both labels ELI, EL)

  I think we have to understand why RFC 6790 is requiring the TTL of EL MUST be set to 0.  So based on implementations of MPLS their maybe possibilities that that even though the ELI must proceed the EL, maybe they are corner cases where the ELI is missing label imposition on the LER and thus the EL is exposed and required the EL to have its TTL set to 0.
[Bruno] “May be there are corner cases” looks like FUD to me. If there is a corner case causing problem, somebody should state it. Otherwise, it seems a bit to easy to block a proposal. E.g. “May be there are corner cases where an LSR looks at the Label Stack and may be unhappy with the ancillary data inserted in the stack (because clearly, as per MPLS architecture, the label stack is intended to be a stack of labels, not whatever we want)
  I think for this we would have to dig into implementations of MPLS label stack and see what implementations have done and if they are setting the TTL to 0 or not.
[Bruno] A compliant implementation MUST set the EL TTL to zero.
Is there anyone having an implementation not setting the TTL to zero? Please speak now.

  If we find that most all implementations of RFC 6790 are not setting the EL TTL to 0 then it’s possible we can reuse the TTL field as entropy label control field.
[Bruno] Agreed.  So far we have not found out that.

  We would have to really understand the corner case with missing ELI and how the EL can possibly be exposed and that possibility as reasons the authors of RFC 6790 stated the verbiage that has drawn my attention.
“b” Do we agree that this how reserved field are defined at the IETF (e.g. https://datatracker.ietf.org/doc/html/rfc7176#section-2.1.1) ?
   Gyan> So this I am guessing in RFC 7176 is an example of how Reserved field is encoded with TRILL.  I don’t see the TTL field being marked as reserved in RFC 3032.  I am missing your point here.
[Bruno] I have not said that the EL TTL has been defined as reserved per RFC 6790. I’m saying that it’s semantic is the one of an IETF reserved field (MUST be sent as zero, MUST be ignored on receipt). What matters is the behavior specified, not really the name.

“c” Do we agree that the way this field is specified (cf “a”) has always allowed the IETF to further extend this field?
 Gyan> RFC 3032 defines the label stack and RFC 6790 defines an entropy label for load balancing two new 4 byte label shims ELI and EL which use the same label stack encoding and processing rules defined in RFC 3032 MPLS-SHIM.  I don’t see anywhere in RFC 3031 or 3032 that states that the TTL field can be further extended or repurposed for other uses.  I am afraid that if we use the field repurposed for something else it may have dire consequences with load balancing.
[Bruno] As per the above, you should be more concerned with draft-kompella-mpls-mspl4fa which is happy repurpose many more TTL in the label stack. https://datatracker.ietf.org/doc/html/draft-kompella-mpls-mspl4fa-02#section-2.2

2)    Implicit null /PHP

So in summary, the EL is never exposed (as top label) and can never be used for forwarding.
 Gyan>  Please see my first long comment related to RFC 6790 section 4.2 bullet 4 states “The TTL for the EL MUST be zero to ensure that it is not used inadvertently for forwarding.”
[Bruno] ack. And please see my answer.

    Gyan> Section 3.5 of NS packet draft mentions how a IETF network slices map to a slice flow aggregate.
“ the routers correlate markers

[Bruno] OK. Note that even with the slice ID in the entropy label (which is orthogonal to the Indicators that we were discussing above), the size of the entropy field is still 20 bits as the slice ID is an information specific (constant) to a flow, and that will vary according to flow (otherwise we would only have a single slice). So we still have 20 bits for the entropy field. That been said, I agree that the N bits of the slice ID is, in general, less likely to carry the same entropy than the one of a good crypto hash. So there is possible degradation depending on the size of the slice ID (and assuming that the Entropy is currently generated by a good crypto hash, although RFC 6790 is silent on this (any function is compliant)

[Bruno] RFC 8662 “examines and describes how ELs are to be applied to Segment Routing MPLS.” It does not re-specify ELI, EL semantics

Regards

--Bruno

Regards,
--Bruno

Orange Restricted
From: Gyan Mishra <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, March 31, 2022 4:58 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]<mailto:[email-masked]>>
Cc: Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>; John E Drake <[email-masked]<mailto:[email-masked]>>; detnet WG <[email-masked]<mailto:[email-masked]>>; mpls <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet] [mpls] [Pals] draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno

Kind Regards

Gyan

Kind Regards

Gyan

Yours Irrespectively,

John

--Bruno

Tony

Orange Restricted

Thank you.

Gyan Mishra

M 301 502-1347

--

Gyan Mishra

M 301 502-1347

Thank you.
--

Gyan Mishra

M 301 502-1347

---

From: Gyan Mishra <[email-masked]>
Date: Tue, 5 Apr 2022 10:59:30 -0400

Hi  Bruno

Please see in-line Gyan2>

I agree with the idea and concepts of being able to reuse the existing
fields such as TTL which  provides flexibility for SPL and ancillary data
indicators and ease of time to market for deployments.  Agreed.

I am also in agreement on the idea of being able to use the 20 bit EL label
field for both entropy and possibly IETF Network slice aggregate.

We should make this an agenda topic for the next DT meeting and come to
hash this subject out and come consensus with the MPLS experts on the reuse
of TTL as well as other fields and impact and feasibility statement as well
as path forward so we can all make progress.

Orange Restricted

*From:* Gyan Mishra <[email-masked]>
*Sent:* Saturday, April 2, 2022 9:33 PM
*To:* DECRAENE Bruno INNOV/NET <[email-masked]>
*Cc:* Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; John
E Drake <[email-masked]>; detnet WG <[email-masked]>; mpls <
[email-masked]>; [email-masked]
*Subject:* Re: [Detnet] [mpls] [Pals]
draft-decraene-mpls-slid-encoded-entropy-label-id (was RE: Please review
the PALS/MPLS/DetNet Joint Session minutes)

Hi Bruno

Responses in-line

Hi Gyan,

“a” So do we agree that RFC 6790 says “MUST be sent as zero and MUST
ignored on receipt” ?

[Bruno] Agreed.

 Gyan2> Ack

seems as stated in RFC 6790.

[Bruno] I agree that your interpretation may be seem as logical. IMHO,
authors of 6790 have added the EL TTL set to zero as an additional  safe
guard in case of buggy LSR, and because that this decision cost was zero.
(not because it was technically needed for correct operation).

      Gyan2> Agreed for safeguard for corner case related to implementation
where EL may get exposed from a technical standpoint and for correct
operation to work, so the EL if exposed is not inadvertently used for
forwarding.  Agreed.

> On this, I guess that the reference would be Kireeti.
>
      Gyan2> I see his draft reuses TTL as well.  We would have the same
concern with Kireeti mspl4fa draft or any other draft repurpose of TTl
fields for a new use case.  We should take this topic up on reuse of TTL
and other fields for SPLs and indicators .

> That been said, let’s agree that so far nobody has found a text in a spec
that would lead a compliant LSR to forward based on an exposed EL. Quite
the contrary, we both agree that RFC 6790 is quite explicit on the behavior
(pop both labels ELI, EL)

 Gyan2> The way I read RFC 6790 the issue is not about buggy code or an LSR
being 100% compliant to tbe specification.  The issue is about a potential
corner case that exists could occur with vendor compliant code where the EL
could inadvertently get exposed as TL and used for forwarding and for that
reason the TTL must be set to 0.  This is more a physics issue not
compliance.

  I think we have to understand why RFC 6790 is requiring the TTL of EL
> MUST be set to 0.  So based on implementations of MPLS their maybe
> possibilities that that even though the ELI must proceed the EL, maybe they
> are corner cases where the ELI is missing label imposition on the LER and
> thus the EL is exposed and required the EL to have its TTL set to 0.
>
> [Bruno] “May be there are corner cases” looks like FUD to me. If there is
> a corner case causing problem, somebody should state it. Otherwise, it
> seems a bit to easy to block a proposal. E.g. “May be there are corner
> cases where an LSR looks at the Label Stack and may be unhappy with the
> ancillary data inserted in the stack (because clearly, as per MPLS
> architecture, the label stack is intended to be a stack of labels, not
> whatever we want)
>
>      Gyan2> Agreed.  We definitely don’t want to use fud by any means to
block a valuable proposal as this one and others that reuse the TTL and
other fields.

>   I think for this we would have to dig into implementations of MPLS label
> stack and see what implementations have done and if they are setting the
> TTL to 0 or not.
>

     Gyan2> Agreed

>  Gyan2> I don’t know of any.  We would have to research the vendor
> implementations.
>
>   If we find that most all implementations of RFC 6790 are not setting the
> EL TTL to 0 then it’s possible we can reuse the TTL field as entropy label
> control field.
>
>  Gyan2> Agreed
>   We would have to really understand the corner case with missing ELI and
> how the EL can possibly be exposed and that possibility as reasons the
> authors of RFC 6790 stated the verbiage that has drawn my attention.
>
> [Bruno] I have not said that the EL TTL has been defined as reserved per
> RFC 6790. I’m saying that it’s semantic is the one of an IETF reserved
> field (MUST be sent as zero, MUST be ignored on receipt). What matters is
> the behavior specified, not really the name.
>
>  Gyan2> Understood
>
> [Bruno] As per the above, you should be more concerned with
> draft-kompella-mpls-mspl4fa which is happy repurpose many more TTL in the
> label stack.
> https://datatracker.ietf.org/doc/html/draft-kompella-mpls-mspl4fa-02#section-2.2
>
>
       Gyan> We need to review with the MPLS experts on the DT and discuss
reuse of TTL and other fields for extension header semantics for path
forward and come to a consensus with the team.

>     Gyan2> Ack
>  Gyan2> Ack
>  Gyan2> Ack

*Gyan Mishra*

*M 301 502-1347*