Subject: [mpls] Indicators in the stack and ancillary data after the BoS

From: Loa Andersson <[email-masked]>
Date: Tue, 15 Jun 2021 14:31:22 +0200

Design Team,

When the Associated Channel first were desing it was assume that one 
associated channel per packet would be enough for the future.

The rules for where the GAL (indicator for an associated channel) should 
be located in the stack and that there could only be one GAL in the 
stack was strictly specified.

Now we are where the development has taken us to a point where the GAL 
can be found aqt almost any position in the stack.

Further changes are discussed.

- As label stack has been growing there is a risk that some LSRs that
   need to find the GAL might find that the GAL is below readable depth.
   The proposed remedy is to allow entering a copy of the first GAL
   higher in the stack, i.e. more than one GAL in the stack point to the
   same ACH.

- It has also been discussed to carry more than one ACH after the BoS,
   i.e. you'll need more than one different GALs in the stack pointing to
   different ACH's.
   (I think the if this statement is true that would require a "slight",
   but backwards compatible redesign of the GAL.)

- It has also been discussed to carry more then set of (the ACH is one)
   ancillary data after the BoS, and e.g. FAI, GAL, EHI, OAM indicator,
   etc. mixed in the label stack.

The issue is that the set of ancillary data after the BoS might will 
interfere and that we need a strong mechanism to find the ancillary data 
that "belongs to" a certain indicator.

I think that we first need to decide what is required and after that
design a solution.

The correctness of statements about need to be verified and if necessary 
corrected.

/Loa

PS

Can someone tell me if I should use anxillary or ancillary?

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Bronze Dragon Consulting             phone: +46 739 81 21 64

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 15 Jun 2021 13:08:24 +0000

Loa and all,

I have to admit that I have not been following the work of the DT closely for the last month or so.

Therefore I may be asking some questions that have already been asked and even answered. In which case you can simply say so(hopefully with some reference).

However I still think these questions should be asked.

Please see these questions - with some clarifications - inline below.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Email:   [email-masked]

-----Original Message-----
From: mpls <[email-masked]> On Behalf Of Loa Andersson
Sent: Tuesday, June 15, 2021 3:31 PM
To: [email-masked]
Subject: [EXTERNAL] [mpls] Indicators in the stack and ancillary data after the BoS

Design Team,

When the Associated Channel first were desing it was assume that one associated channel per packet would be enough for the future.

[[Sasha]] Actually the initial GACh design (RFC 5586<https://datatracker.ietf.org/doc/html/rfc5586>) has provided for a TLV to follow the GACh fixed header, with the provision that the presence of such a TLV is inferred from the GACh type in the fixed header. However, this mechanism has never been used, and eventually GACh TLVs have been retired (RFC 7026<https://datatracker.ietf.org/doc/html/rfc7026>).

I wonder if the need for multiple GACh in a packet could be addressed by GACh TLVs?

The rules for where the GAL (indicator for an associated channel) should be located in the stack and that there could only be one GAL in the stack was strictly specified.

[[Sasha]] GAL has been restricted to be just the BoS label in the MPLS-TP environments, but this restriction does not apply  "to other environments".

However, I am not aware of any rules for handling GAL if it is exposed to the MPLS forwarder (i.e. becomes top of the stack) while being followed by some other labels. RFC 5586 only says that "A receiving LSR, LER, or PE MUST NOT forward a G-ACh packet to  another node based on the GAL label". This is quite different from the definition of handling for the Routing Alert label. From my POV:

*         Lack of these definitions effectively precludes usage of GAL in a non-BoS position

*         Any such definition, if introduced today, will not be backward compatible with the existing implementations.

Now we are where the development has taken us to a point where the GAL can be found at almost any position in the stack.

[[Sasha]] Please see my previous comment.

   same ACH.[[Sasha]] The LSR normally looks just at the top label in the stack. If GAL is not in the top position, why should it react to it? And if it is in the top position but not BoS, how will the packet be forwarded? In addition, GAL does not "point to an ACH". It says that ACH immediately follows the BoS label, so I fail to understand how a GAL can point to a specific ACH.

   different ACH's.[[Sasha]] By different GALs do you mean different reserved - or extended reserved - label values?

The issue is that the set of ancillary data after the BoS might will interfere and that we need a strong mechanism to find the ancillary data that "belongs to" a certain indicator.

I think that we first need to decide what is required and after that design a solution.

The correctness of statements about need to be verified and if necessary corrected.

/Loa

PS

--

Loa Andersson                        email: [email-masked]<mailto:[email-masked]>

Senior MPLS Expert                          [email-masked]<mailto:[email-masked]>

_______________________________________________

mpls mailing list

[email-masked]<mailto:[email-masked]>

https://clicktime.symantec.com/389rPq1ADpF9WRstEF2jdmj6H2?u=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls

Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.

Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.

---

From: <[email-masked]>
Date: Thu, 17 Jun 2021 03:23:55 +0800 (CST)

Hi Sasha,
I agree with all your comments regarding the GAL. Indeed, GAL in non-MPLS-TP environments may be placed anywhere in a label stack. But how an LSR acts in the case GAL is not BoS has been left under-defined in RFC 5586. As I understand it, you are concerned that any work on closing that gap would not be backward compatible with the existing implementations handling GAL. While that might be the case, I think that the Open DT may give it a try and investigate how the existing systems will handle GAL being not the BoS label.

Regards,
Greg Mirsky
Sr. Standardization Expert
预研标准部/有线研究院/有线产品经营部  Standard Preresearch Dept./Wireline Product R&D Institute/Wireline Product Operation Division
E: [email-masked]
www.zte.com.cn
------------------Original Mail------------------
Sender: AlexanderVainshtein
To: Loa Andersson;
CC: [email-masked];
Date: 2021/06/15 06:09
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS
_______________________________________________
mpls mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mpls

Loa and all,
Please see these questions – with some clarifications –  inline below.
-----Original Message-----
From: mpls <[email-masked]> On Behalf Of Loa Andersson
Sent: Tuesday, June 15, 2021 3:31 PM
To: [email-masked]
Subject: [EXTERNAL] [mpls] Indicators in the stack and ancillary data after the BoS
Design Team,
When the Associated Channel first were desing it was assume that one associated channel per packet would be enough for the future.
[[Sasha]] Actually the initial GACh design (RFC 5586) has provided for a TLV to follow the GACh fixed header,  with the provision that the presence of such a TLV is inferred from the GACh type in the fixed header. However, this mechanism has never been used, and eventually GACh TLVs have been retired (RFC  7026).
I wonder if the need for multiple GACh in a packet could be addressed by GACh TLVs?
The rules for where the GAL (indicator for an associated channel) should be located in the stack and that there could only be one GAL in the stack was strictly specified.
[[Sasha]] GAL has been restricted to be just the BoS label in the MPLS-TP environments, but this restriction does not apply  “to other environments”.
However, I am not aware of any rules for handling GAL if it is exposed to the MPLS forwarder (i.e. becomes top of the stack) while being followed by some other labels. RFC 5586 only says that “A  receiving LSR, LER, or PE MUST NOT forward a G-ACh packet to  another node based on the GAL label”. This is quite different from the definition of handling for the Routing Alert label. From my POV:
·         Lack of these definitions effectively precludes usage of GAL in a non-BoS position
·         Any such definition, if introduced today, will not be backward compatible with the existing implementations.
Now we are where the development has taken us to a point where the GAL can be found at almost any position in the stack.
[[Sasha]] Please see my previous comment.
Further changes are discussed.
- As label stack has been growing there is a risk that some LSRs that
need to find the GAL might find that the GAL is below readable depth.
The proposed remedy is to allow entering a copy of the first GAL
higher in the stack, i.e. more than one GAL in the stack point to the
same ACH.[[Sasha]] The LSR normally looks just at the top label in the stack. If GAL is not in the top position, why should it react to it? And if it is in the top position but not BoS, how will the  packet be forwarded? In addition, GAL does not “point to an ACH”. It says that ACH immediately follows the BoS label, so I fail to understand how a GAL can point to a specific ACH.
- It has also been discussed to carry more than one ACH after the BoS,
i.e. you'll need more than one different GALs in the stack pointing to
different ACH's.[[Sasha]] By different GALs do you mean different reserved – or extended reserved – label values?
(I think the if this statement is true that would require a "slight",
but backwards compatible redesign of the GAL.)
- It has also been discussed to carry more then set of (the ACH is one)
ancillary data after the BoS, and e.g. FAI, GAL, EHI, OAM indicator,
etc. mixed in the label stack.
The issue is that the set of ancillary data after the BoS might will interfere and that we need a strong mechanism to find the ancillary data that "belongs to" a certain indicator.
I think that we first need to decide what is required and after that design a solution.
The correctness of statements about need to be verified and if necessary corrected.
/Loa
PS
Can someone tell me if I should use anxillary or ancillary?
--
Loa Andersson                        email:  [email-masked]
Senior MPLS Expert                           [email-masked]
Bronze Dragon Consulting             phone: +46 739 81 21 64
_______________________________________________
mpls mailing list
[email-masked]
https://clicktime.symantec.com/389rPq1ADpF9WRstEF2jdmj6H2?u=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance  or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 17 Jun 2021 06:45:15 +0000

Hi Greg,

Lots of thanks for a prompt and highly relevant response to my email.

Please see some comments inline below.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: [email-masked] <[email-masked]>
Sent: Wednesday, June 16, 2021 10:24 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re:[mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi Sasha,

I agree with all your comments regarding the GAL. Indeed, GAL in non-MPLS-TP environments may be placed anywhere in a label stack. But how an LSR acts in the case GAL is not BoS has been left under-defined in RFC 5586.

[[Sasha]] From my POV “under-defined” above is a gross understatement.

As I understand it, you are concerned that any work on closing that gap would not be backward compatible with the existing implementations handling GAL.

[[Sasha]] I am not aware of any implementations that handle GAL that is not  at the bottom of the label stack. Did I miss some?

While that might be the case, I think that the Open DT may give it a try and investigate how the existing systems will handle GAL being not the BoS label.

[[Sasha]] Great minds think alike! One useful step could be collecting the known actual behavior of popular implementations in this case, say, by running a survey among the vendors – what do you think?

Regards,

Greg Mirsky

E: [email-masked]<mailto:[email-masked]>

https://clicktime.symantec.com/3XViqah8L6FZgSaNkbm23jd6H2?u=www.zte.com.cn

------------------Original Mail------------------

To: Loa Andersson;

CC: [email-masked]<mailto:[email-masked]>;

Date: 2021/06/15 06:09

Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS _______________________________________________

mpls mailing list

https://clicktime.symantec.com/35rH1i9V3fZqennDh8n1jpv6H2?u=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls

Loa and all,

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Email:   [email-masked]<mailto:[email-masked]>

-----Original Message-----

From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Loa Andersson

To: [email-masked]<mailto:[email-masked]>

Subject: [EXTERNAL] [mpls] Indicators in the stack and ancillary data after the BoS Design Team, When the Associated Channel first were desing it was assume that one associated channel per packet would be enough for the future.

- As label stack has been growing there is a risk that some LSRs that need to find the GAL might find that the GAL is below readable depth.

The proposed remedy is to allow entering a copy of the first GAL higher in the stack, i.e. more than one GAL in the stack point to the same ACH.[[Sasha]] The LSR normally looks just at the top label in the stack. If GAL is not in the top position, why should it react to it? And if it is in the top position but not BoS, how will the  packet be forwarded? In addition, GAL does not “point to an ACH”. It says that ACH immediately follows the BoS label, so I fail to understand how a GAL can point to a specific ACH.

- It has also been discussed to carry more than one ACH after the BoS, i.e. you'll need more than one different GALs in the stack pointing to different ACH's.[[Sasha]] By different GALs do you mean different reserved – or extended reserved – label values?

(I think the if this statement is true that would require a "slight", but backwards compatible redesign of the GAL.)

- It has also been discussed to carry more then set of (the ACH is one) ancillary data after the BoS, and e.g. FAI, GAL, EHI, OAM indicator, etc. mixed in the label stack.

/Loa

PS

--

mpls mailing list

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 17 Jun 2021 08:35:58 +0100

> On 17 Jun 2021, at 07:45, Alexander Vainshtein <[email-masked]> wrote:
> 

That is actually a considerable amount of work that will take a while.

It seems to me that an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable.

The original assumed processing model is to take the context of the PW label or PW+FAT label, discover the GAL and then process the GAL in the context of the PW label.

When we extended GAL to apply to LSPs we again had the model that the GAL operated in the context of the LSP label that preceded it for context. It was still BoS.

Putting the GAL further up the stack is a new behaviour.

If it arrives at an LSR that knows the new semantic all is good.

If it arrives at an LSR that does not know the new semantic then 

a) An error has occurred either in setting up the LSP, or in forwarding.

b) The behaviour at the receiving node is unpredictable, but in any well written implementation should just result in the packet being dropped and counted as with any other Mal-formed packet.

So I would think that it is OK to have multiple GALs and GALs not at BoS IFF the creator of the LSP ensured that all LSRs on the LSP, including ECMP and FRR paths that found the GAL at ToS were known to be able to process it correctly.

A GAL not at BoS and not at ToS should not be inspected or processed by any LSR that did not know what it was doing, and to attempt to precess it would be a violation of the normal MPLS processing model.

- Stewart

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 17 Jun 2021 09:06:37 +0000

Stewart,
I fully agree with your statement that “an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable”.

Regarding your statement “it is OK to have multiple GALs and GALs not at BoS IFF the creator of the LSP ensured that all LSRs on the LSP, including ECMP and FRR paths that found the GAL at ToS were known to be able to process it correctly”:

1.       I fully agree with this statement as a general restriction

2.       Quite a lot of things have to be done in order to make this restriction work including at least:

a.       The definition of correct processing of GAL at ToS but not at BoS must be provided

b.       Advertisement of ability to process GAL not at BoS correctly in IGP and BGP must be defined

c.       Ability to set up network-wide paths that only cross nodes that process GAL correctly must be provided for different techniques (RSVP-TE, SR-TE, FlexAlgo. BGP-LU etc.)
It is still possible that, after all this work, we shall find out  that the benefits of supporting GAL at ToS but not BoS will be only available in the networks where all the nodes support the new functionality because presence of non-supporting nodes imposes too many restrictions on connectivity and/or resilience.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Stewart Bryant <[email-masked]>
Sent: Thursday, June 17, 2021 10:36 AM
To: Alexander Vainshtein <[email-masked]>
Cc: Stewart Bryant <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

On 17 Jun 2021, at 07:45, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:

- Stewart

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Thu, 17 Jun 2021 14:15:53 +0000

Hi,

It’s not clear how we could put a GAL not at a BoS:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |                              ACH                              |

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |                         ACH TLV Header                        |

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |                                                               ~

   ~                     zero or more ACH TLVs                     ~

   ~                                                               |

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |                                                               ~

   ~                        G-ACh Message                          ~

   ~                                                               |

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                      Figure 2: G-ACh Packet Payload

If the GAL does not have S-bit set, wouldn’t a transit LSR treat any 4-ocet field (i.e. those in the above Figure) after that GAL as a label+TOS+S+TTL? If that 4-octet field has the S-bit set, the transit LSR will think the label stack ends there even though that’s just part of the ACH.

Or are you saying that a GAL not at the BoS will not have the ACH following it?

Jeffrey

From: mpls <[email-masked]> On Behalf Of Alexander Vainshtein
Sent: Thursday, June 17, 2021 5:07 AM
To: Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

[External Email. Be cautious of content]

Stewart,

     *   The definition of correct processing of GAL at ToS but not at BoS must be provided
     *   Advertisement of ability to process GAL not at BoS correctly in IGP and BGP must be defined
     *   Ability to set up network-wide paths that only cross nodes that process GAL correctly must be provided for different techniques (RSVP-TE, SR-TE, FlexAlgo. BGP-LU etc.)
It is still possible that, after all this work, we shall find out  that the benefits of supporting GAL at ToS but not BoS will be only available in the networks where all the nodes support the new functionality because presence of non-supporting nodes imposes too many restrictions on connectivity and/or resilience.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, June 17, 2021 10:36 AM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: Stewart Bryant <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

- Stewart

Juniper Business Use Only

---

From: John E Drake <[email-masked]>
Date: Thu, 17 Jun 2021 14:44:48 +0000

Hi,

Comment inline.

Yours Irrespectively,

John

Stewart,
I fully agree with your statement that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".

Regarding your statement "it is OK to have multiple GALs and GALs not at BoS IFF the creator of the LSP ensured that all LSRs on the LSP, including ECMP and FRR paths that found the GAL at ToS were known to be able to process it correctly":

[JD]  As we start to add all sorts of nifty functions both within the stack and after it, we need to add these functions in such a way that a transit LSR that does not understand one or more of them still will be able to forward a packet containing them.  For example, if we were to add ancillary data in the label stack, it needs to be done in such a way as to allow a transit router to skip over the ancillary data that it doesn't understand.

I.e., incremental deployment is a fact of life and we have to live with it.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

While that might be the case, I think that the Open DT may give it a try and investigate how the existing systems will handle GAL being not the BoS label.
[[Sasha]] Great minds think alike! One useful step could be collecting the known actual behavior of popular implementations in this case, say, by running a survey among the vendors - what do you think?

- Stewart

---

From: Loa Andersson <[email-masked]>
Date: Thu, 17 Jun 2021 16:46:18 +0200

DT,

Responded to Jeffrey's mail, but it is intended to address the entire 
discussion.

There seem to be enough issues to sort out around the GAL/ACH pair, and 
I was worried about a set of other indicators and the data that they 
might want to put "after the BoS". So far I have seen no real effort to 
address the interference's this might lead to.

Further inline

On 17/06/2021 16:15, Jeffrey (Zhaohui) Zhang wrote:
> If the GAL does not have S-bit set, wouldn’t a transit LSR treat any 
> 4-ocet field (i.e. those in the above Figure) after that GAL as a 
> label+TOS+S+TTL? If that 4-octet field has the S-bit set, the transit 
> LSR will think the label stack ends there even though that’s just part 
> of the ACH.
> 

Well, as far as I understand a GAL which does not have the NoS-bit set 
will have other labels after itself. The BoS-bit will be found deeper 
down stack and the ACH will immediately fo9llow the BoS.

Yes there are issues here, but I'd like to see the DT address multiple 
indicators in the stack and multiple sets of ancillary data after the BoS.

I think we need to nail down the relevant questiuons first, and start 
working on solutions after that.

/Loa
> 
> Jeffrey
> 
> *From:*mpls <[email-masked]> *On Behalf Of *Alexander Vainshtein
> *Sent:* Thursday, June 17, 2021 5:07 AM
> *To:* Stewart Bryant <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary 
> data after the BoS
> 
> *[External Email. Be cautious of content]*
> 
> Stewart,
> 
> I fully agree with your statement that “an old implementation that 
> received a ToS GAL not at BoS would at best throw an exception or worst 
> be unpredictable”.
> 
> Regarding your statement “it is OK to have multiple GALs and GALs not at 
> BoS IFF the creator of the LSP ensured that all LSRs on the LSP, 
> including ECMP and FRR paths that found the GAL at ToS were known to be 
> able to process it correctly”:
> 
>      1. The definition of correct processing of GAL at ToS but not at
>         BoS must be provided
>      2. Advertisement of ability to process GAL not at BoS correctly in
>         IGP and BGP must be defined
>      3. Ability to set up network-wide paths that only cross nodes that
>         process GAL correctly must be provided for different techniques
>         (RSVP-TE, SR-TE, FlexAlgo. BGP-LU etc.)
> 
> It is still possible that, after all this work, we shall find out  that 
> the benefits of supporting GAL at ToS but not BoS will be only available 
> in the networks where all the nodes support the new functionality 
> because presence of non-supporting nodes imposes too many restrictions 
> on connectivity and/or resilience.
> 
> Regards,
> 
> Sasha
> 
> Office: +972-39266302
> 
> Cell:      +972-549266302
> 
> Email: [email-masked] <mailto:[email-masked]>
> 
> *From:*Stewart Bryant <[email-masked] 
> <mailto:[email-masked]>>
> *Sent:* Thursday, June 17, 2021 10:36 AM
> *To:* Alexander Vainshtein <[email-masked] 
> <mailto:[email-masked]>>
> *Cc:* Stewart Bryant <[email-masked] 
> <mailto:[email-masked]>>; [email-masked] 
> <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary 
> data after the BoS
> 
>     On 17 Jun 2021, at 07:45, Alexander Vainshtein
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>     While that might be the case, I think that the Open DT may give it a
>     try and investigate how the existing systems will handle GAL being
>     not the BoS label.
> 
>     */[[Sasha]] Great minds think alike! One useful step could be
>     collecting the known actual behavior of popular implementations in
>     this case, say, by running a survey among the vendors – what do you
>     think?/*
> 
> It seems to me that an old implementation that received a ToS GAL not at 
> BoS would at best throw an exception or worst be unpredictable.
> 
> The original assumed processing model is to take the context of the PW 
> label or PW+FAT label, discover the GAL and then process the GAL in the 
> context of the PW label.
> 
> When we extended GAL to apply to LSPs we again had the model that the 
> GAL operated in the context of the LSP label that preceded it for 
> context. It was still BoS.
> 
> b) The behaviour at the receiving node is unpredictable, but in any well 
> written implementation should just result in the packet being dropped 
> and counted as with any other Mal-formed packet.
> 
> So I would think that it is OK to have multiple GALs and GALs not at BoS 
> IFF the creator of the LSP ensured that all LSRs on the LSP, including 
> ECMP and FRR paths that found the GAL at ToS were known to be able to 
> process it correctly.
> 
> A GAL not at BoS and not at ToS should not be inspected or processed by 
> any LSR that did not know what it was doing, and to attempt to precess 
> it would be a violation of the normal MPLS processing model.
> 
> - Stewart
> 
> 
> Notice: This e-mail together with any attachments may contain 
> information of Ribbon Communications Inc. and its Affiliates that is 
> confidential and/or proprietary for the sole use of the intended 
> recipient. Any review, disclosure, reliance or distribution by others or 
> forwarding without express permission is strictly prohibited. If you are 
> not the intended recipient, please notify the sender immediately and 
> then delete all copies, including any attachments.
> 
> 

--

---

From: Loa Andersson <[email-masked]>
Date: Thu, 17 Jun 2021 16:53:35 +0200

DT,

Quick comments on John's mail:

On 17/06/2021 16:44, John E Drake wrote:
> *//*
> 
> */[JD] As we start to add all sorts of nifty functions both within the 
> stack and after it, we need to add these functions in such a way that a 
> transit LSR that does not understand one or more of them still will be 
> able to forward a packet containing them.  

Yes, we must have an architecture that supports that.

> For example, if we were to 
> add ancillary data in the label stack, it needs to be done in such a way 
> as to allow a transit router to skip over the ancillary data that it 
> doesn’t understand./*

Ancillary "data in the stack"?  Does not all ancillary data go after the 
BoS?

/Loa
> 
> *//*
> 
> */I.e., incremental deployment is a fact of life and we have to live 
> with it./*
> 

--

---

From: John E Drake <[email-masked]>
Date: Thu, 17 Jun 2021 14:55:34 +0000

Loa,

I thought the FAI proposal put ancillary data in the stack.

Yours Irrespectively,

John

> From: Loa Andersson <[email-masked]>
> Sent: Thursday, June 17, 2021 10:54 AM
> To: John E Drake <[email-masked]>; Alexander Vainshtein
> <[email-masked]>; Stewart Bryant
> <[email-masked]>
> Cc: [email-masked]
> > stack and after it, we need to add these functions in such a way that
> > a transit LSR that does not understand one or more of them still will
> > be able to forward a packet containing them.
> 
> > add ancillary data in the label stack, it needs to be done in such a
> > way as to allow a transit router to skip over the ancillary data that
> > it doesn't understand./*
> 
> Ancillary "data in the stack"?  Does not all ancillary data go after the BoS?
> 
> /Loa
> >
> > *//*
> >

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Thu, 17 Jun 2021 15:01:32 +0000

Hi Loa,

> but I'd like to see the DT address multiple indicators in the stack and multiple sets of ancillary data after the BoS.

I think the earlier emails of this email thread were talking about multiple indicators in the stack; for multiple set of ancillary data after the BoS, either the extended ACH or the proposed MPLS/generic extension headers or a merge of those proposals should be able to handle it. This is alluded to the DataAfterBOS wiki page.

Thanks.

Jeffrey

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Thursday, June 17, 2021 10:46 AM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

DT,

Further inline

/Loa
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/mpls__;!!NEt6yMaO-gk!RVgTGVbknjgIjv3x-q8ob1JglFKOP6qKkgAcCSPbeBMMj2AnexFnPevXopeK1a6u$
>

--

---

From: Loa Andersson <[email-masked]>
Date: Thu, 17 Jun 2021 17:03:49 +0200

John,

I might have missed something, but I have not seen that defined 
anywhere; if we want a stand alone bit set in the FAI to be ancillary 
data, what is a bit that points to data after the BoS? In my mind any 
bit in the FAI is anindicator, that is why it named FAI.

But I'm willing to changed my mind about the naming as long as we have 
an agreement.

/Loa

--

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 17 Jun 2021 16:07:41 +0100

Hi Sasha

I completely agree on all points.

Stewart

> On 17 Jun 2021, at 10:06, Alexander Vainshtein <[email-masked]> wrote:
> 
> Stewart,
> On 17 Jun 2021, at 07:45, Alexander Vainshtein <[email-masked] <mailto:[email-masked]>> wrote:
>

---

From: Haoyu Song <[email-masked]>
Date: Thu, 17 Jun 2021 16:56:09 +0000

My opinion is to not tamper with ACH anymore because it's designed for control channel only and so far for a special scenario. The constraints on GAL and format of ACH are hard to adapt to the new use case requirements. 

Thanks!
Haoyu

-----Original Message-----
From: mpls <[email-masked]> On Behalf Of Jeffrey (Zhaohui) Zhang
Sent: Thursday, June 17, 2021 8:02 AM
To: Loa Andersson <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi Loa,

Thanks.

Jeffrey

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Thursday, June 17, 2021 10:46 AM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

DT,

Responded to Jeffrey's mail, but it is intended to address the entire discussion.

There seem to be enough issues to sort out around the GAL/ACH pair, and I was worried about a set of other indicators and the data that they might want to put "after the BoS". So far I have seen no real effort to address the interference's this might lead to.

Further inline

On 17/06/2021 16:15, Jeffrey (Zhaohui) Zhang wrote:
> Hi,
>
> It's not clear how we could put a GAL not at a BoS:
>
> If the GAL does not have S-bit set, wouldn't a transit LSR treat any 
> LSR will think the label stack ends there even though that's just part 
> of the ACH.
>

Well, as far as I understand a GAL which does not have the NoS-bit set will have other labels after itself. The BoS-bit will be found deeper down stack and the ACH will immediately fo9llow the BoS.

Yes there are issues here, but I'd like to see the DT address multiple indicators in the stack and multiple sets of ancillary data after the BoS.

I think we need to nail down the relevant questiuons first, and start working on solutions after that.

/Loa
> I fully agree with your statement that "an old implementation that 
> received a ToS GAL not at BoS would at best throw an exception or 
> worst be unpredictable".
>
> Regarding your statement "it is OK to have multiple GALs and GALs not 
> at BoS IFF the creator of the LSP ensured that all LSRs on the LSP, 
> including ECMP and FRR paths that found the GAL at ToS were known to 
> be able to process it correctly":
>
>  1. I fully agree with this statement as a general restriction  2. 
> Quite a lot of things have to be done in order to make this
>     restriction work including at least:
>
> It is still possible that, after all this work, we shall find out  
> that the benefits of supporting GAL at ToS but not BoS will be only 
> available in the networks where all the nodes support the new 
> functionality because presence of non-supporting nodes imposes too 
> many restrictions on connectivity and/or resilience.
>
> Regards,
>
> Sasha
>
> Office: +972-39266302
>
> Cell:      +972-549266302
>
> <mailto:[email-masked]>
>
>     this case, say, by running a survey among the vendors - what do you
>     think?/*
>
> It seems to me that an old implementation that received a ToS GAL not 
> at BoS would at best throw an exception or worst be unpredictable.
>
> label or PW+FAT label, discover the GAL and then process the GAL in 
> the context of the PW label.
>
> b) The behaviour at the receiving node is unpredictable, but in any 
> well written implementation should just result in the packet being 
> dropped and counted as with any other Mal-formed packet.
>
> So I would think that it is OK to have multiple GALs and GALs not at 
> including ECMP and FRR paths that found the GAL at ToS were known to 
> be able to process it correctly.
>
> A GAL not at BoS and not at ToS should not be inspected or processed 
> by any LSR that did not know what it was doing, and to attempt to 
> precess it would be a violation of the normal MPLS processing model.
>
> - Stewart
>
>
> recipient. Any review, disclosure, reliance or distribution by others 
> or forwarding without express permission is strictly prohibited. If 
> you are not the intended recipient, please notify the sender 
> immediately and then delete all copies, including any attachments.
>
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furld
> efense.com%2Fv3%2F__https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2F
> mpls__%3B!!NEt6yMaO-gk!RVgTGVbknjgIjv3x-q8ob1JglFKOP6qKkgAcCSPbeBMMj2A
> nexFnPevXopeK1a6u%24&amp;data=04%7C01%7Chsong%40futurewei.com%7Ccc49de
> 9585a24092e29708d931a0e327%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%
> 7C637595389337881384%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI
> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=5et4Juc3IjG
> dfux%2FR5MsJnuTYDWL6S4pZ8uz3F6h34Q%3D&amp;reserved=0
>

--

Juniper Business Use Only
_______________________________________________
mpls mailing list
[email-masked]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls&amp;data=04%7C01%7Chsong%40futurewei.com%7Ccc49de9585a24092e29708d931a0e327%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637595389337881384%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=XQlRpwkgODLRxcIjyMYyPMiCF2KDC0Y7GG4O8VGESnw%3D&amp;reserved=0

---

From: Huub van Helvoort <[email-masked]>
Date: Fri, 18 Jun 2021 11:06:53 +0200

Hi Stewad, Sasha,

I second that.

Cheers, Huub.

=====

On 17/06/2021 17:07, Stewart Bryant wrote:
>> On 17 Jun 2021, at 10:06, Alexander Vainshtein 
>> <[email-masked] <mailto:[email-masked]>> 
>> wrote:
>>
>> Stewart,
>> worst be unpredictable”.
>> Regarding your statement “it is OK to have multiple GALs and GALs not 
>> be able to process it correctly”:
>> 1.I fully agree with this statement as a general restriction
>> 2.Quite a lot of things have to be done in order to make this 
>> a.The definition of correct processing of GAL at ToS but not at BoS 
>> must be provided
>> b.Advertisement of ability to process GAL not at BoS correctly in IGP 
>> and BGP must be defined
>> c.Ability to set up network-wide paths that only cross nodes that 
>> *Sent:*Thursday, June 17, 2021 10:36 AM
>> *To:*Alexander Vainshtein <[email-masked] 
>> *Cc:*Stewart Bryant <[email-masked] 
>> *Subject:*Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary 
>> data after the BoS
>>
>>
>>     While that might be the case, I think that the Open DT may give it
>>     a try and investigate how the existing systems will handle GAL
>>     being not the BoS label.
>>     this case, say, by running a survey among the vendors – what do
>>     you think?/*
>>

-- 
================================================================
Always remember that you are unique...just like everyone else...

---

From: Loa Andersson <[email-masked]>
Date: Sun, 20 Jun 2021 13:11:25 +0200

Jeffrey,

> I think the earlier emails of this email thread were talking about multiple indicators in the stack; for multiple set of ancillary data after the BoS, either the extended ACH or the proposed MPLS/g…

hmm - yes partly, but there are several indicators proposed in several 
drafts

  draft-gandhi-mpls-ioam-sr has an E"E indicaor and an HBH indicator

  draft-kompella-mpls-mspl4fa make use of TC field and TTL of a special 
purpose label (FAI) as indicators

  there has also been discussion about putting more than one GAL in the 
stack, i.e. differerent GALs pointing to different ACHs.

  draft-many-mpls-multiple-gal proposes to add a copy of the GAL higher
uop the stack so that LSRs with a too shallow maximun readable depth 
might reach the GAL

  there has also been discussion about putting more than one GAL in the 
stack, i.e. differerent GALs pointing to different ACHs.

  draft-song-mpls-eh-indicator have a list of potential indicators, that 
is also telling if the EH should be processed on every EH capable node 
or "just" at ingress and egress

The FAI might put ancillary data after the BoS.

I think we need to have a comprehensive discussion

- first what we want to have
- second how when re-direct by an indicator we find the
   ancillary data that belongs to that indicator.

/Loa

--

---

From: <[email-masked]>
Date: Mon, 21 Jun 2021 08:50:00 +0000

[+ authors of draft-song-mpls-eh-indicator]

> From: mpls [mailto:[email-masked]] On Behalf Of Loa Andersson
> 
> Jeffrey,
> 
> 
> > I think the earlier emails of this email thread were talking about multiple indicators
> in the stack; for multiple set of ancillary data after the BoS, either the extended
> ACH or the proposed MPLS/generic extension headers or a merge of those
> proposals should be able to handle it. This is alluded to the DataAfterBOS wiki
> page.
> 

The following draft proposes a way to carry indicators.
https://datatracker.ietf.org/doc/html/draft-decraene-mpls-slid-encoded-entropy-label-id#section-2

It's short (less than one page) and backward compatible for LSR & LER.

Referring to table 5 of draft-song-mpls-eh-indicator it also
-  does not require additional label assuming Entropy Label is already used for load balancing 
-  does not require an additional scarce resource (Special-Purpose MPLS Label value)
- allows location freedom
- does not need control plane extension

Could the authors of draft-song-mpls-eh-indicator update their table 5 in order to include the above draft?

Thanks,
Regards,
--Bruno
 
> > To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein
> <[email-masked]>; Stewart Bryant <[email-masked]>
> > Cc: [email-masked]
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/mpls__;!!NEt6yM
> aO-gk!RVgTGVbknjgIjv3x-
> q8ob1JglFKOP6qKkgAcCSPbeBMMj2AnexFnPevXopeK1a6u$
> >>
> >
> > --
> >

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Loa Andersson <[email-masked]>
Date: Mon, 21 Jun 2021 11:39:57 +0200

Haoyu, DT

> My opinion is to not tamper with ACH anymore because it's designed for control channel only and so far for a special scenario. The constraints on GAL and format of ACH are hard to adapt to the new…
I think this is a position that is possible to defend.

One question though.

RFC 5586 specifies "that the ACH appears immediately after the bottom of 
the label stack."

If the slot immediately after the label stack is reserved for the ACH 
does this mean the no other ancillary data maybe inserted in this 
position, e.g. MPLS EH's, given that there is a GAL in the stack?

/Loa

--

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 21 Jun 2021 09:55:25 +0000

Loa and all,

I fully agree with the proposal "to not tamper with ACH anymore".

From my POV, this includes (by implication) not tampering also with GAL as well.

As for the question " If the slot immediately after the label stack is reserved for the ACH does this mean the no other ancillary data may be inserted in this position, e.g. MPLS EH's, given that there is a GAL in the stack" the answer, IMHO, is YES.

However, it is quite possible to carry any kind of new information in the ACH, similar to the way this has been done in Section 3 of RFC 8169<https://datatracker.ietf.org/doc/html/rfc8169#section-3> where G-ACH is used for residence time measurement.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Monday, June 21, 2021 12:40 PM
To: Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Haoyu, DT

One question though.

RFC 5586 specifies "that the ACH appears immediately after the bottom of the label stack."

If the slot immediately after the label stack is reserved for the ACH does this mean the no other ancillary data maybe inserted in this position, e.g. MPLS EH's, given that there is a GAL in the stack?

/Loa

> From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Jeffrey (Zhaohui)

> Zhang

> To: Loa Andersson <[email-masked]<mailto:[email-masked]>>; Alexander Vainshtein

> <[email-masked]<mailto:[email-masked]>>; Stewart Bryant

> <[email-masked]<mailto:[email-masked]>>

> Cc: [email-masked]<mailto:[email-masked]>

> Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary

> From: Loa Andersson <[email-masked]<mailto:[email-masked]>>

> To: Jeffrey (Zhaohui) Zhang <[email-masked]<mailto:[email-masked]>>; Alexander Vainshtein

> <[email-masked]<mailto:[email-masked]>>; Stewart Bryant

> <[email-masked]<mailto:[email-masked]>>

> Cc: [email-masked]<mailto:[email-masked]>

> Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary

>> LSR will think the label stack ends there even though that's just

>> part of the ACH.

>> Or are you saying that a GAL not at the BoS will not have the ACH

>> following it?

>> *From:*mpls <[email-masked]<mailto:[email-masked]>> *On Behalf Of *Alexander

>> Vainshtein

>> *To:* Stewart Bryant <[email-masked]<mailto:[email-masked]>>

>> *Cc:* [email-masked]<mailto:[email-masked]>

>> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and

>> ancillary data after the BoS

>> <mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

>> <mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

>> <mailto:[email-masked]>

>> *Subject:* Re: [mpls] [EXTERNAL] Indicators in the stack and

>> ancillary data after the BoS

>> The original assumed processing model is to take the context of the

>> PW label or PW+FAT label, discover the GAL and then process the GAL

>> in the context of the PW label.

>> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%25>

>> 2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252

>> F%252Furld

>> efense.com%2Fv3%2F__https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2

>> F

>> mpls__%3B!!NEt6yMaO-gk!RVgTGVbknjgIjv3x-q8ob1JglFKOP6qKkgAcCSPbeBMMj2

>> A

>> nexFnPevXopeK1a6u%24&amp;data=04%7C01%7Chsong%40futurewei.com%7Ccc49d

>> e

>> 9585a24092e29708d931a0e327%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0

>> %

>> 7C637595389337881384%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ

>> I

>> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=5et4Juc3Ij

>> G

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%252>

> F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%

> 252Fwww.ietf.org%252Fmailman%252Flistinfo%252Fmpls%26data%3D04%257C01%

> 257Chsong%2540futurewei.com%257Ccc49de9585a24092e29708d931a0e327%257C0

> fee8ff2a3b240189c753a1d5591fedc%257C1%257C0%257C637595389337881384%257

> CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I

> k1haWwiLCJXVCI6Mn0%253D%257C1000%26sdata%3DXQlRpwkgODLRxcIjyMYyPMiCF2K

> DC0Y7GG4O8VGESnw%253D%26reserved%3D0

--

---

From: Loa Andersson <[email-masked]>
Date: Mon, 21 Jun 2021 12:15:56 +0200

Sasha,

>  From my POV, this includes (by implication) not tampering also with GAL 

Would you include adding a copy of the GAL higher up in the stack to 
make sure that it is within readable depth for any LSR?
> 
> As for the question " If the slot immediately after the label stack is 
> reserved for the ACH does this mean the no other ancillary data may be 
> inserted in this position, e.g. MPLS EH's, given that there is a GAL in 
> the stack" the answer, IMHO, is YES.
> 
> However, it is quite possible to carry any kind of new information in 
> the ACH, similar to the way this has been done in Section 3 of RFC 8169 
> <https://datatracker.ietf.org/doc/html/rfc8169#section-3> where G-ACH is 
> used for residence time measurement.

Logically this means that we can carry everything in the associated 
channel. However there can only one ACH per packet, right?

/Loa
> To: Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang 
> <zzhang=[email-masked]>; Alexander Vainshtein 
>  > My opinion is to not tamper with ACH anymore because it's designed 
> for control channel only and so far for a special scenario. The 
> constraints on GAL and format of ACH are hard to adapt to the new use 
> case requirements.
> 
>  >
> 
>  > From: mpls <[email-masked] <mailto:[email-masked]>> On 
> Behalf Of Jeffrey (Zhaohui)
> 
>  > Zhang
> 
>  > To: Loa Andersson <[email-masked] <mailto:[email-masked]>>; Alexander Vainshtein
> 
> <mailto:[email-masked]>>; Stewart Bryant
> 
>  > <[email-masked] <mailto:[email-masked]>>
> 
>  > Cc: [email-masked] <mailto:[email-masked]>
> 
>  >> but I'd like to see the DT address multiple indicators in the stack 
> and multiple sets of ancillary data after the BoS.
> 
>  >
> 
>  > I think the earlier emails of this email thread were talking about 
> multiple indicators in the stack; for multiple set of ancillary data 
> after the BoS, either the extended ACH or the proposed MPLS/generic 
> extension headers or a merge of those proposals should be able to handle 
> it. This is alluded to the DataAfterBOS wiki page.
> 
>  >
> 
>  > Thanks.
> 
>  >
> 
>  > Jeffrey
> 
>  >
> 
>  > From: Loa Andersson <[email-masked] <mailto:[email-masked]>>
> 
>  > To: Jeffrey (Zhaohui) Zhang <[email-masked] 
> <mailto:[email-masked]>>; Alexander Vainshtein
> 
> <mailto:[email-masked]>>; Stewart Bryant
> 
>  > <[email-masked] <mailto:[email-masked]>>
> 
>  > Cc: [email-masked] <mailto:[email-masked]>
> 
>  > There seem to be enough issues to sort out around the GAL/ACH pair, 
> and I was worried about a set of other indicators and the data that they 
>  > Well, as far as I understand a GAL which does not have the NoS-bit 
> set will have other labels after itself. The BoS-bit will be found 
> deeper down stack and the ACH will immediately fo9llow the BoS.
> 
>  >
> 
>  > Yes there are issues here, but I'd like to see the DT address 
> multiple indicators in the stack and multiple sets of ancillary data 
> after the BoS.
> 
>  >
> 
>  >> *From:*mpls <[email-masked] <mailto:[email-masked]>> 
> *On Behalf Of *Alexander
> 
>  >> Vainshtein
> 
>  >> *Cc:* [email-masked] <mailto:[email-masked]>
> 
>  >> <mailto:[email-masked] 
>  >> <mailto:[email-masked] <mailto:[email-masked]>>>
> 
>  >> <mailto:[email-masked] 
> <mailto:[email-masked]>>>
> 
>  >> <mailto:[email-masked] 
> <mailto:[email-masked]>>>; [email-masked] 
> <mailto:[email-masked]>
> 
>  >> <mailto:[email-masked] <mailto:[email-masked]>>; 
> [email-masked] <mailto:[email-masked]>
> 
>  >> <mailto:[email-masked] <mailto:[email-masked]>>
> 
>  >>      <mailto:[email-masked] 
> <mailto:[email-masked]>>> wrote:
> 
>  >>
> 
>  >> [email-masked] <mailto:[email-masked]>
> 
>  >> 
> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A% 
> <https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%25>
> 
>  > Loa Andersson                        email: [email-masked] <mailto:[email-masked]>
> 
>  > Senior MPLS Expert [email-masked] <mailto:[email-masked]>
> 
>  > [email-masked] <mailto:[email-masked]>
> 
>  > 
> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2 
> <https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%252>
> 
> Loa Andersson                        email: [email-masked] <mailto:[email-masked]>
> 
> Senior MPLS Expert [email-masked] <mailto:[email-masked]>
> 

--

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 21 Jun 2021 10:54:39 +0000

Loa,

Regarding your question “Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?”  my answer is NO.

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined – not in RFC 5586<https://datatracker.ietf.org/doc/html/rfc5586> and not anywhere else. Stewart has responded that “an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable”.  Neither of these options sounds optimistic to me.

I also do not favor investing into a technique that would guarantee that packets with GAL in the middle of the stack only pass thru new LSRs that know how to handle them .

However, it is quite possible to do the following IMHO:

1.       Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS

2.       Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, “we can carry everything in the associated channel”, including TLVs and Sub-TLVs, if necessary – it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).

3.       Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way

4.       Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://datatracker.ietf.org/doc/html/rfc8169>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://en.wikipedia.org/wiki/Primum_non_nocere>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Stewart Bryant <[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

>  From my POV, this includes (by implication) not tampering also with

> GAL as well.

Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?

> inserted in this position, e.g. MPLS EH's, given that there is a GAL

> in the stack" the answer, IMHO, is YES.

> the ACH, similar to the way this has been done in Section 3 of RFC

> 8169

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3> where G-ACH is used for residence time measurement.

Logically this means that we can carry everything in the associated channel. However there can only one ACH per packet, right?

/Loa

> To: Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang

> <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Alexander Vainshtein

> RFC 5586 specifies "that the ACH appears immediately after the bottom

> of the label stack."

>  > From: mpls <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

> On Behalf Of Jeffrey (Zhaohui)

>  > To: Loa Andersson <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>; Alexander

>  > <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

>  > Cc: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

>  > Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and

> ancillary

>  >> but I'd like to see the DT address multiple indicators in the

> stack and multiple sets of ancillary data after the BoS.

> extension headers or a merge of those proposals should be able to

> handle it. This is alluded to the DataAfterBOS wiki page.

>  > From: Loa Andersson <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

>  > <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

>  > Cc: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

>  > Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and

> ancillary

>  > Responded to Jeffrey's mail, but it is intended to address the

> entire discussion.

> and I was worried about a set of other indicators and the data that

> they might want to put "after the BoS". So far I have seen no real

> effort to address the interference's this might lead to.

>  >>      |                              ACH

> |

>  >>      |                         ACH TLV Header

> |

>  >>      |

> ~

>  >>      ~                     zero or more ACH TLVs

> ~

>  >>      ~

> |

>  >>      |

> ~

>  >>      ~                        G-ACh Message

> ~

>  >>      ~

> |

>  >> If the GAL does not have S-bit set, wouldn't a transit LSR treat

> any

>  >> label+TOS+S+TTL? If that 4-octet field has the S-bit set, the

> transit

>  > I think we need to nail down the relevant questiuons first, and

> start working on solutions after that.

>  >> *From:*mpls <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

>  >> *To:* Stewart Bryant <[email-masked]

>  >> *Cc:* [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

>  >> Regarding your statement "it is OK to have multiple GALs and GALs

> not

>  >> at BoS IFF the creator of the LSP ensured that all LSRs on the

> LSP,

>  >> including ECMP and FRR paths that found the GAL at ToS were known

> to

>  >>       1. The definition of correct processing of GAL at ToS but

> not at

>  >>       2. Advertisement of ability to process GAL not at BoS

> correctly in

>  >>       3. Ability to set up network-wide paths that only cross

> nodes that

>  >>          process GAL correctly must be provided for different

> techniques

>  >> that the benefits of supporting GAL at ToS but not BoS will be

> only

> <mailto:[email-masked]>>>

> <mailto:[email-masked]>>>; [email-masked]<mailto:[email-masked]>

>  >> <mailto:[email-masked]

> <mailto:[email-masked]>>;

> [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

>  >> <mailto:[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>>>

>  >>      While that might be the case, I think that the Open DT may

> give it a

>  >>      try and investigate how the existing systems will handle GAL

> being

>  >>      collecting the known actual behavior of popular

> implementations in

>  >>      this case, say, by running a survey among the vendors - what

> do you

>  >> It seems to me that an old implementation that received a ToS GAL

> not

>  >> The original assumed processing model is to take the context of

> the

>  >> PW label or PW+FAT label, discover the GAL and then process the

> GAL

>  >> When we extended GAL to apply to LSPs we again had the model that

> the

>  >> b) The behaviour at the receiving node is unpredictable, but in

> any

>  >> So I would think that it is OK to have multiple GALs and GALs not

> at

>  >> including ECMP and FRR paths that found the GAL at ToS were known

> to

>  >> A GAL not at BoS and not at ToS should not be inspected or

> processed

>  >> information of Ribbon Communications Inc. and its Affiliates that

> is

>  >> recipient. Any review, disclosure, reliance or distribution by

> others

>  >> or forwarding without express permission is strictly prohibited.

> If

>  >> [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

> <https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%

> 25>

> <mailto:[email-masked]>

>  > Senior MPLS Expert [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

>  > [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

> <https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%

> 252>

> <mailto:[email-masked]>

> Senior MPLS Expert [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

--

---

From: John E Drake <[email-masked]>
Date: Mon, 21 Jun 2021 12:51:34 +0000

Hi,

I agree with Sasha's email, below, which is proposing what I was proposing several months ago.

As an aside, RFC 8662 does not define what a transit LSR should do when it finds an [ELI, EL] pair at the top of the MPLS label stack, either when it understands the label pair or when it doesn't.

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]> On Behalf Of Alexander Vainshtein
Sent: Monday, June 21, 2021 6:55 AM
To: Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Loa,

Regarding your question "Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?"  my answer is NO.

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc5586__;!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588$> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8169__;!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg$>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://urldefense.com/v3/__https:/en.wikipedia.org/wiki/Primum_non_nocere__;!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0$>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://urldefense.com/v3/__https:/clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https*3A*2F*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc8169*23section-3__;JSUlJSUlJQ!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0$>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://urldefense.com/v3/__https:/clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https*3A*25__;JSU!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs$>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://urldefense.com/v3/__https:/clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https*3A*252__;JSU!!NEt6yMaO-gk!QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA$>

--

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 21 Jun 2021 15:45:55 +0000

John,
"Great minds think alike".

Regarding your aside comment on RFC 8662:
The last para of Section 4.3 of RFC 6790<https://datatracker.ietf.org/doc/html/rfc6790#section-4.3> says:

   As stated in Sections 4.1<https://datatracker.ietf.org/doc/html/rfc6790#section-4.1> and 5<https://datatracker.ietf.org/doc/html/rfc6790#section-5>, an egress LSR that signals both ELC

   and implicit null MUST pop the ELI and the next label (which should

   be the EL), if it encounters a packet with the ELI as the topmost

   label.  Any other LSR (including PHP LSRs) MUST drop such packets, as

   per Section 3.18 of [RFC3031]<https://datatracker.ietf.org/doc/html/rfc3031#section-3.18>.

All that is needed is to clarify the (rather, self-evident) rules for SIDs for which the originating routers  effectively signal Implicit Null  (explicitly or implicitly), including:

1.       All Adj-SIDs

2.       All EPE SIDs

3.       IGP Prefix SIDs that have been advertised with PHP (P-flag cleared in IS-IS, NP-flag cleared in OSPF)

4.       BGP Prefix SIDs advertised with Implicit Null in the NLRI of  the BGP-LU route.

I do not think that I have missed anything (Binding SIDs are not involved in PHP).

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: John E Drake <[email-masked]>
Sent: Monday, June 21, 2021 3:52 PM
To: Alexander Vainshtein <[email-masked]>; Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Subject: RE: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi,

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Alexander Vainshtein
Sent: Monday, June 21, 2021 6:55 AM
To: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Loa,

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://clicktime.symantec.com/3LpX2kp84U26BibbLu6xFzk6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc5586__%3B%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588%24> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://clicktime.symantec.com/389RV1YUirVu9q6t88snhkP6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169__%3B%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg%24>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://clicktime.symantec.com/3GdXcZMxHHLvCC23kWNFkFG6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fen.wikipedia.org%2Fwiki%2FPrimum_non_nocere__%3B%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0%24>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://clicktime.symantec.com/3UR1A1MPDSqDJ5gouUDZ8i86H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3FFh4tSjBeGN2kf7C3a3Sa76H2%3Fu%3Dhttps%2A3A%2A2F%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc8169%2A23section-3__%3BJSUlJSUlJQ%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0%24>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://clicktime.symantec.com/3G18FNxso3yysVeGE2oYas86H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F32ELHVPxdZe1NeGCU5oipbG6H2%3Fu%3Dhttps%2A3A%2A25__%3BJSU%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs%24>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://clicktime.symantec.com/3R52RdsrwRGTaBhuR2Sd5Qw6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F353Ka7ifLCb9e7KAzjZ4fsf6H2%3Fu%3Dhttps%2A3A%2A252__%3BJSU%21%21NEt6yMaO-gk%21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA%24>

--

---

From: <[email-masked]>
Date: Mon, 21 Jun 2021 16:07:59 +0000

> As an aside, RFC 8662 does not define what a transit LSR should do when it finds an [ELI, EL] pair at the top of the MPLS label stack, either when it understands the label pair or when it doesn't.

I believe it does:

- if it does not understand ELI (i.e. does not support RFC 8662) it drops the packet as per Section 3.18 of [RFC3031]<https://datatracker.ietf.org/doc/html/rfc3031#section-3.18>.
- if it understands ELI (i.e. supports RFC 8662), it processes the EL, ELI pair as defined in https://datatracker.ietf.org/doc/html/rfc6790#section-4.1 (The 2nd paragraph explicitly covers the case where the top label is the ELI). (Note that in this case, since the ELI is the top label that LSR is actually an LER with PHP enabled)

Regards,
--Bruno

From: mpls [mailto:[email-masked]] On Behalf Of John E Drake
Sent: Monday, June 21, 2021 2:52 PM
To: Alexander Vainshtein <[email-masked]>; Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi,

Yours Irrespectively,

John

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

/Loa

--

---

From: Tarek Saad <[email-masked]>
Date: Mon, 21 Jun 2021 16:19:36 +0000

Hi Sasha,

Thanks.

1.       >> Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, “we can carry everything in the associated channel”, including TLVs and Sub-TLVs, if necessary – it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
[TS]: Yes, this proposal has floated already (with an increment to the version in the ACH header). Couple of comments:

1.      How to deal with co-existence of already defined ACH type(s) with the new “ancillary data” ACH type at the same time? i.e. carrying multiple ACH(s) in same packet?

2.      Packaging all ancillary data insider one ACH does it become hindrance to allowing some of the ancillary data be processed by some of the nodes along the LSP path?

3.      Would mixing hop-by-hop ancillary data and edge-to-edge ancillary data in same ACH have further limitations?

4.      There is an open item (from last week) on the DT to investigate whether insertion at a transit router (i.e. adding ancillary data on transit node) is doable by HW or even desirable. Would that be a new “ancillary data” ACH appended to an existing ACH in the same packet?

Regards,
Tarek

On 6/21/21, 6:56 AM, "mpls" <[email-masked]> wrote:

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Stewart Bryant <[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

/Loa

--

---

From: Haoyu Song <[email-masked]>
Date: Mon, 21 Jun 2021 16:34:12 +0000

Hi Loa and all,

Yes, I think ACH and EH are mutual exclusive, so are the GAL and EHI.

Haoyu

From: Alexander Vainshtein <[email-masked]>
Sent: Monday, June 21, 2021 2:55 AM
To: Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Stewart Bryant <[email-masked]>
Subject: RE: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Loa and all,

However, it is quite possible to carry any kind of new information in the ACH, similar to the way this has been done in Section 3 of RFC 8169<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3&data=04%7C01%7Chsong%40futurewei.com%7C42cd0f1c948b4a6ee41f08d9349ab686%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637598661371423261%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wnP%2BgLiYEMGkbwZgNb%2BjtySMbsJ6HPDLYOvjpWh6gZE%3D&reserved=0> where G-ACH is used for residence time measurement.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 12:40 PM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Haoyu, DT

One question though.

/Loa

>> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclicktime.symantec.com%2F32ELHVPxdZe1NeGCU5oipbG6H2%3Fu%3Dhttps%253A%2525&data=04%7C01%7Chsong%40futurewei.com%7C42cd0f1c948b4a6ee41f08d9349ab686%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637598661371423261%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rjJlDBAtnPuKvyqJHlsVWCDDYyO9Xbq5hfh4TT64q3s%3D&reserved=0>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclicktime.symantec.com%2F353Ka7ifLCb9e7KAzjZ4fsf6H2%3Fu%3Dhttps%253A%25252&data=04%7C01%7Chsong%40futurewei.com%7C42cd0f1c948b4a6ee41f08d9349ab686%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637598661371433221%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9NTIe5iaRxlv7Nhm5S37gsJJHyrqXUa33QBoY8LeKEA%3D&reserved=0>

--

---

From: Haoyu Song <[email-masked]>
Date: Mon, 21 Jun 2021 16:37:54 +0000

Hi Bruno,

For clarification, does it mean an entropy label must be present in the packet? What if it doesn't have such a label?
Thanks!

Haoyu

-----Original Message-----
From: [email-masked] <[email-masked]> 
Sent: Monday, June 21, 2021 1:50 AM
To: Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

> > multiple indicators
> in the stack; for multiple set of ancillary data after the BoS, either 
> the extended ACH or the proposed MPLS/generic extension headers or a 
> merge of those proposals should be able to handle it. This is alluded 
> to the DataAfterBOS wiki page.
> 
>   draft-kompella-mpls-mspl4fa make use of TC field and TTL of a 
> special purpose label (FAI) as indicators
> 
>   there has also been discussion about putting more than one GAL in 
> the stack, i.e. differerent GALs pointing to different ACHs.
> 
>   draft-many-mpls-multiple-gal proposes to add a copy of the GAL 
> higher uop the stack so that LSRs with a too shallow maximun readable 
> depth might reach the GAL
> 
>   there has also been discussion about putting more than one GAL in 
> the stack, i.e. differerent GALs pointing to different ACHs.
> 
>   draft-song-mpls-eh-indicator have a list of potential indicators, 
> that is also telling if the EH should be processed on every EH capable 
> node or "just" at ingress and egress

The following draft proposes a way to carry indicators.
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-decraene-mpls-slid-encoded-entropy-label-id%23section-2&amp;data=04%7C01%7Chsong%40futurewei.com%7C08d7220727794b6895f808d934919550%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637598622176314998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=VR7DCVMaNM8cGK0XuDLugXG6yP6jRYIlgbDuYrh%2B7uM%3D&amp;reserved=0

Thanks,
Regards,
--Bruno
 
> > data after
> the BoS
> >> any 4-ocet field (i.e. those in the above Figure) after that GAL as 
> >> a
> >> label+TOS+S+transit
> > multiple indicators in the stack and multiple sets of ancillary data after the BoS.
> >
> >> not at BoS IFF the creator of the LSP ensured that all LSRs on the 
> >> LSP, including ECMP and FRR paths that found the GAL at ToS were 
> >> known to be able to process it correctly":
> >>
> >> not at BoS would at best throw an exception or worst be unpredictable.
> >>
> >> the GAL operated in the context of the LSP label that preceded it 
> >> for context. It was still BoS.
> >>
> >> to be able to process it correctly.
> >>
> >> processed by any LSR that did not know what it was doing, and to 
> >> attempt to precess it would be a violation of the normal MPLS processing model.
> >>
> >> - Stewart
> >>
> >>
> >> is confidential and/or proprietary for the sole use of the intended 
> >> others or forwarding without express permission is strictly 
> >> prohibited. If you are not the intended recipient, please notify 
> >> the sender immediately and then delete all copies, including any attachments.
> >>
> >>
> mpls__%3B!!NEt6yM&amp;data=04%7C01%7Chsong%40futurewei.com%7C08d722072
> 7794b6895f808d934919550%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C6
> 37598622176314998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=zE3QHyImI86j7K
> zn9PKE5qasAU01YKm%2FbbpXg6g4hAc%3D&amp;reserved=0
> aO-gk!RVgTGVbknjgIjv3x-
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fmpls&amp;data=04%7C01%7Chsong%40future
> wei.com%7C08d7220727794b6895f808d934919550%7C0fee8ff2a3b240189c753a1d5
> 591fedc%7C1%7C1%7C637598622176324952%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM
> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;s
> data=b7TztzRbKspQdyrJ8%2Btps6IeLQlCl3E0mrgvJRydlTk%3D&amp;reserved=0

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: <[email-masked]>
Date: Mon, 21 Jun 2021 16:49:10 +0000

Hi Haoyu,

> From: Haoyu Song [mailto:[email-masked]]
> 
> Hi Bruno,
> 
> For clarification, does it mean an entropy label must be present in the packet?

Yes. (Both ELI and EL.)

> What if it doesn't have such a label?

The node requiring the indicator adds the Entropy Label (ELI, EL). I guess same principle as the other options discussed in the Figure 5 of draft-song-mpls-eh-indicator.
In which case you get Entropy information for free (I mean no extra label). But from the entropy standpoint, it would be better to have the EL, ELI pushed by the ingress as this is the node which should have the most entropy information.

Thanks,
--Bruno

> To: Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>;
> Alexander Vainshtein <[email-masked]>; Stewart Bryant
> <[email-masked]>; [email-masked]
> Cc: [email-masked]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker
> .ietf.org%2Fdoc%2Fhtml%2Fdraft-decraene-mpls-slid-encoded-entropy-label-
> id%23section-
> 2&amp;data=04%7C01%7Chsong%40futurewei.com%7C08d7220727794b6895f808
> d934919550%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C63759862217
> 6314998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=VR7DCVMaNM8cGK
> 0XuDLugXG6yP6jRYIlgbDuYrh%2B7uM%3D&amp;reserved=0
> 
> ______________________________________________________________________
> ___________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou
> copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le
> signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si
> ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed, used or
> copied without authorisation.

---

From: Haoyu Song <[email-masked]>
Date: Mon, 21 Jun 2021 17:44:26 +0000

Hi Bruno, 

Thanks for the clarification. We might have some issues using the forced binding in some cases. For example, in our EH architecture, we allow an EH to be inserted or removed at any nodes on an LSP. Also, ELI+EI are two labels, which is more expensive than one SPL solution. Having said this, I could list your proposal as a possible solution.

BTW, Kireeti and I have discussed a possible option working in a different way.
We would use a new SPL to indicate multiples things (also by encoding the unused TTL/CoS bits): the existence of other special labels in the label stack (e.g., ELI+EL) and the existence/location of extension headers after the label stack. Since we have a new SPL, we have more freedom to define its behavior. 

Best,
Haoyu

-----Original Message-----
From: [email-masked] <[email-masked]> 
Sent: Monday, June 21, 2021 9:49 AM
To: Haoyu Song <[email-masked]>; Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

Hi Haoyu,

Yes. (Both ELI and EL.)

Thanks,
--Bruno

> To: Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang 
> <[email-masked]>; Alexander Vainshtein 
> > in the stack; for multiple set of ancillary data after the BoS, 
> > either the extended ACH or the proposed MPLS/generic extension 
> > headers or a merge of those proposals should be able to handle it. 
> > This is alluded to the DataAfterBOS wiki page.
> >
> > hmm - yes partly, but there are several indicators proposed in 
> > several drafts
> >
> > higher uop the stack so that LSRs with a too shallow maximun 
> > readable depth might reach the GAL
> >
> > that is also telling if the EH should be processed on every EH 
> > capable node or "just" at ingress and egress
> 
> -  does not require an additional scarce resource (Special-Purpose 
> MPLS Label
> value)
> > > There seem to be enough issues to sort out around the GAL/ACH 
> > > pair, and I was worried about a set of other indicators and the 
> > > data that they might want to put "after the BoS". So far I have 
> > > seen no real effort to address the interference's this might lead to.
> > >
> > > Further inline
> > >
> > >
> > >> any 4-ocet field (i.e. those in the above Figure) after that GAL 
> > >> as a
> > > multiple indicators in the stack and multiple sets of ancillary 
> > > data after the
> BoS.
> > >
> > >> I fully agree with your statement that "an old implementation 
> > >> that received a ToS GAL not at BoS would at best throw an 
> > >> exception or worst be unpredictable".
> > >>
> > >> not at BoS IFF the creator of the LSP ensured that all LSRs on 
> > >> the LSP, including ECMP and FRR paths that found the GAL at ToS 
> > >> were known to be able to process it correctly":
> > >>
> > >> only available in the networks where all the nodes support the 
> > >> new functionality because presence of non-supporting nodes 
> > >> imposes too many restrictions on connectivity and/or resilience.
> > >>
> > >> Regards,
> > >>
> > >> Sasha
> > >>
> > >> Office: +972-39266302
> > >>
> > >> Cell:      +972-549266302
> > >>
> > >> the PW label or PW+FAT label, discover the GAL and then process 
> > >> the GAL in the context of the PW label.
> > >>
> > >> any well written implementation should just result in the packet 
> > >> being dropped and counted as with any other Mal-formed packet.
> > >>
> > >> known to be able to process it correctly.
> > >>
> > >> attempt to precess it would be a violation of the normal MPLS 
> > >> processing
> model.
> > >>
> > >> - Stewart
> > >>
> > >>
> > >> is confidential and/or proprietary for the sole use of the 
> > >> intended recipient. Any review, disclosure, reliance or 
> > >> distribution by others or forwarding without express permission 
> > >> is strictly prohibited. If you are not the intended recipient, 
> > >> please notify the sender immediately and then delete all copies, including any attachments.
> > >>
> > >>
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi 
> que les pieces jointes. Les messages electroniques etant susceptibles 
> d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not 
> be distributed, used or copied without authorisation.

---

From: John E Drake <[email-masked]>
Date: Mon, 21 Jun 2021 18:41:22 +0000

Hi,

I'm familiar with the references, comments inline.

Yours Irrespectively,

John

Juniper Business Use Only
From: [email-masked] <[email-masked]>
Sent: Monday, June 21, 2021 12:08 PM
To: John E Drake <[email-masked]>; Alexander Vainshtein <[email-masked]>; Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

I believe it does:

- if it does not understand ELI (i.e. does not support RFC 8662) it drops the packet as per Section 3.18 of [RFC3031]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc3031*section-3.18__;Iw!!NEt6yMaO-gk!WZ0qLMfPs0aI4D8cgFh5CDyOKgPt3bJJohVHurnbcvOW12eGRaQxzkmJXGCvWO0$>.

[JD]  This section does not define the behavior of a transit LSR that receives a reserved label at the top of the MPLS label stack.

- if it understands ELI (i.e. supports RFC 8662), it processes the EL, ELI pair as defined in https://datatracker.ietf.org/doc/html/rfc6790#section-4.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!WZ0qLMfPs0aI4D8cgFh5CDyOKgPt3bJJohVHurnbcvOW12eGRaQxzkmJKW3xpzU$> (The 2nd paragraph explicitly covers the case where the top label is the ELI). (Note that in this case, since the ELI is the top label that LSR is actually an LER with PHP enabled)

[JD]  This section defines the behavior of an egress LSR that receives an [ELI, EL] pair.  It does not define the behavior of a transit LSR that receives an [ELI, EL] at the top of the MPLS label stack.  At the very least, RFC 8662 should have defined which transit LSR, the LSR popping the SID above the [ELI, EL] pair or the LSR receiving it should remove it and stipulating that once that behavior is defined, ensuring that the subject transit router must understand entropy labels.

My bigger point is that we are defining all sorts of nifty new functions while ignoring incremental deployment.  If we edict that every LSR needs to understand every nifty new function before those functions may be deployed, then we are ensuring that those nifty new functions will never be deployed.

Regards,
--Bruno

To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>; Loa Andersson <[email-masked]<mailto:[email-masked]>>

Hi,

Yours Irrespectively,

John

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

/Loa

--

Thank you.

---

From: John E Drake <[email-masked]>
Date: Mon, 21 Jun 2021 18:56:02 +0000

Comment inline.

Yours Irrespectively,

John

Juniper Business Use Only
From: Alexander Vainshtein <[email-masked]>
Sent: Monday, June 21, 2021 11:46 AM
To: John E Drake <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Loa Andersson <[email-masked]>
Subject: RE: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

John,

Regarding your aside comment on RFC 8662:
The last para of Section 4.3 of RFC 6790<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.3__;Iw!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgEYFFvI$> says:

   As stated in Sections 4.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-4.1__;Iw!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHfo9sMUI$> and 5<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc6790*section-5__;Iw!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHyGgu2ZY$>, an egress LSR that signals both ELC

   per Section 3.18 of [RFC3031]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc3031*section-3.18__;Iw!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgqlxMBs$>.

[JD]  If that's the behavior that we want, then RFC 8662 is not deployable.

  1.  All Adj-SIDs
  2.  All EPE SIDs

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: John E Drake <[email-masked]<mailto:[email-masked]>>

Hi,

Yours Irrespectively,

John

Loa,

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://urldefense.com/v3/__https:/clicktime.symantec.com/3LpX2kp84U26BibbLu6xFzk6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc5586__*3B*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588*24__;JSUlJSUlJSUlJSUlJSUl!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHeNCd6vA$> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://urldefense.com/v3/__https:/clicktime.symantec.com/389RV1YUirVu9q6t88snhkP6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc8169__*3B*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg*24__;JSUlJSUlJSUlJSUlJSUl!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvz6RrTE$>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://urldefense.com/v3/__https:/clicktime.symantec.com/3GdXcZMxHHLvCC23kWNFkFG6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fen.wikipedia.org*2Fwiki*2FPrimum_non_nocere__*3B*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0*24__;JSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHSI9qHKg$>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://urldefense.com/v3/__https:/clicktime.symantec.com/3UR1A1MPDSqDJ5gouUDZ8i86H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3FFh4tSjBeGN2kf7C3a3Sa76H2*3Fu*3Dhttps*2A3A*2A2F*2A2Fdatatracker.ietf.org*2A2Fdoc*2A2Fhtml*2A2Frfc8169*2A23section-3__*3BJSUlJSUlJQ*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvEw3iqk$>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://urldefense.com/v3/__https:/clicktime.symantec.com/3G18FNxso3yysVeGE2oYas86H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F32ELHVPxdZe1NeGCU5oipbG6H2*3Fu*3Dhttps*2A3A*2A25__*3BJSU*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs*24__;JSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHiE0sPVM$>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://urldefense.com/v3/__https:/clicktime.symantec.com/3R52RdsrwRGTaBhuR2Sd5Qw6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F353Ka7ifLCb9e7KAzjZ4fsf6H2*3Fu*3Dhttps*2A3A*2A252__*3BJSU*21*21NEt6yMaO-gk*21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA*24__;JSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mH2RXrzIw$>

--

---

From: <[email-masked]>
Date: Tue, 22 Jun 2021 03:25:22 +0800 (CST)

Hi Sasha,
great discussion, thank you!
Please find my notes below in-line tagged by GIM>>.

Best regards,
Greg Mirsky
------------------Original Mail------------------
Sender: AlexanderVainshtein
To: Loa Andersson;
CC: [email-masked];Haoyu Song;Jeffrey (Zhaohui) Zhang;
Date: 2021/06/21 03:56
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Loa,
Regarding your question “Would you include adding a copy of the GAL higher up in the stack to make sure that it is within readable depth for any LSR?”  my answer is NO.
I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, 
GIM>> I would not refer to existing implementation that supports GAL only as BoS being standards-complaint. I believe that earlier, we've established that the requirement in RFC 5586 "GAL MUST be BoS" applies only and only to an MPLS-TP environment. RFC 55886 and all standards up to date are underspecified for other scenarios. I understand that developers used their best judgment by implementing GAL as strictly BoS, but that doesn't have any foundation in current RFCs. We seem to agree on this characterization of the state of the use of GAL in a non-MPLS-TP environment. Hence, the next step could be closing the gap in the GAL specification. It appears there are two possible paths:
- have a short document extending GAL is only BoS to all MPLS environments;
- allow GAL as ToS while != BoS.
In either case, GAL indicates that a single ACH immediately follows the BoS.

 it will not know how to handle it since such handling has not ever been defined – not in RFC 5586 and not anywhere else. Stewart has responded that “an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable”.  Neither of these options  sounds optimistic to me.
I also do not favor investing into a technique that would guarantee that packets with GAL in the middle of the stack only pass thru new LSRs that know how to handle them .
However, it is quite possible to do the following IMHO:
1.       Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
2.       Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, “we can carry everything in the associated channel”, including TLVs and Sub-TLVs,  if necessary – it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
GIM>> That, to me, is a logical conclusion of #1. I might propose to allocate a new, different from GAL, SPL to signal that the "new ACH immediately follows the BoS". And we might avoid terminology confusion in the future if the new ACH is named differently, e.g., "MPLS Extension Header (MEH)".

3.       Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
GIM>> If we use a new SPL (why not call it MEHI), we can allow MEHI to be not BoS, and thus an LSR can react to its presence in the stack. Consequently, MEHI  can appear in the stack more than once.

4.       Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169) even if they cannot detect presence of GAL at BoS due to the depth of the stack.
I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere.
My 2c,
Sasha
Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]
-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>; Stewart Bryant <[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS
Sasha,
>  > From: mpls <[email-masked] <mailto:[email-masked]>>

---

From: <[email-masked]>
Date: Tue, 22 Jun 2021 08:22:29 +0000

Hi,

Please see inline [Bruno]

From: John E Drake [mailto:[email-masked]]
Sent: Monday, June 21, 2021 8:41 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>; Alexander Vainshtein <[email-masked]>; Loa Andersson <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

Hi,

I'm familiar with the references, comments inline.
[Bruno] I'm pretty sure you are. I'm only providing the reference as the source i.e. reference for anyone to check.

Yours Irrespectively,

John

Juniper Business Use Only
From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 12:08 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>; Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>; Loa Andersson <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <[email-masked]<mailto:[email-masked]>>
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

I believe it does:

[Bruno]

- this section does not say that reserved label are to be treated differently

- RFC 8662 builds on it and nobody complained at the time

Now if you are saying that in the MPLS architecture the forwarding of unknown reserved label is not defined, I leave this to the WG. That would be a 20 years old issue.

[JD]  This section defines the behavior of an egress LSR that receives an [ELI, EL] pair.  It does not define the behavior of a transit LSR that receives an [ELI, EL] at the top of the MPLS label stack.

[Bruno] My reading of https://datatracker.ietf.org/doc/html/rfc3031#section-3.15 is that in your scenario (ELI on top while the node is not an LER) the node is not forwarding along an LSP. As such I would not called it a transit LSR of an LSP.

In all cases, if PHP is enabled, the LSR does not know whether it was supposed to the LER of that LSP. All it sees is the ELI, EL and RFC 6790 tells it to process the ELI, EL including when the ELI is on not.

But why are we having this conversation?

[JD] At the very least, RFC 8662 should have defined which transit LSR, the LSR popping the SID above the [ELI, EL] pair or the LSR receiving it should remove it and stipulating that once that behavior is defined, ensuring that the subject transit router must understand entropy labels.

RFC 8662 does not change data plane processing of the Entropy Label on transit or egress LSR.

To answer your question, the ELI, EL is removed by the "LSR receiving" the [ELI, EL pair]. There is no changed compared to LDP (and in both cases, PHP may be enabled or not; LSP may be stacked on not). BTW that node is an egress LSR, not a transit LSR.

[Bruno] Thanks for the clarification.

I'm fine with your bigger point and I support it.

I'm all for incremental deployment. I'm even in favor of incremental deployment with incremental benefit in order to get early incentive for deploying the new feature.

Regards,
--Bruno

Hi,

Yours Irrespectively,

John

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

/Loa

--

Thank you.

---

From: <[email-masked]>
Date: Tue, 22 Jun 2021 08:33:10 +0000

Hi Haoyu,

Thanks for replying.

> Sent: Monday, June 21, 2021 7:44 PM
> To: DECRAENE Bruno INNOV/NET <[email-masked]>; Loa
> Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>;
> Thanks for the clarification. We might have some issues using the forced binding
> in some cases.
[Bruno] I'm not familiar with this 'forced binding' term.

>  For example, in our EH architecture, we allow an EH to be inserted
> or removed at any nodes on an LSP.

[Bruno] As the thread is about EH _indicator_ I'll focus on the indicator. Draft [1] does not forbid the indicator to be modified en route.
[1] https://datatracker.ietf.org/doc/html/draft-decraene-mpls-slid-encoded-entropy-label-id-01#section-2 

> Also, ELI+EI are two labels, which is more
> expensive than one SPL solution.

[Bruno]
It would be more expensive only in the case that EL is not needed/useful AND that one SPL be allocated for signaling the EH.
In all other cases, it's less expensive. 

> Having said this, I could list your proposal as a
> possible solution.
 
[Bruno] OK. I believe it should, as per the abstract of draft-song-mpls-eh-indicator

> We would use a new SPL to indicate multiples things (also by encoding the unused
> TTL/CoS bits): the existence of other special labels in the label stack (e.g., ELI+EL)
> and the existence/location of extension headers after the label stack. Since we
> have a new SPL, we have more freedom to define its behavior.

[Bruno] Sure it also works. Different tradeoffs.

Best regards,
--Bruno 

> To: Haoyu Song <[email-masked]>; Loa Andersson <[email-masked]>; Jeffrey
> (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein
> <[email-masked]>; Stewart Bryant <[email-masked]>;
> [email-masked]
> Cc: [email-masked]
> The node requiring the indicator adds the Entropy Label (ELI, EL). I guess same
> principle as the other options discussed in the Figure 5 of draft-song-mpls-eh-
> indicator.
> In which case you get Entropy information for free (I mean no extra label). But from
> the entropy standpoint, it would be better to have the EL, ELI pushed by the
> ingress as this is the node which should have the most entropy information.
> 
> Thanks,
> --Bruno
> 
> > Thanks!
> >
> > Haoyu
> >

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 22 Jun 2021 12:36:51 +0000

Greg,

Lots of thanks for a prompt response.

Regarding your statement "I would not refer to existing implementation that supports GAL only as BoS being standards-compliant":

As we have established, RFC 5586 does not defined any rules for handling of GAL that is exposed as ToS to the MPLS forwarding plane but is not marked as BoS at the same time. To me this means that its nominal acceptance of GAL in these positions in non-MPLS-TP environments meaningless for all practical purposes: it allows something but does not say what to do with it if it happens, and there is simply no way to implement this part of the standard in an interoperable way IMHO.

Existing implementations are standard-compliant in the parts that are fully specified, and de-fact ignore the parts that  have not been specified and therefore cannot be implemented in an interoperable way.  And, what is most important, any attempts to add the missing definitions would at this stage (11 years after RFC 5586 has been published), with high probability, result in backward compatibility issues.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: [email-masked] <[email-masked]>
Sent: Monday, June 21, 2021 10:25 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; zzhang=[email-masked]
Subject: Re:[mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi Sasha,

Best regards,

Greg Mirsky

https://clicktime.symantec.com/3QEJM7g1nQZHEgxqFN4Bu5t6H2?u=www.zte.com.cn

------------------Original Mail------------------

To: Loa Andersson;

CC: [email-masked];Haoyu<mailto:[email-masked];Haoyu> Song;Jeffrey (Zhaohui) Zhang;

Date: 2021/06/21 03:56

mpls mailing list

https://clicktime.symantec.com/3B8keUakyNitCUw7ALbJZXv6H2?u=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----

/Loa

> <mailto:[email-masked]>

--

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 22 Jun 2021 13:27:09 +0000

John,
Lots of thanks for your comment.

A few points:

1.       I have sent the previous email as a response to your aside claim that "RFC 8662 does not define what a transit LSR should do when it finds an [ELI, EL] pair at the top of the MPLS label stack, either when it understands the label pair or when it doesn't". As I see it, both Bruno and I have demonstrated that this behavior is defined

2.       My reading of your comment to my response "If that's the behavior that we want, then RFC 8662 is not deployable" is that:

a.       You agree that some behavior has been defined in RFC 8662

b.       You claim that the defined behavior makes RFC 8662 non-deployable

3.       I respectfully disagree with (2b) above because:

a.       My reading of RFC 6790 is that {ELI, EL} pairs in any case can be only inserted if the node that receives ELI at the top of the stack has signaled its capability to handle such a pair

b.       While RFC 8662 does not define any mechanisms for signaling ability to handle {ELI, EL} pair at the top of the stack, it provides Informational references to a couple of drafts that define such signaling in IS-IS and OSPF, and these drafts have been already approved for publication as RFCs

c.       With this signaling in place in an SR domain, {ELI, EL} pairs can be inserted in the label stack in the following positions:

                                                               i.      Immediately following an IGP Prefix SID that has been advertised (with PHP) by a node that has signaled its capability to support {ELI, EL}

                                                             ii.      Immediately following an Adjacency SID that represents an IGP adjacency between the advertising node and the node that signaled its capability to support {ELI, EL}

d.       Depending on specific network topology and the specific stack of SIDs, the rules above may be too restrictive to provide effective load balancing, or not. But IMHO and FWIW this is not really different from the situation with "vanilla" RFC 6790 if, e.g., some of the PEs where a given L3VPN service is represented, have advertised their ability to handle {ELI, EL} via LDP or RSVP-TE, and some did not advertise such capability.
My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Sent: Monday, June 21, 2021 9:56 PM

Comment inline.

Yours Irrespectively,

John

Juniper Business Use Only
From: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 11:46 AM
To: John E Drake <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Loa Andersson <[email-masked]<mailto:[email-masked]>>
Subject: RE: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

John,

Regarding your aside comment on RFC 8662:
The last para of Section 4.3 of RFC 6790<https://clicktime.symantec.com/3PDwoeDbzi3ETXLKkVKHdzL6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc6790%2Asection-4.3__%3BIw%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgEYFFvI%24> says:

   As stated in Sections 4.1<https://clicktime.symantec.com/3Q8XJjsAd3fUwja7w7zBq576H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc6790%2Asection-4.1__%3BIw%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHfo9sMUI%24> and 5<https://clicktime.symantec.com/3W3scAvqRKZF6Hd9aBjBfFo6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc6790%2Asection-5__%3BIw%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHyGgu2ZY%24>, an egress LSR that signals both ELC

   per Section 3.18 of [RFC3031]<https://clicktime.symantec.com/3WQ3TqWDKFmsEnTM4eLJ4SG6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc3031%2Asection-3.18__%3BIw%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgqlxMBs%24>.

  1.  All Adj-SIDs
  2.  All EPE SIDs

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Hi,

Yours Irrespectively,

John

Loa,

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://clicktime.symantec.com/3LFH4qVDAVUpzHHufyXwucN6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3LpX2kp84U26BibbLu6xFzk6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc5586__%2A3B%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588%2A24__%3BJSUlJSUlJSUlJSUlJSUl%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHeNCd6vA%24> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://clicktime.symantec.com/3UtLBGnixTrdCEoquSjkP16H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F389RV1YUirVu9q6t88snhkP6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc8169__%2A3B%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg%2A24__%3BJSUlJSUlJSUlJSUlJSUl%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvz6RrTE%24>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://clicktime.symantec.com/3QMnW2BUztofokYXW2fRkBn6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3GdXcZMxHHLvCC23kWNFkFG6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fen.wikipedia.org%2A2Fwiki%2A2FPrimum_non_nocere__%2A3B%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0%2A24__%3BJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHSI9qHKg%24>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://clicktime.symantec.com/3QzLxrBKpSCaXxZ9WTPFXb46H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3UR1A1MPDSqDJ5gouUDZ8i86H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3FFh4tSjBeGN2kf7C3a3Sa76H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Fdatatracker.ietf.org%2A2A2Fdoc%2A2A2Fhtml%2A2A2Frfc8169%2A2A23section-3__%2A3BJSUlJSUlJQ%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvEw3iqk%24>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://clicktime.symantec.com/3LbDZkybzjLKby6DAk88iUh6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3G18FNxso3yysVeGE2oYas86H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F32ELHVPxdZe1NeGCU5oipbG6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A25__%2A3BJSU%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHiE0sPVM%24>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://clicktime.symantec.com/3RHS3C2UW8kquG4WDQJFwvp6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3R52RdsrwRGTaBhuR2Sd5Qw6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F353Ka7ifLCb9e7KAzjZ4fsf6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A252__%2A3BJSU%2A21%2A21NEt6yMaO-gk%2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mH2RXrzIw%24>

--

---

From: John E Drake <[email-masked]>
Date: Tue, 22 Jun 2021 14:05:42 +0000

Hi,

Comments inline.

Yours Irrespectively,

John

Sent: Tuesday, June 22, 2021 9:27 AM

John,

A few points:

[JD]  Perhaps I missed it, but nowhere in RFC 8662 do I see a statement of the form:  "This document updates the transit LSR behavior defined in RFC 6790 to allow a transit LSR that understands entropy labels to remove an [ELI, EL] pair that it sees at the top of an MPLS label stack.  This requires a transit LSR to indicate this capability to other LSRs in the SR domain and it requires the ingress LSR to ensure that when it inserts [ELI, EL] pairs in the MPLS label stack they will only be at the top of the MPLS label stack when received by a transit LSR with this capability.".

  1.  My reading of your comment to my response "If that's the behavior that we want, then RFC 8662 is not deployable" is that:

     *   You agree that some behavior has been defined in RFC 8662

[JD]  Your email, to which I responded, referenced RFC 6790 not RFC 8662 and I was commenting on this sentence:  "Any other LSR (including PHP LSRs) MUST drop such packets.".  I.e., it is referring to non-egress, or transit, LSRs.

     *   You claim that the defined behavior makes RFC 8662 non-deployable

[JD]  See my previous comment.

  1.  I respectfully disagree with (2b) above because:

     *   My reading of RFC 6790 is that {ELI, EL} pairs in any case can be only inserted if the node that receives ELI at the top of the stack has signaled its capability to handle such a pair

[JD]   RFC 6970 only talks about egress LSRs advertising this capability.

[JD]  See my first comment above.

     *   With this signaling in place in an SR domain, {ELI, EL} pairs can be inserted in the label stack in the following positions:

[JD]  See my first comment above.  I am not saying that RFC 8662 doesn't work.  Rather, like RFC 5886 it is under-specified.

     *   Depending on specific network topology and the specific stack of SIDs, the rules above may be too restrictive to provide effective load balancing, or not. But IMHO and FWIW this is not really different from the situation with "vanilla" RFC 6790 if, e.g., some of the PEs where a given L3VPN service is represented, have advertised their ability to handle {ELI, EL} via LDP or RSVP-TE, and some did not advertise such capability.
My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Comment inline.

Yours Irrespectively,

John

John,

Regarding your aside comment on RFC 8662:
The last para of Section 4.3 of RFC 6790<https://urldefense.com/v3/__https:/clicktime.symantec.com/3PDwoeDbzi3ETXLKkVKHdzL6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc6790*2Asection-4.3__*3BIw*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgEYFFvI*24__;JSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMw-aGDt8$> says:

   As stated in Sections 4.1<https://urldefense.com/v3/__https:/clicktime.symantec.com/3Q8XJjsAd3fUwja7w7zBq576H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc6790*2Asection-4.1__*3BIw*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHfo9sMUI*24__;JSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMre1vkh0$> and 5<https://urldefense.com/v3/__https:/clicktime.symantec.com/3W3scAvqRKZF6Hd9aBjBfFo6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc6790*2Asection-5__*3BIw*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHyGgu2ZY*24__;JSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMISv8PZA$>, an egress LSR that signals both ELC

   per Section 3.18 of [RFC3031]<https://urldefense.com/v3/__https:/clicktime.symantec.com/3WQ3TqWDKFmsEnTM4eLJ4SG6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fdoc*2Fhtml*2Frfc3031*2Asection-3.18__*3BIw*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgqlxMBs*24__;JSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMCPiuYwU$>.

  1.  All Adj-SIDs
  2.  All EPE SIDs

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Hi,

Yours Irrespectively,

John

Loa,

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://urldefense.com/v3/__https:/clicktime.symantec.com/3LFH4qVDAVUpzHHufyXwucN6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3LpX2kp84U26BibbLu6xFzk6H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fdatatracker.ietf.org*2A2Fdoc*2A2Fhtml*2A2Frfc5586__*2A3B*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588*2A24__*3BJSUlJSUlJSUlJSUlJSUl*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHeNCd6vA*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMfgunyzA$> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://urldefense.com/v3/__https:/clicktime.symantec.com/3UtLBGnixTrdCEoquSjkP16H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F389RV1YUirVu9q6t88snhkP6H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fdatatracker.ietf.org*2A2Fdoc*2A2Fhtml*2A2Frfc8169__*2A3B*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg*2A24__*3BJSUlJSUlJSUlJSUlJSUl*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvz6RrTE*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMz23oBcc$>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://urldefense.com/v3/__https:/clicktime.symantec.com/3QMnW2BUztofokYXW2fRkBn6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3GdXcZMxHHLvCC23kWNFkFG6H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fen.wikipedia.org*2A2Fwiki*2A2FPrimum_non_nocere__*2A3B*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0*2A24__*3BJSUlJSUlJSUlJSUlJSU*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHSI9qHKg*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMA9liAC0$>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://urldefense.com/v3/__https:/clicktime.symantec.com/3QzLxrBKpSCaXxZ9WTPFXb46H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3UR1A1MPDSqDJ5gouUDZ8i86H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fclicktime.symantec.com*2A2F3FFh4tSjBeGN2kf7C3a3Sa76H2*2A3Fu*2A3Dhttps*2A2A3A*2A2A2F*2A2A2Fdatatracker.ietf.org*2A2A2Fdoc*2A2A2Fhtml*2A2A2Frfc8169*2A2A23section-3__*2A3BJSUlJSUlJQ*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0*2A24__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvEw3iqk*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMMFVywrw$>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://urldefense.com/v3/__https:/clicktime.symantec.com/3LbDZkybzjLKby6DAk88iUh6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3G18FNxso3yysVeGE2oYas86H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fclicktime.symantec.com*2A2F32ELHVPxdZe1NeGCU5oipbG6H2*2A3Fu*2A3Dhttps*2A2A3A*2A2A25__*2A3BJSU*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs*2A24__*3BJSUlJSUlJSUlJSUlJSUlJSU*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHiE0sPVM*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMg35ivIo$>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://urldefense.com/v3/__https:/clicktime.symantec.com/3RHS3C2UW8kquG4WDQJFwvp6H2?u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fclicktime.symantec.com*2F3R52RdsrwRGTaBhuR2Sd5Qw6H2*3Fu*3Dhttps*2A3A*2A2F*2A2Furldefense.com*2A2Fv3*2A2F__https*2A3A*2A2Fclicktime.symantec.com*2A2F353Ka7ifLCb9e7KAzjZ4fsf6H2*2A3Fu*2A3Dhttps*2A2A3A*2A2A252__*2A3BJSU*2A21*2A21NEt6yMaO-gk*2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA*2A24__*3BJSUlJSUlJSUlJSUlJSUlJSU*21*21NEt6yMaO-gk*21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mH2RXrzIw*24__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMct3koic$>

--

---

From: John E Drake <[email-masked]>
Date: Tue, 22 Jun 2021 14:11:26 +0000

Doh, RFC 5586 not RFC 5886

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]> On Behalf Of John E Drake
Sent: Tuesday, June 22, 2021 10:06 AM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Haoyu Song <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Hi,

Comments inline.

Yours Irrespectively,

John

John,

A few points:

Office: +972-39266302
Cell:      +972-549266302

Comment inline.

Yours Irrespectively,

John

John,

  1.  All Adj-SIDs
  2.  All EPE SIDs

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Hi,

Yours Irrespectively,

John

Loa,

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

/Loa

--

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 22 Jun 2021 15:01:35 +0000

John,
Again, lots of thanks for your comments.

Please see inline below.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Sent: Tuesday, June 22, 2021 5:06 PM

Hi,

Comments inline.

Yours Irrespectively,

John

John,

A few points:

[JD]  Perhaps I missed it, but nowhere in RFC 8662 do I see a statement of the form:  "This document updates the transit LSR behavior defined in RFC 6790 to allow a transit LSR that understands entropy labels to remove an [ELI, EL] pair that it sees at the top of an MPLS label stack.  This requires a transit LSR to indicate this capability to other LSRs in the SR domain and it requires the ingress LSR to ensure that when it inserts [ELI, EL] pairs in the MPLS label stack they will only be at the top of the MPLS label stack when received by a transit LSR with this capability."
[[Sasha]] I have not seen any such statement either. However, I think that we may be dealing with a terminology issue. I believe that in SR a node that originates advertisement, say, of an IGP Prefix SID, is considered as the egress node for the segment represented by this SID, even if it is a transit node for an LSP that includes this SID in the middle of the stack of SIDs. This is why they have assumed that considerations of 6790 pertaining to egress LERs are sufficient.

[JD]  Your email, to which I responded, referenced RFC 6790 not RFC 8662 and I was commenting on this sentence:  "Any other LSR (including PHP LSRs) MUST drop such packets.".  I.e., it is referring to non-egress, or transit, LSRs.  [[Sasha]] Please see my previous comment

[JD]   RFC 6970 only talks about egress LSRs advertising this capability.[[Sasha]] Please see my previous comment about terminology. Please note also that the signaling drafts I have mentioned, define signaling of Entropy Label Capability (ELC) by all router. In particular, the Signaling Entropy Label Capability and Entropy Readable Label Depth Using ISIS<https://datatracker.ietf.org/doc/html/draft-ietf-isis-mpls-elc-13> draft says:

   Bit 3 in the Prefix Attribute Flags [RFC7794<https://datatracker.ietf.org/doc/html/rfc7794>] is used as the ELC Flag

   (E-flag), as shown in Figure 1.  If a router has multiple interfaces,

   the router MUST NOT announce the ELC for any local host prefixes

   unless all of its interfaces are capable of processing ELs.  If a

   router supports ELs on all of its interfaces, it SHOULD set the ELC

   for every local host prefix it advertises in IS-IS.
>From my POV, this specification means that the {ELI, EL} tuple may be safely inserted into the label stack immediately after any IGP Prefix SID advertised with PHP by a router that has also advertised ELC capability in IS-IS.

[JD]  See my first comment above.  I am not saying that RFC 8662 doesn't work.  Rather, like RFC 5886 it is under-specified.
[[Sasha]] From my POV, the above-mentioned signaling drafts provide the missing specification. This is quite different from RFC 5586.

Office: +972-39266302
Cell:      +972-549266302

Comment inline.

Yours Irrespectively,

John

John,

Regarding your aside comment on RFC 8662:
The last para of Section 4.3 of RFC 6790<https://clicktime.symantec.com/3EvxQY3dZE6MCzRCky9z2rr6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3PDwoeDbzi3ETXLKkVKHdzL6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc6790%2A2Asection-4.3__%2A3BIw%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgEYFFvI%2A24__%3BJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMw-aGDt8%24> says:

   As stated in Sections 4.1<https://clicktime.symantec.com/3BjFjLGdKqJxXP2G6xuL1M76H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3Q8XJjsAd3fUwja7w7zBq576H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc6790%2A2Asection-4.1__%2A3BIw%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHfo9sMUI%2A24__%3BJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMre1vkh0%24> and 5<https://clicktime.symantec.com/3Nq2VVAPq9XCTjsgwnp77NH6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3W3scAvqRKZF6Hd9aBjBfFo6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc6790%2A2Asection-5__%2A3BIw%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHyGgu2ZY%2A24__%3BJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMISv8PZA%24>, an egress LSR that signals both ELC

   per Section 3.18 of [RFC3031]<https://clicktime.symantec.com/3ENxypfQnY6TWQf4EL29AHf6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3WQ3TqWDKFmsEnTM4eLJ4SG6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fdatatracker.ietf.org%2A2Fdoc%2A2Fhtml%2A2Frfc3031%2A2Asection-3.18__%2A3BIw%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHgqlxMBs%2A24__%3BJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMCPiuYwU%24>.

  1.  All Adj-SIDs
  2.  All EPE SIDs

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Hi,

Yours Irrespectively,

John

Loa,

I have already said on this thread that if GAL is exposed as ToS but not BoS to an existing standards-compliant MPLS forwarder, it will not know how to handle it since such handling has not ever been defined - not in RFC 5586<https://clicktime.symantec.com/3B4HUFbyoxVqmUo7QNkPWDS6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3LFH4qVDAVUpzHHufyXwucN6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3LpX2kp84U26BibbLu6xFzk6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fdatatracker.ietf.org%2A2A2Fdoc%2A2A2Fhtml%2A2A2Frfc5586__%2A2A3B%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMt9fY588%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSUl%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHeNCd6vA%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMfgunyzA%24> and not anywhere else. Stewart has responded that "an old implementation that received a ToS GAL not at BoS would at best throw an exception or worst be unpredictable".  Neither of these options sounds optimistic to me.

  1.  Retain the existing definitions of GAL just at BoS and ACH that immediately follows the BoS
  2.  Define new ACH types that can carry new ancillary data, and the structures that can be used for this purpose (as you have said, "we can carry everything in the associated channel", including TLVs and Sub-TLVs, if necessary - it will be up to the specific applications to process such structures in ACH, but at least this would not affect MPLS forwarding).
  3.  Allow LERs that (a) can detect presence of GAL at BoS and (b) recognize new ACH types to meddle with the information carried in the ACH while forwarding labeled packets in the usual way
  4.  Also allow usage of TTL to help LERs that recognize new ACH types to meddle with the information carried in the ACH (similar to what has been done in RFC 8169<https://clicktime.symantec.com/3LjkcAVhuwRQJqtrxjrf92G6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3UtLBGnixTrdCEoquSjkP16H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F389RV1YUirVu9q6t88snhkP6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fdatatracker.ietf.org%2A2A2Fdoc%2A2A2Fhtml%2A2A2Frfc8169__%2A2A3B%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMB4Q7qmg%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSUl%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvz6RrTE%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMz23oBcc%24>) even if they cannot detect presence of GAL at BoS due to the depth of the stack.

I cannot say whether this approach is good enough for the specific set of applications. But it looks to me as reasonably safe since it does not require any new forwarding functionality in existing LERs - primum non nocere<https://clicktime.symantec.com/3GKsCwGpjw974kU3D48QDJb6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3QMnW2BUztofokYXW2fRkBn6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3GdXcZMxHHLvCC23kWNFkFG6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fen.wikipedia.org%2A2A2Fwiki%2A2A2FPrimum_non_nocere__%2A2A3B%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMI1UZKH0%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSU%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHSI9qHKg%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMA9liAC0%24>.

My 2c,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: Loa Andersson <[email-masked]<mailto:[email-masked]>>
Sent: Monday, June 21, 2021 1:16 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Haoyu Song <[email-masked]<mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]<mailto:zzhang=[email-masked]>>; Stewart Bryant <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Sasha,

> <https://clicktime.symantec.com/3FFh4tSjBeGN2kf7C3a3Sa76H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc8169%23section-3<https://clicktime.symantec.com/3UGQjSTm4XpXjLBxA9zJqcr6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3QzLxrBKpSCaXxZ9WTPFXb46H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3UR1A1MPDSqDJ5gouUDZ8i86H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fclicktime.symantec.com%2A2A2F3FFh4tSjBeGN2kf7C3a3Sa76H2%2A2A3Fu%2A2A3Dhttps%2A2A2A3A%2A2A2A2F%2A2A2A2Fdatatracker.ietf.org%2A2A2A2Fdoc%2A2A2A2Fhtml%2A2A2A2Frfc8169%2A2A2A23section-3__%2A2A3BJSUlJSUlJQ%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtM0KyFNp0%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHvEw3iqk%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMMFVywrw%24>> where G-ACH is used for residence time measurement.

/Loa

> https://clicktime.symantec.com/32ELHVPxdZe1NeGCU5oipbG6H2?u=https%3A%<https://clicktime.symantec.com/3AgFeqrtXP4t1gnnks7D75x6H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3LbDZkybzjLKby6DAk88iUh6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3G18FNxso3yysVeGE2oYas86H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fclicktime.symantec.com%2A2A2F32ELHVPxdZe1NeGCU5oipbG6H2%2A2A3Fu%2A2A3Dhttps%2A2A2A3A%2A2A2A25__%2A2A3BJSU%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMnORLvEs%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSUlJSU%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mHiE0sPVM%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMg35ivIo%24>

> https://clicktime.symantec.com/353Ka7ifLCb9e7KAzjZ4fsf6H2?u=https%3A%2<https://clicktime.symantec.com/3Twd6qr3hcrCv35XXASuzA16H2?u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fclicktime.symantec.com%2F3RHS3C2UW8kquG4WDQJFwvp6H2%3Fu%3Dhttps%2A3A%2A2F%2A2Furldefense.com%2A2Fv3%2A2F__https%2A3A%2A2Fclicktime.symantec.com%2A2F3R52RdsrwRGTaBhuR2Sd5Qw6H2%2A3Fu%2A3Dhttps%2A2A3A%2A2A2F%2A2A2Furldefense.com%2A2A2Fv3%2A2A2F__https%2A2A3A%2A2A2Fclicktime.symantec.com%2A2A2F353Ka7ifLCb9e7KAzjZ4fsf6H2%2A2A3Fu%2A2A3Dhttps%2A2A2A3A%2A2A2A252__%2A2A3BJSU%2A2A21%2A2A21NEt6yMaO-gk%2A2A21QiStnftbs7rzJ6JZRtxhV6LZks_wNvQJ-rNe5phnYEW6lEzzVD0vSHtMG_cybmA%2A2A24__%2A3BJSUlJSUlJSUlJSUlJSUlJSU%2A21%2A21NEt6yMaO-gk%2A21Uk8p6mR2CzJJFXxJuweTWC-0CDe7odXjIzcOP6yLGVqs-46um02ro1mH2RXrzIw%2A24__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU%21%21NEt6yMaO-gk%21RH_x4Yy-jTEqPMKiyMGCocKS-X0kXu_C0ArR_wJSIAgx76WXM8Mm5xSMct3koic%24>

--

---

From: John E Drake <[email-masked]>
Date: Tue, 22 Jun 2021 15:35:06 +0000

Hi,

Snipped, comment inline [JD1}.

Yours Irrespectively,

John

[JD1]  This seems like quite a stretch.  Also, as I mentioned in my first email, there is no discussion of how an [ELI, EL] pair at the top on a MPLS label stack is to be processed or ensuring, when building an MPLS label stack, that [ELI, EL] pairs need to be inserted such that they rise to the top of an MPLS label stack only when they are received by a transit LSR that understands entropy labels.

[JD1}  Do you think that it would be safe to re-use the RFC 8662 techniques for GAL?  I.e., a node indicates whether it can receive a GAL at the top of an MPLS label stack and the ingress LSR constructs MPLS label stacks in which a GAL rises to the top of the stack only at such nodes.  Or do implementations of RFC 5586 explicitly scan an MPLS label stack for GAL labels that are not at the bottom of the stack?  This seems like silly behavior but one never knows.

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 22 Jun 2021 17:10:09 +0000

John,
Regarding your question "Do you think that it would be safe to re-use the RFC 8662 techniques for GAL?  I.e., a node indicates whether it can receive a GAL at the top of an MPLS label stack and the ingress LSR constructs MPLS label stacks in which a GAL rises to the top of the stack only at such nodes":

IMHO the answer depends on what the node that detects GAL at ToS but not BoS is handing this. I see two options:

1.       Option 1: Pop GAL, process the ACH and forward based on the next label in the stack. This would be safe but probably useless for the specific applications people are trying to address

2.       Option 2:  Same as above, but push GAL on top of the new label stack when forwarding the packet based on the next label in the stack) - i.e., similar to what is defined or the Router Alert label. This would probably work for the targeted applications, but would not be safe - unless you only pass thru the new nodes that understand GAL at ToS but not BoS.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Sent: Tuesday, June 22, 2021 6:35 PM

Hi,

Yours Irrespectively,

John

---

From: John E Drake <[email-masked]>
Date: Tue, 22 Jun 2021 17:57:56 +0000

Hi,

Comment inline.

Yours Irrespectively,

John

Sent: Tuesday, June 22, 2021 1:10 PM

John,

[JD]  Why?  The ingress node would have placed multiple GALs within the MPLS label stack and subsequent transit LSRs would operate on the next GAL.  This is how RFC 8662 works.

  1.  Option 2:  Same as above, but push GAL on top of the new label stack when forwarding the packet based on the next label in the stack) - i.e., similar to what is defined or the Router Alert label. This would probably work for the targeted applications, but would not be safe - unless you only pass thru the new nodes that understand GAL at ToS but not BoS.

[JD]  This is not how RFC 8662 works

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Hi,

Yours Irrespectively,

John

---

From: Haoyu Song <[email-masked]>
Date: Tue, 22 Jun 2021 18:10:20 +0000

Hi Bruno, 

Thanks for the response. 
Just one clarification. What I mean for "forced binding" is that:  EL and EH are two independent use cases but if we require encoding ELI to indicate EH, we need both in a packet which may not be necessary. In other words, if we don't need (or support) EL at all, we still need to find another way to indicate EH.

Regards,
Haoyu

-----Original Message-----
From: [email-masked] <[email-masked]> 
Sent: Tuesday, June 22, 2021 1:33 AM
To: Haoyu Song <[email-masked]>; [email-masked]
Cc: [email-masked]; Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Subject: RE: [mpls] Indicators in the stack and ancillary data after the BoS

Hi Haoyu,

Thanks for replying.

> Thanks for the clarification. We might have some issues using the 
> forced binding in some cases.

>  For example, in our EH architecture, we allow an EH to be inserted or 
> removed at any nodes on an LSP.

[Bruno] As the thread is about EH _indicator_ I'll focus on the indicator. Draft [1] does not forbid the indicator to be modified en route.
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-decraene-mpls-slid-encoded-entropy-label-id-01%23section-2&amp;data=04%7C01%7Chsong%40futurewei.com%7Cc4ee94527683473cb22408d935586024%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637599475954543231%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=pM57vSqNe51mL1PGdXKahJjdyM6B1yzl23ftktvlYmQ%3D&amp;reserved=0 

> Also, ELI+EI are two labels, which is more expensive than one SPL 
> solution.

[Bruno]

> Having said this, I could list your proposal as a possible solution.

> We would use a new SPL to indicate multiples things (also by encoding 
> the unused TTL/CoS bits): the existence of other special labels in the 
> label stack (e.g., ELI+EL) and the existence/location of extension 
> headers after the label stack. Since we have a new SPL, we have more freedom to define its behavior.

Best regards,
--Bruno 

> The node requiring the indicator adds the Entropy Label (ELI, EL). I 
> guess same principle as the other options discussed in the Figure 5 of 
> draft-song-mpls-eh- indicator.
> In which case you get Entropy information for free (I mean no extra 
> label). But from the entropy standpoint, it would be better to have 
> the EL, ELI pushed by the ingress as this is the node which should have the most entropy information.
> 
> Thanks,
> --Bruno
> 
> > Thanks!
> >
> > Haoyu
> >
> > >   there has also been discussion about putting more than one GAL 
> > > in the stack, i.e. differerent GALs pointing to different ACHs.
> > >
> > >   there has also been discussion about putting more than one GAL 
> > > in the stack, i.e. differerent GALs pointing to different ACHs.
> > >
> > >   draft-song-mpls-eh-indicator have a list of potential 
> > > indicators, that is also telling if the EH should be processed on 
> > > every EH capable node or "just" at ingress and egress
> >
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fda
> > ta
> > tracker
> > .ietf.org%2Fdoc%2Fhtml%2Fdraft-decraene-mpls-slid-encoded-entropy-la
> > be
> > l-
> > id%23section-
> >
> > > >> If the GAL does not have S-bit set, wouldn't a transit LSR 
> > > >> treat any 4-ocet field (i.e. those in the above Figure) after 
> > > >> that GAL as a
> > > >> LSR will think the label stack ends there even though that's 
> > > >> just part of the ACH.
> > > >>
> > > > Well, as far as I understand a GAL which does not have the 
> > > > NoS-bit set will have other labels after itself. The BoS-bit 
> > > > will be found deeper down stack and the ACH will immediately fo9llow the BoS.
> > > >
> > > >> Regarding your statement "it is OK to have multiple GALs and 
> > > >> GALs not at BoS IFF the creator of the LSP ensured that all 
> > > >> LSRs on the LSP, including ECMP and FRR paths that found the 
> > > >> GAL at ToS were known to be able to process it correctly":
> > > >>
> > > >> It is still possible that, after all this work, we shall find 
> > > >> out that the benefits of supporting GAL at ToS but not BoS will 
> > > >> be only available in the networks where all the nodes support 
> > > >> the new functionality because presence of non-supporting nodes 
> > > >> It seems to me that an old implementation that received a ToS 
> > > >> GAL not at BoS would at best throw an exception or worst be unpredictable.
> > > >>
> > > >> When we extended GAL to apply to LSPs we again had the model 
> > > >> that the GAL operated in the context of the LSP label that 
> > > >> preceded it for context. It was still BoS.
> > > >>
> > > >> any well written implementation should just result in the 
> > > >> packet being dropped and counted as with any other Mal-formed packet.
> > > >>
> > > >> So I would think that it is OK to have multiple GALs and GALs 
> > > >> were known to be able to process it correctly.
> > > >>
> > > >> processed by any LSR that did not know what it was doing, and 
> > > >> to attempt to precess it would be a violation of the normal 
> > > >> MPLS processing
> > model.
> > > >>
> > > >> - Stewart
> > > >>
> > > >>
> > > >> information of Ribbon Communications Inc. and its Affiliates 
> > > >> that is confidential and/or proprietary for the sole use of the 
> > > >> please notify the sender immediately and then delete all 
> > > >> copies, including
> any attachments.
> > > >>
> > > >>
> > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > ur
> > > ld
> > >
> >
> > confidentielles ou privilegiees et ne doivent donc pas etre 
> > diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> > ce message par erreur, veuillez le signaler a l'expediteur et le 
> > detruire ainsi que les pieces jointes. Les messages electroniques 
> > etant susceptibles d'alteration, Orange decline toute responsabilite 
> > si ce message a ete altere,
> deforme ou falsifie. Merci.
> >
> > If you have received this email in error, please notify the sender 
> > and delete this message and its attachments.
> > As emails may be altered, Orange is not liable for messages that 
> > have been modified, changed or falsified.
> > Thank you.
> 
>

---

From: <[email-masked]>
Date: Wed, 23 Jun 2021 12:41:57 +0000

Hi Haoyu,

> Just one clarification. What I mean for "forced binding" is that:  EL and EH are two
> independent use cases but if we require encoding ELI to indicate EH, we need
> both in a packet which may not be necessary.

Thanks for the clarification.
Agreed.

> In other words, if we don't need (or
> support) EL at all, we still need to find another way to indicate EH.

More simply, rather than defining and supporting another way, you need to support/advertise the EL.

Regards,
Bruno
> Cc: [email-masked]; Loa Andersson <[email-masked]>; Jeffrey (Zhaohui) Zhang
> <[email-masked]>; Alexander Vainshtein <[email-masked]>;
> Stewart Bryant <[email-masked]>
> [Bruno] As the thread is about EH _indicator_ I'll focus on the indicator. Draft [1]
> does not forbid the indicator to be modified en route.
> [1]
> .ietf.org%2Fdoc%2Fhtml%2Fdraft-decraene-mpls-slid-encoded-entropy-label-id-
> 01%23section-
> 2&amp;data=04%7C01%7Chsong%40futurewei.com%7Cc4ee94527683473cb22408
> d935586024%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C63759947595
> 4543231%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=pM57vSqNe51mL1P
> GdXKahJjdyM6B1yzl23ftktvlYmQ%3D&amp;reserved=0
> 
> It would be more expensive only in the case that EL is not needed/useful AND that
> one SPL be allocated for signaling the EH.

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Wed, 30 Jun 2021 21:26:14 +0000

Hi,

Pardon me that I was not able to timely follow the discussions. I am slowly catching up - please allow me to reply to this old email even though there were a lot of newer ones because this one has better context than latter deeply-nested email. However I will refer to some latter email in this reply.

Please see zzh> below.

-----Original Message-----
From: Loa Andersson <[email-masked]>
Sent: Sunday, June 20, 2021 7:11 AM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Alexander Vainshtein <[email-masked]>; Stewart Bryant <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] [EXTERNAL] Indicators in the stack and ancillary data after the BoS

Jeffrey,

zzh> I consider this as a one of the extension header use case so does not need to be considered separately.

zzh> Except Bruno's proposal of using ELI/EL as an indicator, I think generally we are leaning towards using an SPL or XL+ESPL as a logical indicator. Draft-kompella allows a single SPL to achieve functions of otherwise multiple SPL or XL+ESPL, so for the purpose of discussions on the indicators, we can just focus on an SPL, which could either be a new SPL (w/ or w/o draft-kompella tricks) or we can extend the GAL.
Zzh> Several people have said we should not mess with GAL/ACH for this purpose and I am fine with that. However, when there is GAL(s) in the stack, since the data after the BoS must be ACH, we started talking about encoding ancillary data in ACH again. So - are we going to use GAL/ACH or not?

Zzh> I thought we could not have different GALs pointing to different ACHs as mentioned in the email below (a non-BOS GAL would not have ACH following it).

zzh> I think Haoyu said later that he is now leaning towards the draft-kompella method.

Zzh> My understanding is this: A) we want  to indicate ancillary data after the BoS;  B) *Some people may* want to have different ancillary data with different indicators.
Zzh> I think A) is not difficult (it's still not clear to me whether GAL/ACH is still on the table).  B) is not feasible, for the same reason that ACH can only be after BoS. However, different ancillary data can be associated with different EHs which are altogether following a single indicator.

Zzh> Jeffrey

/Loa

--