Subject: [bess] Signaling Control Word in EVPN

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Fri, 5 Oct 2018 09:44:32 +0530

RFC 8214 (EVPN VPWS) introduces a new EVPN Layer 2 Attributes extended
community for signaling the L2 MTU and other control flags, including the
one to signal that the control word needs to be included when sending
packets to this PE. It further describes how MTU checking is to be
performed when signaled using this extended community.

RFC 7432 however is completely silent about it. Is the extended community
described in RFC 8214 expected to used in EVPN (VPLS) as well to signal
things like the usage of control word?

Regards,
Muthu

---

From: Alexander Vainshtein <[email-masked]>
Date: Sun, 7 Oct 2018 14:28:12 +0000

Resending because the previous message is has gone to the BESS list moderator

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]

From: Alexander Vainshtein
Sent: Sunday, October 7, 2018 5:25 PM
To: 'Muthu Arul Mozhi Perumal' <[email-masked]>; [email-masked]' <[email-masked]>; [email-masked]' <[email-masked]>; [email-masked]' <[email-masked]>; [email-masked]' <[email-masked]>; [email-masked]' <[email-masked]>
Cc: [email-masked]; Michael Gorokhovsky <[email-masked]>; Yechiel Rosengarten ([email-masked]) <[email-masked]>; Ron Sdayoor ([email-masked]) <[email-masked]>; Shell Nakash <[email-masked]>; Rotem Cohen ([email-masked]) <[email-masked]>; Dmitry Valdman <[email-masked]>
Subject: RE: [bess] Signaling Control Word in EVPN

Muthu, authors of 8214, and all,
I fully agree that RFC 7432 does not define any way to exchange information about usage or non-usage of the Control Word (CW) in the EVPN encapsulation of Ethernet frames via the EVPN Control Plane (CP). It only RECOMMDNDS its usage or non-usage in specific deployment scenarios.

I also think that a generic (not limited to VPWS) EVPN-CP mechanism for exchanging information about usage (or non-usage) of CW must handle several issues that are not relevant for VPWS services:

1.      Usage or non-usage of CW in EVPN encapsulation of BUM packets:

a.      If ingress replication is used as the P-tunneling technology, usage of CW can be requested by each egress MAC-VRF of a given EVI.

b.      If P2MP MPLS LSPs are used as the P-tunneling technology, all egress MAC-VRFs of the given  EVI will receive BUM packets either with or without the CW – the decision would be taken by the ingress MAC-VRF, and egress MAC-VRFs would have to cope with whatever they get

c.       One possible way to address these  two scenarios could be by using the EVPN Layer 2 Extended Community with EVPN Inclusive Multicast Ethernet Tag Routes (Type 3 EVPN routes):

                                                              i.      Since RFRC 7432 explicitly states in Section 12.2 that in this case BUM traffic may experience reordering when sent over P2MP LSPs, by default the CW SHOULD NOT be included in EVPN encapsulation of BUM traffic (since its only purpose is to prevent reordering)

                                                             ii.      In the case of ingress replication, the C flag in this extended community would represent a request to include the CW in the EVPN encapsulation of the copies of BUM frames sent to the egress MAC-VRF that has advertised this route. However, even this may be superfluous, and we may agree not to use the CW in EVPN encapsulation of BUM traffic

2.      Usage or non-usage of CW in EVPN encapsulation of “known unicast” packets:

a.      In the case of VPWS each MAC-VRF is attached to just one ES, so advertising usage or non-usage of the CW in per-EVI Ethernet A-D route makes sense

b.      In the case of a generic MP2MP EVI, each MAC-VRF can be attached to multiple ES. In this case, implementations SHOULD advertise the same C flag in all per-EVI Ethernet A-D routes they advertise. Alternatively, implementations may use this extended community with per-EVI Ethernet A-D Route with MAX-ESI representing all attached Ethernet Segments

3.      In any case, all implementations MUST be able:

a.      To send and receive BUM packets without the CW

b.      To send “known unicast” traffic with the CW if so requested.

I am not sure I’ve covered all aspects of signaling usage or non-usage of the CW in generic EVPN services, but, IMHO,  the above-mentioned points must be addressed in any solution.

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]<mailto:[email-masked]>

From: BESS [mailto:[email-masked]] On Behalf Of Muthu Arul Mozhi Perumal
Sent: Friday, October 5, 2018 7:15 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [bess] Signaling Control Word in EVPN

RFC 8214 (EVPN VPWS) introduces a new EVPN Layer 2 Attributes extended community for signaling the L2 MTU and other control flags, including the one to signal that the control word needs to be included when sending packets to this PE. It further describes how MTU checking is to be performed when signaled using this extended community.

RFC 7432 however is completely silent about it. Is the extended community described in RFC 8214 expected to be used in EVPN (VPLS) as well to signal things like the usage of control word?

Regards,
Muthu

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is 
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this 
transmission in error, please inform us by e-mail, phone or fax, and then delete the original 
and all copies thereof.
___________________________________________________________________________

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 8 Oct 2018 14:59:53 +0000

Muthu and all,
Please note also that RFC 7432 explicitly states that the CW SHOULD NOT be used in the EVPN encapsulation of Ethernet frames that are delivered across P2P or P2MP LSPs.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Alexander Vainshtein
Sent: Sunday, October 7, 2018 5:28 PM
To: Muthu Arul Mozhi Perumal <[email-masked]>
Cc: [email-masked]; Michael Gorokhovsky <[email-masked]>; Yechiel Rosengarten ([email-masked]) <[email-masked]>; Ron Sdayoor ([email-masked]) <[email-masked]>; Shell Nakash <[email-masked]>; Rotem Cohen ([email-masked]) <[email-masked]>; Dmitry Valdman <[email-masked]>
Subject: FW: [bess] Signaling Control Word in EVPN

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Alexander Vainshtein
Sent: Sunday, October 7, 2018 5:25 PM
To: 'Muthu Arul Mozhi Perumal' <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Michael Gorokhovsky <[email-masked]<mailto:[email-masked]>>; Yechiel Rosengarten ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>>; Ron Sdayoor ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>>; Shell Nakash <[email-masked]<mailto:[email-masked]>>; Rotem Cohen ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>>; Dmitry Valdman <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] Signaling Control Word in EVPN

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Regards,
Muthu

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 8 Oct 2018 12:33:25 -0400

Sasha,

In the light of draft-ietf-pals-ethernet-cw (very soon to be published as
an RFC), we might want to take a second look at the recommendations in
7432. I think 8214 has it right, where it recommends the control word in
the absence of an entropy label to prevent ECMP reordering. 7432 does
recommend the control word be used for MP2P LSPs, but I would certainly
recommend it whenever Ethernet frames are being sent without an entropy
label, whether MP2P, P2P, or P2MP, unless it is known that ECMP is not in
use in the network. As you know, ECMP reordering of Ethernet frames isn't
just theoretical, it's actually happening in the field.

Cheers,
Andy

On Mon, Oct 8, 2018 at 11:00 AM Alexander Vainshtein <
[email-masked]> wrote:

> Muthu and all,
>
> Please note also that RFC 7432 explicitly states that the CW SHOULD NOT be
> used in the EVPN encapsulation of Ethernet frames that are delivered across
> P2P or P2MP LSPs.
>
>
>
> Regards,
>
> Sasha
>
>
>
> Office: +972-39266302
>
> Cell:      +972-549266302
>
> *From:* Alexander Vainshtein
> *Sent:* Sunday, October 7, 2018 5:28 PM
> *To:* Muthu Arul Mozhi Perumal <[email-masked]>
> *Cc:* [email-masked]; Michael Gorokhovsky <[email-masked]>;
> Yechiel Rosengarten ([email-masked]) <
> [email-masked]>; Ron Sdayoor ([email-masked]) <
> [email-masked]>; Shell Nakash <[email-masked]>; Rotem
> Cohen ([email-masked]) <[email-masked]>; Dmitry Valdman
> <[email-masked]>
> *Subject:* FW: [bess] Signaling Control Word in EVPN
>
>
>
> *From:* Alexander Vainshtein
> *Sent:* Sunday, October 7, 2018 5:25 PM
> *To:* 'Muthu Arul Mozhi Perumal' <[email-masked]>; '
> [email-masked]' <[email-masked]>; [email-masked]' <
> [email-masked]>; [email-masked]' <[email-masked]>; '
> [email-masked]' <[email-masked]>; [email-masked]' <
> [email-masked]>
> *Cc:* [email-masked]; Michael Gorokhovsky <[email-masked]>;
> Yechiel Rosengarten ([email-masked]) <
> [email-masked]>; Ron Sdayoor ([email-masked]) <
> [email-masked]>; Shell Nakash <[email-masked]>; Rotem
> Cohen ([email-masked]) <[email-masked]>; Dmitry Valdman
> <[email-masked]>
> *Subject:* RE: [bess] Signaling Control Word in EVPN
>
>
>
> I fully agree that RFC 7432 does not define any way to exchange
> information about usage or non-usage of the Control Word (CW) in the EVPN
> encapsulation of Ethernet frames via the EVPN Control Plane (CP). It only
> RECOMMDNDS its usage or non-usage in specific deployment scenarios.
>
>
>
> I also think that a generic (not limited to VPWS) EVPN-CP mechanism for
> exchanging information about usage (or non-usage) of CW must handle several
> issues that are not relevant for VPWS services:
>
>
>
> a.      If ingress replication is used as the P-tunneling technology,
> usage of CW can be requested by each egress MAC-VRF of a given EVI.
>
> b.      If P2MP MPLS LSPs are used as the P-tunneling technology, all
> egress MAC-VRFs of the given  EVI will receive BUM packets either with or
> without the CW – the decision would be taken by the ingress MAC-VRF, and
> egress MAC-VRFs would have to cope with whatever they get
>
> c.       One possible way to address these  two scenarios could be by
> using the EVPN Layer 2 Extended Community with EVPN Inclusive Multicast
> Ethernet Tag Routes (Type 3 EVPN routes):
>
>                                                               i.      Since
> RFRC 7432 explicitly states in Section 12.2 that in this case BUM traffic
> may experience reordering when sent over P2MP LSPs, by default the CW
> SHOULD NOT be included in EVPN encapsulation of BUM traffic (since its only
> purpose is to prevent reordering)
>
>                                                              ii.      In
> the case of ingress replication, the C flag in this extended community
> would represent a request to include the CW in the EVPN encapsulation of
> the copies of BUM frames sent to the egress MAC-VRF that has advertised
> this route. However, even this may be superfluous, and we may agree not to
> use the CW in EVPN encapsulation of BUM traffic
>
> a.      In the case of VPWS each MAC-VRF is attached to just one ES, so
> advertising usage or non-usage of the CW in per-EVI Ethernet A-D route
> makes sense
>
> b.      In the case of a generic MP2MP EVI, each MAC-VRF can be attached
> to multiple ES. In this case, implementations SHOULD advertise the same C
> flag in all per-EVI Ethernet A-D routes they advertise. Alternatively,
> implementations may use this extended community with per-EVI Ethernet A-D
> Route with MAX-ESI representing all attached Ethernet Segments
>
> I am not sure I’ve covered all aspects of signaling usage or non-usage of
> the CW in generic EVPN services, but, IMHO,  the above-mentioned points
> must be addressed in any solution.
>
>
>
> My 2c,
>
> Sasha
>
>
>
> Office: +972-39266302
>
> Cell:      +972-549266302
>
> *From:* BESS [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Muthu Arul Mozhi Perumal
> *Sent:* Friday, October 5, 2018 7:15 AM
> *To:* [email-masked]
> *Subject:* [bess] Signaling Control Word in EVPN
>
>
>
> described in RFC 8214 expected to be used in EVPN (VPLS) as well to signal
> _______________________________________________
> BESS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bess
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 9 Oct 2018 10:28:37 +0000

Andy hi!
As always, very glad to hear from you.

I fully agree that reordering due to lack of the CW happens, and that usage of the CW is the right way to eliminate that.

However, I think that the situation with usage of the CW MP2MP EVPN services is somewhat different from the situation with P2P services or with PW-based VPLS (RFC 4762).

In the case of the latter, two endpoints of a given PW can negotiate usage of the CW (BTW, 8214 does not specify any rules for such negotiation, but this should not be too difficult to specify following the general logic defined in Section 7.2 of RFC 8022). From my POV this ability is really the enabler for the draft-ietf-pals-ethernet-cw<https://tools.ietf.org/html/draft-ietf-pals-ethernet-cw-07>  that says ”where both the ingress PE and the egress PE support the Ethernet pseudowire control  word, then the CW MUST be used”.

In the case of the former, I do not see how such negotiation can be successfully performed because the egress MAC-VRF of an MP2MP EVI cannot infer the ingress MAC-VRF of the EVPN-encapsulated packet from the EVPN encapsulation.  I.e., in order to use the CW in an MP2MP EVPN service instance, all MAC-VRFs that participate in the EVI in question must be able to send and receive the CW in the EVPN encapsulation of Ethernet frames. The scenario where a new PE that does not support CW in the EVPN encapsulation is added to an already existing EVI that has been using the CW is a nice illustration of the complexity of the problem. At the same quite a few deployed EVPN-MPLS implementations do not support usage of the CW (or at least do not announce ability to configure its usage), while some implementations impose restrictions on support of the CW (e.g., do not support it in EVPN encapsulation of BUM traffic). To me this means that we cannot ignore the backward compatibility problem – and I do not see any way to resolve it using the EVPN CP.

Did I miss something substantial here? Is there some “BGP voodoo”  trick that would facilitate the required negotiation?
Or should we agree that consistent usage of the CW in the EVPN encapsulation has to be delegated to some network-wide management mechanism/LSO? For the reference, the latest (expired) version of the EVPN YANG data model<https://www.ietf.org/archive/id/draft-ietf-bess-evpn-yang-05.txt> does not include any attributes that indicate usage or non-usage of the CW in the encapsulation...

Your feedback would be highly appreciated.

Regards, and lots of thanks in advance,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Andrew G. Malis [mailto:[email-masked]]
Sent: Monday, October 8, 2018 7:33 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Shell Nakash <[email-masked]>; Yechiel Rosengarten <[email-masked]>; Michael Gorokhovsky <[email-masked]>; Dmitry Valdman <[email-masked]>; Ron Sdayoor <[email-masked]>; [email-masked]; Rotem Cohen <[email-masked]>
Subject: Re: [bess] Signaling Control Word in EVPN

Sasha,

In the light of draft-ietf-pals-ethernet-cw (very soon to be published as an RFC), we might want to take a second look at the recommendations in 7432. I think 8214 has it right, where it recommends the control word in the absence of an entropy label to prevent ECMP reordering. 7432 does recommend the control word be used for MP2P LSPs, but I would certainly recommend it whenever Ethernet frames are being sent without an entropy label, whether MP2P, P2P, or P2MP, unless it is known that ECMP is not in use in the network. As you know, ECMP reordering of Ethernet frames isn't just theoretical, it's actually happening in the field.

Cheers,
Andy

On Mon, Oct 8, 2018 at 11:00 AM Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Muthu and all,
Please note also that RFC 7432 explicitly states that the CW SHOULD NOT be used in the EVPN encapsulation of Ethernet frames that are delivered across P2P or P2MP LSPs.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

To: Muthu Arul Mozhi Perumal <[email-masked]<mailto:[email-masked]>>

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

To: 'Muthu Arul Mozhi Perumal' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>' <[email-masked]<mailto:[email-masked]>>

My 2c,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Regards,
Muthu

[email-masked]<mailto:[email-masked]>

---

From: James Bensley <[email-masked]>
Date: Tue, 9 Oct 2018 13:48:55 +0100

> I fully agree that reordering due to lack of the CW happens, and that usage of the CW is the right way to eliminate that.

Hi Sasha,

If I may rudely interject here. You have to be careful with your
wording. The PWMCW doesn't eliminate reordering.

https://tools.ietf.org/html/draft-ietf-pals-ethernet-cw-07#section-4:

4.  Recommendation

   The ambiguity between an MPLS payload that is an Ethernet PW and one
   that is an IP packet is resolved when the Ethernet PW control word is
   used.  This document updates [RFC4448] to state that where both the
   ingress PE and the egress PE support the Ethernet pseudowire control
   word, then the CW MUST be used.

The CW doesn't fix the problem in every case, but it does fix it in most cases.

The only way to truly fix this problem is to use entropy labels. The
problem with the CW draft is that it only recommends entropy labels or
FAT labels when ECMP is required, not when it isn't require *but*
exists anyway within the packet switched core:

   Where the application of ECMP to an Ethernet PW traffic is required,
   and where both the ingress and the egress PEs support [RFC6790]
   (Entropy Label Indicator/Entropy Label (ELI/EL)) or both the ingress
   and the egress PEs support [RFC6391] (FAT PW), then either method may
   be used.

For this reason my personal opinion is that only EL should be
recommended by the EVPN draft. I don't think a solution that doesn't
completely fix the problem (PWMCW) should be recommend when we have
one that does fully fix the problem (El/ELI or ELC/RLD in SR).

> Or should we agree that consistent usage of the CW in the EVPN encapsulation has to be delegated to some network-wide management mechanism/LSO? For the reference, the latest (expired) version of the EVPN YANG data model does not include any attributes that indicate usage or non-usage of the CW in the encapsulation...

This is perhaps the more difficult question ^ what to recommend in the
case that EL isn't supported? You could recommend that all devices
should be configured to *not* hash on the VPN payload and only the
label stack, then there can be no mistaking what the VPN contents are
(Ethernet/IPv4/IPv6/other) but, some devices might not support that.
Alternatively, you could recommend that the CW is used only when EL
isn't available but, some devices might not support that. I'll leave
this up to you guys as you're the experts here, I just wanted to point
out the wording issue above regarding CW and EL and "fixing"
reordering.

Cheers,
James.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 9 Oct 2018 10:16:06 -0400

James,

It's much harder to mandate use of EL than the CW for several reasons:
- CW implementation is much more common than EL implementation
- PWs and/or EVPN are rarely the only traffic in an MPLS traffic tunnel,
rather, they will be multiplexed with other MPLS-based applications that
are using the traffic tunnel to reach a common destination. Thus, by using
the CW, you can disable ECMP only for those MPLS packets that cannot
tolerate reordering.

That said, I'm also concerned that because of the existing text in 7432,
implementations may not be using the CW even for P2P EVPN.

And we still don't have a good answer for Muthu's original question. :-)

Cheers,
Andy

On Tue, Oct 9, 2018 at 8:49 AM James Bensley <[email-masked]> wrote:

> > I fully agree that reordering due to lack of the CW happens, and that
> usage of the CW is the right way to eliminate that.
>
> Hi Sasha,
>
> > Or should we agree that consistent usage of the CW in the EVPN
> encapsulation has to be delegated to some network-wide management
> mechanism/LSO? For the reference, the latest (expired) version of the EVPN
> YANG data model does not include any attributes that indicate usage or
> non-usage of the CW in the encapsulation...
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 9 Oct 2018 14:32:35 +0000

Andy, James and all,
One answer for Muthu’s question could be that the EVPN CP (including the EVPN Layer 2 Attributes Extended Community) cannot provide support for negotiating the CW usage in the case of MP2MP EVPN services.

I agree that this can hardly be called a good answer, but I suspect that there is no other one.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Andrew G. Malis [mailto:[email-masked]]
Sent: Tuesday, October 9, 2018 5:16 PM
To: [email-masked]
Cc: Alexander Vainshtein <[email-masked]>; [email-masked]; Shell Nakash <[email-masked]>; Yechiel Rosengarten <[email-masked]>; Michael Gorokhovsky <[email-masked]>; Dmitry Valdman <[email-masked]>; Ron Sdayoor <[email-masked]>; [email-masked]; Rotem Cohen <[email-masked]>
Subject: Re: [bess] Signaling Control Word in EVPN

James,

It's much harder to mandate use of EL than the CW for several reasons:
- CW implementation is much more common than EL implementation
- PWs and/or EVPN are rarely the only traffic in an MPLS traffic tunnel, rather, they will be multiplexed with other MPLS-based applications that are using the traffic tunnel to reach a common destination. Thus, by using the CW, you can disable ECMP only for those MPLS packets that cannot tolerate reordering.

That said, I'm also concerned that because of the existing text in 7432, implementations may not be using the CW even for P2P EVPN.

Cheers,
Andy

On Tue, Oct 9, 2018 at 8:49 AM James Bensley <[email-masked]<mailto:[email-masked]>> wrote:
On Tue, 9 Oct 2018 at 11:29, Alexander Vainshtein

Hi Sasha,

4.  Recommendation

Cheers,
James.

---

From: James Bensley <[email-masked]>
Date: Tue, 9 Oct 2018 15:34:52 +0100

> James,

Hi Andy,

> It's much harder to mandate use of EL than the CW for several reasons:

I didn't say it should be mandated, but recommended.

> - PWs and/or EVPN are rarely the only traffic in an MPLS traffic tunnel, rather, they will be multiplexed with other MPLS-based applications that are using the traffic tunnel to reach a common dest…

The CW does not disable ECMP. Any LSR on the path between ingress and
egress LER is free to look beyond the MPLS label stack and
misinterpret the 0x00 0x00 at the start of a control-word as a valid
MAC that starts 00:00:XX:XX:XX:XX and try to hash on Ethernet headers
starting directly after the MPLS label stack, and not label stack + 4
bytes. This is my point. The PWMCW doesn't stop re-ording in all
cases, but it does in most. So yes, not all devices support EL, but CW
doesn't stop re-ordering in all cases, so?

> And we still don't have a good answer for Muthu's original question. :-)

Sorry my intention is not to send this thread off-topic.

Cheers,
James.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 9 Oct 2018 10:54:17 -0400

James,

Agreed. We touched on that in section 7 of draft-ietf-pals-ethernet-cw,
where we advised operators that enabling post-CW DPI for ECMP calculations
could cause misordering.

Cheers,
Andy

---

From: James Bensley <[email-masked]>
Date: Tue, 23 Oct 2018 09:06:08 +0100

Hi All,

I've removed all the individual contacts as per mod request and
changed the subject as this conversation was diverging away from the
OPs original query (sorry if any of that was my fault).

On Tue, 9 Oct 2018 at 15:54, Andrew G. Malis <[email-masked]> wrote:
>
> James,
>
> Agreed. We touched on that in section 7 of draft-ietf-pals-ethernet-cw, where we advised operators that enabling post-CW DPI for ECMP calculations could cause misordering.

Hi Andy,

Perhaps I wasn't clear enough - and LSR (not the ingress/egress LER)
even trying to detect CW is a source of problems within the network,
this is why I was querying the use of the PWMCW.

On Tue, 9 Oct 2018 at 16:44, Yutianpeng (Tim) <[email-masked]> wrote:
>
> Personally I support having control word capability which is more common. It is said in RFC 8214:
>
>    “It is recommended that the control word be included in the absence of an entropy label”
>
> So I think we can still have EL capability on EVPN VPWS if I understand correct. Only if the existing "BGP Path  Attributes" does not work well.
>
> I support adapt CW capability into EVPN also as EVPN itself is facing same challenge with EVPN VPWS.
>
> By the way, rfc4385 mentioned PW controlling ECMP.

The CW doesn't control ECMP - this is my gripe here. It's one thing to
recommend a technology that not all devices support (EL/ELI or FAT)
but which fully fixes the problem if it supported, it's another thing
to recommend a technology that doesn't fully fix the problem if it is
supported at all. My concern is that by continuing to suggest a
partial solution (PWMCW) we help to keep that partial solution in
circulation when surely the WG should look to deprecate it for a
solution that fully fixes the problem?

Cheers,
James.

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 23 Oct 2018 08:27:53 +0000

James,

I am adding PALS WG to the list of addressees because, AFAIK, the PW CW is defined in this WG.

I think that the really observed problem with incorrect ECMP behavior exists, but it is different from your description in your earlier email:

Any LSR on the path between ingress and egress LER is free to look beyond the MPLS label stack and misinterpret the 0x00 0x00 at the start of a control-word as a valid MAC that starts 00:00:XX:XX:XX:XX and try to hash on Ethernet headers starting directly after the MPLS label stack.

I have not seen (or heard about) such behavior in any deployed networks.

However, I am aware of some modern forwarding chipsets that (correctly) treat the ‘0000’ in the first nibble of the payload of a labeled packet (i.e., immediately following the bottom of the label stack) as the indication of a 32-bit PW control word but (incorrectly), consider this as a CW of an Ethernet PW (as if no other PWs exist!) and try to hash on the presumed MAC addresses, Ethertype etc.  Such behavior is really deadly for, say TDM PWs that, AFAIK, are still widely deployed in many places.

I wonder if IETF (or, specifically, the PALS WG) can do anything about that.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: BESS [mailto:[email-masked]] On Behalf Of James Bensley
Sent: Tuesday, October 23, 2018 11:06 AM
To: [email-masked]
Subject: [bess] CW in EVPN: Was Signaling Control Word in EVPN

Hi All,

I've removed all the individual contacts as per mod request and changed the subject as this conversation was diverging away from the OPs original query (sorry if any of that was my fault).

On Tue, 9 Oct 2018 at 15:54, Andrew G. Malis <[email-masked]<mailto:[email-masked]>> wrote:

Hi Andy,

Perhaps I wasn't clear enough - and LSR (not the ingress/egress LER) even trying to detect CW is a source of problems within the network, this is why I was querying the use of the PWMCW.

On Tue, 9 Oct 2018 at 16:44, Yutianpeng (Tim) <[email-masked]<mailto:[email-masked]>> wrote:

> By the way, rfc4385 mentioned PW controlling ECMP.

The CW doesn't control ECMP - this is my gripe here. It's one thing to recommend a technology that not all devices support (EL/ELI or FAT) but which fully fixes the problem if it supported, it's another thing to recommend a technology that doesn't fully fix the problem if it is supported at all. My concern is that by continuing to suggest a partial solution (PWMCW) we help to keep that partial solution in circulation when surely the WG should look to deprecate it for a solution that fully fixes the problem?

Cheers,

James.

BESS mailing list

---

From: James Bensley <[email-masked]>
Date: Mon, 29 Oct 2018 13:33:52 +0000

Hi Sasha,

Well in either case we have both provided different examples of when
the PWMCW doesn't prevent reordering when ECMP is present in the PSN.
Perhaps I need to actually step back a bit here and ask a different
question, to further understand the non-technical problem first.

What is the scope/remit of the WG in this scenario? To be clear on
what I mean by this, the PWMCW is a flawed method for preventing
reordering when ECMP is present (see our two examples above). It also
doesn't add any entropy to improve ECMP when ECMP is required. Entropy
labels help to prevent reordering and help to add entropy, despite
these technical benefits but they may have other non-technical
disadvantages. So what is the WGs remit with recommending one
technology over another?

In this specific case, is it:

Option 1. The WG's remit is to phase out or discourage flawed
technologies if a superior one exists, so it should look to deprecate
CWs because ELs are superior from a purely technical view?
Option 2. The WG's remit is to support as many technologies as
possible, so it shouldn't look to deprecate CWs because ELs may have
other non-technical draw backs?
Option 3. The WG's remit is to remain neutral on the subject of CWs vs
other methods, and simply ensure that all drafts follow the correct
due diligence process regardless of whether one technology is
technically "better" than another?
Option 4. The WG's remit is something else?

Cheers,
James.

---

From: "Yutianpeng (Tim)" <[email-masked]>
Date: Mon, 29 Oct 2018 22:38:47 +0000

Hi,
Some comments below.
Regards,
Tim
-----Original Message-----
From: BESS [mailto:[email-masked]] On Behalf Of James Bensley
Sent: 29 October 2018 13:34
To: [email-masked]; [email-masked]
Subject: Re: [bess] CW in EVPN: Was Signaling Control Word in EVPN

> I have not seen (or heard about) such behavior in any deployed networks.
[Tim]: This behavior exists on some devices, but they should at least ignore 0x0 MAC.
> However, I am aware of some modern forwarding chipsets that (correctly) treat the ‘0000’ in the first nibble of the payload of a labeled packet (i.e., immediately following the bottom of the label…
[Tim]: Yes, this does exist, at least ever :), but this is a behavior not compliance with RFC and should be corrected. Chipset with such capabilities is usually programmable and should have the capability to fix the limit, otherwise will be a disaster for pretending to be intelligent. 

Hi Sasha,

Well in either case we have both provided different examples of when the PWMCW doesn't prevent reordering when ECMP is present in the PSN.
Perhaps I need to actually step back a bit here and ask a different question, to further understand the non-technical problem first.

What is the scope/remit of the WG in this scenario? To be clear on what I mean by this, the PWMCW is a flawed method for preventing reordering when ECMP is present (see our two examples above). It also doesn't add any entropy to improve ECMP when ECMP is required. Entropy labels help to prevent reordering and help to add entropy, despite these technical benefits but they may have other non-technical disadvantages. So what is the WGs remit with recommending one technology over another?

Option 1. The WG's remit is to phase out or discourage flawed technologies if a superior one exists, so it should look to deprecate CWs because ELs are superior from a purely technical view?
Option 2. The WG's remit is to support as many technologies as possible, so it shouldn't look to deprecate CWs because ELs may have other non-technical draw backs?
Option 3. The WG's remit is to remain neutral on the subject of CWs vs other methods, and simply ensure that all drafts follow the correct due diligence process regardless of whether one technology is technically "better" than another?
Option 4. The WG's remit is something else?

[Tim]: My opinion is: 
Control word is proven technology for years and has been deployed a lot. I cannot find a reason we should abandon it. ELs can be better but it is also difficult to achieve.
Many of the chipsets nowadays are still not capable to implement ELs. I believe both techniques will exit for a long period of time.

Cheers,
James.