Subject: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

From: <[email-masked]>
Date: Mon, 3 May 2021 09:20:58 +0200

Hi WG,

We got final updates from authors on draft-ietf-bess-evpn-unequal-lb.
 
I'm opening a new short Working Group Last Call (to be closed on 5/10) to
get any last comments before moving to the next step.
However, the document having normative references to EVPN PREF DF, and
PER-MCAST-FLOW-DF, the draft will not be sent to IESG until these drafts are
ready.

Feel free to send comments to the list before next Monday.

Thanks,

Stephane
 
https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-unequal-lb/

---

From: <[email-masked]>
Date: Mon, 3 May 2021 09:20:16 +0000

Hi Stéphane, authors,

I have not followed the discussions on this document, but I'll nonetheless raise one point  regarding the bandwidth community (better safe than sorry).
- why has [BGP-LINK-BW] been moved to informational reference while its reading seem mandatory?
- A new EVPN Link Bandwidth extended community is defined, but I could not find its specification. I guess that this is the same format as [BGP-LINK-BW] but transitive. Could this be explicitly stated?
- [BGP-LINK-BW] advertises the bandwidth in unit of bytes (not bits!) per second. Could the unit of the new EVPN Link Bandwidth extended community be also clearly spelled out? Especially give the history on this (cf below). Also in order to avoid misleading the readers could the examples use the correct unit (vs bits per seconds as writen)
- 10 years ago or so, I had raised a similar point (distinction between bits and bytes) on [BGP-LINK-BW] in the IDR WG. And it turned out that 1 major implementation had implemented and deployed "bytes per second" as per the spec, while another implementation had implemented and deployed "bits per second" which is the typical unit of link bandwidth. Given the deployments, none was willing to change its implementation as it would be a non-backward compatible change with themselves. What's the status on this? Could we have an implementation status on this?

Thanks
Regards,
--Bruno

From: BESS [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, May 3, 2021 9:21 AM
To: [email-masked]
Subject: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi WG,

However, the document having normative references to EVPN PREF DF, and PER-MCAST-FLOW-DF, the draft will not be sent to IESG until these drafts are ready.

Thanks,

Stephane

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Neeraj Malhotra <[email-masked]>
Date: Wed, 5 May 2021 22:40:56 -0700

Hi Bruno,

Many thanks for the review comments. We have revised the draft addressing
your comments.

More inline.

Thanks,
Neeraj

On Mon, May 3, 2021 at 2:20 AM <[email-masked]> wrote:

> Hi Stéphane, authors,
>
>
>
> I have not followed the discussions on this document, but I’ll nonetheless
> raise one point  regarding the bandwidth community (better safe than
> sorry).
>

[NM]: There was a leftover reference to this in one of the sections that
has been fixed now to use new EVPN EC. With this, reference to
[BGP-LINK-BW] is purely informational (as was intended).

> - A new EVPN Link Bandwidth extended community is defined, but I could not
> find its specification. I guess that this is the same format as
> [BGP-LINK-BW] but transitive. Could this be explicitly stated?
>

[NM]: clarified in section 4.

> - [BGP-LINK-BW] advertises the bandwidth in unit of bytes (not bits!) per
> second. Could the unit of the new EVPN Link Bandwidth extended community be
> also clearly spelled out? Especially give the history on this (cf below).
> Also in order to avoid misleading the readers could the examples use the
> correct unit (vs bits per seconds as writen)
>

[NM]: done.

> - 10 years ago or so, I had raised a similar point (distinction between
> bits and bytes) on [BGP-LINK-BW] in the IDR WG. And it turned out that 1
> major implementation had implemented and deployed “bytes per second” as per
> the spec, while another implementation had implemented and deployed “bits
> per second” which is the typical unit of link bandwidth. Given the
> deployments, none was willing to change its implementation as it would be a
> non-backward compatible change with themselves. What’s the status on this?
> Could we have an implementation status on this?
>

[NM]: I don't have this information. Perhaps someone else could comment.

>
>
> Thanks
>
> Regards,
>
> --Bruno
>
>
>
>
>
> *From**:* BESS [mailto:[email-masked]] *On Behalf Of *
> [email-masked]
> *Sent:* Monday, May 3, 2021 9:21 AM
> *To:* [email-masked]
> *Subject:* [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
>
>
> Hi WG,
>
>
>
>
>
> _______________________________________________
> BESS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bess
>

---

From: <[email-masked]>
Date: Thu, 6 May 2021 08:03:43 +0000

Hi Neeraj,

Thanks for considering my comments.
Much better from my perspective. Thank you.

I have two comments on the changes:
- Regarding deployments
§4.1 allows two rather incompatible encodings/usages with no way to detect which one is used: some PE could advertise the bandwidth in bytes, while some other PE could advertise a general weight. I understand that both works, but to me there is a significant risk of issues over time or between domain/SP. I’d prefer that you only chose one in order to favour consistency in deployments and usage and I would prefer the real bandwidth (at least for consistency with the name of the community, but also because this is not subjective)  (And if a SP really wants to put an arbitrary value, I think he will figure out by himself, that it can do so).
If you disagree with the above, then I would have a comment on the two below sentences:
An implementation may support one or more of the above ways of
   encoding the value.  Operator MUST ensure consistent encoding of this
   value across all PEs in an ethernet segment.
Logic dictates that the second sentence (MUST) can only be fulfilled if the first sentence mandates that all implementations MUST support both options, or one specifically defined.

- Regarding existing implementations:
previous version of the draft did not really specify the format of the EVPN EC. I had personally assumed that the format was similar to the draft-ietf-idr-link-bandwidth link bandwidth community hence encoded in IEEE floating point format. Latest version of the draft defines it in unsigned integer. Integer looks good to me, but for an existing implementation this may be seen as an incompatible change very late in the process. Obviously, if there are no implementation, there is no issue. In which case, you could also express the bandwidth in unit of bit/s _if you_ wish to. (I have no preference). However given that the draft had indicated a codepoint, there seem to be a risk of existing implementations hence incompatible change. BTW the codepoint is squatted even though the registry is FCFS hence easy to request.

Thanks,
--Bruno

From: Neeraj Malhotra [mailto:[email-masked]]
Sent: Thursday, May 6, 2021 7:41 AM
To: DECRAENE Bruno TGI/OLN <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Bruno,

Many thanks for the review comments. We have revised the draft addressing your comments.

More inline.

Thanks,
Neeraj

On Mon, May 3, 2021 at 2:20 AM <[email-masked]<mailto:[email-masked]>> wrote:
Hi Stéphane, authors,

I have not followed the discussions on this document, but I’ll nonetheless raise one point  regarding the bandwidth community (better safe than sorry).
- why has [BGP-LINK-BW] been moved to informational reference while its reading seem mandatory?

[NM]: There was a leftover reference to this in one of the sections that has been fixed now to use new EVPN EC. With this, reference to [BGP-LINK-BW] is purely informational (as was intended).

[NM]: done.

- 10 years ago or so, I had raised a similar point (distinction between bits and bytes) on [BGP-LINK-BW] in the IDR WG. And it turned out that 1 major implementation had implemented and deployed “bytes per second” as per the spec, while another implementation had implemented and deployed “bits per second” which is the typical unit of link bandwidth. Given the deployments, none was willing to change its implementation as it would be a non-backward compatible change with themselves. What’s the status on this? Could we have an implementation status on this?

Thanks
Regards,
--Bruno

From: BESS [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Monday, May 3, 2021 9:21 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi WG,

Thanks,

Stephane

Thank you.
_______________________________________________
BESS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/bess

---

From: "Rabadan, Jorge (Nokia - US/Mountain View)" <[email-masked]>
Date: Thu, 6 May 2021 08:35:54 +0000

Hi Bruno,

Thanks for your comments.

About the first point, we do have use cases where the bandwidth is not what we want to encode in the EC but rather a generalized weight that is derived from the link count, logical weight or simply a configured value. Among the co-authors we also discussed the possibility of defining two ECs: one for BW and one for a generalized-weight, so that the remote PE can catch if the multi-homed PEs were indeed using the same meaning of the weight. However, we thought it was easier/simpler to use a generalized value in a single EC sub-type, and add the sentence below.

The sentence can be modified/fixed. But the gist is that the multi-homed PEs may support multiple meanings for the weight (BW, link-count, etc), but at least one of those MUST be common across all PEs and the multi-homed routes must use it consistently. Would it be enough if we fix it?

About existing implementations, a new EVPN sub-type was defined only a couple of revisions ago, where, before, the existing non-transitive link BW EC was used, so there’s been some churn in the use of the EC anyway. I think it is important to get it as soon as possible, but get it right rather than finding gaps later once the document is done. But let us know your thoughts too.

Thank you.
Jorge

From: BESS <[email-masked]> on behalf of [email-masked] <[email-masked]>
Date: Thursday, May 6, 2021 at 10:04 AM
To: Neeraj Malhotra <[email-masked]>
Cc: [email-masked] <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 12:45:59 +0000

Hi Jorge,

Thanks for the feedback.

Regarding the first point, I can live with the current text. But I think I would prefer that the text favour one option, and leave it to the responsibility of the SP for others usages. E.g.

OLD:
EVPN Link Bandwidth Extended Community value field is to be treated
   as a 6 octet unsigned integer that may be set to:

   o  total bandwidth of PE's all physical links in an ethernet segment,
      expressed in bytes/sec.

   o  or a generalized weight that may be set to link count, locally
      configured weight, or a value computed based on an attribute other
      than link bandwidth.

   An implementation may support one or more of the above ways of
   encoding the value.  Operator MUST ensure consistent encoding of this
   value across all PEs in an ethernet segment.  Procedures related to
   signaling and handling of this extended community defined in this
   document use "total bandwidth in bytes/sec" encoding as an example to
   illustrate its usage.

NEW:
   EVPN Link Bandwidth Extended Community value field is to be treated
   as a 6 octet unsigned integer representing total bandwidth of PE's all physical links in an ethernet segment,
      expressed in bytes/sec.

Note however that the load balancing algorithm defines in this document uses ratio of Link Bandwidths hence the operator may choose a different unit or use the community as
    a generalized weight that may be set to link count, locally
      configured weight, or a value computed based on an attribute other
      than link bandwidth. In such case, the operator MUST ensure consistent usage of the unit
across all PEs in an ethernet segment. This may involve multiple routing domains/Autonomous Systems.

But I leave this to you.

Thanks,
--Bruno

From: Rabadan, Jorge (Nokia - US/Mountain View) [mailto:[email-masked]]
Sent: Thursday, May 6, 2021 10:36 AM
To: DECRAENE Bruno TGI/OLN <[email-masked]>; Neeraj Malhotra <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Bruno,

About existing implementations, a new EVPN sub-type was defined only a couple of revisions ago, where, before, the existing non-transitive link BW EC was used, so there's been some churn in the use of the EC anyway. I think it is important to get it as soon as possible, but get it right rather than finding gaps later once the document is done. But let us know your thoughts too.

Thank you.
Jorge

From: BESS <[email-masked]<mailto:[email-masked]>> on behalf of [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Date: Thursday, May 6, 2021 at 10:04 AM
To: Neeraj Malhotra <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
Hi Neeraj,

§4.1 allows two rather incompatible encodings/usages with no way to detect which one is used: some PE could advertise the bandwidth in bytes, while some other PE could advertise a general weight. I understand that both works, but to me there is a significant risk of issues over time or between domain/SP. I'd prefer that you only chose one in order to favour consistency in deployments and usage and I would prefer the real bandwidth (at least for consistency with the name of the community, but also because this is not subjective)  (And if a SP really wants to put an arbitrary value, I think he will figure out by himself, that it can do so).

Thanks,
--Bruno

From: Neeraj Malhotra [mailto:[email-masked]]
Sent: Thursday, May 6, 2021 7:41 AM
To: DECRAENE Bruno TGI/OLN <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

---

From: John E Drake <[email-masked]>
Date: Thu, 6 May 2021 12:54:28 +0000

Bruno,

I had suggested:

"The value field in the link bandwidth EC is to be treated as a 6 octet unsigned integer and it is the provider's  responsibility to encode it consistently across all of the PEs attached to a given ES.  So, for example, if the provider wanted the EC to represent attachment circuit bandwidth, it should decide the units, e.g., 1 GBPS, and then encode the value field as a multiple of that unit.

This ensures that when an ingress PE is doing weighted load balancing, in all cases it is doing simple integer arithmetic on values whose semantics are unknown to it."

Yours Irrespectively,

John

Juniper Business Use Only
From: BESS <[email-masked]> On Behalf Of [email-masked]
Sent: Thursday, May 6, 2021 4:04 AM
To: Neeraj Malhotra <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

[External Email. Be cautious of content]

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-unequal-lb/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-bess-evpn-unequal-lb/__;!!NEt6yMaO-gk!RXfyob-BQAdE2D2qzwgUOjGOeP3zn3ANqH6QLuU9WuDatlUvr_IFtXP90p4KAB8$>

Thank you.
_______________________________________________
BESS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/bess<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/bess__;!!NEt6yMaO-gk!RXfyob-BQAdE2D2qzwgUOjGOeP3zn3ANqH6QLuU9WuDatlUvr_IFtXP9oDCvgec$>

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 13:15:58 +0000

Hi John,

Personally, I would prefer that the text indicates the default/standardized usage, such that by default, if all operators follow this, this just works.
Proposing no default and that everyone be free to pick his own unit seem to me a path for domain1/AS1/VPN1 picks unit 1 and domain2/AS2/VPN2 picks unit 2. Then in case of merge, inter-domain/AS/VPN, we may ends up using inconsistent units.

> This ensures that when an ingress PE is doing weighted load balancing, in all cases it is doing simple integer arithmetic on values whose semantics are unknown to it."
I think that we all agree with this.
But this assumes and hence requires that all egress PEs use the same unit. Having a single unit (e.g., bytes/s) is a simple way to ensure this. If one want to state multiple options, stating the easy default and providing a warning for variations seems to increase the probability of consistency in various cases (including the one above e.g. network merges). IOW, to me the unit/semantic is part of the interoperability and hence standard. It's only about encoding/syntax. e.g. my outdoor temperature is 30°. Does this sound hot or cold to you?

Thanks,
--Bruno

From: John E Drake [mailto:[email-masked]]
Sent: Thursday, May 6, 2021 2:54 PM
To: DECRAENE Bruno TGI/OLN <[email-masked]>; Neeraj Malhotra <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Bruno,

I had suggested:

Yours Irrespectively,

John

From: BESS <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

---

From: John E Drake <[email-masked]>
Date: Thu, 6 May 2021 13:37:13 +0000

Bruno,

It's *not* all egress PEs, it's only the multi-homed PEs attached to the same ES that need to be configured consistently.  This is exactly the same requirement that we have for Ethernet Tag in RFC 8484:  https://datatracker.ietf.org/doc/html/rfc8584#section-1.1.

Yours Irrespectively,

John

Juniper Business Use Only
From: [email-masked] <[email-masked]>
Sent: Thursday, May 6, 2021 9:16 AM
To: John E Drake <[email-masked]>
Cc: [email-masked]; [email-masked]; Neeraj Malhotra <[email-masked]>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi John,

Thanks,
--Bruno

To: DECRAENE Bruno TGI/OLN <[email-masked]<mailto:[email-masked]>>; Neeraj Malhotra <[email-masked]<mailto:[email-masked]>>

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 14:06:25 +0000

John,

> It's *not* all egress PEs, it's only the multi-homed PEs attached to the same ES that need to be configured consistently

Agreed.
But step by step consistency becomes nice to have on all PEs: on a given PE, you probably don't want to mix and match different units on a per ES basis. So consistency per PE. Since an ES may be attached to different PEs, it's easier to have consistency across PEs within a domain. Then you have multi domains scenarios, including a new domain been involved long after the original network design.
In the absence of global consistency, at some point when you have to merge PE/domains/VPNs the inconsistency becomes problematic and requires some special cases/handling.
Seems simpler to ensure consistency by default. At least to me.

We are not even discussing reducing the number of options, not to mention to one. We are only discussing to pick one per default so that we get interop by default.
This point may be moved to a deployment consideration section if you believe that this hurts the specification. But I feel that it may have impact on implementations e.g. one cli/yang model/documentation referring to bit/s while the other one referring to bytes/s... and voilà we are likely to have inconsistencies.

Regards,
--Bruno

Sent: Thursday, May 6, 2021 3:37 PM

Bruno,

Yours Irrespectively,

John

Juniper Business Use Only
From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, May 6, 2021 9:16 AM
To: John E Drake <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Neeraj Malhotra <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

---

From: John E Drake <[email-masked]>
Date: Thu, 6 May 2021 14:23:14 +0000

Bruno,

If you like we could say that the EC value field is an unsigned integer which by default represents the attachment circuit bandwidth in units of 1 GBPS.  (As an aside, if six octet unsigned integer arithmetic is difficult, we could say that the value field is a four octet unsigned integer that is right hand justified within the six octet value field.)

If a value other than attachment circuit bandwidth is being represented, it must be configured consistently on all of the PEs attached to a given ES.

Yours Irrespectively,

John

Sent: Thursday, May 6, 2021 10:06 AM

John,

Agreed.

Regards,
--Bruno

Bruno,

It's *not* all egress PEs, it's only the multi-homed PEs attached to the same ES that need to be configured consistently.  This is exactly the same requirement that we have for Ethernet Tag in RFC 8484:  https://datatracker.ietf.org/doc/html/rfc8584#section-1.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8584*section-1.1__;Iw!!NEt6yMaO-gk!QPl16s6_2hw-s8FGIDB05lj2KQi4jSOPQr2JwnEw8gjz_4E8O8fBA8RQcG2udw4$>.

Yours Irrespectively,

John

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 14:45:59 +0000

John,

My comment is limited to specifying the default unit which provides consistency by default. I had proposed some text, but others word along this line works for me.

Your below text goes beyond this and proposes to additionally change the unit and the syntax. That's your proposal, not mine. I leave this to the authors/WG. But I'm a priori not fine with unsigned integer in units of 1 GBPS as it does not seem to work with links below 1GBPS. (I personally don't see an issue with the encoding currently chosen in the draft but if authors/WG want to change, this also works for me)

Regards,
--Bruno

Sent: Thursday, May 6, 2021 4:23 PM

Bruno,

Yours Irrespectively,

John

John,

Agreed.

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

Thank you.

---

From: John E Drake <[email-masked]>
Date: Thu, 6 May 2021 15:09:21 +0000

Bruno,

The concern I have with bytes/sec is that given current and future link speeds, it wastes a lot of bits describing  link speeds that no longer exist.

Yours Irrespectively,

John

Sent: Thursday, May 6, 2021 10:46 AM

John,

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

John,

Agreed.

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 15:58:04 +0000

John,

Those bits are transmitted in the community/BGP in all cases (whether transmitted as 0 or as significant bits) and probably needs to be locally stored in case the BGP routes needs to be re-advertised to another BGP peer.
So we are talking about integer computation...

I was just saying that (access) link below 1GBPS still exists, at least in some region. If you change your proposal to a different offset (e.g. Mbit/s) that would better work for me and probably still address your point.
Again, I leave this encoding discussion between you and the authors.

Regards,
Bruno

Sent: Thursday, May 6, 2021 5:09 PM

Bruno,

Yours Irrespectively,

John

John,

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

John,

Agreed.

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

Thank you.

Thank you.

---

From: John E Drake <[email-masked]>
Date: Thu, 6 May 2021 16:02:20 +0000

Bruno,

I'm sorry, I was just proposing 1 GBS for purposes of discussion.  I think 1 Mbit/s  would be fine.

Yours Irrespectively,

John

Sent: Thursday, May 6, 2021 11:58 AM

John,

Regards,
Bruno

Bruno,

Yours Irrespectively,

John

John,

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

John,

Agreed.

Regards,
--Bruno

Bruno,

Yours Irrespectively,

John

Hi John,

Thanks,
--Bruno

Bruno,

I had suggested:

Yours Irrespectively,

John

Hi Neeraj,

Thanks,
--Bruno

Hi Bruno,

More inline.

Thanks,
Neeraj

[NM]: done.

Thanks
Regards,
--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

Thank you.

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Thu, 6 May 2021 21:19:10 -0700

Hi Bruno, Jorge, et al.,

Jorge,

> Among the co-authors we also discussed the possibility of defining two ECs: one for BW and one for a generalized-weight, so that the remote PE can catch if the multi-homed PEs were indeed using the same meaning of the weight. However, we thought it was easier/simpler to use a generalized value in a single EC sub-type, and add the sentence below. 

Have you considered reserving a byte in the value field to signal a weight type? Similar to how it is done with DF election framework (and allocate a few right away, e.g. 0 for physical link BW, 1 for link count, 2 for manually configured weight, ...). This would help not only with interop cases, but to prevent & diagnose potential configuration mistakes as well. 

For interop reasons, I would support Bruno & argue that it makes sense to use "SHOULD" normative language at least for one way of calculation (e.g. BW).

A couple of minor comments on newly-introduced text in -09 and -10:

1.	Revision -10 introduced a curious change in section 5.2 “Remote PE Behavior”, replacing “1000 Mbps” with “1000 megabytes” in an example. It is not correct in the current form, since right above it there is an explicit statement that links are Gigabit Ethernet: "As an example, for a CE dual-homed to PE-1, PE-2, PE-3 via 2, 1, and 1 GE physical links respectively".

(keeping the Mbps units there would be appropriate, since this is a weight calculation, not a community encoding example)

2.	Section 4.1 “Usage of EVPN Link Bandwidth Extended Community”:

“An implementation may support one or more of the above ways of encoding the value." -> I would assume this should be "MUST support at least one", since every other possible option is included in bullet#2?

--

Sergey

From: BESS <[email-masked]> On Behalf Of [email-masked]
Sent: Thursday, May 6, 2021 5:46 AM
To: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; Neeraj Malhotra <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

 

But I leave this to you.

Thanks,

--Bruno

From: Rabadan, Jorge (Nokia - US/Mountain View) [mailto:[email-masked]] 
Sent: Thursday, May 6, 2021 10:36 AM
To: DECRAENE Bruno TGI/OLN <[email-masked] <mailto:[email-masked]> >; Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Bruno,

Thank you.

Jorge

From: BESS <[email-masked] <mailto:[email-masked]> > on behalf of [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> >
Date: Thursday, May 6, 2021 at 10:04 AM
To: Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> >, [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> >
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

From: Neeraj Malhotra [mailto:[email-masked]] 
Sent: Thursday, May 6, 2021 7:41 AM
To: DECRAENE Bruno TGI/OLN <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Bruno,

More inline.

Thanks,

Neeraj

On Mon, May 3, 2021 at 2:20 AM <[email-masked] <mailto:[email-masked]> > wrote:

Hi Stéphane, authors,

[NM]: done.

 

Thanks

Regards,

--Bruno

From: BESS [mailto:[email-masked] <mailto:[email-masked]> ] On Behalf Of [email-masked] <mailto:[email-masked]> 
Sent: Monday, May 3, 2021 9:21 AM
To: [email-masked] <mailto:[email-masked]> 
Subject: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi WG,

 

 
Thanks,

Stephane

 

 

_______________________________________________
BESS mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/bess

---

From: Neeraj Malhotra <[email-masked]>
Date: Fri, 7 May 2021 09:17:31 -0700

Hi Sergey,

Thanks for the comments. We would like to avoid adding another code point
within the value field, as every new encoding of the "weight" will require
a new code point. Agree that we should specify the default units (for which
the consensus on the thread appears to be Mbps), but would rather leave it
to implementations to support any additional units such as number of links,
configured weight, or anything else, than have a standardized code point
for each unit. Please let me know if this sounds fine. I will fix the units
in section 5.2, and the text in section 4.1 as suggested by you and Bruno.

Thanks,
Neeraj

On Thu, May 6, 2021 at 9:19 PM <[email-masked]> wrote:

> Hi Bruno, Jorge, et al.,
>
>
>
> Jorge,
>
> *> Among the co-authors we also discussed the possibility of defining two
> ECs: one for BW and one for a generalized-weight, so that the remote PE can
> catch if the multi-homed PEs were indeed using the same meaning of the
> weight. However, we thought it was easier/simpler to use a generalized
> value in a single EC sub-type, and add the sentence below. *
>
> Have you considered reserving a byte in the value field to signal a weight
> type? Similar to how it is done with DF election framework (and allocate a
> few right away, e.g. 0 for physical link BW, 1 for link count, 2 for
> manually configured weight, ...). This would help not only with interop
> cases, but to prevent & diagnose potential configuration mistakes as well.
>
>
>
> For interop reasons, I would support Bruno & argue that it makes sense to
> use "SHOULD" normative language at least for one way of calculation (e.g.
> BW).
>
>
>
>    1. Revision -10 introduced a curious change in section 5.2 “Remote PE
>    Behavior”, replacing “1000 Mbps” with “1000 megabytes” in an example. It is
>    not correct in the current form, since right above it there is an explicit
>    statement that links are Gigabit Ethernet: *"As an example, for a CE
>    dual-homed to PE-1, PE-2, PE-3 via 2, 1, and 1 GE physical links
>    respectively".*
>
> (keeping the Mbps units there would be appropriate, since this is a weight
> calculation, not a community encoding example)
>
>    1. Section 4.1 “Usage of EVPN Link Bandwidth Extended Community”:
>
> *“An implementation may support one or more of the above ways of encoding
> the value."* -> I would assume this should be "MUST support at least
> one", since every other possible option is included in bullet#2?
>
>
>
> --
>
> Sergey
>
> *From:* BESS <[email-masked]> *On Behalf Of *
> [email-masked]
> *Sent:* Thursday, May 6, 2021 5:46 AM
> *To:* Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>;
> Neeraj Malhotra <[email-masked]>
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
>
>
> Hi Jorge,
>
>
>
> Thanks for the feedback.
>
>
>
> Regarding the first point, I can live with the current text. But I think I
> would prefer that the text favour one option, and leave it to the
> responsibility of the SP for others usages. E.g.
>
>
>
> OLD:
>
> Note however that the load balancing algorithm defines in this document
> uses ratio of Link Bandwidths hence the operator may choose a different
> unit or use the community as
>
> *From:* Rabadan, Jorge (Nokia - US/Mountain View) [
> mailto:[email-masked] <[email-masked]>]
> *Sent:* Thursday, May 6, 2021 10:36 AM
> *To:* DECRAENE Bruno TGI/OLN <[email-masked]>; Neeraj Malhotra
> <[email-masked]>
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
>
>
> Hi Bruno,
>
>
>
> About the first point, we do have use cases where the bandwidth is not
> what we want to encode in the EC but rather a generalized weight that is
> derived from the link count, logical weight or simply a configured value.
> Among the co-authors we also discussed the possibility of defining two ECs:
> one for BW and one for a generalized-weight, so that the remote PE can
> catch if the multi-homed PEs were indeed using the same meaning of the
> weight. However, we thought it was easier/simpler to use a generalized
> value in a single EC sub-type, and add the sentence below.
>
>
>
> The sentence can be modified/fixed. But the gist is that the multi-homed
> PEs may support multiple meanings for the weight (BW, link-count, etc), but
> at least one of those MUST be common across all PEs and the multi-homed
> routes must use it consistently. Would it be enough if we fix it?
>
>
>
> About existing implementations, a new EVPN sub-type was defined only a
> couple of revisions ago, where, before, the existing non-transitive link BW
> EC was used, so there’s been some churn in the use of the EC anyway. I
> think it is important to get it as soon as possible, but get it right
> rather than finding gaps later once the document is done. But let us know
> your thoughts too.
>
>
>
> Thank you.
>
> Jorge
>
>
>
>
>
> *From: *BESS <[email-masked]> on behalf of
> [email-masked] <[email-masked]>
> *Date: *Thursday, May 6, 2021 at 10:04 AM
> *To: *Neeraj Malhotra <[email-masked]>
> *Cc: [email-masked] <[email-masked]>, [email-masked] <
> [email-masked]>
> *Subject: *Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
> Hi Neeraj,
>
>
>
> §4.1 allows two rather incompatible encodings/usages with no way to detect
> which one is used: some PE could advertise the bandwidth in bytes, while
> some other PE could advertise a general weight. I understand that both
> works, but to me there is a significant risk of issues over time or between
> domain/SP. I’d prefer that you only chose one in order to favour
> consistency in deployments and usage and I would prefer the real bandwidth
> (at least for consistency with the name of the community, but also because
> this is not subjective)  (And if a SP really wants to put an arbitrary
> value, I think he will figure out by himself, that it can do so).
>
> Logic dictates that the second sentence (MUST) can only be fulfilled if
> the first sentence mandates that all implementations MUST support both
> options, or one specifically defined.
>
>
>
> previous version of the draft did not really specify the format of the
> EVPN EC. I had personally assumed that the format was similar to the
> draft-ietf-idr-link-bandwidth link bandwidth community hence encoded in
> IEEE floating point format. Latest version of the draft defines it in
> unsigned integer. Integer looks good to me, but for an existing
> implementation this may be seen as an incompatible change very late in the
> process. Obviously, if there are no implementation, there is no issue. In
> which case, you could also express the bandwidth in unit of bit/s _*if
> you*_ wish to. (I have no preference). However given that the draft had
> indicated a codepoint, there seem to be a risk of existing implementations
> hence incompatible change. BTW the codepoint is squatted even though the
> registry is FCFS hence easy to request.
>
>
>
> Thanks,
>
> --Bruno
>
>
>
>
>
> *From:* Neeraj Malhotra [mailto:[email-masked]
> <[email-masked]>]
> *Sent:* Thursday, May 6, 2021 7:41 AM
> *To:* DECRAENE Bruno TGI/OLN <[email-masked]>
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
>
>
>
>
> Hi Bruno,
>
>
>
> raise one point  regarding the bandwidth community (better safe than sorry).
>

---

From: Sergey Fomin <[email-masked]>
Date: Fri, 07 May 2021 12:33:00 -0700

Hi Neeraj,

thank you for the feedback.

I agree that allocating a new code point for every possible calculation method would be cumbersome.

How about a middle ground solution, with allocation of just a few values (even 2 would suffice: 1 for BW in whatever units we decide; the other "catch-all" generalized weight)?

This way operators will get at least basic visibility & cfg. error prevention mechanism in place (I have a preference towards explicit signaling from ops point of view; but would be glad to hear operators opinion on this one).

My concern with the current text is, while striving for maximum flexibility, we may end up in a situation where 2 implementations won't be able to interoperate with each other (e.g. vendor X decides to implement link BW as a metric, vendor Y decides to implement link count as a metric - since support for any method of weight calculation is completely optional). I believe we, as a technical community, should give guidance on implementing sensible defaults (i.e. "an implementation SHOULD support BW weight calculation method") where it makes sense.

--

Sergey

07.05.2021, 09:18, "Neeraj Malhotra" <[email-masked]>:

Hi Sergey,

Thanks for the comments. We would like to avoid adding another code point within the value field, as every new encoding of the "weight" will require a new code point. Agree that we should specify the default units (for which the consensus on the thread appears to be Mbps), but would rather leave it to implementations to support any additional units such as number of links, configured weight, or anything else, than have a standardized code point for each unit. Please let me know if this sounds fine. I will fix the units in section 5.2, and the text in section 4.1 as suggested by you and Bruno.

Thanks,

Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

Revision -10 introduced a curious change in section 5.2 “Remote PE Behavior”, replacing “1000 Mbps” with “1000 megabytes” in an example. It is not correct in the current form, since right above it there is an explicit statement that links are Gigabit Ethernet: "As an example, for a CE dual-homed to PE-1, PE-2, PE-3 via 2, 1, and 1 GE physical links respectively".

Section 4.1 “Usage of EVPN Link Bandwidth Extended Community”:

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

expressed in bytes/sec.

than link bandwidth.

illustrate its usage.

NEW:

expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.

,

---

From: Neeraj Malhotra <[email-masked]>
Date: Fri, 7 May 2021 16:29:04 -0700

Hi Sergey, Bruno,

Many thanks for the detailed review and comments. Discussed with co-authors
and have enhanced the 'value' encoding to explicitly encode weight units as
default or generalized as suggested below. I have published a revision that
incorporates all comments discussed on this thread, as well as some
additional editorial comments from Jorge. Please let me know if there are
any additional comments (in particular on section 4).

Thanks,
Neeraj

On Fri, May 7, 2021 at 12:33 PM Sergey Fomin <[email-masked]> wrote:

> How about a middle ground solution, with allocation of just a few values
> (even 2 would suffice: 1 for BW in whatever units we decide; the other
> "catch-all" generalized weight)?
> This way operators will get at least basic visibility & cfg. error
> prevention mechanism in place (I have a preference towards explicit
> signaling from ops point of view; but would be glad to hear operators
> opinion on this one).
>
> My concern with the current text is, while striving for maximum
> flexibility, we may end up in a situation where 2 implementations won't be
> able to interoperate with each other (e.g. vendor X decides to implement
> link BW as a metric, vendor Y decides to implement link count as a metric -
> since support for any method of weight calculation is completely
> optional). I believe we, as a technical community, should give guidance on
> implementing sensible defaults (i.e. "an implementation SHOULD support BW
> weight calculation method") where it makes sense.
>
> --
> Sergey
>
>

---

From: <[email-masked]>
Date: Mon, 10 May 2021 12:54:33 +0000

Hi Neeraj, co-authors,

Thanks for taking into account the comments.
-11 addresses my comments.

Two comments on the new text:

“EVPN Link Bandwidth Extended Community value field is to be treated
   as a 6 octet unsigned integer representing total bandwidth of PE's
   all physical links in an ethernet segment, expressed in Mbits/sec
   (MegabitsPerSecond). »

Actually with your latest change introducing the new field “Value-unit”, bandwidth is encoded in only 5 octets (not 6). (I’m assuming that the new field “Value-unit” is not to be used when computing the ratio of bandwidth as this would make no sense).

§5.2 Does not say anything with regards to the new field “Value-unit”. A priori, the ratio of bandwidth is only to be computed on bandwidth/EC using the same unit, no?

Thanks,
Regards,
--Bruno

From: Neeraj Malhotra [mailto:[email-masked]]
Sent: Saturday, May 8, 2021 1:29 AM
To: Sergey Fomin <[email-masked]>
Cc: DECRAENE Bruno TGI/OLN <[email-masked]>; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Sergey, Bruno,

Many thanks for the detailed review and comments. Discussed with co-authors and have enhanced the 'value' encoding to explicitly encode weight units as default or generalized as suggested below. I have published a revision that incorporates all comments discussed on this thread, as well as some additional editorial comments from Jorge. Please let me know if there are any additional comments (in particular on section 4).

Thanks,
Neeraj

On Fri, May 7, 2021 at 12:33 PM Sergey Fomin <[email-masked]<mailto:[email-masked]>> wrote:
Hi Neeraj,
thank you for the feedback.

--
Sergey

07.05.2021, 09:18, "Neeraj Malhotra" <[email-masked]<mailto:[email-masked]>>:

Hi Sergey,

Thanks,
Neeraj

On Thu, May 6, 2021 at 9:19 PM <[email-masked]<mailto:[email-masked]>> wrote:

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

To: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>; Neeraj Malhotra <[email-masked]<mailto:[email-masked]>>

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

---

From: "Fomin, Sergey (Nokia - US/Mountain View)" <[email-masked]>
Date: Tue, 11 May 2021 00:35:57 +0000

Thank you Neeraj, authors,
-11 changes look good to resolve my previous concerns. (I second Bruno’s comment about 5 vs 6 octets)

  *   Stylistically I don’t think we need to expand Mbps to Mbits/sec every time (one expanded definition should be enough), but up to you.
  *   Additionally, I would add either that other “value-unit” encoding values are reserved and should not be used for non-standard implementations, or explicitly allow this usage.

I have a few other comments, mostly minor and nits.

General/minor comments:

  1.  Can we add that only a single instance of LBW community per route is allowed, and how to handle erroneous situations, such as
     *   Receiving a route with multiple attached LBW community instances
     *   Received routes from different PEs having different value-units for the same ES (e.g. PE1 sends routes with 0x00, PE2 w. 0x01)
Last paragraph in section 5.2 already describes similar case with presence/absence of the community, so it could be expanded. Though the same restrictions would apply to a recently-introduced type 5 usecase (and, perhaps to a lesser extent, to a BUM load-sharing scenarios as well) so it’s either worth moving these considerations to section 4 (community definition and usage) or expanding section 9 with the same considerations.

  1.  Across the text there's an implicit assumption that either aliasing is enabled or mac addresses are learned from all PEs (and, of course, ES is in all-active mode), which may not be the case.
I propose to add a remark/reference to 7432, stating that all "regular" qualifiers still apply (i.e. presence of both Eth A-D per ES + per EVI routes for aliasing, or per ES + type2 for non-aliased load-sharing).
This may have some effect on weight computation procedures, i.e. likely that PEs which advertise neither should be excluded from the computation despite the presence of LBW community. Or we could require aliasing to be enabled.

  1.  Section 10. IRB MH with non-EVPN routing: I think there's too much detail describing the usecase (which is not currently applicable to this document), and it feels like an intro to another draft. Is it possible to shorten it to 1 or 2 sentences?

  1.  Section 5.1 Local PE Behavior:   A PE that is part of an Ethernet Segment's redundancy group would advertise an additional "link bandwidth" extended community attribute with Ethernet A-D per-ES route (EVPN Route Type 1), that represents total bandwidth of PE's physical links in an Ethernet Segment.

     *   Shall we replace "would" with MUST/SHOULD?
     *   Also, "represents total bandwidth of physical links" is probably not accurate anymore and should be rephrased?

  1.  Can we explicitly state that the community should not be attached to A-D per EVI & type 2 mac/mac-ip routes (and ignored by remote PE if received on these route types)

Nits:

  *   It seems that ES & ESI terms are used somewhat interchangeably (starting from the very beginning in terminology section, also section 3 "Solution Overview", etc); could you please check and maybe stick to "ES" in cases where we don't specifically talk about identifiers? Also, there's no definition of ESI in the terminology section.
  *   Could we add a definition of “path-list” to the terminology section? (it is used extensively throughout the document in different variations, such as "weighted path-list", "forwarding path-list", "MAC path-list" etc)
  *   Same thing for “access bandwidth”, please add a definition
  *   Section 2 “Introduction” - first bullet point was split into two separate bullets
  *   Section 5.2, last line in the last sentence, typo  - "attribute t be used" should be "to be used"
  *   Inconsistent hyperlinking to RFC/draft references in sections 6.1, 6.3, 13; also inconsistent hyperlinking in the table of contents
  *   Section 6.2 contains a reference to a non-existing example in Section 3 ("Considering the same example as in Section 3, ...), probably should point to 5.2?
  *   Inconsistent spelling of "per ES/per-ES" (as in "Ethernet A-D per ES"). Probably should be whitespaced, not dashed (it is spelled this way in 7432)

--
Sergey

From: BESS <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, May 10, 2021 5:55 AM
To: Neeraj Malhotra <[email-masked]>
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; Sergey Fomin <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

From: Neeraj Malhotra [mailto:[email-masked]]
Sent: Saturday, May 8, 2021 1:29 AM
To: Sergey Fomin <[email-masked]<mailto:[email-masked]>>
Cc: DECRAENE Bruno TGI/OLN <[email-masked]<mailto:[email-masked]>>; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; BESS <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

---

From: Neeraj Malhotra <[email-masked]>
Date: Mon, 10 May 2021 17:36:22 -0700

Hi Bruno,

Thanks for catching this. Fixed both in the latest revision.

Neeraj

On Mon, May 10, 2021 at 5:54 AM <[email-masked]> wrote:

> Actually with your latest change introducing the new field “Value-unit”,
> bandwidth is encoded in only 5 octets (not 6). (I’m assuming that the new
> field “Value-unit” is not to be used when computing the ratio of bandwidth
> as this would make no sense).
>
>
>
> §5.2 Does not say anything with regards to the new field “Value-unit”. A
> priori, the ratio of bandwidth is only to be computed on bandwidth/EC using
> the same unit, no?
>
>
>
> Thanks,
>
> Regards,
>
> --Bruno
>
>
>
> *From**:* Neeraj Malhotra [mailto:[email-masked]]
> *Sent:* Saturday, May 8, 2021 1:29 AM
> *To:* Sergey Fomin <[email-masked]>
> *Cc:* DECRAENE Bruno TGI/OLN <[email-masked]>; Rabadan, Jorge
> (Nokia - US/Mountain View) <[email-masked]>;
> [email-masked]; BESS <[email-masked]>
> Many thanks for the detailed review and comments. Discussed with
> co-authors and have enhanced the 'value' encoding to explicitly encode
> weight units as default or generalized as suggested below. I have
> published a revision that incorporates all comments discussed on this
> thread, as well as some additional editorial comments from Jorge. Please
> let me know if there are any additional comments (in particular on section
> 4).
>
>
>
> Thanks,
>
> Neeraj
>
>
>
> *From:* BESS [mailto:[email-masked]] *On Behalf Of *
> [email-masked]

---

From: Neeraj Malhotra <[email-masked]>
Date: Mon, 10 May 2021 18:59:11 -0700

Hi Sergey,

Thanks for the detailed comments (missed in flight while publishing the
previous revision). I have addressed all of the comments and nits below in
the latest revision (rev13), except that a couple of hyperlinks for a
reason that I can't immediately figure out are not showing up from XML
source. Let me check and fix that in the next revision.

Again, much appreciate your diligence in catching these.

Thanks,
Neeraj

On Mon, May 10, 2021 at 5:36 PM Fomin, Sergey (Nokia - US/Mountain View) <
[email-masked]> wrote:

>    - Stylistically I don’t think we need to expand Mbps to Mbits/sec
>    every time (one expanded definition should be enough), but up to you.
>    - Additionally, I would add either that other “value-unit” encoding
>    values are reserved and should not be used for non-standard
>    implementations, or explicitly allow this usage.
>
>
>
>    1. Can we add that only a single instance of LBW community per route
>    is allowed, and how to handle erroneous situations, such as
>       - Receiving a route with multiple attached LBW community instances
>       - Received routes from different PEs having different value-units
>       for the same ES (e.g. PE1 sends routes with 0x00, PE2 w. 0x01)
>
> Last paragraph in section 5.2 already describes similar case with
> presence/absence of the community, so it could be expanded. Though the same
> restrictions would apply to a recently-introduced type 5 usecase (and,
> perhaps to a lesser extent, to a BUM load-sharing scenarios as well) so
> it’s either worth moving these considerations to section 4 (community
> definition and usage) or expanding section 9 with the same considerations.
>
>
>
>    1. Across the text there's an implicit assumption that either aliasing
>    is enabled or mac addresses are learned from all PEs (and, of course, ES is
>    in all-active mode), which may not be the case.
>
> I propose to add a remark/reference to 7432, stating that all "regular"
> qualifiers still apply (i.e. presence of both Eth A-D per ES + per EVI
> routes for aliasing, or per ES + type2 for non-aliased load-sharing).
>
> This may have some effect on weight computation procedures, i.e. likely
> that PEs which advertise neither should be excluded from the computation
> despite the presence of LBW community. Or we could require aliasing to be
> enabled.
>
>
>
>    1. Section 10. IRB MH with non-EVPN routing: I think there's too much
>    detail describing the usecase (which is not currently applicable to this
>    document), and it feels like an intro to another draft. Is it possible to
>    shorten it to 1 or 2 sentences?
>
>
>
>    1. Section 5.1 Local PE Behavior:   A PE that is part of an Ethernet
>    Segment's redundancy group would advertise an additional "link bandwidth"
>    extended community attribute with Ethernet A-D per-ES route (EVPN Route
>    Type 1), that represents total bandwidth of PE's physical links in an
>    Ethernet Segment.
>
>
>    - Shall we replace "would" with MUST/SHOULD?
>       - Also, "represents total bandwidth of physical links" is probably
>       not accurate anymore and should be rephrased?
>
>
>
>    1. Can we explicitly state that the community should not be attached
>    to A-D per EVI & type 2 mac/mac-ip routes (and ignored by remote PE if
>    received on these route types)
>
>
>
> Nits:
>
>    - It seems that ES & ESI terms are used somewhat interchangeably
>    (starting from the very beginning in terminology section, also section 3
>    "Solution Overview", etc); could you please check and maybe stick to "ES"
>    in cases where we don't specifically talk about identifiers? Also, there's
>    no definition of ESI in the terminology section.
>    - Could we add a definition of “path-list” to the terminology section?
>    (it is used extensively throughout the document in different variations,
>    such as "weighted path-list", "forwarding path-list", "MAC path-list" etc)
>    - Same thing for “access bandwidth”, please add a definition
>    - Section 2 “Introduction” - first bullet point was split into two
>    separate bullets
>    - Section 5.2, last line in the last sentence, typo  - "attribute t be
>    used" should be "to be used"
>    - Inconsistent hyperlinking to RFC/draft references in sections 6.1,
>    6.3, 13; also inconsistent hyperlinking in the table of contents
>    - Section 6.2 contains a reference to a non-existing example in
>    Section 3 ("Considering the same example as in Section 3, ...), probably
>    should point to 5.2?
>    - Inconsistent spelling of "per ES/per-ES" (as in "Ethernet A-D per
>    ES"). Probably should be whitespaced, not dashed (it is spelled this way in
>    7432)
>
>
>
> --
>
> Sergey
>
>
>
> *Sent:* Monday, May 10, 2021 5:55 AM
> *To:* Neeraj Malhotra <[email-masked]>
> *Cc:* Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>;
> Sergey Fomin <[email-masked]>; [email-masked]; BESS <
> [email-masked]>

---

From: <[email-masked]>
Date: Tue, 11 May 2021 06:39:23 +0000

Thanks Neeraj,

Thanks.

Thanks for the text on error handling. Nothing urgent, but nitpicking on one word. §4.1 & 5.2 have 3 iterations of “this attribute is to be ignored”. I’m afraid one could read this as “this BGP attribute is to be ignored”. Surely we mean “this(theses) EVPN Link Bandwidth extended community(ies) is(are) to be ignored”

One feedback that please feel free to use or ignore.

“A receiving PE MUST ensure that each route contains only a single instance of this extended community attribute sub-type.”
Stricto census, what is needed is a single instance of this sub-type and _Value-Units_. In order to accommodate one ingress PE in a domain using “Weight in units of Mbps” and another ingress PE in a different domain using “Generalized Weight”, one could imagine an egress PE advertising 2 communities/both units. I have a slight preference for allowing both, but up to you (In deployments I would personally advocate using only a single unit and the one whose implementation is mandatory, but when there’s an opportunity to go wrong, the opportunity is rarely missed ;-) )

Thanks again,
--Bruno

From: Neeraj Malhotra [mailto:[email-masked]]
Sent: Tuesday, May 11, 2021 3:59 AM
To: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>
Cc: DECRAENE Bruno TGI/OLN <[email-masked]>; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Sergey,

Thanks for the detailed comments (missed in flight while publishing the previous revision). I have addressed all of the comments and nits below in the latest revision (rev13), except that a couple of hyperlinks for a reason that I can't immediately figure out are not showing up from XML source. Let me check and fix that in the next revision.

Thanks,
Neeraj

On Mon, May 10, 2021 at 5:36 PM Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>> wrote:
Thank you Neeraj, authors,
-11 changes look good to resolve my previous concerns. (I second Bruno’s comment about 5 vs 6 octets)

General/minor comments:

Nits:

--
Sergey

Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>; Sergey Fomin <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; BESS <[email-masked]<mailto:[email-masked]>>

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

---

From: "Fomin, Sergey (Nokia - US/Mountain View)" <[email-masked]>
Date: Tue, 11 May 2021 19:48:09 +0000

Hi Neeraj & Bruno,

Bruno,
               > Stricto census, what is needed is a single instance of this sub-type and _Value-Units_. In order to accommodate one ingress PE in a domain using “Weight in units of Mbps” and another ingress PE in a different domain using “Generalized Weight”, one could imagine an egress PE advertising 2 communities/both units.
By "egress PE", I'm assuming, you mean "local PE" in terms of this RFC (with an attached ES)?
I'm not opposed to the idea, thought it adds a layer of implementation complexity.
If you see this as a valid use case (somewhat exotic, for sure), it might be worth relaxing the requirement; in this case, we also should add a remark along the lines that “remote PE” logic how to select between 2 types/sets of LBW communities is left up to the implementer / may be configurable; and support for attaching 2 types of communities at the same time is optional.

Thank you for your prompt response Neeraj,
Following up on my aliasing/non-aliasing point, I see you added the following statement
   o  Procedures related to bridged unicast and BUM traffic are
      applicable to both aliasing and non-alaising mode as defined in
      [RFC7432].
(btw, a typo in ‘aliasing’)
This is good, could we also expand text in section 5.2 “Remote PE behavior” a little bit to include an example for non-aliased case where a MAC has been learned only from a subset of PEs.
E.g. in the current example there, calculation is performed for all 3 PEs and gives weights 2 / 1 /1 for a mac entry X.
If aliasing is not enabled, and a mac entry Y is known only via PE-1 & PE-3; path-list for this mac would be [PE-1, PE-3] with 2 / 1 LB ratio.

Thank you!
--
Sergey

From: [email-masked] <[email-masked]>
Sent: Monday, May 10, 2021 11:39 PM
To: Neeraj Malhotra <[email-masked]>; Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Thanks Neeraj,

Thanks.

Thanks again,
--Bruno

To: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>

Hi Sergey,

Thanks,
Neeraj

General/minor comments:

Nits:

--
Sergey

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

Thank you.

---

From: <[email-masked]>
Date: Tue, 11 May 2021 20:24:13 +0000

Hi Sergey,

Please see inline [Bruno]

From: Fomin, Sergey (Nokia - US/Mountain View) [mailto:[email-masked]]
Sent: Tuesday, May 11, 2021 9:48 PM
To: DECRAENE Bruno TGI/OLN <[email-masked]>; Neeraj Malhotra <[email-masked]>
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj & Bruno,

Bruno,
By "egress PE", I'm assuming, you mean "local PE" in terms of this RFC (with an attached ES)?
[Bruno] Correct. Thanks for translating.

That been said, in the draft §5.2 uses “remote PE” for the ingress PE while §4.1 uses “remote PE” for the egress PE  (“A receiving PE MUST check for consistent 'Value-Units' received in the EVPN link bandwidth exteneded community from each remote PE in an Ethernet Segment. » I’d assume that terminology in §4.1 need to be fixed.

I'm not opposed to the idea, thought it adds a layer of implementation complexity.
If you see this as a valid use case (somewhat exotic, for sure), it might be worth relaxing the requirement; in this case, we also should add a remark along the lines that “remote PE” logic how to select between 2 types/sets of LBW communities is left up to the implementer / may be configurable; and support for attaching 2 types of communities at the same time is optional.
[Bruno] I’m not pushing for it. Just raising the point. Up to you.
May be finding a wording working for both options. e.g.
OLD:
A receiving PE MUST ensure that each route contains only a single
      instance of this extended community attribute sub-type.  In case
      of more than one instance, this attribute is to be ignored.
NEW:
A receiving PE MUST ensure that each route contains only a single

      instance of this extended community attribute sub-type and Value-Units.  In case
      of more than one instance, this attribute is to be ignored.

(BTW, note sure why “Value-Units” is plural, but English is not my first language)

--Bruno

Thank you!
--
Sergey

From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Monday, May 10, 2021 11:39 PM
To: Neeraj Malhotra <[email-masked]<mailto:[email-masked]>>; Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; BESS <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Thanks Neeraj,

Thanks.

Thanks again,
--Bruno

Hi Sergey,

Thanks,
Neeraj

General/minor comments:

Nits:

--
Sergey

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

Thank you.

---

From: "Fomin, Sergey (Nokia - US/Mountain View)" <[email-masked]>
Date: Wed, 12 May 2021 00:07:32 +0000

Hi Neeraj, Jorge & all,

Just wanted to clarify my previous message after a short chat with Jorge: when I say “non-aliasing” case, I mean a scenario where eth. a-d per EVI route is not used or generated [by PEs-1/2/3 in this example].
It is indeed a relatively rare scenario which doesn’t bring much value, so I’m open to drop support for it in this draft and just require eth. a-d per EVI route to be present (=aliasing enabled) to support the procedures described in this document.

Thank you,
--
Sergey
From: BESS <[email-masked]> On Behalf Of Fomin, Sergey (Nokia - US/Mountain View)
Sent: Tuesday, May 11, 2021 12:48 PM
To: [email-masked]; Neeraj Malhotra <[email-masked]>
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; BESS <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj & Bruno,

Bruno,

Thank you!
--
Sergey

Thanks Neeraj,

Thanks.

Thanks again,
--Bruno

Hi Sergey,

Thanks,
Neeraj

General/minor comments:

Nits:

--
Sergey

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

Thank you.

---

From: "Rabadan, Jorge (Nokia - US/Mountain View)" <[email-masked]>
Date: Wed, 12 May 2021 06:20:38 +0000

Thanks Sergey. I agree with that, for Layer 2 aliasing.
For RT5 routes the extended community goes directly with the RT5 as stated in the document.

The IP aliasing draft, that we will refresh soon, will reference to this draft and will allow the extended community to be advertised with A-D per EVI routes in the IP-VRF context.

Thanks.
Jorge

From: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>
Date: Wednesday, May 12, 2021 at 2:07 AM
To: Neeraj Malhotra <[email-masked]>, Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>
Cc: [email-masked] <[email-masked]>, BESS <[email-masked]>, [email-masked] <[email-masked]>
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
Hi Neeraj, Jorge & all,

Thank you,

Hi Neeraj & Bruno,

Bruno,

Thank you!
--
Sergey

Thanks Neeraj,

Thanks.

Thanks again,
--Bruno

Hi Sergey,

Thanks,
Neeraj

On Mon, May 10, 2021 at 5:36 PM Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]<mailto:[email-masked]>> wrote:
Thank you Neeraj, authors,
-11 changes look good to resolve my previous concerns. (I second Bruno’s comment about 5 vs 6 octets)
·         Stylistically I don’t think we need to expand Mbps to Mbits/sec every time (one expanded definition should be enough), but up to you.
·         Additionally, I would add either that other “value-unit” encoding values are reserved and should not be used for non-standard implementations, or explicitly allow this usage.

General/minor comments:
1.      Can we add that only a single instance of LBW community per route is allowed, and how to handle erroneous situations, such as
o    Receiving a route with multiple attached LBW community instances
o    Received routes from different PEs having different value-units for the same ES (e.g. PE1 sends routes with 0x00, PE2 w. 0x01)
Last paragraph in section 5.2 already describes similar case with presence/absence of the community, so it could be expanded. Though the same restrictions would apply to a recently-introduced type 5 usecase (and, perhaps to a lesser extent, to a BUM load-sharing scenarios as well) so it’s either worth moving these considerations to section 4 (community definition and usage) or expanding section 9 with the same considerations.

2.      Across the text there's an implicit assumption that either aliasing is enabled or mac addresses are learned from all PEs (and, of course, ES is in all-active mode), which may not be the case.
I propose to add a remark/reference to 7432, stating that all "regular" qualifiers still apply (i.e. presence of both Eth A-D per ES + per EVI routes for aliasing, or per ES + type2 for non-aliased load-sharing).
This may have some effect on weight computation procedures, i.e. likely that PEs which advertise neither should be excluded from the computation despite the presence of LBW community. Or we could require aliasing to be enabled.

3.      Section 10. IRB MH with non-EVPN routing: I think there's too much detail describing the usecase (which is not currently applicable to this document), and it feels like an intro to another draft. Is it possible to shorten it to 1 or 2 sentences?

4.      Section 5.1 Local PE Behavior:   A PE that is part of an Ethernet Segment's redundancy group would advertise an additional "link bandwidth" extended community attribute with Ethernet A-D per-ES route (EVPN Route Type 1), that represents total bandwidth of PE's physical links in an Ethernet Segment.
o    Shall we replace "would" with MUST/SHOULD?
o    Also, "represents total bandwidth of physical links" is probably not accurate anymore and should be rephrased?

5.      Can we explicitly state that the community should not be attached to A-D per EVI & type 2 mac/mac-ip routes (and ignored by remote PE if received on these route types)

Nits:
·         It seems that ES & ESI terms are used somewhat interchangeably (starting from the very beginning in terminology section, also section 3 "Solution Overview", etc); could you please check and maybe stick to "ES" in cases where we don't specifically talk about identifiers? Also, there's no definition of ESI in the terminology section.
·         Could we add a definition of “path-list” to the terminology section? (it is used extensively throughout the document in different variations, such as "weighted path-list", "forwarding path-list", "MAC path-list" etc)
·         Same thing for “access bandwidth”, please add a definition
·         Section 2 “Introduction” - first bullet point was split into two separate bullets
·         Section 5.2, last line in the last sentence, typo  - "attribute t be used" should be "to be used"
·         Inconsistent hyperlinking to RFC/draft references in sections 6.1, 6.3, 13; also inconsistent hyperlinking in the table of contents
·         Section 6.2 contains a reference to a non-existing example in Section 3 ("Considering the same example as in Section 3, ...), probably should point to 5.2?
·         Inconsistent spelling of "per ES/per-ES" (as in "Ethernet A-D per ES"). Probably should be whitespaced, not dashed (it is spelled this way in 7432)

--
Sergey

Hi Neeraj, co-authors,

Thanks,
Regards,
--Bruno

Hi Sergey, Bruno,

Thanks,
Neeraj

--
Sergey

Hi Sergey,

Thanks,
Neeraj

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

Thanks

Regards,

--Bruno

Hi WG,

Thanks,

Stephane

Thank you.

Thank you.

Thank you.
,

Thank you.

Thank you.

---

From: Neeraj Malhotra <[email-masked]>
Date: Wed, 12 May 2021 14:07:35 -0700

Hi Bruno, Sergey,

Thanks for the additional comments and discussion. To summarize, I plan to
address the following comments by this weekend:

*[Bruno] §4.1 & 5.2 have 3 iterations of “this attribute is to be ignored”.
I’m afraid one could read this as “this BGP attribute is to be ignored”.
Surely we mean “this(theses) EVPN Link Bandwidth extended community(ies)
is(are) to be ignored”.*

[NM]: ack.

*[Sergey] could we also expand text in section 5.2 “Remote PE behavior” a
little bit to include an example for non-aliased case where a MAC has been
learned only from a subset of PEs.*

[NM]: Although not ultra important, I think for completeness it will be
good to allow inclusion of the EC in RT-2 for non-aliasing ECMP use cases.
It is a simple extension similar to RT-5. Will add.

*[Bruno] in the draft §5.2 uses “remote PE” for the ingress PE while §4.1
uses “remote PE” for the egress PE  (“A receiving PE MUST check for
consistent 'Value-Units' received in the EVPN link bandwidth exteneded
community from each remote PE in an Ethernet Segment. » I’d assume that
terminology in §4.1 need to be fixed.*

[NM]: ack.

Please see inline for the following comment:

*[Bruno] Stricto census, what is needed is a single instance of this
sub-type and _Value-Units_. In order to accommodate one ingress PE in a
domain using “Weight in units of Mbps” and another ingress PE in a
different domain using “Generalized Weight”, one could imagine an egress PE
advertising 2 communities/both units. I have a slight preference for
allowing both, but up to you.*

[NM]: Yes, strictly speaking, this is correct. However, for simplicity, I
am inclined to have local/egress PEs that are part of the Ethernet Segment
specify a single / consistent weight unit they are going to use and have
the receiving/ingress PEs simply follow, rather than give ingress PEs a
choice of which units to choose out of multiple received units based on
some specified or locally provisioned precedence (implementation and
provisioning overhead as Sergey also commented).

Thanks,

Neeraj

On Tue, May 11, 2021 at 11:20 PM Rabadan, Jorge (Nokia - US/Mountain View) <
[email-masked]> wrote:

> The IP aliasing draft, that we will refresh soon, will reference to this
> draft and will allow the extended community to be advertised with A-D per
> EVI routes in the IP-VRF context.
>
>
>
> Thanks.
>
> Jorge
>
>
>
> *From: *Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>
> *Date: *Wednesday, May 12, 2021 at 2:07 AM
> *To: *Neeraj Malhotra <[email-masked]>, Rabadan, Jorge (Nokia -
> US/Mountain View) <[email-masked]>
> *Cc: [email-masked] <[email-masked]>, BESS <
> [email-masked]>, [email-masked] <[email-masked]>
> *Subject: *RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
> Hi Neeraj, Jorge & all,
>
>
>
> Just wanted to clarify my previous message after a short chat with Jorge:
> when I say “non-aliasing” case, I mean a scenario where eth. a-d per EVI
> route is not used or generated [by PEs-1/2/3 in this example].
>
> It is indeed a relatively rare scenario which doesn’t bring much value, so
> I’m open to drop support for it in this draft and just require eth. a-d per
> EVI route to be present (=aliasing enabled) to support the procedures
> described in this document.
>
>
>
> Thank you,
>
> --
>
> Sergey
>
> *From:* BESS <[email-masked]> *On Behalf Of *Fomin, Sergey (Nokia
> - US/Mountain View)
> *Sent:* Tuesday, May 11, 2021 12:48 PM
> *To:* [email-masked]; Neeraj Malhotra <[email-masked]>
> *               > Stricto census, what is needed is a single instance of
> this sub-type and _Value-Units_. In order to accommodate one ingress PE in
> a domain using “Weight in units of Mbps” and another ingress PE in a
> different domain using “Generalized Weight”, one could imagine an egress PE
> advertising 2 communities/both units.*
>
> If you see this as a valid use case (somewhat exotic, for sure), it might
> be worth relaxing the requirement; in this case, we also should add a
> remark along the lines that “remote PE” logic how to select between 2
> types/sets of LBW communities is left up to the implementer / may be
> configurable; and support for attaching 2 types of communities at the same
> time is optional.
>
>
>
> *   o  Procedures related to bridged unicast and BUM traffic are*
>
> *      applicable to both aliasing and non-alaising mode as defined in*
>
> *      [RFC7432].*
>
> This is good, could we also expand text in section 5.2 “Remote PE
> behavior” a little bit to include an example for non-aliased case where a
> MAC has been learned only from a subset of PEs.
>
> E.g. in the current example there, calculation is performed for all 3 PEs
> and gives weights 2 / 1 /1 for a mac entry X.
>
> If aliasing is not enabled, and a mac entry Y is known only via PE-1 &
> PE-3; path-list for this mac would be [PE-1, PE-3] with 2 / 1 LB ratio.
>
>
>
> Thank you!
>
> --
>
> Sergey
>
>
>
> *From:* [email-masked] <[email-masked]>
> *Sent:* Monday, May 10, 2021 11:39 PM
> *To:* Neeraj Malhotra <[email-masked]>; Fomin, Sergey (Nokia -
> US/Mountain View) <[email-masked]>
> *Subject:* RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb
>
>
>
> Thanks Neeraj,
>
>
>
> Thanks.
>
> Stricto census, what is needed is a single instance of this sub-type and _
> *Value-Units*_. In order to accommodate one ingress PE in a domain using
> “Weight in units of Mbps” and another ingress PE in a different domain
> using “Generalized Weight”, one could imagine an egress PE advertising 2
> communities/both units. I have a slight preference for allowing both, but
> up to you (In deployments I would personally advocate using only a single
> unit and the one whose implementation is mandatory, but when there’s an
> opportunity to go wrong, the opportunity is rarely missed ;-) )
>
>
>
> Thanks again,
>
> --Bruno
>
>
>
> *Sent:* Tuesday, May 11, 2021 3:59 AM
> *To:* Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>
> ·         Stylistically I don’t think we need to expand Mbps to Mbits/sec
> ·         Additionally, I would add either that other “value-unit”
> encoding values are reserved and should not be used for non-standard
> o    Received routes from different PEs having different value-units for
> the same ES (e.g. PE1 sends routes with 0x00, PE2 w. 0x01)
>
> 2.      Across the text there's an implicit assumption that either
> aliasing is enabled or mac addresses are learned from all PEs (and, of
> course, ES is in all-active mode), which may not be the case.
>
> 3.      Section 10. IRB MH with non-EVPN routing: I think there's too
> much detail describing the usecase (which is not currently applicable to
> this document), and it feels like an intro to another draft. Is it possible
> to shorten it to 1 or 2 sentences?
>
>
>
> 4.      Section 5.1 Local PE Behavior:   A PE that is part of an Ethernet
> o    Also, "represents total bandwidth of physical links" is probably not
> accurate anymore and should be rephrased?
>
>
>
> 5.      Can we explicitly state that the community should not be attached
> ·         It seems that ES & ESI terms are used somewhat interchangeably
> ·         Could we add a definition of “path-list” to the terminology
> section? (it is used extensively throughout the document in different
> variations, such as "weighted path-list", "forwarding path-list", "MAC
> path-list" etc)
>
> ·         Inconsistent hyperlinking to RFC/draft references in sections
> 6.1, 6.3, 13; also inconsistent hyperlinking in the table of contents
>
> ·         Section 6.2 contains a reference to a non-existing example in
> ·         Inconsistent spelling of "per ES/per-ES" (as in "Ethernet A-D
> per ES"). Probably should be whitespaced, not dashed (it is spelled this
> way in 7432)
>
>
>
> --
>
> Sergey
>
>
>
> 1.      Revision -10 introduced a curious change in section 5.2 “Remote
> PE Behavior”, replacing “1000 Mbps” with “1000 megabytes” in an example. It
> is not correct in the current form, since right above it there is an
> explicit statement that links are Gigabit Ethernet: *"As an example, for
> a CE dual-homed to PE-1, PE-2, PE-3 via 2, 1, and 1 GE physical links
> respectively".*
>

---

From: Neeraj Malhotra <[email-masked]>
Date: Fri, 14 May 2021 15:58:18 -0700

Hi,

fyi, have addressed all comments on this thread and some more comments from
co-authors in the latest revision (rev14).

Thanks,
Neeraj

On Wed, May 12, 2021 at 2:07 PM Neeraj Malhotra <[email-masked]>
wrote:

> *[Bruno] §4.1 & 5.2 have 3 iterations of “this attribute is to be
> ignored”. I’m afraid one could read this as “this BGP attribute is to be
> ignored”. Surely we mean “this(theses) EVPN Link Bandwidth extended
> community(ies) is(are) to be ignored”.*
>
>
> [NM]: ack.
>
>
> On Tue, May 11, 2021 at 11:20 PM Rabadan, Jorge (Nokia - US/Mountain View)
> <[email-masked]> wrote:
>
>> It is indeed a relatively rare scenario which doesn’t bring much value,
>> so I’m open to drop support for it in this draft and just require eth. a-d
>> per EVI route to be present (=aliasing enabled) to support the procedures
>> ·         Stylistically I don’t think we need to expand Mbps to
>> Mbits/sec every time (one expanded definition should be enough), but up to
>> you.
>>
>> 1.      Can we add that only a single instance of LBW community per
>> route is allowed, and how to handle erroneous situations, such as
>>
>> 4.      Section 5.1 Local PE Behavior:   A PE that is part of an
>> Ethernet Segment's redundancy group would advertise an additional "link
>> bandwidth" extended community attribute with Ethernet A-D per-ES route
>> (EVPN Route Type 1), that represents total bandwidth of PE's physical links
>> in an Ethernet Segment.
>>
>> o    Also, "represents total bandwidth of physical links" is probably
>> 5.      Can we explicitly state that the community should not be
>> attached to A-D per EVI & type 2 mac/mac-ip routes (and ignored by remote
>> PE if received on these route types)
>>
>>
>>
>> Nits:
>>
>> ·         Section 5.2, last line in the last sentence, typo  -
>> "attribute t be used" should be "to be used"
>>
>> Have you considered reserving a byte in the value field to signal a
>> weight type? Similar to how it is done with DF election framework (and
>> allocate a few right away, e.g. 0 for physical link BW, 1 for link count, 2
>> for manually configured weight, ...). This would help not only with interop
>> (keeping the Mbps units there would be appropriate, since this is a
>> weight calculation, not a community encoding example)
>>
>> Regarding the first point, I can live with the current text. But I think
>> I would prefer that the text favour one option, and leave it to the
>> *To:* DECRAENE Bruno TGI/OLN <[email-masked]>; Neeraj
>> Malhotra <[email-masked]>
>> §4.1 allows two rather incompatible encodings/usages with no way to
>> detect which one is used: some PE could advertise the bandwidth in bytes,
>> while some other PE could advertise a general weight. I understand that
>> both works, but to me there is a significant risk of issues over time or
>> between domain/SP. I’d prefer that you only chose one in order to favour
>> I have not followed the discussions on this document, but I’ll
>> nonetheless raise one point  regarding the bandwidth community (better safe
>> than sorry).
>>
>> - A new EVPN Link Bandwidth extended community is defined, but I could
>> not find its specification. I guess that this is the same format as

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 15 May 2021 00:16:59 -0400

I support publication of this important EVPN enhancement to all active
multi homed PE-CE AC  ESI backup path aliasing ECMP per flow load balancing
to now support weighed and bandwidth or link count based uneven load
balancing in case of MLAG link failures or uneven bandwidth upgrades.

Kind Regards

Gyan

On Fri, May 14, 2021 at 6:58 PM Neeraj Malhotra <[email-masked]>
wrote:

>
> Hi,
>
> fyi, have addressed all comments on this thread and some more comments
> from co-authors in the latest revision (rev14).
>
> Thanks,
> Neeraj
>
>
>> Thanks for the additional comments and discussion. To summarize, I plan
>> to address the following comments by this weekend:
>>
>> On Tue, May 11, 2021 at 11:20 PM Rabadan, Jorge (Nokia - US/Mountain
>> View) <[email-masked]> wrote:
>>
>>> *From: *Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]
>>> >
>>> Just wanted to clarify my previous message after a short chat with
>>> Jorge: when I say “non-aliasing” case, I mean a scenario where eth. a-d per
>>> EVI route is not used or generated [by PEs-1/2/3 in this example].
>>>
>>> *From:* BESS <[email-masked]> *On Behalf Of *Fomin, Sergey
>>> (Nokia - US/Mountain View)
>>> If you see this as a valid use case (somewhat exotic, for sure), it
>>> might be worth relaxing the requirement; in this case, we also should add a
>>> E.g. in the current example there, calculation is performed for all 3
>>> PEs and gives weights 2 / 1 /1 for a mac entry X.
>>>
>>> Stricto census, what is needed is a single instance of this sub-type and
>>> _*Value-Units*_. In order to accommodate one ingress PE in a domain
>>> using “Weight in units of Mbps” and another ingress PE in a different
>>> domain using “Generalized Weight”, one could imagine an egress PE
>>> allowing both, but up to you (In deployments I would personally advocate
>>> using only a single unit and the one whose implementation is mandatory, but
>>> when there’s an opportunity to go wrong, the opportunity is rarely missed
>>> ;-) )
>>>
>>>
>>>
>>> Thanks again,
>>>
>>> --Bruno
>>>
>>>
>>>
>>> *Cc:* DECRAENE Bruno TGI/OLN <[email-masked]>; Rabadan,
>>> Jorge (Nokia - US/Mountain View) <[email-masked]>;
>>> On Mon, May 10, 2021 at 5:36 PM Fomin, Sergey (Nokia - US/Mountain View)
>>> <[email-masked]> wrote:
>>>
>>> o    Received routes from different PEs having different value-units
>>> ·         It seems that ES & ESI terms are used somewhat
>>> interchangeably (starting from the very beginning in terminology section,
>>> also section 3 "Solution Overview", etc); could you please check and maybe
>>> stick to "ES" in cases where we don't specifically talk about identifiers?
>>> Also, there's no definition of ESI in the terminology section.
>>>
>>> *Cc:* DECRAENE Bruno TGI/OLN <[email-masked]>; Rabadan,
>>> Jorge (Nokia - US/Mountain View) <[email-masked]>;
>>> Thanks for the comments. We would like to avoid adding another code
>>> point within the value field, as every new encoding of the "weight" will
>>> require a new code point. Agree that we should specify the default units
>>> (for which the consensus on the thread appears to be Mbps), but would
>>> rather leave it to implementations to support any additional units such as
>>> number of links, configured weight, or anything else, than have a
>>> standardized code point for each unit. Please let me know if this sounds
>>> fine. I will fix the units in section 5.2, and the text in section 4.1 as
>>> suggested by you and Bruno.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Neeraj
>>>
>>>
>>>
>>> *> Among the co-authors we also discussed the possibility of defining
>>> two ECs: one for BW and one for a generalized-weight, so that the remote PE
>>> can catch if the multi-homed PEs were indeed using the same meaning of the
>>> For interop reasons, I would support Bruno & argue that it makes sense
>>> to use "SHOULD" normative language at least for one way of calculation
>>> (e.g. BW).
>>>
>>>
>>>
>>> explicit statement that links are Gigabit Ethernet: *"As an example,
>>> for a CE dual-homed to PE-1, PE-2, PE-3 via 2, 1, and 1 GE physical links
>>> respectively".*
>>>
>>> *“An implementation may support one or more of the above ways of
>>> encoding the value."* -> I would assume this should be "MUST support at
>>> least one", since every other possible option is included in bullet#2?
>>>
>>>
>>>
>>> --
>>>
>>> Sergey
>>>
>>> all physical links in an ethernet segment,
>>>
>>>       expressed in bytes/sec.
>>>
>>>
>>>
>>> - [BGP-LINK-BW] advertises the bandwidth in unit of bytes (not bits!)
>>> per second. Could the unit of the new EVPN Link Bandwidth extended
>>> community be also clearly spelled out? Especially give the history on this
>>> (cf below). Also in order to avoid misleading the readers could the
>>> examples use the correct unit (vs bits per seconds as writen)
>>>
>>>
>>>
>>> [NM]: done.
>>>
>>>
>>>
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: <[email-masked]>
Date: Thu, 20 May 2021 09:08:40 +0200

Ok thanks Neeraj, if there is no more comment from the group, I’m closing the WGLC and will move fwd.

Thanks,

Stephane

From: Neeraj Malhotra <[email-masked]> 
Sent: samedi 15 mai 2021 00:58
To: BESS <[email-masked]>
Cc: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked]>; [email-masked]; [email-masked]; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked]>
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi,

fyi, have addressed all comments on this thread and some more comments from co-authors in the latest revision (rev14).

Thanks,

Neeraj

On Wed, May 12, 2021 at 2:07 PM Neeraj Malhotra <[email-masked] <mailto:[email-masked]> > wrote:

Hi Bruno, Sergey,

Thanks for the additional comments and discussion. To summarize, I plan to address the following comments by this weekend:

[Bruno] §4.1 & 5.2 have 3 iterations of “this attribute is to be ignored”. I’m afraid one could read this as “this BGP attribute is to be ignored”. Surely we mean “this(theses) EVPN Link Bandwidth extended community(ies) is(are) to be ignored”.

[NM]: ack.

[Sergey] could we also expand text in section 5.2 “Remote PE behavior” a little bit to include an example for non-aliased case where a MAC has been learned only from a subset of PEs.

[NM]: Although not ultra important, I think for completeness it will be good to allow inclusion of the EC in RT-2 for non-aliasing ECMP use cases. It is a simple extension similar to RT-5. Will add.

[Bruno] in the draft §5.2 uses “remote PE” for the ingress PE while §4.1 uses “remote PE” for the egress PE  (“A receiving PE MUST check for consistent 'Value-Units' received in the EVPN link bandwidth exteneded community from each remote PE in an Ethernet Segment. » I’d assume that terminology in §4.1 need to be fixed.

[NM]: ack.

[Bruno] Stricto census, what is needed is a single instance of this sub-type and _Value-Units_. In order to accommodate one ingress PE in a domain using “Weight in units of Mbps” and another ingress PE in a different domain using “Generalized Weight”, one could imagine an egress PE advertising 2 communities/both units. I have a slight preference for allowing both, but up to you.

[NM]: Yes, strictly speaking, this is correct. However, for simplicity, I am inclined to have local/egress PEs that are part of the Ethernet Segment specify a single / consistent weight unit they are going to use and have the receiving/ingress PEs simply follow, rather than give ingress PEs a choice of which units to choose out of multiple received units based on some specified or locally provisioned precedence (implementation and provisioning overhead as Sergey also commented).

Thanks,

Neeraj 

On Tue, May 11, 2021 at 11:20 PM Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> > wrote:

Thanks.

Jorge

From: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >
Date: Wednesday, May 12, 2021 at 2:07 AM
To: Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >, Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> >, BESS <[email-masked] <mailto:[email-masked]> >, [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> >
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj, Jorge & all,

Thank you,

--

Sergey

From: BESS <[email-masked] <mailto:[email-masked]> > On Behalf Of Fomin, Sergey (Nokia - US/Mountain View)
Sent: Tuesday, May 11, 2021 12:48 PM
To: [email-masked] <mailto:[email-masked]> ; Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; BESS <[email-masked] <mailto:[email-masked]> >
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj & Bruno,

Bruno, 

      [RFC7432].

Thank you!

--

Sergey

From: [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> > 
Sent: Monday, May 10, 2021 11:39 PM
To: Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >; Fomin, Sergey (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; BESS <[email-masked] <mailto:[email-masked]> >
Subject: RE: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Thanks Neeraj,

Thanks. 

Thanks again,

--Bruno

From: Neeraj Malhotra [mailto:[email-masked]] 
Sent: Tuesday, May 11, 2021 3:59 AM
To: Fomin, Sergey (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >
Cc: DECRAENE Bruno TGI/OLN <[email-masked] <mailto:[email-masked]> >; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; BESS <[email-masked] <mailto:[email-masked]> >
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Sergey,

Thanks,

Neeraj

On Mon, May 10, 2021 at 5:36 PM Fomin, Sergey (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> > wrote:

General/minor comments:

Nits:

--

Sergey

From: BESS <[email-masked] <mailto:[email-masked]> > On Behalf Of [email-masked] <mailto:[email-masked]> 
Sent: Monday, May 10, 2021 5:55 AM
To: Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >
Cc: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; Sergey Fomin <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; BESS <[email-masked] <mailto:[email-masked]> >
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Neeraj, co-authors,

   (MegabitsPerSecond). »

Thanks,

Regards,

--Bruno

From: Neeraj Malhotra [mailto:[email-masked]] 
Sent: Saturday, May 8, 2021 1:29 AM
To: Sergey Fomin <[email-masked] <mailto:[email-masked]> >
Cc: DECRAENE Bruno TGI/OLN <[email-masked] <mailto:[email-masked]> >; Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; BESS <[email-masked] <mailto:[email-masked]> >
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Sergey, Bruno,

Thanks,

Neeraj

On Fri, May 7, 2021 at 12:33 PM Sergey Fomin <[email-masked] <mailto:[email-masked]> > wrote:

Hi Neeraj,

-- 

Sergey

07.05.2021, 09:18, "Neeraj Malhotra" <[email-masked] <mailto:[email-masked]> >:

Hi Sergey,

Thanks,

Neeraj

On Thu, May 6, 2021 at 9:19 PM <[email-masked] <mailto:[email-masked]> > wrote:

Hi Bruno, Jorge, et al.,

Jorge,

--

Sergey

From: BESS <[email-masked] <mailto:[email-masked]> > On Behalf Of [email-masked] <mailto:[email-masked]> 
Sent: Thursday, May 6, 2021 5:46 AM
To: Rabadan, Jorge (Nokia - US/Mountain View) <[email-masked] <mailto:[email-masked]> >; Neeraj Malhotra <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: Re: [bess] New short WGLC for draft-ietf-bess-evpn-unequal-lb

Hi Jorge,

Thanks for the feedback.

OLD:

      expressed in bytes/sec.

      than link bandwidth.

   illustrate its usage.

NEW:

      expressed in bytes/sec.

 

But I leave this to you.

Thanks,

--Bruno

Hi Bruno,

Thank you.

Jorge

Hi Neeraj,

- Regarding deployments

Thanks,

--Bruno

 

Hi Bruno,

More inline.

Thanks,

Neeraj

Hi Stéphane, authors,

[NM]: done.

 

Thanks

Regards,

--Bruno

Hi WG,

 

 
Thanks,

Stephane

 

 

 

 

,