Subject: [bess] NVO3 VXLAN Source port entropy

From: Gyan Mishra <[email-masked]>
Date: Wed, 14 Jul 2021 18:11:17 -0400

Dear BESS Experts

?? On NVO3 VXLAN overlay encapsulation

My understanding of VXLAN source port entropy is to provide uniform load
balancing similar to RFC 6437 IPv6 flow label uniform stateful load
balancing, in NVO3 VXLAN context, using header 5-tuple L2/L3/L4 hash and
generating source port input key to hash function for per packet per flow
 uniform load balancing as achieved with EVPN ECMP pr weighted UCMP MHD
MLAG PE-CE AC.

The problem with L3 ECMP and weighed UCMP is the Day 1 well known TCP
polarization of flows where high bandwidth flows are not evenly distributed
between L3 paths.

So the question is does source port entropy provide per RFC 7348 excerpt
below provide per packet per flow load balancing or flow based where all
packets that are part of the same flow get hashed to the same path.

Outer UDP Header:  This is the outer UDP header with a source port
      provided by the VTEP and the destination port being a well-known
      UDP port.

      -  Destination Port: IANA has assigned the value 4789 for the
         VXLAN UDP port, and this value SHOULD be used by default as the
         destination UDP port.  Some early implementations of VXLAN have
         used other values for the destination port.  To enable
         interoperability with these implementations, the destination
         port SHOULD be configurable.

      -  Source Port:  It is recommended that the UDP source port number
         be calculated using a hash of fields from the inner packet --
         one example being a hash of the inner Ethernet frame's headers.
         This is to enable a level of entropy for the ECMP/load-
         balancing of the VM-to-VM traffic across the VXLAN overlay.
         When calculating the UDP source port number in this manner, it
         is RECOMMENDED that the value be in the dynamic/private port
         range 49152-65535 [RFC6335
<https://datatracker.ietf.org/doc/html/rfc6335>].

Kind Regards

Gyan

-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: Gyan Mishra <[email-masked]>
Date: Wed, 14 Jul 2021 21:20:59 -0400

All

I think I figured out how “source port entropy” works and provides “better”
load balancing then traditional IGP & EGP based ECMP underlay algorithms
that
are subject to polarization.

So normally w/o source port entropy vxlan feature the overlay NVE Anycast
vtep tunnel as a tunnel source and destination so as it’s a single source /
destination IP for the vtep tunnel termination, so that would get pinned to
a single path similar to L2 VPN service label, single source / destination
PE to PE single path.  So just as with L2 VPN you have the FAT PW which now
you read into the payload and extract the src/dest flows and can now load
balance the flows over Ethernet bundles.

Similarly with VXLAN “source port entropy” per RFC 7348  the L2/L3/L4
headers 5-tuple hash is used to generate the outer header udp source port
which is used as the input key to the hashing function.  So one of the
major advantages of VXLAN is now traffic can be much more uniformly evenly
load balanced now with 5-tuple info over L3 ECMP path as compare to
traditional IPv4 flow based per session source / destination hash load
balancing.

The vxlan 5-tuple hash input key to hash function is also very similar
analogous to IPv6 flow label RFC 6437 5-tuple header hash input key to hash
function stateless mode flow label uniform load balancing.

Kind Regards

Gyan
On Wed, Jul 14, 2021 at 6:11 PM Gyan Mishra <[email-masked]> wrote:

>          range 49152-65535 [RFC6335 <https://datatracker.ietf.org/doc/html/rfc6335>].
>
>
>
> Kind Regards
>
> Gyan
>
>
> --
>

*Gyan Mishra*

*M 301 502-1347*

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 14 Jul 2021 20:01:53 -0700

I’m planning to have a presentation in RTGWG that would explain RTO based header changes/ aka self healing fabie   (flow label in v6/ overlay transport layer source port in v4, in linux kernel since 2016), stay tuned.
This is mostly relevant to DC domain and while host based still a very important improvement.

Cheers,
Jeff

> On Jul 14, 2021, at 18:21, Gyan Mishra <[email-masked]> wrote:
> 
> 
> All
> 
> I think I figured out how “source port entropy” works and provides “better” load balancing then traditional IGP & EGP based ECMP underlay algorithms that
> So normally w/o source port entropy vxlan feature the overlay NVE Anycast vtep tunnel as a tunnel source and destination so as it’s a single source / destination IP for the vtep tunnel termination, so that would get pinned to a single path similar to L2 VPN service label, single source / destination PE to PE single path.  So just as with L2 VPN you have the FAT PW which now you read into the payload and extract the src/dest flows and can now load balance the flows over Ethernet bundles.  
> 
> Similarly with VXLAN “source port entropy” per RFC 7348  the L2/L3/L4 headers 5-tuple hash is used to generate the outer header udp source port which is used as the input key to the hashing function.  So one of the major advantages of VXLAN is now traffic can be much more uniformly evenly load balanced now with 5-tuple info over L3 ECMP path as compare to traditional IPv4 flow based per session source / destination hash load balancing.  
> 
> The vxlan 5-tuple hash input key to hash function is also very similar analogous to IPv6 flow label RFC 6437 5-tuple header hash input key to hash function stateless mode flow label uniform load balancing.
> 
> 
> Kind Regards 
> 
> Gyan 
>> My understanding of VXLAN source port entropy is to provide uniform load balancing similar to RFC 6437 IPv6 flow label uniform stateful load balancing, in NVO3 VXLAN context, using header 5-tuple L2/L3/L4 hash and generating source port input key to hash function for per packet per flow  uniform load balancing as achieved with EVPN ECMP pr weighted UCMP MHD MLAG PE-CE AC.  
>> 
>> The problem with L3 ECMP and weighed UCMP is the Day 1 well known TCP polarization of flows where high bandwidth flows are not evenly distributed between L3 paths.
>> 
>> So the question is does source port entropy provide per RFC 7348 excerpt below provide per packet per flow load balancing or flow based where all packets that are part of the same flow get hashed to the same path.
>> 
>>          range 49152-65535 [RFC6335].
>> 
>> 
>> Kind Regards 
>> 
>> Gyan
>> 
>> 
>> -- 
>> 
>> 
>> Gyan Mishra
>> Network Solutions Architect 
>> Email [email-masked]
>> M 301 502-1347
>> 
>> 
> -- 
> 
> 
> Gyan Mishra
> Network Solutions Architect 
> Email [email-masked]
> M 301 502-1347
> 
> 
> _______________________________________________
> BESS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bess

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 15 Jul 2021 00:03:44 -0400

Perfect!  Looking forward to it!

Kind Regards

Gyan

On Wed, Jul 14, 2021 at 11:01 PM Jeff Tantsura <[email-masked]>
wrote:

> I’m planning to have a presentation in RTGWG that would explain RTO based
> header changes/ aka self healing fabie   (flow label in v6/ overlay
> transport layer source port in v4, in linux kernel since 2016), stay tuned.
> I think I figured out how “source port entropy” works and provides
> “better” load balancing then traditional IGP & EGP based ECMP underlay
> algorithms that

*Gyan Mishra*

*M 301 502-1347*