Subject: [nvo3] Extend hash in VXLAN to also cover inner 5-tuple (if present)?

From: Bruno Rijsman <[email-masked]>
Date: Mon, 1 Oct 2012 15:50:16 -0400

A question / suggestion for draft-mahalingam-dutt-dcops-vxlan-02.txt

The draft says:

*It is recommended that the source port be a hash of the inner Ethernet
frame's headers. This to enable a level of entropy for ECMP/load balancing
of the VM to VM traffic across the VXLAN overlay.*

One thing to take into consideration is that some Network Interface Cards
(NICs) support "Receive Side Scaling (RSS)" and "Scalable I/O" which allows
the received traffic to be load balanced over multiple CPUs and/or CPU
cores based on a hash of the header of the received packet.

If the UDP source port contains a hash of only the inner Ethernet frame
header, then all packets destined to a particular VM will have the same
ethernet header, and hence the same UDP source port, and hence the same
5-tuple hash for the outer header, and will hence land on the same CPU (*).

Would it be worth extending the definition of the hash as follows?

*It is recommended that the source port be a hash of the inner Ethernet
frame's headers plus the inner IPv4 or IPv6 source address, destination
address, and protocol (if present) plus the inner UDP or TCP source port
and destination port (if present).  This to enable a level of entropy for
ECMP/load balancing of the VM to VM traffic across the VXLAN overlay and
for load balancing of the traffic across multiple CPUs or CPU cores in the
receiving VM.*

(*) Unless RSS hashing is smart enough to also hash on the header of the
inner header which I don't think it is.