Subject: Re: [Int-area] I-D Action: draft-xu-intarea-ip-in-udp-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Tue, 21 Apr 2015 16:05:10 +1200

Hi,

> IPv6 flow label has been proposed as an entropy field for load
> balancing in IPv6 network environment [RFC6438].  However, as stated
> in [RFC6936], the end-to-end use of flow labels for load balancing is
> a long-term solution and therefore the use of load balancing using
> the transport header fields would continue until any widespread
> deployment is finally achieved.

That is a false argument. RFC6438 describes a model where the
tunnel end point sets the flow label; that is a much smaller fix
to a tunnel end point that converting it to use a new encapsulation
such as IP-in-UDP. Also, the code to generate the "entropy" value
that you propose as a bogus source port number is no simpler than
the code to generate a pseudo-random flow label; in fact it's virtually
the same code, except that it generates a 16-bit port number instead
of a 20-bit flow label.

(The argument is RFC6936 is also false, since it says you need a stateful
solution, but you don't; 6438 makes it clear that you use a stateless
hash.)

> This field contains a 16-bit entropy value that is generated by
> the encapsulator to uniquely identify a flow.  What constitutes
> a flow is locally determined by the encapsulator and therefore
> is outside the scope of this document.  What algorithm is
> actually used by the encapsulator to generate an entropy value
> is outside the scope of this document.

That is ducking the hard part. I recommend the discussions in
RFC 6437 and 6438, where we tried to avoid completely ducking it.
You don't even discuss whether the algorithm is stateful or
stateless. For a stateless hash, I am quite fond of FNV.

> In case the tunnel does not need entropy, this field of all
> packets belonging to a given flow SHOULD be set to a randomly
> selected constant value so as to avoid packet reordering.

That is very confusing. Yes, of course, all packets that belong to
the same microflow must have the same ECMP hash, therefore in
your model they must carry the same bogus source port number. But
what is the difference between "a randomly selected constant value"
and the result of your unspecified "entropy value"? They are
both pseudorandom values that are held constant for a given
microflow.

Bottom line, I don't understand why the world needs this hack
(pseudorandom source port). It is no easier to deploy than the
correct solution (pseudorandom flow label).

   Brian

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 21 Apr 2015 08:19:53 +0000

Hi Brain,

Thanks for your comments and please see my response inline.

> -----Original Message-----
> From: Int-area [mailto:[email-masked]] On Behalf Of Brian E
> Carpenter
> Sent: Tuesday, April 21, 2015 12:05 PM
> To: [email-masked]
> Subject: Re: [Int-area] I-D Action: draft-xu-intarea-ip-in-udp-00.txt
> 
> Hi,
> 
> > a long-term solution and therefore the use of load balancing using the
> > transport header fields would continue until any widespread deployment
> > is finally achieved.
> 
> That is a false argument. RFC6438 describes a model where the tunnel end point
> sets the flow label; that is a much smaller fix to a tunnel end point that
> converting it to use a new encapsulation such as IP-in-UDP. Also, the code to
> generate the "entropy" value that you propose as a bogus source port number is
> no simpler than the code to generate a pseudo-random flow label; in fact it's
> virtually the same code, except that it generates a 16-bit port number instead of
> a 20-bit flow label.

Agree with your argument that it's virtually the same code for generating the entropy value.

> (The argument is RFC6936 is also false, since it says you need a stateful solution,
> but you don't; 6438 makes it clear that you use a stateless
> hash.)

My guess is that the co-authors of RFC6936 had been fully convinced with the logic of RFC6438 (i.e., including transport-layer information as input keys to a hash may be a problem, especially for the IPv6 case and therefore using the {dest addr, source addr, flow label} 3-tuple of the inner packet would be a future-proof option). As such, in the case where the inner packet use IPv6 with the flow label of zero, to provide good   entropy in the flow label field of the outer IPv6 header, the only reasonable way that the co-authors of RFC6936 could think of is to allow the tunnel ingress to create a random flow label value and keep corresponding state so that all packets that were associated with a flow would be consistently given the same flow label. ^_^

> > This field contains a 16-bit entropy value that is generated by the
> > encapsulator to uniquely identify a flow.  What constitutes a flow is
> > locally determined by the encapsulator and therefore is outside the
> > scope of this document.  What algorithm is actually used by the
> > encapsulator to generate an entropy value is outside the scope of this
> > document.
> 
> That is ducking the hard part. I recommend the discussions in RFC 6437 and
> 6438, where we tried to avoid completely ducking it.
> You don't even discuss whether the algorithm is stateful or stateless. For a
> stateless hash, I am quite fond of FNV.

It's stateless. By the way, I believe the implementers would have enough intelligence to prefer a statelss algorithm to a stateful algorithm unless there was extraordinary reasons.

> > In case the tunnel does not need entropy, this field of all packets
> > belonging to a given flow SHOULD be set to a randomly selected
> > constant value so as to avoid packet reordering.
> 
> That is very confusing. Yes, of course, all packets that belong to the same
> microflow must have the same ECMP hash, therefore in your model they must
> carry the same bogus source port number. But what is the difference between
> "a randomly selected constant value"
> and the result of your unspecified "entropy value"? They are both
> pseudorandom values that are held constant for a given microflow.
> 
> Bottom line, I don't understand why the world needs this hack (pseudorandom
> source port). It is no easier to deploy than the correct solution (pseudorandom
> flow label).

Bottom line is flow label is only contained in IPv6 header:) By the way, you may be curious to know why the RTG DP Encapsulation design team has the following assessment in its report (https://tools.ietf.org/html/draft-rtg-dt-encap-01#page-7):

   "It has been suggested that when IPv6 is used it would not be
   necessary to add a UDP header for entropy, since the IPv6 flow label
   can be used for entropy.... While such an approach would save 8 bytes of headers
   when the underlay is IPv6, it does assume that the underlay routers
   use the flow label for ECMP, and it also would make the IPv6 approach
   different than the IPv4 approach.  Currently the leaning is towards
   recommending using the UDP encaps for both IPv4 and IPv6 underlay."

Best regards,
Xiaohu

>    Brian
> 
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Tom Herbert <[email-masked]>
Date: Tue, 21 Apr 2015 08:42:20 -0700

On Mon, Apr 20, 2015 at 9:05 PM, Brian E Carpenter
<[email-masked]> wrote:
> such as IP-in-UDP.

Brian,

Updating end hosts to set flow labels per RFC6438 is easy (e.g. this
is supported in Linux stack now). Upgrading all of our switches in the
network to use flow labels for ECMP and updating all of our NICs to
use flow labels for RSS is *not* easy-- this assumes that would could
even find HW that supports labels and are already using IPv6. Deployed
devices commonly support hashing UDP tuple for ECMP and RSS. This
works reliably for both IPv6 and IPv4, and in fact is a major
motivation for  proposed encapsulation protocols using UDP
(MPLS-in-UDP, GRE-in-UDP, VXLAN, GUE, Geneve, etc.). Intermediate
device support for flow labels is still needed, for instance we only
get 14 bits of entropy in UDP source port (ephemeral port range) which
is too little for some use cases.

Tom

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 22 Apr 2015 08:43:31 +1200

Tom,
On 22/04/2015 03:42, Tom Herbert wrote:
> even find HW that supports labels and are already using IPv6. 

True. But we only make it less likely to happen by proposing a work-around
with bogus port numbers. I don't think that is the IETF showing industry
leadership, exactly. (The same goes for Xiaohu's quote from
https://tools.ietf.org/html/draft-rtg-dt-encap-01#page-7 .) If we cop
out, we can certainly rely on the industry following us.

Rgds
   Brian

> Deployed

---

From: Tom Herbert <[email-masked]>
Date: Wed, 22 Apr 2015 08:29:20 -0700

Brian,

We would have more leverage with the vendors if flow labels were
enabled by default on hosts, but the initial ambiguity in their
specification has complicated that. In the initially proposed patches
for Linux that implemented RFC6438, we had enabled flow labels by
default for all IPv6 packets. There was pushback because of concerns
that stateless flow labels could conflict with those made by flow
label manager which is a stateful mechanism that includes a socket
option to set flow labels on connections. This mechanism predated
RFC6437 and RFC6438, so the part in section 4, RFC6437 that stateful
flow labels can't "disturb" stateless ones was after the fact. The
upshot is that we were not able to turn on flow labels by default and
that fact limits their potential value.

A solution to this might be to split the number space into an RFC6438
range, and stateful labels range. This is a not backwards compatible
with possible uses of stateful flow labels, but if the range RFC6438
is indicated by the high order bit that might be workable based on the
assumption that most stateful flow labels are probably assigned low
values.

Tom

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 23 Apr 2015 08:19:53 +1200

Hi Tom,
On 23/04/2015 03:29, Tom Herbert wrote:
> range, and stateful labels range. 

I personally might agree, but 6man objected to suggestions along those lines
during the process that led to RFC6437.

   Brian

> This is a not backwards compatible