Subject: Re: [Gen-art] Gen-ART review of draft-ietf-opsawg-large-flow-load-balancing-11

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 6 May 2014 19:16:59 -0700

Hi Martin,

Thanks for the review and comments.  Please see inline.

We need some clarification on the security portion.

Anoop

On Thu, Apr 24, 2014 at 2:51 PM, Martin Thomson <[email-masked]>wrote:

> I am the assigned Gen-ART reviewer for this draft. For background on
> Gen-ART, please see the FAQ at
>
> <http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.
>
> Please resolve these comments along with any other Last Call comments
> you may receive.
>
> Document: draft-ietf-opsawg-large-flow-load-balancing-11
> Reviewer: Martin Thomson
> Review Date: 2014-04-24
> IETF LC End Date: 2014-05-06
> IESG Telechat date: (if known)
>
> Summary: Basically ready.  Looks like a pretty straightforward, even
> commonsense, coverage of the ways that flow distribution might be
> achieved and a need for it identified.
>
> Major issues: None
>
> Minor issues: Or questions, really.
>
> There's not a lot of discussion about the costs of maintaining an
> exception list for rebalanced flows.  A hash-based distribution is
> going to cost essentially zero state because the outbound path can be
> determined on a per-packet basis, but as soon as you start
> redistributing, there is an added state cost (and potential increase
> in lookup times).  That probably needs some discussion.
>
>
We could probably add another subsection to Section 6 which discusses
operational considerations.  Would the following address this concern?

6.3 Forwarding Resources

Hash-based techniques used for load balancing with LAG/ECMP
are usually stateless.  The mechanisms described in this document
require additional resources in the forwarding plane of routers for creating
PBR rules that are capable of overriding the forwarding decision from
the hash-based approach.  These resources may limit the number of
flows that can be rebalanced and may also impact the latency experienced
by packets due to the additional lookups that are required.

> This plays into the security considerations, which I think need to
> highlight this as a potential DoS vector.  Implementing automatic
> redistribution on top of a hash-based stateless distribution is
> vulnerable to attack if the hash function used is predictable.
>
>
Can you say something more about this attack?  Is it that an attacker
would send large flows that map to the same link in order to consume
resources in the PBR table?

> In Section 5.1 the recommended formula for determining imbalance puts
> the average utilization on the divisor, which leads to large
> variations in output when the overall utilization is low.  In that
> state, it's probably best to avoid redistribution entirely, since no
> single link is likely to be close to capacity.  I'd recommend a
> simpler formula of max_i(|U_i - U_ave|).  (Note: You are missing an
> ellipsis in the calculation of U_ave in the divisor part.)
>
> Thanks for catching the missing ellipsis.

Also, thanks for the suggestion.  It makes sense to remove u_ave
from the denominator since it's already a normalized number.

> Nits/editorial comments:
>
> You don't rely on the definition of COTS, which I think is good.  It
> can probably go.  There may be others.  I don't tend to check these
> things.
>

Will remove it.  I believe that's the only one that is not used.

>
> Some diagrams have some alignment issues, see Figure 2.
>

Yes, we have had issues with this when converting from Word and
keep forgetting to adjust the .txt before submitting.

>
> I've never encountered a Fat-Tree before.  So the use of the name
> actually hindered comprehension.  It's harmless, but probably
> unnecessary.
>

We'll change it to clos...fat tree is a term that is commonly used
by data center folks.

>
> Section 5.3 essentially includes a copy of Section 4.3.1.
>

Thanks for catching that.  We'll modify it to remove the redundancy
and simply refer to 4.3.1.

>
> The formulae in Section 5.6.1 are incomprehensible.  I assume that
> there is a missing solidus '/' character on each.
>

Yes, it is missing that character.  Thanks for catching it.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 7 May 2014 13:25:05 -0700

Inline...

> by packets due to the additional lookups that are required.

Yep, that's perfect.

> resources in the PBR table?

That's exactly the concern.  Similar concerns arise in hash tables
that are predictable.  If an attacker wants to mount a DoS attack, one
vector they have open to them is to overload a single bucket, forcing
lookup times to increase from O(1) to O(N).  There's some research on
the matter, http://cr.yp.to/papers/surf.pdf for example.

> keep forgetting to adjust the .txt before submitting.

It's probably too late for this, but you might find this useful in
future: https://github.com/cabo/kramdown-rfc2629 (see it in action
here: https://github.com/tlswg/tls13-spec)