Subject: Re: [aqm] [AQM Evaluation Guidelines]

From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" <[email-masked]>
Date: Thu, 17 Apr 2014 07:14:06 +0000

All,

I'm not sure if we can reach good test coverage if we apply more and more detailed "realistic" test cases, particularly if we are looking into the future. The more detailed the test cases are, the more relevant others we leave out.

Shouldn't we more concentrate on what we expect from a good AQM?

The only thing we might expect from an AQM is to prevent greedy TCP sources from drawing buffers permanently towards full state, instead of the much better almost empty state, while maintaining full link utilization. In all other situations, I'm convinced, AQM cannot improve a lot. But at least, in these cases, it should not make things worse.

Of course the greedy TCP case can be overlaid by others (unresponsive flows, application paced flows, a renewal process of short lived flows, synchronized start of flows etc.etc.), and we should take this into account. But we should not expect a lot if those "others" dominate the scenario.

Example: An overlay of finite flows arriving as a renewal process with inter arrival time T (seconds) and flow size S (bits). And a link capacity of C. In cases with C < T*S the link collapses with an infinite number of uncompleted flows, each of almost zero rate. In the opposite case, C > T*S the link cannot be loaded permanently at 100% since the traffic offer is simply not there. There is little to gain by AQM. It is only a thin range short before the collapse.

Wolfram

---

From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" <[email-masked]>
Date: Thu, 17 Apr 2014 07:29:33 +0000

Sorry, 
It should be C < S/T and C > S/T, respectively, see inline

Wolfram

> -----Ursprüngliche Nachricht-----
> Von: aqm [mailto:[email-masked]] Im Auftrag von
> LAUTENSCHLAEGER, Wolfram (Wolfram)
> Gesendet: Donnerstag, 17. April 2014 09:14
> An: [email-masked]
> Betreff: Re: [aqm] [AQM Evaluation Guidelines]
> 
> All,
> 
> I'm not sure if we can reach good test coverage if we apply more and more
> detailed "realistic" test cases, particularly if we are looking into the future.
> The more detailed the test cases are, the more relevant others we leave out.
> 
> The only thing we might expect from an AQM is to prevent greedy TCP
> sources from drawing buffers permanently towards full state, instead of the
> much better almost empty state, while maintaining full link utilization. In all
> other situations, I'm convinced, AQM cannot improve a lot. But at least, in
> these cases, it should not make things worse.
> 
> Of course the greedy TCP case can be overlaid by others (unresponsive
> flows, application paced flows, a renewal process of short lived flows,
> synchronized start of flows etc.etc.), and we should take this into account.
> But we should not expect a lot if those "others" dominate the scenario.
> 
> Example: An overlay of finite flows arriving as a renewal process with inter
> arrival time T (seconds) and flow size S (bits). And a link capacity of C. In cases
> with C < T*S the link collapses with an infinite number of uncompleted flows,

Should be C < S/T

> each of almost zero rate. In the opposite case, C > T*S the link cannot be

Should be C > S/T

> loaded permanently at 100% since the traffic offer is simply not there. There
> is little to gain by AQM. It is only a thin range short before the collapse.
> 
> Wolfram
> 
> _______________________________________________
> aqm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/aqm

---

From: John Leslie <[email-masked]>
Date: Thu, 17 Apr 2014 11:11:28 -0400

> Shouldn't we more concentrate on what we expect from a good AQM?

   I'd love to...

> sources from drawing buffers permanently towards full state,

   I'm not the least bit sure that's possible. :^(

   However, it might be possible to separate the queue(s) of "greedy"
TCP flows from other queues. There's a lot of room for discussion about
how to do that...

> instead of the much better almost empty state,

   Consider the class of real-time flows, which hardly ever want to
stand in line for more than a millisecond or two. (They can generate
redundancy to compensate for packet loss, or ideally have ECN-marked
packets get through quickly.)

> while maintaining full link utilization.

   That's a concern of _some_ traffic, but not real-time flows.

> In all other situations, I'm convinced, AQM cannot improve a lot.

   "All other situations" covers a lot of ground... :^(

   And we may not all mean the same thing by "greedy TCP". Standard
TCPs probe for additional capacity if they have anything to send. I
consider that enough to call them "greedy"; but I'm sure others don't
agree.

> But at least, in these cases, it should not make things worse.

   "Make things worse" is perhaps insufficiently defined.

   Any delay at all "makes things worse" for real-time traffic.

   Failure to discover unused capacity "makes things worse" for TCP.

> synchronized start of flows etc.etc.),

   These cover quite different situations...

   Unresponsive flows are a problem iff we buffer too much/many of them.

   Application-paced flows are a problem only if they're probing for
increased capacity.

   Revewal flows are a problem if they guess badly at remaining capacity.

> and we should take this into account. But we should not expect a lot
> if those "others" dominate the scenario.

   I speculate that none of these would be a problem if we limited the
extent which we buffer them.

--
John Leslie <[email-masked]>

---

From: "LAUTENSCHLAEGER, Wolfram (Wolfram)" <[email-masked]>
Date: Tue, 22 Apr 2014 07:39:08 +0000

See comments in line

> TCP flows from other queues. There's a lot of room for discussion about how
> to do that...

You are talking about scheduling, not AQM. I agree with you that
perfect scheduling everywhere could obsolete any AQM. But as long ...

>    Consider the class of real-time flows, which hardly ever want to stand in line
> for more than a millisecond or two. (They can generate redundancy to
> compensate for packet loss, or ideally have ECN-marked packets get through
> quickly.)
> 

That's a concern for _all_ traffic if _some_ greedy traffic is involved, 
no matter if real-time flows are there, too.

>    And we may not all mean the same thing by "greedy TCP". Standard TCPs
> probe for additional capacity if they have anything to send. I consider that
> enough to call them "greedy"; but I'm sure others don't agree.

I mean with "greedy TCP" a flow that is probing for additional capacity
_at_this_ queue. Many TCP flows are bandwidth restricted elsewhere
(up- or downstream), and as such not greedy to my understanding.

>    Failure to discover unused capacity "makes things worse" for TCP.

That's what I meant.

> > flows, application paced flows, a renewal process of short lived
> > flows, synchronized start of flows etc.etc.),
> 

Unresponsive flows are a problem if they absorb the AQM's 
drop/mark signals, so that the responsive flows miss these and continue 
to grow their sending rates.

> 
>    Application-paced flows are a problem only if they're probing for increased
> capacity.
> 

You missed my point: A renewal process of TCP flows is not a problem for AQM.
But it is a problem for AQM _evaluation_. The 100% capacity utilization cannot
be reached, no matter how good or bad the AQM is. The capacity utilization is
solely defined by the traffic definition and not by the AQM. Nevertheless we
have such traffic in reality. If we want to evaluate this, we first should be
clear what we expect.

>    I speculate that none of these would be a problem if we limited the extent
> which we buffer them.
> 
> --

Wolfram

---

From: David Collier-Brown <[email-masked]>
Date: Tue, 22 Apr 2014 08:36:27 -0400

Just a clarification: a greedy TCP will try to send more packets than
will fit, and with a bad queuing algorithm will fill buffers, while with
a good one will cause drops.

The buffer-filling is a secondary effect, and can be avoided.  The harm
to other streams caused as a direct effect is the problem, and may
indeed be hard to deal with, or distinguish from a flow enthusiastically
probing to see if it can go faster through the bottleneck link.

The fix for the latter is drops, of course. The fix for a TCP who
ignores drops is ... maybe *lots* of drops? (:-))

--dave
[I'm not being /totally/ serious, but improving the existing bottleneck
signalling is worth discussing. Someone with knowledge of a bad TCP,
please comnent]

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
[email-masked]           |                      -- Mark Twain

---

From: John Leslie <[email-masked]>
Date: Tue, 22 Apr 2014 14:47:36 -0400

David Collier-Brown <[email-masked]> wrote:
> will fit,

   Clarification is good!

   However, I'd like to work out a sufficiently precise definition of
"greedy TCP" here.

   My personal preference would be to call a TCP "greedy" when it is
probing for additional capacity, and only then.

   (I'm quite willing to work with a different definition; but I think
we need _one_ definition when we talk of "greedy TCP".

> and with a bad queuing algorithm will fill buffers, while with a
> good one will cause drops.

   Please forgive me if you can: I like to think in terms of whether
packets are arriving faster than they can be forwarded, or slower --
not in terms of what's in the buffer(s).

> The buffer-filling is a secondary effect, and can be avoided.

   I'd say it SHOULD be avoided, except in limited cases.

   When TCP is _not_ probing for additional capacity, buffering helps
TCP.

   When TCP _is_ probing for additional capacity, buffering increases
the depth of the sawtooth.

   Buffering _always_ tends to increase delay for real-time flows.

> The harm to other streams caused as a direct effect is the problem,
> and may indeed be hard to deal with,

   Maintaining multiple buffers can help _a_lot_ here.

> or distinguish from a flow enthusiastically probing to see if it can
> go faster through the > bottleneck link.

   Yes, distinguishing the probing is hard. :^(

   I suggest we think in terms of assuming TCP is probing, absent any
actual indicator that it isn't.

> The fix for the latter is drops, of course.

   Dropping packets is a _poor_ fix -- though often it's the best fix
available. :^(

   In an "ideal" world, we'd be able to identify the next packet of
that flow to go out and ECN-mark it as Congestion Experienced. That's
hard, of course, but it would be a better fix if everything worked
that way.

> The fix for a TCP who ignores drops is ... maybe *lots* of drops? (:-))

   Excellent question!

   But first we should ask, can we tell that's happening?

   If we could, I'd suggest turning off all buffering for that flow.
That would probably drop *lots* of packets when we turn it off. And
it seems quite fair to deny the benefits of buffering to any TCP flow
that doesn't cut back its sending rate.

---

From: David Collier-Brown <[email-masked]>
Date: Tue, 22 Apr 2014 15:17:11 -0400

Strongly agree!!!

I suspect it might be useful to think of the probing as a "greedy
algorithm" that is stopped by drops, so that a nutso-evil  scheme gets
stopped by drops the same way.  I just don't know much about what I've
now defined as nutso-evil TCP (:-))

--dave
[This is *not* suggested as a new term of art]