Subject: [aqm] analysis paper on PIE...

From: "Rong Pan (ropan)" <[email-masked]>
Date: Tue, 11 Nov 2014 07:15:55 +0000

Please see our analysis paper on PIE…

Thanks,

Rong

---

From: Martin Stiemerling <[email-masked]>
Date: Wed, 12 Nov 2014 13:46:49 -1000

[writing not as AD, but as random IETF participant]

Sorry for this blunt question:

Is there any other analysis made by an independent source, i.e., where 
not the PIE authors are running an analysis of PIE?

Thanks,

   Martin

Am 10.11.14 um 21:15 schrieb Rong Pan (ropan):
> _______________________________________________
> aqm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/aqm
>

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 13 Nov 2014 00:03:44 +0000

Hi Martin,

I believe these papers may qualify that requirement:

http://ipv6.cablelabs.com/wp-content/uploads/2014/06/DOCSIS-AQM_May2014.pdf
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6925768
https://www.duo.uio.no/handle/10852/37381

tl;dr - both pie and codel camps did some independent implementations and testing of the respective other algorithm, with discussions and denting out some poorly described aspects in that process. It's my understanding that this lead to a better quality of the drafts in both instances.

Richard Scheffenegger

> -----Original Message-----
> From: aqm [mailto:[email-masked]] On Behalf Of Martin Stiemerling
> Sent: Mittwoch, 12. November 2014 13:47
> To: [email-masked]
> Subject: Re: [aqm] analysis paper on PIE...
> 
> Is there any other analysis made by an independent source, i.e., where not
> the PIE authors are running an analysis of PIE?
> 
> Thanks,
> 
>    Martin
> 
> > Please see our analysis paper on PIE...
> >
> > Thanks,
> >
> > Rong
> >
> >
> >

---

From: Martin Stiemerling <[email-masked]>
Date: Wed, 12 Nov 2014 14:05:28 -1000

> tl;dr - both pie and codel camps did some independent implementations
> and testing of the respective other algorithm, with discussions and
> denting out some poorly described aspects in that process. It's my
> understanding that this lead to a better quality of the drafts in
> both instances.

Cool & thanks!

   Martin

>
>
>
> Richard Scheffenegger
>
>
>
>
>> -----Original Message----- From: aqm [mailto:[email-masked]]
>> On Behalf Of Martin Stiemerling Sent: Mittwoch, 12. November 2014
>> 13:47 To: [email-masked] Subject: Re: [aqm] analysis paper on PIE...
>>
>> Is there any other analysis made by an independent source, i.e.,
>> where not the PIE authors are running an analysis of PIE?
>>
>> Thanks,
>>
>> Martin
>>
>>> _______________________________________________ aqm mailing list
>>> [email-masked] https://www.ietf.org/mailman/listinfo/aqm
>>>
>>
>> _______________________________________________ aqm mailing list
>> [email-masked] https://www.ietf.org/mailman/listinfo/aqm

---

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Wed, 12 Nov 2014 14:19:20 -1000

> I believe these papers may qualify that requirement:

I'll add to that the results I presented yesterday in ICCRG:

http://www.ietf.org/proceedings/91/slides/slides-91-iccrg-4.pdf

However, I don't believe there has been any *theoretical* analysis of
either algorithm apart from the one in the original PIE paper...

-Toke

---

From: Dave Taht <[email-masked]>
Date: Wed, 12 Nov 2014 20:40:38 -0800

> http://ipv6.cablelabs.com/wp-content/uploads/2014/06/DOCSIS-AQM_May2014.pdf

This documents the docsis-pie implementation which has rather a few
basic improvements on pie by itself, notably bytemode, some predictive
stuff, and drop-semi-derandomization.

It also uses overlarge and not-recomended-by-the-inventors constants
for codel and sfq_codel, and lumps together all results at all
bandwidths, where, as we've shown, current aqm implementations perform
differently at different bandwidths and RTTs.

The earlier paper covered the bandwidth scenarios more broadly and in
depth, with less twiddling of the constants.

I also had a very long post on this list going into the problems with
the testing done here. which I'll search for unless someone beats me
to it.

Some of the tools used in this evaluation landed in ns2 earlier this
year, and I would certainly like these tests reproduced independently,
with sane values for codel and fq_codel, and preferably against a
simulator I trust more, like ns3.

> http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6925768

Yep. Toke points to codel's falloff at higher rates in his paper also
- this is mostly due to a problem in the control law introduced in the
linux version that isn't in the ns2 version, and is nearly invisible
in the fq_codel version. I do fear a similar problem is in PIE when
dealing with TSO packets but have not tested.

And the load transients are a problem with any straight aqm system.

> https://www.duo.uio.no/handle/10852/37381

I have picked apart this paper elsewhere also. I would have liked it,
if in particular, fq_codel had been used throughout the tests in
comparison, particularly with ecn....

To add to the comparisons:

http://caia.swin.edu.au/reports/140630A/CAIA-TR-140630A.pdf

While quite good, this was kind of limited to steady state
performance, and at rates below 10mbit.

I was delighted to see someone actually use cerowrt for it's intended
purpose, evaluating
each new algorithm on real hardware in their bachelor's thesis, but I
can't find that url right now.

> tl;dr - both pie and codel camps did some independent implementations and testing of the respective other algorithm,

I tracked codel, fq_codel, pie, improvements to red (ared), sfq,
sfqred, and SFB closely for the past 4 years. All of these are in
linux 3.14 or later (with pie entering last). The ability to do basic
testing of everything on the table is a download away, on nearly every
linux distro, and cerowrt has 'em all - and openwrt several...

I spent GSOC2014 getting ns3 up to speed. For some reason I don't seem
to have any time to write papers myself...

>with discussions and denting out some poorly described aspects in that process. It's my understanding that this lead to a better quality of the drafts in both instances.

Ya know, I'm not part of the codel or pie "camp". I'm pretty firmly in
the low knobs "fq + an aqm with ecn support" camp.There aren't a lot
of people in a codel-only camp. The algorithm is a toolkit, much like
pie is a toolkit for docsis-pie, and setting up the debate as codel vs
pie feels like an exercise in dialectical dualism for sake of
excluding the third alternative.

Certainly codel the algorithm, and codel the stand-alone aqm can be
improved (and I have patches for that available for a long time now)
but it has taken a long time to have an adaquate suite of test tools
to be able to analyze the often microscopic differences between
versions of the base aqm algorithms, be the pie or codel or red
derived. (for example I was unaware until I got a preview of toke's
paper of the degree of decline in effectiveness in codel alone at
100Mbit, being focused primarily on finding improvements at 5mbit or
below (the speed most of the edge of the internet runs at - and the
hardware I use to test peaking out at about 60mbits before htb rolls
over and dies on a cpu designed in 1989. So instead of fiddling with
codel I've been fiddling with a better rate shaper embedded into
fq_codel)

I do certainly hope that work can move forward on the evaluation
guidelines based on a wide variety of scenarios. In my own case, my
biases are towards managing slow start better (vs steady state TCPs),
capturing all the latency sources (e.g DNS, tcp syns), and enabling
voip, gaming, web, and videoconferencing traffic better to the expense
of full single flow goodput, at rates well below 100mbit, and
typically at baseline physical rtts in the 4-50ms range.

I utterly agree that more testing is needed. However the only open
source test suite (netperf-wrapper) only implements a few of the tests
in the pie and docsis-pie papers, making reproduction difficult, and
the earlier

-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks