Subject: [aqm] Fwd: packet loss (and therefore ECN) might not happen much actually

From: "Eggert, Lars" <[email-masked]>
Date: Tue, 4 Mar 2014 17:12:53 +0000

Forwarded with permission

Begin forwarded message:

> From: Jon Crowcroft <[email-masked]>
> Subject: packet loss (and therefore ECN) might not happen much actually
> Date: March 4, 2014 at 16:39:22 GMT
> To: "Eggert, Lars" <[email-masked]>
> Cc: <[email-masked]>
> 
> see also this upcoming trace driven study:
> http://www.ee.ucl.ac.uk/~uceejoa/publications/infocom.pdf
> 
> it looks like (in japan at least) TCP is very rarely controlled by packet loss (dupack or timeout) but more by
> sender or receiver rate limiting (or just being too short lived:)
> 
> In missive <[email-masked]>, "Eggert, Lars" typed:
> 
>>> 
>>> Begin forwarded message:
>>> 
>>>> From: Jon Crowcroft <[email-masked]>
>>>> Subject: sorry, dont have chairs emails to hand so...metrics for AQM
>>>> Date: March 4, 2014 at 15:40:12 GMT
>>>> To: "Eggert, Lars" <[email-masked]>
>>>> =20
>>>> there were a stack of metrics for comparing ABR mechanisms back in the =
>>> ATM days - many would translate quite well for AQM (with or without =
>>> ECN:)
>>>> see for example this thesis advised by Raj Jain (he of the Fairness =
>>> index people talk about in context of tcp a lot:)
>>>> http://www.shivkumar.org/research/papers/shivdissertation97.pdf
>>>> =20
>>>> j.
>>> 
>>> 
>>> --Apple-Mail=_06212177-1BE8-4B46-9802-A93C9A97A569
>>> Content-Transfer-Encoding: quoted-printable
>>> Content-Type: text/html;
>>> 	charset=iso-8859-1
>>> 
>>> <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>>> charset=3Diso-8859-1"></head><body style=3D"word-wrap: break-word; =
>>> -webkit-nbsp-mode: space; -webkit-line-break: =
>>> after-white-space;"><br><div><br><div>Begin forwarded message:</div><br =
>>> class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>> margin-left: 0px;"><span style=3D"font-family:'Helvetica'; color:rgba(0, =
>>> 0, 0, 1.0);"><b>From: </b></span><span =
>>> style=3D"font-family:'Helvetica';">Jon Crowcroft &lt;<a =
>>> href=3D"mailto:[email-masked]">[email-masked]</a>&=
>>> gt;<br></span></div><div style=3D"margin-top: 0px; margin-right: 0px; =
>>> margin-bottom: 0px; margin-left: 0px;"><span =
>>> style=3D"font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>Subject: =
>>> </b></span><span style=3D"font-family:'Helvetica';"><b>sorry, dont have =
>>> chairs emails to hand so...metrics for AQM</b><br></span></div><div =
>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>> margin-left: 0px;"><span style=3D"font-family:'Helvetica'; color:rgba(0, =
>>> 0, 0, 1.0);"><b>Date: </b></span><span =
>>> style=3D"font-family:'Helvetica';">March 4, 2014 at 15:40:12 =
>>> GMT<br></span></div><div style=3D"margin-top: 0px; margin-right: 0px; =
>>> margin-bottom: 0px; margin-left: 0px;"><span =
>>> style=3D"font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>To: =
>>> </b></span><span style=3D"font-family:'Helvetica';">"Eggert, Lars" =
>>> &lt;<a =
>>> href=3D"mailto:[email-masked]">[email-masked]</a>&gt;<br></span></div><b=
>>> r><div><meta http-equiv=3D"Content-Type" content=3D"text/html; =
>>> charset=3Diso-8859-1"><div dir=3D"ltr">there were a stack of metrics for =
>>> comparing ABR mechanisms back in the ATM days - many would translate =
>>> quite well for AQM (with or without ECN:)<div>see for example this =
>>> thesis advised by Raj Jain (he of the Fairness index people talk about =
>>> in context of tcp a lot:)</div>
>>> <div><a =
>>> href=3D"http://www.shivkumar.org/research/papers/shivdissertation97.pdf">h=
>>> ttp://www.shivkumar.org/research/papers/shivdissertation97.pdf</a><br></di=
>>> v><div><br></div><div>j.</div></div>
>>> </div></blockquote></div><br></body></html>=
>>> 
>>> --Apple-Mail=_06212177-1BE8-4B46-9802-A93C9A97A569--
>>> 
>>> --Apple-Mail=_E069BE08-CFCB-49D9-ACF2-341B4333EBF4
>>> Content-Transfer-Encoding: 7bit
>>> Content-Disposition: attachment; filename="signature.asc"
>>> Content-Type: application/pgp-signature; name="signature.asc"
>>> Content-Description: Message signed with OpenPGP using GPGMail
>>> 
>>> -----BEGIN PGP SIGNATURE-----
>>> 
>>> iQCVAwUBUxX8PtZcnpRveo1xAQIp+wP+IWIS0CKa8tSTCt/TToAmfmilB7V4CgQo
>>> 2hByWToZr94vcPCpzduZ1AWymGcDh0bDTS4nL7gKxruPzqpoKGC6wLnfFCBqm450
>>> YYZMfJjD8pM9JnkQu46QdcQ03CUK+r4MO6ZCA7VPO22/pCDweTw6TtQWEYqRzaMs
>>> AwyLMUuIjBQ=
>>> =UDgK
>>> -----END PGP SIGNATURE-----
>>> 
>>> --Apple-Mail=_E069BE08-CFCB-49D9-ACF2-341B4333EBF4--
> 
> cheers
> 
>   jon
>

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 21 Mar 2014 21:27:36 +0000

>> sender or receiver rate limiting (or just being too short lived:)

It would be interesting to know their delay variation. You’ve seen my famous “9 second delay” graphic. There was no packet loss at all in that… You have also seen, I believe, my annotated Shepherd Diagram of an upload to Picasa. That was from Akasaka, and had three drops in a five second window, resulting in the session spending 40% of its duration underrunning available capacity.

It would be interesting to know the traffic mix, the line speeds and latencies end to end, and so on. From my perspective, it’s Really Hard to say “the internet acts this way”; consider the problem of the six blind philosophers and the elephant... What I think I *can* say is that I measured something in a certain way in a particular topological place at a particular time and with a particular workload, I analyzed it in a certain way, and in that measurement and analysis I observed ... something.

If you want my guess at what the Japanese trace measured, it had upwards of 50 MBPS end to end and "enough" buffer at that rate in the bottleneck switch to prevent tail-drop loss in the ambient workload. Short sessions, which predominate, would not touch that, and high volume sessions might, as you say, self-limit in one of several ways.

For comparison, yesterday, I took 24 hours of tcpdump trace on my laptop and wrote a reduction script. I started out by capturing 38 hours of traces earlier in the week in one hour chunks, and discovered that tcpdump zero-bases its data structures when it switches output files. Then I took a single 24 hour trace file.

In that reduction, I distinguished between microflows *from* me and microflows *to* me (where “me” might be my IPv4 or my IPv6 address or name), which would be the two halves of a TCP session. I also threw out sessions that didn’t make sense to me, such as ones that might have already been open when I started the trace. Reason? I have asymmetric bandwidth (12 MBPS down and 2 MBPS up, sez the contract, and I think that’s interpreted as “at least”, as I have seen higher), and I expect the two directions to behave a little differently. 

Rates are in kilobits/second, and all numbers are for a session. I have TCP sessions that are as short as a single packet each way (data/RST, for whatever reason I might receive such things, and maybe SYN/SYN-ACK) and pipelined tcp connections lasting the better part of an hour (I opened all of my face:b00c friends’ pages, which moved quite a bit of data, all using IPv6).

my flows: 10548
my retransmissions: 4009
my packets:	min=1	        median=10	        95%=33	        max=73732
my bytes:	min=1	        median=2493	        95%=19608	max=697486314
my durations:	min=0.002751	median=58.096561	95%=120.355108	max=35764.656936
my kbps:	min=0.000074	median=0.577529	        95%=17.171851	max=1049048788.929813

his flows: 14977
his retransmissions: 2859
his packets:	min=1	median=9	95%=104	max=181542
his bytes:	min=1	median=3795	95%=110354	max=221579702
his durations:	min=0.000015	median=46.146412	95%=148.102106	max=35764.620901
his kbps:	min=0.000459	median=0.928163	95%=114.575466	max=22604.513177

There are some weird questions I want to understand about the “max” fields. I edited out sessions that were open when I started the trace, of which there were a few. There are a couple of other strange sessions. One of these days I might sort out the difference between 14977 and 10548. But I think the bottom line is that while the median session in my home office probably doesn’t incur a loss, it looks to me like the ones at the 95th percentile for size probably does - and maybe several.

---

From: Dave Taht <[email-masked]>
Date: Fri, 21 Mar 2014 21:54:28 +0000

I'd be very interest in DNS request/reply analysis of that traffic.

On Fri, Mar 21, 2014 at 9:27 PM, Fred Baker (fred) <[email-masked]> wrote:
> It would be interesting to know their delay variation. You've seen my famous "9 second delay" graphic. There was no packet loss at all in that... You have also seen, I believe, my annotated Shepherd Diagram of an upload to Picasa. That was from Akasaka, and had three drops in a five second window, resulting in the session spending 40% of its duration underrunning available capacity.
>
> It would be interesting to know the traffic mix, the line speeds and latencies end to end, and so on. From my perspective, it's Really Hard to say "the internet acts this way"; consider the problem of the six blind philosophers and the elephant... What I think I *can* say is that I measured something in a certain way in a particular topological place at a particular time and with a particular workload, I analyzed it in a certain way, and in that measurement and analysis I observed ... something.
>
> In that reduction, I distinguished between microflows *from* me and microflows *to* me (where "me" might be my IPv4 or my IPv6 address or name), which would be the two halves of a TCP session. I also threw out sessions that didn't make sense to me, such as ones that might have already been open when I started the trace. Reason? I have asymmetric bandwidth (12 MBPS down and 2 MBPS up, sez the contract, and I think that's interpreted as "at least", as I have seen higher), and I expect the two directions to behave a little differently.
>
> Rates are in kilobits/second, and all numbers are for a session. I have TCP sessions that are as short as a single packet each way (data/RST, for whatever reason I might receive such things, and maybe SYN/SYN-ACK) and pipelined tcp connections lasting the better part of an hour (I opened all of my face:b00c friends' pages, which moved quite a bit of data, all using IPv6).
>
> my flows: 10548
> my retransmissions: 4009
> There are some weird questions I want to understand about the "max" fields. I edited out sessions that were open when I started the trace, of which there were a few. There are a couple of other strange sessions. One of these days I might sort out the difference between 14977 and 10548. But I think the bottom line is that while the median session in my home office probably doesn't incur a loss, it looks to me like the ones at the 95th percentile for size probably does - and maybe several.
>
> _______________________________________________
> aqm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/aqm
>

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

---

From: "Eggert, Lars" <[email-masked]>
Date: Sat, 22 Mar 2014 08:09:27 +0000

>>> sender or receiver rate limiting (or just being too short lived:)

This was actually from a mail I forwarded for Jon. CC'ed him, since you're discussing this.

Lars

---

From: Jon Crowcroft <[email-masked]>
Date: Sat, 22 Mar 2014 14:06:47 +0000

so the context is this large study done by 
Joao Taveira Araujo  as part of his PhD with
Richard G. Clegg, and partly reported in this paper:
http://www.richardclegg.org/previous/pubs/rgc_icc_2013.pdf

In missive <[email-masked]>, "Eggert, Lars" ty
ped:

 >>--Apple-Mail=_23C6E70C-B35C-41A1-AAF0-FE2CAFABB5A5
 >>Content-Type: text/plain;
 >>	charset=windows-1252
 >>
 >>>>> it looks like (in japan at least) TCP is very rarely controlled by =
 >>packet loss (dupack or timeout) but more by
 >>This was actually from a mail I forwarded for Jon. CC'ed him, since =
 >>you're discussing this.
 >>
 >>Lars
 >>
 >>--Apple-Mail=_23C6E70C-B35C-41A1-AAF0-FE2CAFABB5A5
 >>iQCVAwUBUy1FN9ZcnpRveo1xAQLg9wQAl4PicOKz5eXJhU7jxin5UoKiREaZxKEK
 >>B94RSeWjDyCrANm6FHGPTW81t4T79x6ACS7DU/1MkNQgnTpWiSkAD729Q7TlliXO
 >>z0SVpeKfaX7MjbWhCVNOGz3VrJ/BHhVl4+SYTyVhoxV5bRgez0oZU9/vpq+MTk18
 >>KbQH5uQna+o=
 >>=0miE
 >>--Apple-Mail=_23C6E70C-B35C-41A1-AAF0-FE2CAFABB5A5--

 cheers

   jon