Subject: [tcpm] usage for timestamp options in the wild

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 26 Sep 2018 10:49:03 -0700

Hello,
this is just out of my curiosity.
I've checked traffic archives in CAIDA (https://data.caida.org/datasets/)
and WIDE (http://mawi.wide.ad.jp/mawi/) on a whim and tried to see how many
connections utilize timestamps.

As far as I've checked some archives recently captured, it seems that
around 60-70% TCP connections use timestamp option. But, this ratio seems
to be a bit lower as I thought most of implementations these days support
TS option and activate it by default.
Does anyone have some ideas about it? Am I looking at uncommon data? Or
there are still many old implementations around? Or, many users have
disabled the option for some reasons?
--
Yoshi

---

From: Praveen Balasubramanian <[email-masked]>
Date: Wed, 26 Sep 2018 18:21:15 +0000

Windows doesn’t do timestamp option by default. It’s 10 byte per packet overhead for marginal benefits.

From: tcpm <[email-masked]> On Behalf Of Yoshifumi Nishida
Sent: Wednesday, September 26, 2018 10:49 AM
To: [email-masked]
Subject: [tcpm] usage for timestamp options in the wild

Hello,
this is just out of my curiosity.
I've checked traffic archives in CAIDA (https://data.caida.org/datasets/<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.caida.org%2Fdatasets%2F&data=02%7C01%7Cpravb%40microsoft.com%7Ccba356afa7e34779babf08d623d86481%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735809644749695&sdata=wUz0HExJ2m3RePYxM0abwYwPhl2UeANaV8NAg%2ByKk9c%3D&reserved=0>) and WIDE (http://mawi.wide..ad.jp/mawi/<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmawi.wide.ad.jp%2Fmawi%2F&data=02%7C01%7Cpravb%40microsoft.com%7Ccba356afa7e34779babf08d623d86481%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735809644759707&sdata=XgSjCziCC%2BuoJAynXixBQObdqmAk%2FlBxZ%2FTR2CMF3Gk%3D&reserved=0>) on a whim and tried to see how many connections utilize timestamps.

As far as I've checked some archives recently captured, it seems that around 60-70% TCP connections use timestamp option. But, this ratio seems to be a bit lower as I thought most of implementations these days support TS option and activate it by default.
Does anyone have some ideas about it? Am I looking at uncommon data? Or there are still many old implementations around? Or, many users have disabled the option for some reasons?
--
Yoshi

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 26 Sep 2018 11:37:09 -0700

On Wed, Sep 26, 2018 at 11:21 AM, Praveen Balasubramanian
<pravb=[email-masked]> wrote:
> I've checked traffic archives in CAIDA (https://data.caida.org/datasets/) and WIDE (http://mawi.wide..ad.jp/mawi/) on a whim and tried to see how many connections utilize timestamps.
>
>
>
how recent are the archives?

AFAIK TS is enabled by default on Linux, FreeBSD, and iOS. Linux has
that by default more than a decade

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 26 Sep 2018 12:43:55 -0700

On Wed, Sep 26, 2018 at 11:37 AM, Yuchung Cheng <[email-masked]> wrote:

> and WIDE (http://mawi.wide..ad.jp/mawi/) on a whim and tried to see how
> many connections utilize timestamps.
> >
> >
> >

They are 2018 data.

> that by default more than a decade

that's why I was wondering.
--
Yoshi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 26 Sep 2018 12:48:07 -0700

Hi Praveen,

> overhead for marginal benefits.

Interesting. So, windows derives RTO only from ACK arrival time?
--
Yoshi

>
>
> *From:* tcpm <[email-masked]> *On Behalf Of * Yoshifumi Nishida
> *Sent:* Wednesday, September 26, 2018 10:49 AM
> *To:* [email-masked]
> *Subject:* [tcpm] usage for timestamp options in the wild
>
>
>
> Hello,
>
> I've checked traffic archives in CAIDA (https://data.caida.org/datasets/
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.caida.org%2Fdatasets%2F&data=02%7C01%7Cpravb%40microsoft.com%7Ccba356afa7e34779babf08d623d86481%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735809644749695&sdata=wUz0HExJ2m3RePYxM0abwYwPhl2UeANaV8NAg%2ByKk9c%3D&reserved=0>)
> and WIDE (http://mawi.wide..ad.jp/mawi/
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmawi.wide.ad.jp%2Fmawi%2F&data=02%7C01%7Cpravb%40microsoft.com%7Ccba356afa7e34779babf08d623d86481%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735809644759707&sdata=XgSjCziCC%2BuoJAynXixBQObdqmAk%2FlBxZ%2FTR2CMF3Gk%3D&reserved=0>)
> on a whim and tried to see how many connections utilize timestamps.
>
>
>

---

From: Praveen Balasubramanian <[email-masked]>
Date: Wed, 26 Sep 2018 20:36:58 +0000

It follows RFC 6298 and RTO is based on SRtt and RttVar. RTT sample measurement will make use of timestamps when present or just use ACK arrival time. As a side result of RACK implementation, the number of RTT samples taken per RTT was increased. See Section 2 and 3 in RFC 6298.

From: Yoshifumi Nishida <[email-masked]>
Sent: Wednesday, September 26, 2018 12:48 PM
To: Praveen Balasubramanian <[email-masked]>
Cc: Yoshifumi Nishida <[email-masked]>; [email-masked]
Subject: Re: [tcpm] usage for timestamp options in the wild

Hi Praveen,

On Wed, Sep 26, 2018 at 11:21 AM, Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>> wrote:
Windows doesn’t do timestamp option by default. It’s 10 byte per packet overhead for marginal benefits.

From: tcpm <[email-masked]<mailto:[email-masked]>> On Behalf Of Yoshifumi Nishida
Sent: Wednesday, September 26, 2018 10:49 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [tcpm] usage for timestamp options in the wild

Hello,
this is just out of my curiosity.
I've checked traffic archives in CAIDA (https://data.caida.org/datasets/<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.caida.org%2Fdatasets%2F&data=02%7C01%7Cpravb%40microsoft.com%7C27cca6cac8fd49d98f6908d623ee27fb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735903152884299&sdata=lMPj3nuM%2FiNU8HBirxJQ6L%2Bg5N5ABT3McVOpVq5ZOW0%3D&reserved=0>) and WIDE (http://mawi.wide..ad.jp/mawi/<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmawi.wide.ad.jp%2Fmawi%2F&data=02%7C01%7Cpravb%40microsoft.com%7C27cca6cac8fd49d98f6908d623ee27fb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735903152894308&sdata=JE%2Ba6m7ElaS6bU4ITuoLb48ack%2FNfQTjOYkHJp7Ax6A%3D&reserved=0>) on a whim and tried to see how many connections utilize timestamps.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 26 Sep 2018 20:43:47 -0700

Hi Praveen,
Sorry, I was not clear enough... I was interested in how windows samples
RTT.

I mean, I am interesting in knowing if the conventional once per RTT
samplings can archive sufficient results
or recording the transmission time of sending packets like RACK can
compensate the lack of TS options.
--
Yoshi

On Wed, Sep 26, 2018 at 1:36 PM, Praveen Balasubramanian <
[email-masked]> wrote:

> It follows RFC 6298 and RTO is based on SRtt and RttVar. RTT sample
> measurement will make use of timestamps when present or just use ACK
> arrival time. As a side result of RACK implementation, the number of RTT
> samples taken per RTT was increased. See Section 2 and 3 in RFC 6298.
>
>
>
> *From:* Yoshifumi Nishida <[email-masked]>
> *Sent:* Wednesday, September 26, 2018 12:48 PM
> *To:* Praveen Balasubramanian <[email-masked]>
> *Cc:* Yoshifumi Nishida <[email-masked]>; [email-masked]
> *Subject:* Re: [tcpm] usage for timestamp options in the wild
>
>
>
> Hi Praveen,
>
>
>
> *From:* tcpm <[email-masked]> *On Behalf Of *Yoshifumi Nishida
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.caida.org%2Fdatasets%2F&data=02%7C01%7Cpravb%40microsoft.com%7C27cca6cac8fd49d98f6908d623ee27fb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735903152884299&sdata=lMPj3nuM%2FiNU8HBirxJQ6L%2Bg5N5ABT3McVOpVq5ZOW0%3D&reserved=0>)
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmawi.wide.ad.jp%2Fmawi%2F&data=02%7C01%7Cpravb%40microsoft.com%7C27cca6cac8fd49d98f6908d623ee27fb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636735903152894308&sdata=JE%2Ba6m7ElaS6bU4ITuoLb48ack%2FNfQTjOYkHJp7Ax6A%3D&reserved=0>)

---

From: Praveen Balasubramanian <[email-masked]>
Date: Thu, 27 Sep 2018 16:35:20 +0000

For just computing RTO, once per RTT is sufficient. Based on lab measurements, any delay based algorithms like HyStart or delay based congestion control all work better if more RTT samples are taken throughout the RTT. RACK requires send timestamps and with that infrastructure in place it is easier to compute more RTT samples per RTT.

It's certainly a worthwhile A/B/C experiment to compare accuracy of RTT using timestamps, one sample per RTT, and multiple samples (N) spread across the RTT. Windows has a constant N today to minimize memory and CPU usage. If anyone has any data on this, please share.

Sent: Wednesday, September 26, 2018 8:44 PM

Hi Praveen,
Sorry, I was not clear enough... I was interested in how windows samples RTT.

I mean, I am interesting in knowing if the conventional once per RTT samplings can archive sufficient results
or recording the transmission time of sending packets like RACK can compensate the lack of TS options.
--
Yoshi

On Wed, Sep 26, 2018 at 1:36 PM, Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>> wrote:
It follows RFC 6298 and RTO is based on SRtt and RttVar. RTT sample measurement will make use of timestamps when present or just use ACK arrival time. As a side result of RACK implementation, the number of RTT samples taken per RTT was increased. See Section 2 and 3 in RFC 6298.

From: Yoshifumi Nishida <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, September 26, 2018 12:48 PM
To: Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>>
Cc: Yoshifumi Nishida <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [tcpm] usage for timestamp options in the wild

Hi Praveen,

Hello,
this is just out of my curiosity.
I've checked traffic archives in CAIDA (https://data.caida.org/datasets/<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.caida.org%2Fdatasets%2F&data=02%7C01%7Cpravb%40microsoft.com%7Cecf887b1e89041da30e008d624703d0d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636736461848296751&sdata=OffGwYFhj%2FwqTbsT7P7%2BaYi88WtT5ne1MI6k79unxTQ%3D&reserved=0>) and WIDE (http://mawi.wide..ad.jp/mawi/<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmawi.wide.ad.jp%2Fmawi%2F&data=02%7C01%7Cpravb%40microsoft.com%7Cecf887b1e89041da30e008d624703d0d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636736461848296751&sdata=Q1M0qVe7u%2B%2F40spn2%2B746Ir7WUGllyLiIr2tBtNFwhU%3D&reserved=0>) on a whim and tried to see how many connections utilize timestamps.

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 27 Sep 2018 16:55:42 -0700

>> how recent are the archives?
There's some number dated in 2011

"2. GOOGLE MEASUREMENTS We sampled TCP statistics on unmodified Google
Web servers world-wide for one week in May 2011. The servers use a
recent version of Linux 2.6 with settings listed in Table 4. The
servers do not include the changes proposed in this paper. The global
statistics for interactive Web services are summarized in Table 1.
Among the billions of connections sampled, 96% of the connections
negotiated SACK but only 12% of the connections negotiated Timestamps.
The majority of clients are Microsoft Windows which by default do not
use TCP Timestamps."
https://storage.googleapis.com/pub-tools-public-publication-data/pdf/37486.pdf

While this is comparing apple (google) to oranges (caida), I would say
70% is already a significant jump.

However I am sensing that the value of TCP timestamp options is
under-valued - Linux for example uses TS for many things:

1. RTT estimation on retransmission - this is crucial when a TCP is in
constant recovery mode such as encountering a traffic policer. Note
that the standard ACK time approach no longer works if any sequence
acked has been transmitted, thus RTO may stale during extended
recovery period.

2. Undo operations (i.e. TCP Eifel RFC4015) requires TCP timestamps -
being able to detect false reactions to losses is particularly key for
"loss-based" congestion control. Such false positives are common in
wireless or even wired networks due to all sorts of dark queues and L2
optimizations. While DSACK and F-RTO can also be used to detect false
recoveries - both of them take at least one more round-trip and/or
require new application data, while timestamp can detect false
retransmission instantly on the original ACK

3. Receiver buffer auto-tuning uses TCP ACKs' TS value to echo to
measure RTT from the receiver to tune the buffer size.

4. Potentially TCP timestamps being generated with known units, can be
used to gauge the data arrival rate. The measurement can be valuable
for congestion control b/c it's not subject to hectic ACK delays in
modern networks. I know folks at NetFlix is doing some deep research
into this aspect.

QUIC has emphasized the importance of unique packet identifier - TCP
TS provides a close equivalent and is super useful, especially w/ more
precision / unit vs an opaque value.