Subject: [tcpm] Disabling PAWS when possible

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 20 Jun 2018 23:49:58 -0700

Hello,

I have been thinking about PAWS and TS option for a while and prepared a
simple short draft.
The basic ideas in the draft are like this.

1: There are several technologies (such as tcpinc, mptcp, tls) that can be
used as a replacement of PAWS
    They can even provide stronger protections than PAWS, which might be
able to contribute to recycling connections in TIME_WAIT.

2: Some implementations have records of transmission times on each segment
which won't require TS option for RTTM.

3: When 1 is available, we don't have to put a TS option in every segment.
    Also, when 1 and 2 are available, we don't have to use TS option at all.
    Since we already have base technologies to replace PAWS and TS, all we
need here is a simple signaling mechanism for feature negotiation.

The draft is still very premature and I may overlook something, but it
would be great if I could get some feedback.

Thanks,
--
Yoshi

---------- Forwarded message ----------
From: <[email-masked]>
Date: Wed, Jun 20, 2018 at 9:28 PM
Subject: I-D Action: draft-nishida-tcpm-disabling-paws-00.txt
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

        Title           : Disabling PAWS When Other Protections Are
Available
        Author          : Yoshifumi Nishida
        Filename        : draft-nishida-tcpm-disabling-paws-00.txt
        Pages           : 7
        Date            : 2018-06-20

Abstract:
   PAWS provides protection against old duplicated segments caused by
   wrapped sequence or earlier incarnated connections.  One drawback of
   PAWS is that it requires to place timestamp option in all segments,
   which consumes 10-12 bytes in the option space of TCP.  In addition,
   since PAWS just checks if timestamps is older or not, the protection
   logic is not very strong against malicious attacks or cannot work
   properly in some situations.  On the other hand, some other
   technologies which can provide stronger protections than PAWS are
   becoming available these days.  In this document, we propose to
   utilize other protection mechanisms as replacements of PAWS when they
   are available.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-nishida-tcpm-disabling-paws/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-nishida-tcpm-disabling-paws-00
https://datatracker.ietf.org/doc/html/draft-nishida-tcpm-disabling-paws-00

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Neal Cardwell <[email-masked]>
Date: Fri, 22 Jun 2018 12:56:57 -0400

On Thu, Jun 21, 2018 at 2:50 AM Yoshifumi Nishida
<[email-masked]> wrote:
>
> Hello,
>
> I have been thinking about PAWS and TS option for a while and prepared a simple short draft.
> 1: There are several technologies (such as tcpinc, mptcp, tls) that can be used as a replacement of PAWS
>     They can even provide stronger protections than PAWS, which might be able to contribute to recycling connections in TIME_WAIT.
>
> 2: Some implementations have records of transmission times on each segment which won't require TS option for RTTM.
>
>     Since we already have base technologies to replace PAWS and TS, all we need here is a simple signaling mechanism for feature negotiation.
>
> The draft is still very premature and I may overlook something, but it would be great if I could get some feedback.
>
> Thanks,
> --
> Yoshi

Hi Yoshifumi,

Thanks for this draft! Personally, I am fine with the general
direction of disabling PAWS. But I would argue strongly for keeping
TCP timestamps.

The draft mentions "suppressing timestamp options" here:

  ... The goal of the
   proposal in the draft is to provide stronger protections for old
   duplicated segments while facilitating the use of TCP option space by
   suppressing timestamp options.

I would argue strongly against suppressing/omitting TCP timestamp
options, even if other mechanisms are available that subsume PAWS
protection.

There are a number of significant advantages to TCP timestamps, beyond PAWS:

(1) quickly detecting spurious loss recoveries (using the Eifel
algorithm - https://tools.ietf.org/html/rfc3522 - implemented by
Linux)

(2) architecturally, they allow TCP to have an approximation to the
"packet number" that QUIC has (enabling (1)):
  https://tools.ietf.org/html/draft-ietf-quic-recovery-08#section-2

(3) allowing receivers to estimate RTT (used by Linux receive buffer autotuning)

(4) allowing more precise bandwidth and RTT measurements by congestion
control and passive monitoring systems

And there are probably other nice uses that I'm forgetting. :-)

>From my perspective, one huge advantage from disabling PAWS is that it
enables the timestamps to be an opaque identifier, so that the sender
can use them in any way it chooses. In particular, it would allow the
sender to use microsecond timestamps, without worrying about the
remote side dropping packets due to PAWS checks. This was the main
challenge we found in implementing microsecond TCP timestamps inside
Google datacenters, as we discussed at TCPM a while back (slide 6):

  https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tcp-options-for-low-latency-00.pdf

My main comment would be, if some proposal is going to standardize a
negotiation scheme for using timestamps but disabling PAWS, IMHO there
could be huge value in having that negotiation mechanism optionally
specify the semantics of a sender's timestamp values, so that
congestion control and passive monitoring tools could make use of the
timestamps for bandwidth and latency measurements. This could be
something as simple as 2 bits, e.g.:

 bits : meaning
  00: traditional RFC 7323 timestamps: use PAWS
  01: microsecond timestamps, disable PAWS
  10: millisecond timestamps, disable PAWS
  11: values with other semantics, disable PAWS

Regarding passive measurements using TCP timestamps, Kathleen Nichols
has some great work using TCP timestamps to analyze TCP traffic. For
example:
  https://datatracker.ietf.org/meeting/97/materials/slides-97-maprg-video-at-the-edge-passive-delay-measurements-kathleen-nichols-01.pdf

cheers,
neal

> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>
>         Title           : Disabling PAWS When Other Protections Are Available
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Fernando Gont <[email-masked]>
Date: Fri, 22 Jun 2018 18:26:19 +0300

> able to contribute to recycling connections in TIME_WAIT.

Not sure if I understood correctly, but you may be able to recycle
connections in the TIME-WAIT state if the initial timestamps are
monotonically-increasing.

Cheers,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 22 Jun 2018 16:18:58 -0700

Hi Fernando,

On Fri, Jun 22, 2018 at 8:26 AM, Fernando Gont <[email-masked]>
wrote:

> Yes, you're right as long as initial timestamps are monotonically
increasing.
But, I think this is not true on some implementations (e.g.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=95a22caee396cef0bb2ca8fafdd82966a49367bb
)
I think it would be better if we can have other ways than PAWS to support
this kind of situations.
--
Yoshi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 22 Jun 2018 16:23:17 -0700

Hi Neal,

On Fri, Jun 22, 2018 at 9:56 AM, Neal Cardwell <[email-masked]> wrote:

> > 2: Some implementations have records of transmission times on each
> segment which won't require TS option for RTTM.
> >
> >     Since we already have base technologies to replace PAWS and TS, all
> we need here is a simple signaling mechanism for feature negotiation.
> >

> And there are probably other nice uses that I'm forgetting. :-)

Thanks. These are very useful pointers for me to think about the draft
further.

I think you argument is fair enough.

My intention is to relax the requirement "The TSopt MUST be sent in every
non-<RST> segment for the duration of the connection" in RFC7323
so that TSopt can be sent at arbitrary intervals. Arbitrary intervals can
contain infinite interval, but you can only do so when you really want,
otherwise you shouldn't.
So, disabling TS entirely is just an option.

>   https://www.ietf.org/proceedings/97/slides/slides-
> 97-tcpm-tcp-options-for-low-latency-00.pdf
>

Yes, I am aware of this draft.
One thing I'm curious is that I thought servers in google may have records
of transmission time per packet as it's required by RACK.
However, this draft utilizes TS option with fine granularity.
Do you still need TS option even when you have records of transmission
time, or this feature is not used in the datacenter?

Anyway, both changing timestamp granularity and disabling PAWS change TS
option semantics.
It can make sense to have an unified negotiation scheme for both.
I will think about it more.

Thank you so much!
--
Yoshi

---

From: Lawrence Stewart <[email-masked]>
Date: Mon, 25 Jun 2018 13:08:56 +1000

Greetings Yoshifumi and all,

On 23/06/2018 09:23, Yoshifumi Nishida wrote:
> Hi Neal,
> 
> On Fri, Jun 22, 2018 at 9:56 AM, Neal Cardwell <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     >
>     > Hello,
>     >
>     algorithm - https://tools.ietf.org/html/rfc3522
>     <https://tools.ietf.org/html/rfc3522> - implemented by
>     Linux)
> 
>     <https://tools.ietf.org/html/draft-ietf-quic-recovery-08#section-2>
> 
> My intention is to relax the requirement "The TSopt MUST be sent in
> every non-<RST> segment for the duration of the connection" in RFC7323
> so that TSopt can be sent at arbitrary intervals. Arbitrary intervals
> can contain infinite interval, but you can only do so when you really
> want, otherwise you shouldn't.
>     <https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tcp-options-for-low-latency-00.pdf>
> 
> One thing I'm curious is that I thought servers in google may have
> records of transmission time per packet as it's required by RACK.
On this topic, bits of my PhD thesis may be of tangential interest:

https://researchbank.swinburne.edu.au/file/67be6dea-9503-43d4-bb6b-8d3f28d74272/1/lawrence_stewart_thesis.pdf

Chapter 6 (page 100) in particular explores a simplistic PAWS-compatible
scheme that uses the TSVal as a token so that arbitrary granularity
clocks can be associated with different connections (e.g. to facilitate
path-appropriate RTT measurement and in turn aid incast mitigation in
low-latency IP networks).

Cheers,
Lawrence

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 25 Jun 2018 11:32:48 +0200

Hi Yoshifumi, Neal,

>   >  Google datacenters, as we discussed at TCPM a while back (slide 6):
[...]

 >   >  My main comment would be, if some proposal is going to
 >   >  standardize a negotiation scheme for using timestamps but
 >   >  disabling PAWS, IMHO there could be huge value in having that
 >   >  negotiation mechanism optionally specify the semantics of a
 >   >  sender's timestamp values, so that congestion control and
 >   >  passive monitoring tools could make use of the timestamps
 >   >  for bandwidth and latency measurements. This could be

TSopt is today semi-opaque - a receiver can not rely on timestamps to 
have a specific granularity or monotonicity today.

It seems to me, that two paths should be discussed here: making TSopt 
completely opaque - that is, disallowing the receiver to make any use of 
it, only allowing it to be a signal from the sender in the past (1RTT) 
to the sender in the future.

Or making it more transparent, but disabling PAWS - meaning that the 
content of TSopt can actually be interpreted by a receiver, potentially 
allowing additional capabilities (one-way delay variation measurement, 
"TCP Chirp", springs to mind).

Provided the clock granularity is fine enough, both points (unique 
packet identifier, and allowing fine-grained timing data to be 
extracted) could be done.

Some of these functionalities were touched upon in this draft some time 
ago:
https://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp-negotiation-05

Best regards,
   Richard

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 25 Jun 2018 13:28:39 -0700

> Do you still need TS option even when you have records of transmission time, or this feature is not used in the datacenter?

RACK's packet sent time and TS options are orthogonal. but yes we do
use both. In particular, the TS-opt val in Google DCs is in
microsecond unit, which we negotiated internally using a private
option like what Neal proposed. We do have to relax the PAWs check for
that, so your proposal would make this feature standardize-able for
public Internet :-)

RACK's packet sent time is essential to RACK. But TS-opt is a nice to
have feature to measure RTT on retransmitted packets.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 26 Jun 2018 21:01:53 -0700

Hi Yuchung,

> have feature to measure RTT on retransmitted packets.

Thanks for the useful info. It seems that people still want to keep TS.

Just to be clear, the main point of the draft is to disable PAWS as the
title indicates.
I guess I should focus on this point more explicitly in the next version.
--
Yoshi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 26 Jun 2018 21:05:37 -0700

Hi Richard,

On Mon, Jun 25, 2018 at 2:32 AM, Scheffenegger, Richard <[email-masked]>
wrote:

> TSopt is today semi-opaque - a receiver can not rely on timestamps to have
> a specific granularity or monotonicity today.
>
> it, only allowing it to be a signal from the sender in the past (1RTT) to
> the sender in the future.
>
> allowing additional capabilities (one-way delay variation measurement, "TCP
> Chirp", springs to mind).
>
>
> Provided the clock granularity is fine enough, both points (unique packet
> identifier, and allowing fine-grained timing data to be extracted) could be
> done.
>

Yep, signaling mechanism to change the semantics of TS is an interesting
point for discussions.
We can extend TS option like you proposed in the draft or integrating into
existing negotiation mechanisms or developing a new option.
Developing a new option seems to be a straightforward approach, but it will
consume extra option space and may be discarded by middleboxes.

BTW, the draft above is cited in the draft as a good starting point of
discussion.
--
Yoshi