Subject: [Taps] I-D Action: draft-ietf-taps-transports-06.txt

From: [email-masked]
Date: Mon, 06 Jul 2015 15:06:31 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Transport Services Working Group of the IETF.

        Title           : Services provided by IETF transport protocols and congestion control mechanisms
        Authors         : Godred Fairhurst
                          Brian Trammell
                          Mirja Kuehlewind
	Filename        : draft-ietf-taps-transports-06.txt
	Pages           : 39
	Date            : 2015-07-06

Abstract:
   This document describes services provided by existing IETF protocols
   and congestion control mechanisms.  It is designed to help
   application and network stack programmers and to inform the work of
   the IETF TAPS Working Group.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-taps-transports/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-taps-transports-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-taps-transports-06

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 15 Jul 2015 16:28:24 +0200

Hi,

Please find a very few comments to the document. Nothing major. Please use
if you see fit and only then :-)

Section 1: End First Paragraph:

Here it is said that a transport service feature may be minimal latency.
Opposed to the other features listed here,
like e.g. in-order or reliable delivery, then minimal latency seems a more
delicate obtainable feature; How one best obtain it
within the services provided by a certain transport protocol, may depend
quite heavily on the application traffic pattern (unless CPU and network
resources are unlimited).
Also minimal latency is not a feature which any of the protocols described
in the document has within its list of features (I think) or
are you thinking minimal latency and best effort transport service (like
UDP) - Or are you thinking some smart protocol of the future which will
deduce how to best
provide a minimal latency service for a particular application flow over a
specific E-to-E path ?

Section 3.1.2:

Not sure if this way of describing the TCP Protocol Components is the
right way to do.

I think that it would be nice to have the list here use Transport Service
Features names
and then possibly - if relevant as it is done now - describe in more
detail how the TCP Component implement the Transport Service Feature.
I further think that one should strive -  best possibly - to use the same
Transport Service Features for the different protocols when applicable.

Like (some easy examples) for TCP:

o  Single stream-oriented transmission:

      The stream abstraction atop
      the datagram service provided by IP is implemented by dividing the
      stream into segments.

o  Flow control:

      Window-based flow control, with receiver-side window management
      and signaling of available window: Scaling the flow control window
      beyond 64kB requires the use of an optional feature, which has
      performance implications in environments where this option is not
      supported; this can be the case either if the receiver does not
      implement window scaling or if a network node on the path strips
      the window scaling option.

o  Congestion Control:

      Window-based congestion control reacting to loss, delay,
      retransmission timeout, or an explicit congestion signal (ECN):
      Most commonly used is a loss signal from the reliability
      component's retransmission mechanism.  TCP reacts to a congestion
      signal by reducing the size of the congestion window;
      retransmission timeout is generally handled with a larger reaction
      than other signals.

I am not sure if Connection oriented  and Feature negotiation should be
split into two features ?

I am not sure if one wants something called Fast Transmission Start - or
simply Fast Open - as a feature ?

I am not sure what the slogan of the present bullet about Nagle should be.
Also I am not so sure about the content of this section.
Disabling Nagle does not always give the lowest latency and further then I
think that we are missing the good side of Nagle in the present text.
(the bundling of multiple stream segments into packets  and its benefits)

I do not think that it is relevant to describe how SACK helps reduce the
latency of Fast Recovery.
 If so, one could also describe how it improves the congestion control
handling - or ? . Is it at all relevant to consider TCP without SACK
enabled ?
Generally I am not so sure what the latency discussion in this section is
about. It says that "the retransmission timeout defines the upper
bound...". But exp back-off _is_ used in TCP, further the statement is
really only true if RTO-restart is used.
Should one not rather say that RTO-based loss detection reacts based on
timeouts - or why does one say anything at all here ? Is it to try to
describe if the protocol provides some upper bound for
the latency - if so that is more provided via MAXRT ?. The RTO gives an
upper bound for when to retransmit - not as such on the reliable delivery.
Then on top one have the delay in the SND buffer.
Generally the protocol is continuously being evolved to provide low
latency loss recovery operation. Perhaps this is relevant to state ?

General:

I assume that once one has settled a good way of describing the TCP
Components, then one will try to align the other sections.
Or alternatively one go back now and try to settle the list of Abstract
Transport Service Features that existing transport protocol provides - and
then afterwards
updates the various  Transport Protocol Components sections in this doc,
including the TCP one.

Thanks.
BR,  Karen

>-----Original Message-----
>From: Taps [mailto:[email-masked]] On Behalf Of internet-
>[email-masked]
>Sent: Tuesday, July 07, 2015 12:07 AM
>To: [email-masked]
>Cc: [email-masked]
>Subject: [Taps] I-D Action: draft-ietf-taps-transports-06.txt
>
>
>        Title           : Services provided by IETF transport protocols
and congestion
>control mechanisms
>_______________________________________________
>Taps mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/taps

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 15 Jul 2015 16:51:16 +0200

Hi,

Section 3.1.2:

o  Flow control:

o  Congestion Control:

General:

Thanks.
BR,  Karen

and congestion

---

From: Joe Touch <[email-masked]>
Date: Wed, 15 Jul 2015 11:14:51 -0700

> provide a minimal latency service for a particular application flow over a

FWIW, UDP doesn't minimize latency; it (arguably) minimizes the latency
created by the transport layer at the expense of ordered, reliable
delivery and congestion control.

So there are several levels of the latency issue:

	- minimize the latency ADDED by the transport layer
	- reduce the latency seen by the app
	(which can involve more active interaction with the network
	layer, anticipation, etc.)

Note also that latency is not a single metric nor a meaningful
standalone metric:

	1. latency between two parties is a property *of a message*,
	i.e., it requires knowing the message size

	2. latency includes both basic and higher order effects;
	the basic is "delay", the higher order include "jitter" etc.
	Many common delay-sensitive use cases are really mostly
	jitter sensitive.

...
> split into two features ?

If you don't share state (i.e., what we tend to call a "connection"),
what is the meaning of feature negotiation?

...
> think that we are missing the good side of Nagle in the present text.

Nagle reduces the number of small packets at the expense of introducing
transport delay. Turning it off increases the number of small packets
and reduces the transport delay.

Although it's theoretically possible for the larger number of packets to
cause delays larger than those that Nagle aggregation would induce, in
practice the Nagle delays are several orders of magnitude larger.

Using byte-based congestion control, the larger number of packets would
have no effect. Using packet-based congestion control, the larger number
of packets would reduce latency (by opening up the window faster,
keeping it open longer, etc.) unless the increased number of packets
itself is the bottleneck.

So the general assumption that turning off Nagle will reduce latency
should be reasonable. Do you have a counterexample seen in the wild?

Joe

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 16 Jul 2015 09:27:10 +0200

HI Joe,

>> Please find a very few comments to the document. Nothing major. Please
>> use if you see fit and only then :-)
>>
>> Opposed to the other features listed here, like e.g. in-order or
>> reliable delivery, then minimal latency seems a more delicate
>> obtainable feature; How one best obtain it within the services
>> provided by a certain transport protocol, may depend quite heavily on
>> the application traffic pattern (unless CPU and network resources are
>> unlimited).
>> Also minimal latency is not a feature which any of the protocols
>> described in the document has within its list of features (I think) or
>> UDP) - Or are you thinking some smart protocol of the future which
>> will deduce how to best provide a minimal latency service for a
>> particular application flow over a specific E-to-E path ?
>
delivery and
>congestion control.
[Karen ] I agree.

>Note also that latency is not a single metric nor a meaningful standalone
>metric:
>
[Karen ] I agree so what do we mean in this document when we say that
a transport service feature can be to provide minimal latency ?
Are we thinking about Nagle OFF ?

>> I am not sure if Connection oriented  and Feature negotiation should
>> be split into two features ?
>
what is the
>meaning of feature negotiation?
>
[Karen ] Yes, but does connection oriented necessarily means that you can
negotiate features ?

>...
>> Disabling Nagle does not always give the lowest latency and further
>> then I think that we are missing the good side of Nagle in the present
text.
>transport delay. Turning it off increases the number of small packets and
>reduces the transport delay.
>
cause
>delays larger than those that Nagle aggregation would induce, in practice
the
>Nagle delays are several orders of magnitude larger.
>
of
>packets would reduce latency (by opening up the window faster, keeping it
>open longer, etc.) unless the increased number of packets itself is the
>bottleneck.
>
should be
>reasonable.
[Karen ] I agree with this as a general assumption.
But when I read the paragraph in the doc it almost sounds as if turning
Nagle off comes with no price.

 Do you have a counterexample seen in the wild?
[Karen ] Yes we have examples with NO Nagle where the number of packets
then becomes the bottleneck.
In the local ends (CPU) as well as in the infrastructure elements. I am
not sure how much of this information I can share.
I can try to find out. That is why I was saying that how to provide
minimal latency may depend on the application traffic.
How to solve also depends on whether it is the average latency that needs
to be minimal or whether the latency for some particular application
"messages" should be kept
as low as possible.

For SCTP we allow for that Nagle is disabled on some streams (streams with
high scheduling priority) and not on others. This is done exactly for this
purpose.
(There is an IPR on this I have to say).

BR, Karen

---

From: Joe Touch <[email-masked]>
Date: Thu, 16 Jul 2015 10:48:32 -0700

Hi, Karen,

On 7/16/2015 12:27 AM, Karen Elisabeth Egede Nielsen wrote:
...
> a transport service feature can be to provide minimal latency ?

There should probably be two levels:

	- minimize what you add at the transport layer

	- reduce even further

> Are we thinking about Nagle OFF ?

That is how TCP might react to a request to minimizing added latency,
but I don't expect a TAPS API should expose the specific knob of "Nagle".

> [Karen ] Yes, but does connection oriented necessarily means that you can

Absolutely. Connection oriented means there is state that persists
between messages. To be useful, that state needs to be controllable.
Negotiating features *is* controlling that state.

...
> Nagle off comes with no price.

I would think that should be clarified.

> minimal latency may depend on the application traffic.

Because latency depends on the sizes of the messages, it should not be a
surprise that latency can't be minimized without that context.

> "messages" should be kept as low as possible.

This is difficult for TCP to handle, because TCP thinks of the input
stream as an ordered sequence of bytes, and the issue of latency is
related to message boundaries within the TCP stream that are NOT part of
the TCP API (there's no way for an application to indicate those
boundaries to TCP - and no, the SEND boundaries are explicitly NOT those
markers).

> high scheduling priority) and not on others. This is done exactly for this

Sure, but that's also why it doesn't make sense for TCP.

Joe

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Fri, 17 Jul 2015 09:17:36 +0200

Hi Joe,

>From: Joe Touch [mailto:[email-masked]]
>Sent: Thursday, July 16, 2015 7:49 PM
>To: Karen Elisabeth Egede Nielsen; [email-masked]
>Cc: [email-masked]
>Subject: Re: [Taps] I-D Action: draft-ietf-taps-transports-06.txt
>
>Hi, Karen,
>
>>> standalone
>That is how TCP might react to a request to minimizing added latency, but I
>don't expect a TAPS API should expose the specific knob of "Nagle".
>
>> [Karen ] Yes, but does connection oriented necessarily means that you
>> can negotiate features ?
>
>Absolutely. Connection oriented means there is state that persists between
>messages. To be useful, that state needs to be controllable.
[Karen ] Yes. OK.
>> But when I read the paragraph in the doc it almost sounds as if
>> turning Nagle off comes with no price.
>
[Karen ] Good. Perhaps a clarification also can come with enforcing the
bundling aspects of the segmentation paragraph.

>> [Karen ] Yes we have examples with NO Nagle where the number of
>> packets then becomes the bottleneck.
>> In the local ends (CPU) as well as in the infrastructure elements. I
>> am not sure how much of this information I can share.
>> How to solve also depends on whether it is the average latency that
>> needs to be minimal or whether the latency for some particular
>> application "messages" should be kept as low as possible.
>
>This is difficult for TCP to handle, because TCP thinks of the input stream
>as an
>ordered sequence of bytes, and the issue of latency is related to message
>boundaries within the TCP stream that are NOT part of the TCP API (there's
>no
>way for an application to indicate those boundaries to TCP - and no, the
>SEND
>boundaries are explicitly NOT those markers).
>
[Karen ] sure.

>> For SCTP we allow for that Nagle is disabled on some streams (streams
>> with high scheduling priority) and not on others. This is done exactly
>> for this purpose.
>
[Karen ] Yes and then also why Nagle off for TCP may give undesired and
contra-productive results from a latency perspective.
But then TCP applications may not typically have the application traffic
patterns that gives these undesired latency effects of disabling Nagle.

BR, Karen

---

From: Joe Touch <[email-masked]>
Date: Fri, 17 Jul 2015 10:27:55 -0700

Hi, Kasren,

> patterns that gives these undesired latency effects of disabling Nagle.

The one thing we should have learned about TCP is to not make
assumptions about traffic patterns:

- (paraphrasing) after going idle, we need to decide whether to slam
CWND down or keep it open, and we need a "time since last sent"; we
don't have that timer, but because TCP traffic is mostly symmetric,
let's use "time since last received" (VJ88)
	Result: burst the entire CWND on the next HTTP request

- TCP congestion control can assume infinite sources
	Result: good performance for elephants but bad for mice
	or transaction sequences (web)

- TCP should not send large amounts of small packets because they create
congestion, e.g., Telnet (Nagle)
	Result: very bad delays on protocols where multibyte
	messages can straddle packet boundaries, e.g.,
	Telnet with Unicode, Web, etc.

	i.e., Nagle fails exactly for the kinds of uses
	it was intended

Joe