Subject: [Taps] Eric Rescorla's No Objection on draft-ietf-taps-minset-08: (with COMMENT)

From: Eric Rescorla <[email-masked]>
Date: Wed, 12 Sep 2018 22:35:01 -0700

Eric Rescorla has entered the following ballot position for
draft-ietf-taps-minset-08: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-taps-minset/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Rich version of this review at:
https://mozphab-ietf.devsvcdev.mozaws.net/D4808

IMPORTANT
S 3.1.
>        - Is any of the following useful to the application?
>          *  Specify checksum coverage used by the sender
>          *  Specify minimum checksum coverage required by receiver
>   
>          Yes: UDP-Lite is preferred.
>          No: UDP is preferred.

there seems to be something missing here wrt penetration rates. I.e.,
SCTP  (absent UDP) does not reliably work for any client behind NATs,
which means that it's not preferred for those clients regardless of
the other benefits in this tree.

COMMENTS
S 1.
>      of libraries to use this transport feature without exposing it, based
>      on knowledge about the applications -- but this is not the general
>      case).  In most situations, in the interest of being as flexible and
>      efficient as possible, the best choice will be for a library to
>      expose at least all of the transport features that are recommended as
>      a "minimal set" here.

What is the bar for the requirements here. I.e., do you believe that
all of these can be implemented with a standard sockets API.

S 3.
>   
>      Based on the categorization, reduction, and discussion in Appendix A,
>      this section describes a minimal set of transport features that end
>      systems should offer.  The described transport system can be
>      implemented over TCP.  Elements of the system that are not marked
>      with "!UDP" can also be implemented over UDP.

Does this mean over native UDP with no other session protocol. Because
you can have TCP over UDP.

S 3.2.
>      multiplexed as streams on a single SCTP association when SCTP may not
>      be available).  The transport system must therefore ensure that
>      group- versus non-group-configurations are handled correctly in some
>      way (e.g., by applying the configuration to all grouped connections
>      even when they are not multiplexed, or informing the application
>      about grouping success or failure).

How do you group connections in TCP? Or is this text saying it
doesn't?

S 7.2.
>      o  Request to negotiate interleaving of user messages
>         Protocols: SCTP
>         Automatable because it requires using multiple streams, but
>         requesting multiple streams in the CONNECTION.ESTABLISHMENT
>         category is automatable.
>         Implementation: via a parameter in CONNECT.SCTP.

I'm not sure I follow how this is automatable. Is the argument that
SCTP will always do it, and so once you have asked for SCTP...?

S 7.2.
>   
>      o  Disable MPTCP
>         Protocols: MPTCP
>         Automatable because the usage of multiple paths to communicate to
>         the same end host relates to knowledge about the network, not the
>         application.

I don't think I understand how this is automatable. Is the theory that
the host auto-negotiates MPTCP? But what if the app doesn't want it no
matter what.

---

From: Michael Welzl <[email-masked]>
Date: Thu, 13 Sep 2018 16:34:30 +0200

Dear Eric,

> the other benefits in this tree.

We added a statement about this below the decision tree.

> all of these can be implemented with a standard sockets API.

I don't fully get this - it CAN be implemented atop the standard
socket API (cf. https://github.com/NEAT-project/neat), if that's
what you're asking?

> you can have TCP over UDP.

Native; we added a disclaimer about this at the end of the introduction.

> How do you group connections in TCP? Or is this text saying it

The text doesn't make any assumption about this being possible with
anything but SCTP. This being said, I can't resist offering an answer
(but that's out of scope of minset):
https://ieeexplore.ieee.org/document/8406887/

> SCTP will always do it, and so once you have asked for SCTP...?

Actually, yes. We added this to the description of the implementation.

> the host auto-negotiates MPTCP? But what if the app doesn't want it no

Then the application wants more than what this design of strictly
"application-specific knowledge" is giving it. That's the trade-off here -
there may be many reasons for applications to want things beyond this
document, but if we weren't strict about these limitations, this would
have hardly become a "minimal set".

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 13 Sep 2018 07:54:45 -0700

> socket API (cf. https://github.com/NEAT-project/neat), if that's

How do you implement limits on retransmission rates with standard sockets?

> https://ieeexplore.ieee.org/document/8406887/

This is behind a paywall, but it appears to just be about congestion
control, so perhaps there is some confusion about "grouping". I had read
that as application visible grouping. Is that not what you mean?

> have hardly become a "minimal set".

That's reasonable, but it seems like a somewhat confusing definition.

-Ekr

---

From: Michael Welzl <[email-masked]>
Date: Thu, 13 Sep 2018 20:02:11 +0200

Hi,

> socket API (cf. https://github.com/NEAT-project/neat <https://github.com/NEAT-project/neat>), if that's
> what you're asking?
> 

It’s an SCTP feature (limiting the number of retransmissions is a part of its partial reliability, specified as one way of doing it in RFC 3758); when you fall back to TCP, you do nothing.

In terms of the document, the following text in appendix A.1 says it:

   o  Configurable Message Reliability
      Protocols: SCTP
(..)
      Implementation over TCP: By using SEND.TCP and ignoring this
      configuration: based on the assumption of the best-effort service
      model, unnecessarily delivering data does not violate application
      expectations.  Moreover, it is not possible to associate the
      requested reliability to a "message" in TCP anyway.
      Implementation over UDP: not possible (UDP is unreliable).

If that seems disappointing: the goal of this document was to describe what can be done while being protocol-independent and allowing to fall back to TCP, or even UDP in some cases. The minute you write code that checks if configuring retransmission limits is possible, you need to implement an exception for the TCP case. I’m not saying this is bad design at all, it just wasn’t the goal of the minset. This doesn’t mean that a TAPS system needs to be limited in this way - it’s just the minimum you can expect from it.

> https://ieeexplore.ieee.org/document/8406887/ <https://ieeexplore.ieee.org/document/8406887/>
> 
> This is behind a paywall, but it appears to just be about congestion control, so perhaps there is some confusion about "grouping". I had read that as application visible grouping. Is that not what you mean?

Sorry for the link problem; it IS application visible grouping, with prioritization. There’s also this draft:
https://tools.ietf.org/html/draft-welzl-tcp-ccc <https://tools.ietf.org/html/draft-welzl-tcp-ccc>
and we have FreeBSD code doing this. Safiqul has various material here: http://safiquli.at.ifi.uio.no/tcp-ccc/ <http://safiquli.at.ifi.uio.no/tcp-ccc/>

I think we're diverging from the minset discussion, so I’ll stop here - but if this really interests you, tell me: I’d be very happy to discuss it further and send you the paper. I’d love for someone to use this!
Since you mentioned TCP over UDP before - that’s actually one of the things we implemented, to ensure that flows traverse the same path.

> That's reasonable, but it seems like a somewhat confusing definition.

What is it that’s confusingly defined? “application-specific knowledge”? “Automatable"? Happy to fix if you tell me what.

Cheers,
Michael

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 13 Sep 2018 11:24:17 -0700

On Thu, Sep 13, 2018 at 11:02 AM, Michael Welzl <[email-masked]> wrote:

> It’s an SCTP feature (limiting the number of retransmissions is a part of
> its partial reliability, specified as one way of doing it in RFC 3758);
> when you fall back to TCP, you do nothing.
>
> If that seems disappointing: the goal of this document was to describe
> what can be done while being protocol-independent and allowing to fall back
> to TCP, or even UDP in some cases. The minute you write code that checks if
> configuring retransmission limits is possible, you need to implement an
> exception for the TCP case. I’m not saying this is bad design at all, it
> just wasn’t the goal of the minset. This doesn’t mean that a TAPS system
> needs to be limited in this way - it’s just the minimum you can expect from
> it.
>

I'm not disappointed, I'm just trying to figure out how to think about this
API and its requirements.

What does it mean to implement this API when you have a transport which
doesn't support the feature? Just that you have an affordance that pretends
to implement it but doesn't?

> https://tools.ietf.org/html/draft-welzl-tcp-ccc
> and we have FreeBSD code doing this. Safiqul has various material here:
> http://safiquli.at.ifi.uio.no/tcp-ccc/
>
> I think we're diverging from the minset discussion, so I’ll stop here -
> but if this really interests you, tell me: I’d be very happy to discuss it
> further and send you the paper. I’d love for someone to use this!
> Since you mentioned TCP over UDP before - that’s actually one of the
> things we implemented, to ensure that flows traverse the same path.
>

I'd love to see the paper.

> What is it that’s confusingly defined? “application-specific knowledge”?
> “Automatable"? Happy to fix if you tell me what.
>

Automatable, I think.

-Ekr

---

From: Michael Welzl <[email-masked]>
Date: Fri, 14 Sep 2018 09:18:39 +0200

> On 13 Sep 2018, at 20:24, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
> I'm not disappointed, I'm just trying to figure out how to think about this API and its requirements.
> 
> What does it mean to implement this API when you have a transport which doesn't support the feature? Just that you have an affordance that pretends to implement it but doesn't?

Exactly - though "pretends to implement it" is maybe not how it would put it: no promises are made. An application says "these messages can be out-of order", or "these connections really belong together, with the following priorities between them", but the priorities are just a hint and might be ignored, and the messages may arrive in-order, nobody promised that there would NOT be ordering.

> I'd love to see the paper.

I'll follow up in a separate email, with TAPS in cc because people there might find it interesting, but removing the IESG to reduce noise.

> What is it that’s confusingly defined? “application-specific knowledge”? “Automatable"? Happy to fix if you tell me what.

Okay, so this is what we have in the text:

" The transport features of IETF transport protocols that do not
   require application-specific knowledge and could therefore be
   utilized by a transport system on its own without involving the
   application are called "Automatable". "

and "application-specific knowledge" is defined as "knowledge that only applications have."

I'm guessing that the issue that you have with "automatable" is that it sounds too much as if the application's input really isn't needed here, no matter what. I'd like to use a "weaker" word in that sense... because of that, we once called it "Potentially automatable", but then that was too long and clumsy.
Do you have any suggestions?

Cheers,
Michael

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 14 Sep 2018 06:22:54 -0700

On Fri, Sep 14, 2018 at 12:18 AM, Michael Welzl <[email-masked]> wrote:

> > It’s an SCTP feature (limiting the number of retransmissions is a part
> of its partial reliability, specified as one way of doing it in RFC 3758);
> Exactly - though "pretends to implement it" is maybe not how it would put
> it: no promises are made. An application says "these messages can be out-of
> order", or "these connections really belong together, with the following
> priorities between them", but the priorities are just a hint and might be
> ignored, and the messages may arrive in-order, nobody promised that there
> would NOT be ordering.
>

All I meant was that the API is a no-op.

> I'll follow up in a separate email, with TAPS in cc because people there
> might find it interesting, but removing the IESG to reduce noise.
>

Thanks.

> I'm guessing that the issue that you have with "automatable" is that it
> sounds too much as if the application's input really isn't needed here, no
> matter what. I'd like to use a "weaker" word in that sense... because of
> that, we once called it "Potentially automatable", but then that was too
> long and clumsy.

Not really. I guess the thing that I'm saying is that MPTCP is something
that the stack could automatically decide to use, but it can't
automatically decide *not* to use if the application wants that. That's
what's puzzling me about "automatable"

-Ekr

Cheers,

---

From: Michael Welzl <[email-masked]>
Date: Sun, 16 Sep 2018 14:04:41 +0200

> Not really. I guess the thing that I'm saying is that MPTCP is something that the stack could automatically decide to use, but it can't automatically decide *not* to use if the application wants that. That's what's puzzling me about “automatable"

You’re right about disabling MPTCP: typically the decision to do so would depend not only on knowledge about the net or OS, but also the type of data that an application has. Sticking with the specs as minset is doing, I searched, and found, in RFC 6897:

Section 3.1: "The following sections summarize the performance effect of MPTCP as seen by an application.”

and in there, we have things like:
**
   The benefits of MPTCP regarding throughput and resilience may come at
   some cost regarding data delivery delay and delay jitter.
(..)
   Applications with high real-time requirements might be affected by
   such a scenario.  One possible remedy is to disable MPTCP for such
   jitter-sensitive applications, either by using the basic API defined
   in this document, or by other means, such as system policies.
**

I guess the decision on whether to use MPTCP or not is actually a mix of knowledge about the network and app requirements. A better interface (IMO) would have the app tell its requirements, such that a system below it could automate its decision. I hope we get to that with draft-ietf-taps-interface, but that’s beyond minset - I think minset should therefore recommend exposure of “Disable MPTCP” and call it “Optimizing”, not “Automatable”. However, I think we should still not recommend exposure of adding and removing paths, as these really don’t make sense without net-specific knowledge.

Do you agree? If so, I’ll make that update in the next version - that was a great catch indeed!

Cheers,
Michael

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 17 Sep 2018 14:15:52 -0500

FWIW,

On Sun, Sep 16, 2018 at 7:03 AM Michael Welzl <[email-masked]> wrote:

>> > What is it that’s confusingly defined? “application-specific
>> knowledge”? “Automatable"? Happy to fix if you tell me what.
>> >
>> > Automatable, I think.
>>
> what's puzzling me about “automatable"
>
>
> You’re right about disabling MPTCP: typically the decision to do so would
> depend not only on knowledge about the net or OS, but also the type of data
> that an application has. Sticking with the specs as minset is doing, I
> searched, and found, in RFC 6897:
>
> I guess the decision on whether to use MPTCP or not is actually a mix of
> knowledge about the network and app requirements. A better interface (IMO)
> would have the app tell its requirements, such that a system below it could
> automate its decision. I hope we get to that
> with draft-ietf-taps-interface, but that’s beyond minset - I think minset
> should therefore recommend exposure of “Disable MPTCP” and call it
> “Optimizing”, not “Automatable”. However, I think we should still not
> recommend exposure of adding and removing paths, as these really don’t make
> sense without net-specific knowledge.
>

Speaking as an individual contributor in PANRG, and not as an AD, I'm not
sure that PANRG would be ready to recommend that yet, even with
net-specific knowledge!

Do you agree? If so, I’ll make that update in the next version - that was a
> great catch indeed!

Speaking as the responsible AD, yes, it was, and thank you, Eric, for
catching it.

Spencer

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 17 Sep 2018 12:30:12 -0700

That SGTM....

-Ekr

On Sun, Sep 16, 2018 at 5:04 AM, Michael Welzl <[email-masked]> wrote:

> automate its decision. I hope we get to that with draft-ietf-taps-interface,
> but that’s beyond minset - I think minset should therefore recommend
> exposure of “Disable MPTCP” and call it “Optimizing”, not “Automatable”.
> However, I think we should still not recommend exposure of adding and
> removing paths, as these really don’t make sense without net-specific
> knowledge.
>

---

From: Michael Welzl <[email-masked]>
Date: Tue, 18 Sep 2018 11:53:41 +0200

Done, with version -10 (just submitted).

Cheers,
Michael

> On 17 Sep 2018, at 21:30, Eric Rescorla <[email-masked]> wrote:
> 
> That SGTM....
> 
> -Ekr
> 
> 
> _______________________________________________
> Taps mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/taps