Subject: [multipathtcp] AD review is coming

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 12 Apr 2019 18:40:19 +0200

Hi authors, hi shepherd/Phil,

Just wanted to quickly let you know that I’ve started the IETF last call just now in order to hopefully get this document on the telechat in 3 weeks. 

I’ve not finished my AD review completely but I am nearly done and convinced that this doc is ready to start IETF last call. So far I have only some editorial or smallish comments and nits which I will send in detail beginning of next week and can be address during or after IETF last call.

Only one request for now (mostly with an eye on the IESG evaluation): The draft says that changes are made which are not backward compatible (see sec 3.1). That’s fine, however, it would be good to also explain a little bit in the draft as well as in the shepherd write up (!) why this is okay. My understanding is that in case of a v0 server the connection will “just" fall-back to non-MPTCP capable and that this is acceptable in current deployment situation. Please confirm and at least update the write-up accordingly as soon as possible!

Thanks!
Mirja

---

From: Alan Ford <[email-masked]>
Date: Mon, 15 Apr 2019 17:55:11 +0100

Hi Mirja,

To be clear, you are looking for some rationale in the draft text as well? The technical discussion of the fallback process is given in Section 3.1:
   The MP_CAPABLE exchange in this specification (v1) is different to
   that specified in v0 [RFC6824].  If a host supports multiple versions
   of MPTCP, the sender of the MP_CAPABLE option SHOULD signal the
   highest version number it supports.  In return, in its MP_CAPABLE
   option, the receiver will signal the version number it wishes to use,
   which MUST be equal to or lower than the version number indicated in
   the initial MP_CAPABLE.  There is a caveat though with respect to
   this version negotiation with old listeners that only support v0.  A
   listener that supports v0 expects that the MP_CAPABLE option in the
   SYN-segment includes the initiator's key.  If the initiator however
   already upgraded to v1, it won't include the key in the SYN-segment.
   Thus, the listener will ignore the MP_CAPABLE of this SYN-segment and
   reply with a SYN/ACK that does not include an MP_CAPABLE, thus
   leading to a fallback to regular TCP.  An initiator MAY cache this
   information about a peer and for future connections, MAY choose to
   attempt using MPTCP v0, if supported, before recording the host as
   not supporting MPTCP.
But you would like to see some discussion mentioning the migration from Experimental to Proposed Standard? And the benefits it brings? Would that cover it?

Many thanks,
Alan

> On 12 Apr 2019, at 17:40, Mirja Kuehlewind <[email-masked]> wrote:
>

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 16 Apr 2019 13:17:22 +0200

Hi Alan,

Yes, I was looking also for eventually a tiny update to the doc (while it would be most important to mention this in the write-up). I think the one thing that could be added below, is one more sentence or maybe even halve a sentence saying that this fallback to plain TCP (instead of v0) is okay, as general deployment is low and there is also an expectation that server get updated first or entities using MPTCP today actually have some control to update both client and server. Or whatever you think is right to say. So the question to answers is why is it okay that fallback is to plain TCP instead of v0…?

Mirja

> On 15. Apr 2019, at 18:55, Alan Ford <[email-masked]> wrote:
> 
> Hi Mirja,
>

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 16 Apr 2019 13:12:42 +0000

Mirja,
> 
> Yes, I was looking also for eventually a tiny update to the doc (while it would be most important to mention this in the write-up). I think the one thing that could be added below, is one more sentence or maybe even halve a sentence saying that this fallback to plain TCP (instead of v0) is okay, as general deployment is low and there is also an expectation that server get updated first or entities using MPTCP today actually have some control to update both client and server. 

Today, the largest deployments are :
- Apple (mainly their own services, but third party apps will appear)
- KT/... using SOCKS proxies
- Hybrid Access networks (where MPTCP is used between CPEs and proxies)

Huawei has started to deploy MPTCP in an end-to-end manner in China.

In December, a zmap scan of the entire IPv4 space onport 443 revealed 
than only about 10k hosts were responding with a valid MP_CAPABLE. Many 
of these were CPE routers that had MPTCP enabled and surprisingly open 
https servers.

> Or whatever you think is right to say. So the question to answers is why is it okay that fallback is to plain TCP instead of v0…?

I think that this would make sense given the limited deployment of MPTCP 
servers.

Olivier

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 16 Apr 2019 16:08:20 +0200

Hi again,

Here are the more detailed comment of my review as promised. Most of these comments are editorial, however, there are a few comments on use of normative language as well as clarifications and also one point about specifying the registration policy for the new registry (comment 13 below) which really must be address in the next version.

Thanks!
Mirja

----------------------------

1) Sec 3.1: “If a responder does not support (or does not want to support)
   any of the initiator's proposals, it can respond without an
   MP_CAPABLE option, thus forcing a fallback to regular TCP.”
   I wonder if this should be normative: “…it MUST respond without an MP_CAPABLE option…”..?

2) Sec 3.3.1: “… and any later data for that sequence space should be ignored.”
   Maybe SHOULD?

3) Sec 3.3.2: “If the DSN received is
   32 bits, it is valid for the implementation to choose whether to send
   a 32-bit or 64-bit Data ACK.”
  Maybe “… the implementation MAY choose whether to send…”

4) Sec 3.3.3: I guess if there is only one subflow it is probably safe to send the TCP FIN together with a DATA_FIN (as an optimisation), or would there be a reason to not do that? Should that be mentioned in the text?

5) Sec 3.3.8: Is the reference to ECN really needed? I feel that this idea is hard to understand without further context.

6) Sec 3.6: Maybe I missed it that but it seems that the error codes “Unacceptable performance” is not further used in the document while I would think further guidance might be needed. 

7) Sec 3.7: Minor editorial point I noticed here but there might be other cases:
“If lost options on data packets occur on any other subflow apart from the initial subflow…”
Not sure if “initial” is correct here because if you move you may not have the initial subflow anymore… I guess it would be something like “all expect one” here…

8) Sec 3.9.2: I wondering about this text, given this is a bis doc and we have an deployment experience report:
“We expect that experience gathered from deployments will provide further guidance on this, and will be affected by particular application characteristics (which are likely to change over time).  […] Results from experimental deployments are needed in order to verify the correctness of this proposal.”
Maybe just rephrase it say something like “We got some experience but more is needed/there is no generics solution.”.. or just remove…

9) Maybe section 3.9.2 should also say again that receiving a second ADD_ADDRESS (with the same ID) is a request from the other end to open a subflow immediately?

10) Sec 3.9.3: Should learned information really be cached “forever”, or should the text maybe says also something about retrying after some time…?

11) Sec 8.1: “This document defines one additional subtype
   (ADD_ADDR) and updates the references to this document for all sub-
   types except ADD_ADDR, which is deprecated.”
   What?

12) Also sec 8.1: RFC6824 also has this half-sentence:
“future assignments are to be defined by Standards Action as defined by [25].”
which got lost in this document. However, given this document obsoletes RFC6824 it should probably also mention the assignment policy for this registry.

13) Sec 8.3. doesn’t specify an assignment policy.

14) Given RFC6824 will be obsoleted by this spec, it does not to be listed in the reference section.

Nits:
- Sec 3.3.6: s/if it so desired for reliability reasons./if it is desired for reliability reasons./
- Sec 3.3.&: s/Multiple retransmissions are triggers that/Multiple retransmissions are triggered that/ or actually
                         s/Multiple retransmissions are triggers that/If multiple retransmissions are triggered that/
- Sec 3.7: s/Middlebox interferance/Middlebox interference/
- Sec 3.9.2: s/If the the same ports are used on all subflows/If the same ports are used on all subflows/

> On 12. Apr 2019, at 18:40, Mirja Kuehlewind <[email-masked]> wrote:
>

---

From: <[email-masked]>
Date: Thu, 18 Apr 2019 15:34:42 +0000

About 
>    attempt using MPTCP v0, if supported, before recording the host as

I think one problem is the phrase "thus leading to a fallback to regular TCP", which is a bit misleading. Suggest deleting and adding more explanation on these lines:

Thus, the listener will ignore the MP_CAPABLE of this SYN-segment and reply with a SYN/ACK that does not include an MP_CAPABLE. The initiator MAY choose to immediately fall back to TCP or MAY choose to attempt a connection using MPTCP v0 (if the initiator supports v0), in order to discover whether the listener supports the earlier version of MPTCP. In general a MPTCP v0 connection is preferred to a TCP one, however in a particular deployment scenario it may be known that the listener is unlikely to support MPTCPv0 and so the initiator may prefer not to attempt a v0 connection. 
An initiator MAY cache information for a peer about what version of MPTCP it supports if any, and use this information for future connection attempts.

phil

-----Original Message-----
From: Mirja Kuehlewind [mailto:[email-masked]] 
Sent: 16 April 2019 12:17
To: Alan Ford <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: AD review is coming

Hi Alan,

Mirja