Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

From: <[email-masked]>
Date: Wed, 4 Jul 2018 07:30:33 +0000

We urgently need your comments about this document. 

It's been in WG last call for just on 4 weeks, but I think the only comments so far are from the WG chairs - clearly we need more comments before the document can advance. We'd like to discuss and close issues before or during the IETF.

So please can you dedicate some time to checking the document.

Thanks,
Phil & Yoshi

-----Original Message-----
From: Eardley,PL,Philip,TUD1 R 
Sent: 26 June 2018 08:33
To: [email-masked]
Subject: RE: WG Last Call for draft-ietf-mptcp-rfc6824bis

Hi,
Just a reminder about the WGLC for the protocol bis. Please send comments, Thanks!
Phil & Yoshi

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 05 June 2018 12:21
To: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

As a reminder, this document Obsoletes: 6824 (if approved) and its Intended status is Standards Track.

As well as suggested changes or edits, positive review comments ("It's ready") are also very valuable. 

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 05 June 2018 09:14
To: [email-masked]
Subject: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

This starts a WG Last Call for draft-ietf-mptcp-rfc6824bis. Please send comments by the end of June. 

Please note there are three IPR disclosures (we're working on getting them added to the rfc6824bis page): 

* two are inherited from RFC6824  https://datatracker.ietf.org/ipr/search/?submit=draft&id=draft-ietf-mptcp-multiaddressed    
* one is inherited from draft-paasch-mptcp-syncookies (which got include in rfc6824bis) https://datatracker.ietf.org/ipr/2678/ 

Thanks,
Phil & Yoshi
_______________________________________________
multipathtcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/multipathtcp

_______________________________________________
multipathtcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 4 Jul 2018 02:10:51 -0700

We also expect the authors to respond the comments sent during WGLC as it
can trigger other comments.

Thanks,
--
Yoshi

On Wed, Jul 4, 2018 at 12:30 AM, <[email-masked]> wrote:

> It's been in WG last call for just on 4 weeks, but I think the only
> comments so far are from the WG chairs - clearly we need more comments
> before the document can advance. We'd like to discuss and close issues
> before or during the IETF.
>
> * two are inherited from RFC6824  https://datatracker.ietf.org/i
> pr/search/?submit=draft&id=draft-ietf-mptcp-multiaddressed
> * one is inherited from draft-paasch-mptcp-syncookies (which got include
> in rfc6824bis) https://datatracker.ietf.org/ipr/2678/
>
> Thanks,
> Phil & Yoshi

---

From: <[email-masked]>
Date: Thu, 5 Jul 2018 08:20:50 +0000

Hi,
Fyi I prepared some text describing the main differences between the bis and rfc6824, and the reasons for those changes. I wrote it in the context of helping the IESG when they get to reviewing it - but I thought it might be useful to helping prompt WG comments at this stage (which we urgently need). The text is draft.
Best wishes,
Phil

--
This summarises the changes in draft-ietf-mptcp-rfc6824bis compared to the RFC6824. 
https://tools.ietf.org/rfcdiff?url1=https://www.ietf.org/id/draft-ietf-mptcp-rfc6824bis-11.txt&url2=https://www.rfc-editor.org/rfc/rfc6824.txt   
1.	It obsoletes RFC6824. This is considered a replacement for RFC6824, as it improves it in several ways. It is not backwards compatible with RFC6824 (the protocol changes in #4 and #9 in particular). 
2.	Its Intended Status is Standards Track. RFC6824 is an Experimental Track document. When it was completed, it was agreed to re-charter the WG with the primary goal to create a bis version of the protocol document on the Standards track. 
a.	The bis incorporates lessons learnt from the various implementations, deployments and experiments. RFC8041 discusses "Use Cases and Operational Experience with Multipath TCP" https://tools.ietf.org/html/rfc8041 and an IETF Journal article describes "Multipath TCP Deployments" https://www.ietfjournal.org/multipath-tcp-deployments/ 
b.	In summary, MPTCP is extensively deployed, especially on smartphones (it is in Apple's iOS and macOS operating systems and is in several models of commercial Android smartphones), and is also deployed in some hybrid access networks.  
c.	The 'reference', maintained Linux implementation of RFC6824 is available at https://multipath-tcp.org/ 
d.	RFC6824bis-11 has been implemented in Linux, but has not been released (yet).
3.	Christoph Paasch is added as an author, as he proposed several of the new elements
4.	Connection initiation, through the exchange of the MP-CAPABLE MPTCP option, is different from RFC6824. Assuming that host A sends the SYN and ACK, and host B sends the SYN-ACK:-
a.	Unlike RFC6824, the SYN doesn't include A's key (it is still sent in the ACK). This change means that A's key, which is sent in the clear, is not sent until B is known to be MPTCP-enabled.
b.	The ACK can include data. This change means that A can send data without having to wait for B to send it data, as only then can it be sure the ACK has been received. To handle this case, the MP-CAPABLE option includes fields with the Data-Level Length and optionally a Checksum. 
c.	In the Flags field, C is now assigned to mean that the sender of this option will not accept additional MPTCP subflows to the source address and port. This is an efficiency improvement, for example where the sender is behind a strict NAT.
d.	In the flags field, D now indicates the use of HMAC-SHA256 (rather than HMAC-SHA1).
5.	Connection initiation also defines the procedure for version negotiation, for implementations that support both v0 (RFC6824) and v1 (RFC6824bis). The initiator indicates the highest version it supports, the responder indicates if it supports this version. To prevent a downgrade attack, the initiator SHOULD fall back to regular TCP, alternatively it MAY establish a new connection with the lower version of MPTCP. 
6.	The HMAC-SHA256 (rather than HMAC-SHA1) algorithm is used, as the algorithm provides better security. It is used to generate the token in the MP_JOIN and ADD_ADDR messages, and to set the initial data sequence number.
7.	As in RFC6824, a TCP RST for the subflow re-sets the subflow, or (in response to MP_JOIN) rejects an attempt to open a subflow. The change is that the process is enhanced by adding a MP_TCPRST Option:-
a.	The new option includes a flag indicating whether the error condition is permanent or temporary, and a field with the "MP_TCPRST Reason Code", which indicates why the TCP RST is being sent. 
b.	The receiving host MAY use this information to decide, for example, whether it tries to re-establish the subflow immediately, later, or never.
c.	IANA is requested to create a further sub-registry with the "MP_TCPRST Reason Codes". 
8.	The MP_PRIO option, which is used to signal a change of priority for a subflow, no longer includes the AddrID field. Its purpose was to allow the changed priority to be  applied on a subflow other than the one it was sent on. However, it has been realised that this could be used by a man-in-the-middle to divert all traffic on to its own path, and this is an easy attack since MP_PRIO does not include a token or other security mechanism. The issue was discussed at IETF-99 and since no implementer was known to use the address identifier on MP-PRIO, it was agreed that the best approach was to remove it.
9.	The ADD_ADDR option, which is used to inform the other host about another potential address, is different in several ways:
a.	It now includes an HMAC of the added address. This is for enhanced security. The RFC6824 procedure for adding an address is open to an attack where an off-path attacker creates a spoofed ADD_ADDR that enables it to hijack an MPTCP-enabled connection. Including an HMAC in the ADD_ADDR message makes it as secure as MP_JOIN. 
b.	This was the primary driver for incrementing the version number. This was first discussed at an interim meeting in 2015 and subsequently at IETF-96. The WG consensus was that this approach was better than incorporating an alternative add address mechanism within an enhanced version 0 of the protocol. The problem and solution is discussed further in RFC7430.
c.	Reliability for the ADD_ADDR option. The IPVer field is replaced with a flag field (the IP version field was not useful, since it is known anyway), and one flag is assigned (E) which is used as an 'Echo' so a host can indicate that it has received the option. Hence this provides reliability for the ADD_ADDR option. 
10.	An additional way of performing a Fast Close is described. The host sends a RST containing the MPT_FASTCLOSE option on all subflows. This allows the host to tear down the subflows and the connection immediately. 
11.	In the IANA registry a new MPTCP subtype option, MP_EXPERIMENTAL, is Reserved for private experiments. However, the document doesn't define how to use the subtype option. 
12.	A new Appendix discusses the usage of both the MPTCP and TCP Fast Open on the same packet. The two main issues are discussed: 
a.	The potential lack of enough option space in a TCP packet. This is more likely to be an issue on the SYN+ACK packet, and the suggestion is to use a shorter TFO cookie if necessary.
b.	A potential problem with data sequence mapping under some circumstances. The suggestion is that the TFO data is not considered part of the Data Sequence Number space.

-----Original Message-----
From: Eardley,PL,Philip,TUD1 R 
Sent: 04 July 2018 08:31
To: [email-masked]' <[email-masked]>
Subject: RE: WG Last Call for draft-ietf-mptcp-rfc6824bis

Thanks,
Phil & Yoshi

-----Original Message-----
From: Eardley,PL,Philip,TUD1 R 
Sent: 26 June 2018 08:33
To: [email-masked]
Subject: RE: WG Last Call for draft-ietf-mptcp-rfc6824bis

Hi,

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 05 June 2018 09:14
To: [email-masked]
Subject: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Thanks,
Phil & Yoshi

---

From: Alan Ford <[email-masked]>
Date: Wed, 11 Jul 2018 07:22:40 +0100

Thank you for your feedback, Phil. All proposed changes look good and we will wrap these into the next version.

@Everyone - we are however awaiting other WGLC feedback before doing a new revision!

Best regards,
Alan

> On 18 Jun 2018, at 14:59, [email-masked] wrote:
> 
> More comments:
> Section 3.5 - 
> Option R (RST) - add (at the end of the bullet) "the Host A enters the CLOSED state directly" (I think)
> 
> << If a host receives a packet with a valid MP_FASTCLOSE option, it
>   shall process it as follows :>>
> I think the text that follows (apart from the last bullet) is only relevant for Option A, where the MP_FASTCLOSE is sent on an ACK - it doesn't apply for option R (where it's sent on a RST). Anyway, things could be laid out slightly differently so the section is clearer. 
> 
> Section 3.6
> I wonder if it would be more logical for this section to be before Fast close?
> 
> This section could do with a read through. A few things:
> << As discussed in Section 3.5 above, the MP_FASTCLOSE option provides a
>   connection-level reset roughly analagous to a TCP RST. Regular TCP
>   RST options remain used to at the subflow-level to indicate the
>   receiving host has no knowledge of the MPTCP subflow or TCP
>   connection to which the packet belongs.>>
> Can this paragraph be deleted? at least I found it a bit confusing that this text is not about this section. 
> 
> "remain used to at" is ugly
> 
> << However, in MPTCP, there may be many reasons for rejecting the
>   opening of a subflow, but these semantics cannot be carried in a
>   standard TCP RST. It would be beneficial for a host to the reasons
>   why its subflow has been closed with a RST, and thus whether it
>   should try to re-establish the subflow immediately, later, or never
>   again.  These semantics are carried in the MP_TCPRST option that can
>   be included on a TCP RST packet.>>
> I presume the new MP_TCPRST can be added to a RST, whenever it's created - the sentence seems to imply it's only when it wants to reject the opening of a subflow (ie in response to MP_JOIN). 
> I assume inclusion of the MP_TCPRST option is optional?
> 
> potential better phrasing:
> A host sends a TCP RST in order to close a subflow or reject an attempt to open a subflow (MP_JOIN). In order to inform the receiving host why a subflow is being closed or rejected, the TCP RST packet MAY include the MP_TCPRST Option. The host MAY use this information to decide, for example, whether it tries to re-establish the subflow immediately, later, or never. 
> 
> <<   o  Unspecified error (code 0x0).  This is the default error implying
>      the subflow is not longer available.  The receiving host SHOULD
>      take account of the 'T' bit in deciding whether to re-estbalish
>      this subflow.  The presence of this option shows that the RST was
>      generated by a MPTCP-aware device.>>
> 
> " The receiving host SHOULD take account of the 'T' bit in deciding whether to re-estbalish this subflow."
> Only this sub-bullet mentions the T bit, which may suggest that it should not be taken account of with the other reason codes.  I think the sentence should be deleted (using the T flag is mentioned earlier). 
> 
> <<enough ressources>>  enough resources
> 
> Section 3.7
> I guess it's OK to refer to it as RST rather than TCP RST?
> 
> << The case described above is a specialized case of fallback, for when
>   the lack of MPTCP support is detected before any data is acknowledged
>   at the connection level on a subflow.  >>
> "The case above" - I'm not sure which of the previous paragraph or paragraphs you're referring to.
> "specialised" - maybe "specific" [is it really specialised?]
> "for when" -> when
> 
> S3.9.2
> << If the two communicating hosts immediately try to set up subflows
>   from all available addresses to all available addresses on the other
>   host, this could end up creating two subflows per path.  This is an
>   inefficient use of resources.>>
> is two subflows correct? Multiple I think
> " could end up creating two subflows per path" -> "could create multiple subflows over the same path"
> 
> << If a host does not support TCP simultaneous open, it
>   is RECOMMENDED that some element of randomization is applied to the
>   time waited before opening new subflows, so that only one subflow
>   exists between a given address pair.  If, however, hosts signal
>   additional ports to use (for example, for leveraging ECMP on-path),
>   this heuristic need not apply.>>
> "time waited" - ugly phrase?
> "exists" -> is created
> "need not apply" -> is not appropriate
> 
> Section 5 - Security
> This section needs updating to reflect the changes in the bis.:
> 
> Add mention of ADD_ADDR, now with HMAC and reliability,
> Discuss the revised MP_CAPABLE
> I'm not sure if the new fast close option (option R) has any security impact?
> Discuss the MP_PRIO change (no off path address indication)
> Discuss downgrade attacks (to v0) during connection initiation
> 
> Section 8 IANA
> << This document updates [RFC6824] and as such IANA is requested to
>   update the TCP option space registry to point to this document for
>   Multipath TCP>>
> This document obsoletes rather than updates...   I suspect this means all the MPTCP options need to be included in this document, but we can leave IANA to decide that. 
> 
> <<  |    C    |   Do not attempt to connect to   |    This document,    |
>   |         |          source address          |     Section 3.1      |
>>> 
> I don't think this is a good summary of flag C's meaning.
> 
> << The
>   contents of this sub-registry are to to this document, as follows:
>>> 
> Is this sentence needed?
> 
> Appendix B TCP Fast Open
> Maybe rephrae the title as "TCP Fast Open and MPTCP" or "Considerations about using TCP Fast Open and MPTCP"?
> 
> This section could do with a read through, it could do with a few minor re-phrasings
> 
> 
> <<   TCP Fast Open (TFO) is an experimental TCP extension, described in
>   [RFC7413], which has been introduced with the objective of gaining
>   one RTT before transmitting data.>>
> " gaining one RTT"?  maybe: which allows data to be sent one RTT earlier than with regular TCP. 
> << It achieves this by sending the SYN-
>   segment together with data>>
> re-phrase (I think data is referring ot the cookie - but you havent yet mentioned it - rather than user data)
> 
> "resp." = ?
> 
> << TFO and MPTCP can be combined provided that the total length of their
>   options does not exceed the maximum 40 bytes possible in TCP>>
> "their" is wrong, as there may be other options. -> "all the"
> 
> << We also note that this does not entail a loss
>   of functionality, because TFO-data is always sent when only one path
>   is active.>>
> -> "TFO-data is only ever sent when only a single path is active"
> How do you guarantee this? Not sure I understand. 
> 
> << To solve this, the TFO data should not be considered part of the Data
>   Sequence Number space:>>
> Is this a SHOULD NOT?  Does it imply a change to rfc7413 (the tfo rfc)? I guess this is really a suggestion to the implementer?
> 
> Section B.3
> I assume this sub-section is an example (so no SHOULD or even MAY are needed)
> 
> << client                                                         server
>     |                                                              |
>     |    S 0(0) <MP_CAPABLE>, <TFO cookie request>                 |
>     | -----------------------------------------------------------> |
>>> 
> Can you explain the S 0(0) nomenclature/ please.
> 
> 
> Appendix D Finite state machine
> I presume we don't need similar FSM for fastclose?
> 
> 
> Section 2:
> Need to add sub-sections on closing subflow & fast close & fallback
> 
> S2.7 - I think the third bullet about notable features could be slightly altered, with enhanced security for ADD_ADDR and MP_CAPABLE. You could also add a mention of rfc7430
> 
> 
> Section 2.1 & 3.1 - can you remind me why we changed MP_CAPABLE so that Host A's key is sent only on the ACK (not on the SYN)? Is this so it's only sent when we know host B is MPTCP capable, which is therefore a slight security improvement?  (Just send a link to earlier emails, if that's easier)
> 
> Thanks
> phil
> Sent: 14 June 2018 11:56
> To: [email-masked]' <[email-masked]>; [email-masked]
> A few comments as I work through the document, basically minor so far
> 
> Section 3.1
> << Given the SYN exchange is different between v1 and v0
>   the exchange cannot be immediately downgraded, and therefore if the
>   far end has requested a lower version then the initiator SHOULD
>   respond with an ACK without any MP_CAPABLE option, to fall back to
>   regular TCP.>>
> I think it should say "to ensure the exchange cannot..."
> 
> 
> << The first 4 bits of the first octet in the MP_CAPABLE option
>   (Figure 4) define the MPTCP option subtype (see Section 8; for
>   MP_CAPABLE, this is 0), and the remaining 4 bits of this octet
>   specify the MPTCP version in use (for this specification, this is 1).
>>> 
> "The first 4 bits of the first octet in the MP_CAPABLE option" Fig 4 shows two earlier octets (Kind & Length) - I know these arent the MP-CAPABLE option, but I think it might be worth spelling out what they are. 
> 
> Would it be better strictly to have "0x0" rather than "0"? (ie match the IANA registry)
> 
> Section 3.4.1
> <<A set of four flags are present after the subtype and before the
>   Address ID.  Only the rightmost bit - labelled 'E' - is assinged
>   today.  The other bits are currently unassigned and MUST be set to
>   zero by a sender and MUST be ignored by the receiver.
> 
>   The 'E' bit exists to provide reliability for this option.  Because
>   this option will often be sent on pure ACKs, there is no guarantee of
>   reliability.  Therefore, a receiver receiving a fresh ADD_ADDR option
>   (where E=0), will send the same option back to the sender, but not
>   including the HMAC, and with E=1.  The lack of this echo can be used
>   by the initial ADD_ADDR sender to retransmit the ADD_ADDR according
>   to local policy.
>>> 
> 4 comments:
> Assinged /assigned
> 
> The 'E' bit - better to say flag?
> 
> Section 2.3 needs to be amended to show the reply /echo message for ADD-ADDR. I suggest the pic in S2.3 also shows the E value. 
> 
> Not sure - does this open security hole, where an attacker pretends an ADD-ADDR has been received? (the Echo doesn't include an hmac, so is easy to spoof)
> 
> And in Fig 12
> |        Truncated HMAC (8 octets, if length > 10 octets)       |
> 
> Could this be made more straightforward? Ie spell out when length is >10 octets ie IPv6 (if I understand it). This also seems to mean that IPv4 doesn't include the HMAC, is this right??
> 
> Thanks
> phil
>

---

From: <[email-masked]>
Date: Sat, 14 Jul 2018 11:08:25 +0000

Hi,

Apart from the pure clarifications, there are a few comments from me and Yoshi that may be worth discussing:

--
2:  As we have removed AddrID from MP_PRIO, we have a certain problem in the situations like the followings.

Let's say a server has two address S1, S2 while the client is behind FW. Since the client is behind firewall, the server uses ADD_ADDR to send the info of S2 through S1, but it wants S2 to be used as a backup.

In this case, if MP_PRIO has addr id field, the server is able to specify S2 is a backup. However, since MP_PRIO doesn't have addr id any more, all the server can do is to wait until the client establishes the connection with S2. But, this connection doesn't look what the server wants. 
I guess using 1 bit in ADD_ADDR might solve the issue.
--

One other high-level comment:  it would be very good if the bis code, implemented in linux, could be released - to allow testing of the protocol changes in the bis, for instance to make sure that no unexpected issues have been introduced

--

--

Section 3.4.1 ... The 'E' bit exists to provide reliability for this option ...
Not sure - does this open security hole, where an attacker pretends an ADD-ADDR has been received? (the Echo doesn't include an hmac, so is easy to spoof)

--

We're waiting for this IPR to get added on the tracker
* one is inherited from draft-paasch-mptcp-syncookies (which got include in rfc6824bis) https://datatracker.ietf.org/ipr/2678/

--

-----Original Message-----
From: Alan Ford [mailto:[email-masked]] 
Sent: 11 July 2018 02:23
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Best regards,
Alan

>   connection to which the packet belongs.>> Can this paragraph be 
> deleted? at least I found it a bit confusing that this text is not about this section.
> 
>   at the connection level on a subflow.  >> "The case above" - I'm not 
> sure which of the previous paragraph or paragraphs you're referring to.
> "specialised" - maybe "specific" [is it really specialised?] "for 
> when" -> when
> 
> S3.9.2
> is two subflows correct? Multiple I think " could end up creating two 
> subflows per path" -> "could create multiple subflows over the same path"
> 
> Add mention of ADD_ADDR, now with HMAC and reliability, Discuss the 
> revised MP_CAPABLE I'm not sure if the new fast close option (option 
> R) has any security impact?
> Discuss the MP_PRIO change (no off path address indication) Discuss 
> downgrade attacks (to v0) during connection initiation
> 
> Section 8 IANA
> re-phrase (I think data is referring ot the cookie - but you havent 
> yet mentioned it - rather than user data)
> 
> "resp." = ?
> 
> S2.7 - I think the third bullet about notable features could be 
> slightly altered, with enhanced security for ADD_ADDR and MP_CAPABLE. 
> You could also add a mention of rfc7430
> 
> 
> Section 2.1 & 3.1 - can you remind me why we changed MP_CAPABLE so 
> that Host A's key is sent only on the ACK (not on the SYN)? Is this so 
> it's only sent when we know host B is MPTCP capable, which is 
> therefore a slight security improvement?  (Just send a link to earlier 
> emails, if that's easier)
> 
> Thanks
> phil
> Not sure - does this open security hole, where an attacker pretends an 
> ADD-ADDR has been received? (the Echo doesn't include an hmac, so is 
> easy to spoof)
> 
> And in Fig 12
> * one is inherited from draft-paasch-mptcp-syncookies (which got 
> include in rfc6824bis) https://datatracker.ietf.org/ipr/2678/
> 
> Thanks,
> Phil & Yoshi

---

From: <[email-masked]>
Date: Sat, 14 Jul 2018 11:35:18 +0000

In case TCPM people missed it, the bis document for the Multipath TCP protocol is in WG last call - all comments would be very much appreciated.
https://tools.ietf.org/html/draft-ietf-mptcp-rfc6824bis-11 

Thank-you!
Phil & Yoshi

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 26 June 2018 03:33
To: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Hi,

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 05 June 2018 09:14
To: [email-masked]
Subject: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Thanks,
Phil & Yoshi

---

From: Christoph Paasch <[email-masked]>
Date: Sun, 15 Jul 2018 15:12:14 -0400

Hello Phil,

some replies to your comments inline:

> I think it should say "to ensure the exchange cannot..."

I'm not sure where you would put the "to ensure the exchange cannot..."?

However, reading through this paragraph, I think the SHOULD should rather be
a MUST, no? Because, if conflicting versions are being requested we anyways
can't do MPTCP.

> "The first 4 bits of the first octet in the MP_CAPABLE option" Fig 4 shows two earlier octets (Kind & Length) - I know these arent the MP-CAPABLE option, but I think it might be worth spelling out…

What about:

"
Like all MPTCP options, the MP_CAPABLE option starts with the Kind and Length
to specify the TCP-option kind and its length. Followed by that is the
MP_CAPABLE option. The first 4 bits of the first octed in the MP_CAPABLE option
(Figure 4) define the MPTCP option subtype ...
"

> Would it be better strictly to have "0x0" rather than "0"? (ie match the IANA registry)

I agree with that.

>    by the initial ADD_ADDR sender to retransmit the ADD_ADDR according

Fixed.

> The 'E' bit - better to say flag?

Agreed.

> Section 2.3 needs to be amended to show the reply /echo message for ADD-ADDR. I suggest the pic in S2.3 also shows the E value. 

Agreed - changed this.

> Not sure - does this open security hole, where an attacker pretends an ADD-ADDR has been received? (the Echo doesn't include an hmac, so is easy to spoof)

I don't think there is a security hole. But worth taking a closer look at.

> Could this be made more straightforward? Ie spell out when length is >10 octets ie IPv6 (if I understand it). This also seems to mean that IPv4 doesn't include the HMAC, is this right??

I'm not sure why we have the "if length > 10 octets" in here.

The HMAC should be there always. I think, we should remove the "if length >
10 octets".

I opened a first pull-request with the changes mentioned in here at
https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/pull/4.

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Sun, 15 Jul 2018 16:14:45 -0400

Hello Phil,

please see inline:

Agreed.

> I wonder if it would be more logical for this section to be before Fast close?

Hmmm... Maybe yes. On the other hand, 3.6 is not vital to full MPTCP
operation. It is purely informational, thus having the MP_FASTCLOSE before
that could make sense.

> Can this paragraph be deleted? at least I found it a bit confusing that this text is not about this section. 

Agreed - I rephrased the beginning of the section, using your suggestion
from below.

> A host sends a TCP RST in order to close a subflow or reject an attempt to open a subflow (MP_JOIN). In order to inform the receiving host why a subflow is being closed or rejected, the TCP RST pac…

I used your text.

Agreed.

Fixed.

> I guess it's OK to refer to it as RST rather than TCP RST?

I think so. What do others think?

> "specialised" - maybe "specific" [is it really specialised?]

I'm actually not sure what this paragraph is adding to the section.
Should we simply remove it?

> is two subflows correct? Multiple I think

It would be two subflows on each path (one being initiated by each host).

> "time waited" - ugly phrase?

Changed to "time to wait"

> "need not apply" -> is not appropriate

Both fixed.

> Discuss downgrade attacks (to v0) during connection initiation

Will work on those.

> This document obsoletes rather than updates...   I suspect this means all the MPTCP options need to be included in this document, but we can leave IANA to decide that. 

Changed "updates" to "obsoletes".

> I don't think this is a good summary of flag C's meaning.

Changed to "Don't reuse source address for new subflows."

>    contents of this sub-registry are to to this document, as follows:

I don't think so.

> Maybe rephrae the title as "TCP Fast Open and MPTCP" or "Considerations about using TCP Fast Open and MPTCP"?

Picked "TCP Fast Open and MPTCP".

> " gaining one RTT"?  maybe: which allows data to be sent one RTT earlier than with regular TCP. 

Changed.

> re-phrase (I think data is referring ot the cookie - but you havent yet mentioned it - rather than user data)

'data' here is referring to user data. I changed it to "with the application's data", to make it more clear.

> "resp." = ?

Changed to respectively.

Fixed.

> How do you guarantee this? Not sure I understand. 

I rephrased this to say that TFO is only ever used on the initial subflow
before any other subflows are attempted.

> Is this a SHOULD NOT?  Does it imply a change to rfc7413 (the tfo rfc)? I guess this is really a suggestion to the implementer?

I think, it should even be a "MUST NOT". Otherwise, implementations won't
interoperate. As this is in the appendix, I change it to "must not".

> I assume this sub-section is an example (so no SHOULD or even MAY are needed)

Indeed - changed the MUST to a must.

>      |    S 0(0) <MP_CAPABLE>, <TFO cookie request>                 |

tcpdump-style sequence number and length of the segment.

Is it ok, if I change the figure for the first time we use this notation to:
S Seq=0(Length=0)

And from there on leave the one we have now?

> Need to add sub-sections on closing subflow & fast close & fallback

I will work on those.

> S2.7 - I think the third bullet about notable features could be slightly altered, with enhanced security for ADD_ADDR and MP_CAPABLE. You could also add a mention of rfc7430

Yes, we should do this.

> Section 2.1 & 3.1 - can you remind me why we changed MP_CAPABLE so that Host A's key is sent only on the ACK (not on the SYN)? Is this so it's only sent when we know host B is MPTCP capable, which…

The reason for this is because now that we send the keys reliably in the
third ACK / first data-segment, we actually don't need to send the key in
the SYN anymore. Thus, we can reduce the size of the MP_CAPABLE in the
SYN-segment and gain some bytes.

I created another pull-request for review with most of the minor edits at https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/pull/5

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Sun, 15 Jul 2018 16:21:04 -0400

Hello,

> Apart from the pure clarifications, there are a few comments from me and Yoshi that may be worth discussing:

I agree - that might be quite useful.

I will work this week on getting this sorted out.

I don't think so, but worth discussing.

I am working with Apple's legal team to get the IPR added to RFC6824bis. ETA
is end of July.

Christoph

---

From: <[email-masked]>
Date: Mon, 16 Jul 2018 14:49:45 +0000

In-line

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 15 July 2018 15:12
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Hello Phil,

However, reading through this paragraph, I think the SHOULD should rather be a MUST, no? Because, if conflicting versions are being requested we anyways can't do MPTCP.

[phil] re-reading, I think I mis-interpreted the phrase "Given the SYN exchange is different between v1 and v0  the exchange cannot be immediately downgraded". -- I think you're saying "A downgrade attack is not immediately possible because the SYN exchange is different in v0 and v1". I don't understand - the downgrade attack is possible, in what sense isn't it immediately possible" ? 

Re SHOULD vs MUST. Are there circumstances when you might want to use mptvp v0 rather than use tcp?. Perhaps where an operator has control of both end points (even in a proxy scenario)

---

From: Christoph Paasch <[email-masked]>
Date: Mon, 16 Jul 2018 11:30:45 -0400

In-line

> [phil] re-reading, I think I mis-interpreted the phrase "Given the SYN exchange is different between v1 and v0  the exchange cannot be immediately downgraded". -- I think you're saying "A downgrade…

The paragraph is not really about attackers trying to downgrade an
MPTCP-version. What this part is describing is when the client supports v1
(and requests v1 in the SYN), while the server only supports v0.

In that case, the server would reply with a SYN/ACK and MP_CAPABLE of
version 0. The client then needs to fallback to regular TCP as the two
versions are not compatible. (because the SYN-exchange has changed).

Re-re-reading again as well, I realize that actually the above would not
happen. If a server only supports v0, he will ignore the MP_CAPABLE option
in the SYN, because its size is not what it expects.

I suggest the following text for this whole paragraph. Please let me know if
that looks good:

   The MP_CAPABLE exchange in this specification (v1) is different to
   that specified in v0 [RFC6824].  If a host supports multiple versions
   of MPTCP, the sender of the MP_CAPABLE option SHOULD signal the
   highest version number it supports.  The passive opener, on receipt
   of this, will signal the version number it wishes to use, which MUST
   be equal to or lower than the version number indicated in the initial
   MP_CAPABLE.
   There is a caveat though with respect to this version negotiation with
   old servers that only support v0. A server that supports v0 expects that
   the MP_CAPABLE option in the SYN-segment includes the client's key. If a
   client however already upgraded to v1, it won't include the key in the
   SYN-segment. Thus, the server will ignore the MP_CAPABLE of this
   SYN-segment and reply with a SYN/ACK that does not include an MP_CAPABLE.
   The client may then retry a new connection with MPTCP v0. Additionally,
   the client can cache this information for future connections.

> Re SHOULD vs MUST. Are there circumstances when you might want to use mptvp v0 rather than use tcp?. Perhaps where an operator has control of both end points (even in a proxy scenario) 

Yes, I think so that one would rather use MPTCP v0 than use TCP.

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Tue, 17 Jul 2018 14:23:34 -0400

Forwarding to mptcp-dev:

Please if you have time and interest, take a look at the RFC 6824bis draft
(https://tools.ietf.org/html/draft-ietf-mptcp-rfc6824bis-11) and send your
comments, suggestions, feedback to the IETF mailing-list.

Cheers,
Christoph

---

From: <[email-masked]>
Date: Tue, 17 Jul 2018 21:02:32 +0000

Christoph,
I think that basically works.

I think we could expand this a bit?
The client MAY re-try a new connection with MPTCP v0 or with TCP, based on considerations such as .... <importance of using multiple paths, danger of MPTCP downgrade attack...> Additionally the client MAY cache this information for future connections. 

Also, the document mostly uses the terms initiator and listener  (client & server as just in the TFO Appendix).

phil

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 16 July 2018 11:31
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

In-line

> [phil] re-reading, I think I mis-interpreted the phrase "Given the SYN exchange is different between v1 and v0  the exchange cannot be immediately downgraded". -- I think you're saying "A downgrade…

The paragraph is not really about attackers trying to downgrade an MPTCP-version. What this part is describing is when the client supports v1 (and requests v1 in the SYN), while the server only supports v0.

In that case, the server would reply with a SYN/ACK and MP_CAPABLE of version 0. The client then needs to fallback to regular TCP as the two versions are not compatible. (because the SYN-exchange has changed).

Re-re-reading again as well, I realize that actually the above would not happen. If a server only supports v0, he will ignore the MP_CAPABLE option in the SYN, because its size is not what it expects.

I suggest the following text for this whole paragraph. Please let me know if that looks good:

> Re SHOULD vs MUST. Are there circumstances when you might want to use 
> mptvp v0 rather than use tcp?. Perhaps where an operator has control 
> of both end points (even in a proxy scenario)

Christoph

---

From: <[email-masked]>
Date: Tue, 17 Jul 2018 23:30:15 +0000

In-line
Thanks
phil
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 15 July 2018 16:15
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

...

[phil] ok. Can the para be re-phrased so it's clear this is about a new situation (compared to the earlier paras). Something like, "Another issue is that both communicating hosts may simultaneously try to set up a subflow between the same pair of addresses. This leads to an inefficient use of resources." (I suppose 'inefficient resources'  refers to resources on the end host?)

...

[phil] yes - I suggest in the first figure you add a key that explains the notation and states it's used in Figures x, y, z.

> emails, if that's easier)

The reason for this is because now that we send the keys reliably in the third ACK / first data-segment, we actually don't need to send the key in the SYN anymore. Thus, we can reduce the size of the MP_CAPABLE in the SYN-segment and gain some bytes.

[phil] ok - if I get it right the motivation for the change in the MP_CAPABLE protocol exchange starts from the observation that option space is most constrained in the SYN, so reducing the size of MP_CAPABLE in the SYN allows other TCP options to be used in the SYn. As a consequence of this decision, Host A doesn't know that Host B has learnt its key, unless the ACK is sent reliably.  

I think it would be useful to tweak the figure in Section 2.1 - just to say the ack is delivered reliably. 
However in Section 3.1 I think it would be good to include a more extensive figure about this - showing the different ways & timings for when the ack is known to have been delivered - ie figure showing somehow that A or B may be one that first sends data - and therefore when A knows that the ACK has indeed been successfully delivered to B.
The actual text (in the para starting " If B has data to send first, then the reliable delivery of the ACK...") is pretty good - however, there are a lot of "this packet", "this is" etc - likely that someone could misinterpret what a "this" refers to. A figure would help clarify and/or breaking the text up .

Also I spotted
<< This option is used to declare the 64-bit keys that the end hosts     
    have generated for this MPTCP connection.  This key is used to     
    authenticate the addition of future subflows to this connection.>>
This key is -> These keys are ...

<< Therefore, A's key must be delivered reliably to B, and in order to     
    do this, the transmission of this packet must be made reliable.>>
is it right to say that the protocol makes transmission of the ACK reliable?   Suppose the Ack is lost. The protocol ensures that A doesn't have the mistaken belief that B has got the Ack. But does it say anything about how /when A sends the Ack again?  (perhaps it should say something about this?)

<< Note that new subflows MUST NOT be established
   (using the process documented in Section 3.2) until a Data Sequence
   Signal (DSS) option has been successfully received across the path
   (as documented in Section 3.3).>>
In the case where A sends data first, then the MP-CAPABLE is sent instead of the DSS. Does the text quoted mean that new subflows are not allowed (until an actual DSS is sent) or are they allowed because DSS has kind of been inferred?

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 18 Jul 2018 10:13:55 -0400

Hello Phil,

> The client MAY re-try a new connection with MPTCP v0 or with TCP, based on considerations such as .... <importance of using multiple paths, danger of MPTCP downgrade attack...> Additionally the cli…

thanks I expanded it in your suggested way.

> Also, the document mostly uses the terms initiator and listener  (client & server as just in the TFO Appendix).

Changed client/server to initiator/listener in the TFO appendix.

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 18 Jul 2018 10:45:28 -0400

Hello Phil,

> [phil] ok. Can the para be re-phrased so it's clear this is about a new situation (compared to the earlier paras). Something like, "Another issue is that both communicating hosts may simultaneously…

thanks for the suggested text. I adopted it.

> [phil] yes - I suggest in the first figure you add a key that explains the notation and states it's used in Figures x, y, z.

Sure, I annotated the figure.

> [phil] ok - if I get it right the motivation for the change in the MP_CAPABLE protocol exchange starts from the observation that option space is most constrained in the SYN, so reducing the size of…

The story actually unfolded the other way around. We identified a need for
reliable key-exchanges when talking to stateless servers. Based on this we
made the key-exchange reliable in the third ACK/first data segment.

Upon this, we realized that we can gain space in the SYN.

Thus, the motivation for the change was the need for the reliable
MP_CAPABLE.

> I think it would be useful to tweak the figure in Section 2.1 - just to say the ack is delivered reliably. 

Incorporating it into the figure seems tricky to me. I changed the first
sentence to mention data as well: "... initial ACK (and data) packets also ..."

> The actual text (in the para starting " If B has data to send first, then the reliable delivery of the ACK...") is pretty good - however, there are a lot of "this packet", "this is" etc - likely th…

Agreed - I will try to work on a proposal for this.

Fixed.

> is it right to say that the protocol makes transmission of the ACK reliable?   Suppose the Ack is lost. The protocol ensures that A doesn't have the mistaken belief that B has got the Ack. But does…

There is not really reliability of the ACK. Rather, piggy-backing of the
MP_CAPABLE on a data-segment, which implicitly makes the MP_CAPABLE reliable
thanks to the data's reliability.

I try to clarify this with the figure I will do.

> In the case where A sends data first, then the MP-CAPABLE is sent instead of the DSS. Does the text quoted mean that new subflows are not allowed (until an actual DSS is sent) or are they allowed b…

It is allowed. It is not so much about the DSS-option itself, but rather any
MPTCP-option that is sent together with data.
I will try to clarify this as well.

I have another comment. In Section 3.1, there is:

If receiving a message with the 'B' flag set to 1, and this is not understood,
then this SYN MUST be silently ignored.

I suggest to change this to "... then the MP_CAPABLE in this SYN MUST be
silently ignored.". Otherwise, hosts setting the B-bit to test for a
feature-capability will incur a SYN-retransmission timeout, which is a huge
latency penalty.

Please let me know if that is acceptable.

I created another Pull-Request at
https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/pull/6.

To not lose track of outstanding tasks, I listed them in the issues-section
of the github: https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/issues

If anyone feels inspired to take onto one of these ... ;-)

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 18 Jul 2018 10:53:45 -0400

Hello Phil,

this summary looks good to me.

Cheers,
Christoph

---

From: <[email-masked]>
Date: Wed, 18 Jul 2018 15:47:43 +0000

Something that needs fixing which Christoph and I just noticed during a chat at the IETF:

S3.7 on Fallback has 
<<Note that this rule essentially prohibits the sending of data on the
   third packet of an MP_CAPABLE or MP_JOIN handshake, since both that
   option and a DSS cannot fit in TCP option space.>>
this needs a mod now data is allowed on the third packet.

.....

> [phil] ok - if I get it right the motivation for the change in the MP_CAPABLE protocol exchange starts from the observation that option space is most constrained in the SYN, so reducing the size of…

The story actually unfolded the other way around. We identified a need for reliable key-exchanges when talking to stateless servers. Based on this we made the key-exchange reliable in the third ACK/first data segment.

Thus, the motivation for the change was the need for the reliable MP_CAPABLE.

> I think it would be useful to tweak the figure in Section 2.1 - just to say the ack is delivered reliably. 

Incorporating it into the figure seems tricky to me. I changed the first sentence to mention data as well: "... initial ACK (and data) packets also ..."

[phil2] the figure in S2.1 (ie in the Operation Overview section) should be modified. As it is compulsory to send the MP_CAPABLE ACK message on the first data packet. Sending MP_CAPABLE ACK without data is done if host A doesn't have data to send, but you still have to send ACK later with the first data. Whereas the natural way to interpret the current fig in S2.1 is that the Ack is sent once.

......

If receiving a message with the 'B' flag set to 1, and this is not understood, then this SYN MUST be silently ignored.

I suggest to change this to "... then the MP_CAPABLE in this SYN MUST be silently ignored.". Otherwise, hosts setting the B-bit to test for a feature-capability will incur a SYN-retransmission timeout, which is a huge latency penalty.

[phil2] fine by me - please make it clear this means fallback to TCP (as in similar text for other cases where we fallback)

---

From: Christoph Paasch <[email-masked]>
Date: Thu, 19 Jul 2018 10:27:34 -0400

> this needs a mod now data is allowed on the third packet.

Created
https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/issues/12.

> [phil2] the figure in S2.1 (ie in the Operation Overview section) should be modified. As it is compulsory to send the MP_CAPABLE ACK message on the first data packet. Sending MP_CAPABLE ACK without…

I will rework this as I review the whole MP_CAPABLE exchange.

> [phil2] fine by me - please make it clear this means fallback to TCP (as in similar text for other cases where we fallback)

Incorporated it into

Cheers,
Christoph

---

From: Alan Ford <[email-masked]>
Date: Thu, 19 Jul 2018 20:29:03 +0100

Hi Phil,

Catching up on these comments, re this:

> On 18 Jul 2018, at 00:30, [email-masked] wrote:
> 

I think the text still holds. We need a DSS to be received (as a DATA ACK) in order to know the path is fully MPTCP capable.

Regards,
Alan

---

From: <[email-masked]>
Date: Thu, 19 Jul 2018 20:14:42 +0000

A small one - S1.5 requirements language - there is new text to use, see https://tools.ietf.org/html/rfc8174

---

From: <[email-masked]>
Date: Thu, 19 Jul 2018 20:20:00 +0000

I'll leave it to you & Christoph to work on some clearer text

-----Original Message-----
From: Alan Ford [mailto:[email-masked]] 
Sent: 19 July 2018 15:29
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [multipathtcp] WG Last Call for draft-ietf-mptcp-rfc6824bis

Hi Phil,

Regards,
Alan

---

From: Christoph Paasch <[email-masked]>
Date: Thu, 26 Jul 2018 11:03:12 -0700

> A small one - S1.5 requirements language - there is new text to use, see https://tools.ietf.org/html/rfc8174 

Updated it - cfr.: https://github.com/multipath-tcp/draft-ietf-mptcp-rfc6824bis/pull/14

Christoph