Subject: Re: [multipathtcp] potential MPTCP proxy charter item

From: <[email-masked]>
Date: Wed, 2 Nov 2016 10:24:35 +0000

Hi Alan, 

Please see inline. 

Cheers,
Med

> -----Message d'origine-----
> De : Alan Ford [mailto:[email-masked]]
> Envoyé : samedi 29 octobre 2016 12:02
> À : BOUCADAIR Mohamed IMT/OLN
> Cc : [email-masked]
> Objet : Re: [multipathtcp] potential MPTCP proxy charter item
> 
> Hi Med,
> 
> Thank you for updating these drafts, I have been awaiting these with great
> interest!

[Med] Cool, thanks.

> 
> Re draft-nam-mptcp-deployment-considerations-00:
> - The three use cases in S3 make sense

[Med] OK. 

> - But in S4, the two deployment scenarios, focuses purely on Scenarios 1 &
> 3, not 2
> 

[Med] The only UC#2-related deployment scenario I'm aware of is the datacenter case, where MPTCP is used inside the DC. This deployment scenario is not listed there because I thought it cannot be considered as ** network-assisted MPTCP ** deployment scenario.   

> In S5.2.1, the usage model is that data is explicitly targeted to the MCP.

[Med] Yes.

> in S5.2.2, implicit, it only works for Scenario 1 of S4.

[Med] It works for both scenarios of S4.

 By the diagrams,
> it needs to now that a destination is not MPTCP capable - how does it have
> this knowledge?

[Med] The default behavior is that it does convert the MPTCP connection into a TCP connection without requiring any knowledge about the destination. This default behavior is motivated by the current support of MPTCP at the servers side. The text can deal with optimization like the one you suggested, but we left those out of scope for the moment.  

 Should it actually be intercepting the SYN/ACK from RM and
> making adjustments there?

[Med] See above. (various optimization triggers can be considered: inspect SYN/ACK, configured servers whitelist, etc.)

> 
> This continues in the 5.2.2.1 discussion - what is the purpose of
> MP_CONVERT in the implicit mode?

[Med] It is there to distinguish native MPTCP connections that can be established from an MPTCP host from those that are required to be proxied. If the option is not included, this flow will experienced:

    _________________MULTIPLE PATH CLIENT/SERVER___________
   /                                                       \
                |                              |
    |           |                              |           |
    |src:c2@    |src:uMCP@1             dst:s1@|src:uMCP@1 |
    |<=MPTCP====|=============================>|<-TCP Leg->|
    |    dst:s2@|                              |    dst:s2@|
    |           |                              |           |
   +--+      +-----+                         +-----+      +--+
   |C2|      | MCP |                         | MCP |      |S2|
   +--+      +-----+                         +-----+      +--+
             Upstream                     Downstream

       Example of a Broken E2E MPTCP Connection (On-path)
 
While we wanted to achieve this:

    _________________MULTIPLE PATH CLIENT/SERVER___________
   /                                                       \
                |                              |
    |           |                              |           |
    |src:c2@    |src:uMCP@1             dst:s1@|src:uMCP@1 |
    |<==========|============MPTCP========================>|
    |    dst:s2@|                              |    dst:s2@|
    |           |                              |           |
   +--+      +-----+                         +-----+      +--+
   |C2|      | MCP |                         | MCP |      |S2|
   +--+      +-----+                         +-----+      +--+
             Upstream                      Downstream

     Example of a Successful E2E MPTCP Connection (On-path)

 Is it just to say “if there’s an MCP on
> this path, I would like it“? This seems different from the use of it as
> described in the plain mode draft (where it appears to be used for
> explicit mode by signalling the target IP address).

[Med] This use is compliant with the plain mode draft: the target IP address is optional. 

 You talk about
> differentiating between “native” and “proxied” MPTCP connections, but how
> would the initiator know this in implicit mode?

[Med] An initiator located behind a CPE does not need to know that.  

> 
> I think this is a valuable document which explains how proxies can be used
> to extend the benefits of MPTCP to connections where one or both end hosts
> are not MPTCP-capable. It think this fits with the proxy charter item
> today.

[Med] Thank you.

> 
> 
> Re draft-boucadair-mptcp-plain-mode-09
> 
> S3.1 - two issues:
> 
> 1) How does the client know the server does not support MPTCP and thus
> should be going through the proxy? Or does everything go through the
> proxy?
> (This is similar to the question in implicit mode of how does the MCP know
> the server does not support MPTCP?)

[Med] Optimization related to the support of MPTCP at the server side are optimizations that are left out of scope for the time being.  

> 
> 2) Why make MP_CONVERT appear to be be part of MPTCP, when it’s not? In S4
> you talk about it being included in the SYN payload - therefore it’s part
> of the TCP payload. This is fine and aligns with what I’ve been saying
> since IETF95 - the proposed protocol is not MPTCP-specific, and is simply
> signalling the target at the start of the TCP payload.

[Med] The option is likely to be included in the payload because of the limited TCP option space. Supplying long options in MPTCP SYN is a generic problem that is worth to be solved by the mptcp wg. A pragmatic approach is to grab some space in the payload to extend the TCP option space. For the sake of interoperability, a meaning will be associated with a flag of MP_CAPBALE to indicate that other options are present in the payload. We are building on that capability.

 This does not need
> to be a MPTCP option to make this happen.
> 
> To be clear - again - I have no issue with this work being done in MPTCP
> WG, it is “how to use proxies to increase deployment of MPTCP”. But this
> particular document is not MPTCP-specific;

[Med] It is MPTCP-specific for various reasons, e.g.,:
* it solves an MPTCP problem (how to make use of MPTCP of one or both peers are not MPTCP compatible). 
* it is deigned to not break native MPTCP connections
* it solves current MPTCP deployments problems with SOCKS.

 whether it is published here or
> elsewhere I don’t care, but it really shouldn’t’ be written as if it’s
> extending the MPTCP spec when there is no reason for it to be.
> 
> Finally, you earlier mentioned the possible effect on the base spec
> regarding a flag, but I see no mention of it in these docs.

[Med] This is indicated as a discussion note in Section 4: 

      DISCUSSION NOTE: ADD DETAILS ABOUT THE NEED FOR AN EXPLICIT SIGNAL
      THAT MPTCP OPTIONS ARE INCLUDED IN THE SYN PAYLOAD

We agreed among authors to initiate a thread on this particular point. 

> 
> Regards,
> Alan
> 
> 
> > On 28 Oct 2016, at 07:44, [email-masked] wrote:
> >
> > Hi Phil,
> >
> > I remember, as well :-)
> >
> > The drafts were completely rewritten to take into considerations the
> comments and also to help the chartering effort. Here is the set of the
> Network-Assisted MPTCP documents:
> >
> > * An MPTCP Option for Network-Assisted MPTCP:
> https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09
> > * Network-Assisted MPTCP: Use Cases, Deployment Scenarios and
> Operational Considerations: https://tools.ietf.org/html/draft-nam-mptcp-
> deployment-considerations-00
> > * DHCP Options for Network-Assisted Multipath TCP:
> https://tools.ietf.org/html/draft-boucadair-mptcp-dhc-06
> > * RADIUS Extensions for Network-Assisted Multipath TCP:
> https://tools.ietf.org/html/draft-boucadair-mptcp-radius-03
> >
> > Questions, comments and suggestions are more than welcome!
> >
> > Cheers,
> > Med
> >
> >> -----Message d'origine-----
> >> De : multipathtcp [mailto:[email-masked]] De la part de
> >> [email-masked]
> >> Envoyé : mercredi 26 octobre 2016 10:13
> >> À : [email-masked]; JACQUENET Christian IMT/OLN
> >> Cc : [email-masked]
> >> Objet : Re: [multipathtcp] potential MPTCP proxy charter item
> >>
> >> Hi,
> >> I remember in Berlin discussion about a new /merged version of Med and
> >> Olivier's drafts - with a nice description of the protocol model to
> make
> >> it easier to understand (both the high-level approach, and the
> assumptions
> >> - especially as these are different to those of RFC6824). I think this
> >> would make it easier to write text for a new charter item. (at least my
> >> attempt to write a new charter item a while back was inaccurate
> >> /inadequate).  Is this possible please?
> >> I agree we want this work to happen - it's great to see MPTCP being
> widely
> >> used and we want standardisation.
> >>
> >> Best wishes
> >> Phil.
> >>
> >> -----Original Message-----
> >> From: multipathtcp [mailto:[email-masked]] On Behalf Of
> >> Mirja Kühlewind
> >> Sent: 25 October 2016 13:07
> >> To: [email-masked]
> >> Cc: [email-masked]
> >> Subject: Re: [multipathtcp] potential MPTCP proxy charter item
> >>
> >> Hi all, hi Christian,
> >>
> >> see below.
> >>
> >>> Am 20.10.2016 um 08:50 schrieb [email-masked]:
> >>>
> >>> WG,
> >>>
> >>> Let me second Med's recent replies to Yoshi, Mirja and Alan.
> >>>
> >>> I too believe the mptcp WG is the right placeholder to work on an
> MPTCP
> >> extension that can accommodate use cases which are currently being
> >> investigated, marketed and - yes - deployed (for some of them, with
> their
> >> procession of already-documented drawbacks (TCP-only communications)
> and
> >> flaws, like QoS-affecting signaling delays) by some operators.
> >>>
> >>> These use cases mainly fall under the so-called hybrid access service
> >> offering umbrella, so that end-users can take advantage of various
> access
> >> network technologies for the sake of optimized resource usage, among
> other
> >> incentives.
> >>
> >> I agree that these use case are interesting and work is needed here.
> See
> >> further below.
> >>
> >>>
> >>> I also believe MPTCP is the right solution to address such use cases.
> >>
> >> As I mention previously there are two very different deployment
> scenarios:
> >> a connection-terminating MPTCP proxy or a MPTCP tunnel solution. In
> >> general using TCP as a tunneling technology is challenging and has a
> >> broader scope than just MPTCP.
> >>
> >>
> >>> As I already mentioned a while ago, working on an extension that can
> >> facilitate the establishment of MPTCP connections in a network-assisted
> >> fashion as we call it in the current Plain Transport Mode option draft
> >> will not hinder MPTCP massive adoption but rather encourage it.
> >>>
> >>> I also know there are already prototype implementations of such
> >> extension that are currently being tested.
> >>>
> >>> I therefore support the adoption of the MPTCP proxy charter item by
> the
> >> mptcp WG, as this work addresses very concrete and short term use cases
> >> that have been publicized by several operators.
> >>>
> >>> And I think that this WG should start working asap on the set of
> >> deliverables mentioned by Med in his recent response to Mirja: MPTCP
> >> extension specification, deployment scenarios and operational
> >> considerations and DHCP/RADIUS-based provisioning means.
> >>
> >> An MPTCP extension specification is clearly in scope for MPTCP and can
> be
> >> done right now without a charter change.
> >>
> >> I’m not sure if deployment scenarios need to be documented (in an RFC).
> As
> >> you said above there are already prototype and some real deployments as
> >> well. Not sure why we need additional documentation here.
> >>
> >> Operational consideration might or might not below in the MPTCP working
> >> group. However, there is the BANANA BoF in Seoul that will discuss
> hybrid-
> >> access services (and different approach to provide this service).
> >> Depending where this effort goes that might be a better home or some wg
> in
> >> intarea.
> >>
> >> DHCP/RADIUS extension go into the DHC working group. They are charter
> to
> >> specify these kind of extensions and you can start this work there
> >> immediately.
> >>
> >> I’m not saying we shouldn’t do this work; I’m just saying that MPTCP
> might
> >> not be the right group for all parts of this work. This is my
> >> recommendation as transport AD but I’m sure a rechartering discussion
> in
> >> the IESG would provide similar feedback.
> >>
> >> Mirja
> >>
> >>
> >>
> >>>
> >>> Cheers,
> >>>
> >>> Christian.
> >>>
> >>> -----Message d'origine-----
> >>> De : multipathtcp [mailto:[email-masked]] De la part de
> >>> [email-masked] Envoyé : jeudi 20 octobre 2016 08:15 À :
> >>> Yoshifumi Nishida; [email-masked] Cc :
> >>> [email-masked] Objet : Re: [multipathtcp] potential MPTCP proxy
> >>> charter item
> >>>
> >>> Hi Yoshi,
> >>>
> >>> The discussion I had with many operators is that they would like to
> see
> >> both TCP and UDP addressed with the ** same solution**. This is why we
> >> came up with the proposal to leverage MPTCP for other protocols (UDP,
> >> mainly).
> >>>
> >>> So, what I would ideally like to see in the mptcp WG proposed charter
> is
> >> (with "X" being MPTCP):
> >>>
> >>>> 1: Protocol X proxy:
> >>>>   It speaks protocol X on behalf of communicating endpoints .
> >>>>   From one end's point of view, it looks as if the other end speaks
> >>>> protocol X.
> >>>
> >>> And
> >>>
> >>>> 3:  Protocol X-Y / Protocol Y-X gateway
> >>>>    Convert protocol X into protocol Y, or convert protocol Y into
> >>>> protocol X for protocol connectivity/compatibility, etc.
> >>>
> >>> I know Olivier/Stefano have a better name to denote both these cases.
> >>>
> >>> Cheers,
> >>> Med
> >>>
> >>>> -----Message d'origine-----
> >>>> De : Yoshifumi Nishida [mailto:[email-masked]] Envoyé :
> >>>> mercredi 19 octobre 2016 22:57 À : [email-masked]
> >>>> Cc
> >>>> : Mirja Kühlewind; BOUCADAIR Mohamed IMT/OLN; [email-masked]
> >>>> Objet : Re: [multipathtcp] potential MPTCP proxy charter item
> >>>>
> >>>> Hello,
> >>>>
> >>>> On Wed, Oct 19, 2016 at 12:42 PM, Olivier Bonaventure
> >>>> <[email-masked]> wrote:
> >>>>> Mirja,
> >>>>>>
> >>>>>>
> >>>>>> there are two cases to distinguish here:
> >>>>>>
> >>>>>> 1) you have one or two MPTCP proxies that terminate the TCP
> >>>>>> connection
> >>>> and
> >>>>>> open a new MPTCP connection
> >>>>>
> >>>>>
> >>>>> There is a very clear demand for this type of solution and there are
> >>>> various
> >>>>> implementations that are available or are being developped. Several
> >>>>> deployments exist and there is a large demand for this type of
> >> services.
> >>>> It
> >>>>> would be silly for the IETF to ignore this use case after having
> >>>>> spent
> >>>> years
> >>>>> to specify the Multipath TCP protocol.
> >>>>
> >>>> I also think we should not ignore the use case.
> >>>> But, I am somehow thinking that we might want to clarify the
> >>>> terminologies a bit more so that we can have more clear focus.
> >>>>
> >>>> I might be wrong, but these words give me the following impressions.
> >>>>
> >>>> 1: Protocol X proxy:
> >>>>   It speaks protocol X on behalf of communicating endpoints .
> >>>>   From one end's point of view, it looks as if the other end speaks
> >>>> protocol X.
> >>>>
> >>>> 2: Tunneling over protocol X
> >>>>    Protocol X carries other protocol Y packets (by using
> >>>> encapsulation)  All info in protocol Y (headers, etc) will be
> >>>> preserved.
> >>>>
> >>>> 3:  Protocol X-Y / Protocol Y-X gateway
> >>>>    Convert protocol X into protocol Y, or convert protocol Y into
> >>>> protocol X for protocol connectivity/compatibility, etc.
> >>>>    Some info in the original protocol might be lost due to the
> >>>> conversion.
> >>>>
> >>>> If I follow these impressions, it seems that some proposals are close
> >>>> to 3. (again, I might be wrong) I might want to check if we want to
> >>>> clarify this point in the charter or leave it ambiguous.
> >>>> I am just wondering if we might need to clarify the terminology or
> >>>> adding some texts to avoid miscommunications.
> >>>>
> >>>> Thanks,
> >>>> --
> >>>> Yoshi
> >>> _______________________________________________
> >>> multipathtcp mailing list
> >>> [email-masked]
> >>> https://www.ietf.org/mailman/listinfo/multipathtcp
> >>>
> >>> ______________________________________________________________________
> >>> ___________________________________________________
> >>>
> >>> Ce message et ses pieces jointes peuvent contenir des informations
> >>> confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
> >>> exploites ou copies sans autorisation. Si vous avez recu ce message
> >>> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi
> que
> >> les pieces jointes. Les messages electroniques etant susceptibles
> >> d'alteration, Orange decline toute responsabilite si ce message a ete
> >> altere, deforme ou falsifie. Merci.
> >>>
> >>> This message and its attachments may contain confidential or
> >>> privileged information that may be protected by law; they should not
> be
> >> distributed, used or copied without authorisation.
> >>> If you have received this email in error, please notify the sender and
> >> delete this message and its attachments.
> >>> As emails may be altered, Orange is not liable for messages that have
> >> been modified, changed or falsified.
> >>> Thank you.
> >>>
> >>> _______________________________________________
> >>> multipathtcp mailing list
> >>> [email-masked]
> >>> https://www.ietf.org/mailman/listinfo/multipathtcp
> >>
> >> _______________________________________________
> >> multipathtcp mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/multipathtcp
> >> _______________________________________________
> >> multipathtcp mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/multipathtcp
> > _______________________________________________
> > multipathtcp mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Alan Ford <[email-masked]>
Date: Wed, 2 Nov 2016 14:06:57 +0000

Hi Med,

> [Med] The only UC#2-related deployment scenario I'm aware of is the datacenter case, where MPTCP is used inside the DC. This deployment scenario is not listed there because I thought it cannot be c…

It’s in a document entitled “Network-Assisted MPTCP” :-)

But in all seriousness it could be just as network-assisted as the other scenarios, if it were to transparently intercept the traffic.

> [Med] It works for both scenarios of S4.

OK, the diagram suggests only Scenario 1, but I can see how Scenario 3 would work as well. Could be worth adding another diagram.

>    |<=MPTCP====|=============================>|<-TCP Leg->|

OK, so this is saying “I’ve already been proxied, don’t proxy me again”.

However, I don’t understand the scenario you’ve drawn above. In the “broken” case, the downstream MCP is converting MPTCP to TCP. But what you ideally want is a direct case to the destination? But you’ve already said above that the sender has no knowledge of whether the far end is MPTCP capable or not, so the downstream MCP should default to TCP (i.e. the “broken” case).

And furthermore, why is the client talking MPTCP, and being translated by an upstream MCP, when it’s already MPTCP? It’s either going to be explicit - and addressed to the MCP - or implicit, and intercepted - and in either way by the spec it will then be converted to TCP.

So what is the actual scenario where the above cases will exist?

> [Med] This use is compliant with the plain mode draft: the target IP address is optional. 

So what does that mean? Does that mean “if there’s an MCP on the path I would like to use it” ?

> [Med] An initiator located behind a CPE does not need to know that.  

So who does? The CPE needs to keep track of what data it sends to the MCP and which it sends direct?

> * it solves an MPTCP problem (how to make use of MPTCP of one or both peers are not MPTCP compatible). 

That does not affect the protocol itself.

> * it is deigned to not break native MPTCP connections

That does not make it MPTCP-specific.

> * it solves current MPTCP deployments problems with SOCKS.

That does not make it MPTCP-specific.

I feel like a broken record here but this signal does not need to be an MPTCP option because:
- It does not signal any information which is specific to the operation of MPTCP
- It does not need to be part of the MPTCP protocol in order to be implemented
- It does not need to be read or understood by any devices which are not MCPs or compliant end hosts (i.e. not by any other devices which implement MPTCP)

As specified, it sits in the TCP payload. This is the right place for it. It is a signal purely between an end host and the MCP.

Regards,
Alan

---

From: <[email-masked]>
Date: Thu, 3 Nov 2016 09:26:04 +0000

Hi Alan, 

Please see inline.

Cheers,
Med

> Envoyé : mercredi 2 novembre 2016 15:07
> > [Med] The only UC#2-related deployment scenario I'm aware of is the
> datacenter case, where MPTCP is used inside the DC. This deployment
> scenario is not listed there because I thought it cannot be considered as
> ** network-assisted MPTCP ** deployment scenario.
> 
> But in all seriousness it could be just as network-assisted as the other
> scenarios, if it were to transparently intercept the traffic.
> 

[Med] Will consider adding some text about the DC deployment scenario. 

> OK, the diagram suggests only Scenario 1, but I can see how Scenario 3
> would work as well. Could be worth adding another diagram.

[Med] will do, happily. 

> > [Med] The default behavior is that it does convert the MPTCP connection
> into a TCP connection without requiring any knowledge about the
> destination. This default behavior is motivated by the current support of
> MPTCP at the servers side. The text can deal with optimization like the
> one you suggested, but we left those out of scope for the moment.
> >
> > [Med] See above. (various optimization triggers can be considered:
> inspect SYN/ACK, configured servers whitelist, etc.)
> >
> >>
> > [Med] It is there to distinguish native MPTCP connections that can be
> established from an MPTCP host from those that are required to be proxied.
> If the option is not included, this flow will experienced:
> >

[Med] Yes.

> 
> However, I don’t understand the scenario you’ve drawn above.

[Med] The scenario is about an MPTCP-capable host located behind a CPE(MCP). This is typically an iPhone connected behind a CPE. We want that MPTCP communications issued from this host are forwarded as such to their destination without soliciting the downstream MCP. 

 In the
> “broken” case, the downstream MCP is converting MPTCP to TCP. But what you
> ideally want is a direct case to the destination?

[Med] Yes.

 But you’ve already said
> above that the sender has no knowledge of whether the far end is MPTCP
> capable or not,

[Med] I confirm. The MPTCP hosted located behind the CPE does not need to have this knowledge.

 so the downstream MCP should default to TCP (i.e. the
> “broken” case).

[Med] It will default to TCP because it has no means to distinguich an MPTCP connection initiated by an MPTCP host from an proxied MPTCP connection, because all these connections will have the same source IP address (CPE's WAN IP address).

> 
> And furthermore, why is the client talking MPTCP, and being translated by
> an upstream MCP, when it’s already MPTCP?

[Med]  The upstream MCP is on the default path (this is your CPE, typically). The upstream MCP does not touch to MPTCP signal. All what the CPE does is to translate the source IP address of the MCTCP client to an external IP address assigned to the CPE.  

 It’s either going to be explicit
> - and addressed to the MCP - or implicit, and intercepted - and in either
> way by the spec it will then be converted to TCP.

[Med] The upstream MCP is implicit; the MPCPT client is not aware there is an MCP on-path. 

> So what is the actual scenario where the above cases will exist?
[Med] I'm not sure to understand the question, but the scenario is characterized as follows:
* an MPTCP-capable client located behind a CPE that embeds an MCP.
* The upstream MCP is instructed to not alter MPTCP connections issued from that MPTCP-capable client (native MPTCP connections)
* The downstream MCP is implicit.
* The downstream will need a trigger to avoid converting native MPTCP connections to TCP.

[Med] Yes.

> So who does?

[Med] The CPE will track the connections it proxies. 

 The CPE needs to keep track of what data it sends to the MCP
> and which it sends direct?

[Med] As per its normal operations, the CPE needs to keep track of NAT sessions. Native MPTCP connections will be handled as any of these NAT sessions. The connections the MCP proxies to MPTCP will be tracked in a dedicated binding tables.

> > [Med] Optimization related to the support of MPTCP at the server side
> are optimizations that are left out of scope for the time being.
> >
> >>
> > [Med] The option is likely to be included in the payload because of the
> limited TCP option space. Supplying long options in MPTCP SYN is a generic
> problem that is worth to be solved by the mptcp wg. A pragmatic approach
> is to grab some space in the payload to extend the TCP option space. For
> the sake of interoperability, a meaning will be associated with a flag of
> MP_CAPBALE to indicate that other options are present in the payload. We
> are building on that capability.
> >
> > This does not need

[Med] It does as the base protocol asumes a deployment model when both endpoints are MPTCP-cabaple.

> That does not make it MPTCP-specific.

[Med] It does because we need a clear indicator to distinguish native MPTCP connections from proxied ones. We don't want to break e2d MPTCP, when it is possible.  

> That does not make it MPTCP-specific.

[Med] It is specific to MPTCP because it is about MPTCP deployment. That's obvious for me!

> - It does not signal any information which is specific to the operation of

[Med] We are not in the e2e MPTCP model, but "network-assisted MPTCP".

---

From: <[email-masked]>
Date: Thu, 3 Nov 2016 09:29:50 +0000

Alan, 

One clarification about this part of the message: 

==
> OK, so this is saying “I’ve already been proxied, don’t proxy me again”.

[Med] Yes.
===

Actually, I wanted to say, "NO".

Sorry.

Cheers,
Med

> [email-masked]
> Envoyé : jeudi 3 novembre 2016 10:26
> À : Alan Ford
> > >> - But in S4, the two deployment scenarios, focuses purely on
> Scenarios
> > 1 &
> > >> 3, not 2
> > >>
> > >
> [Med] The scenario is about an MPTCP-capable host located behind a
> CPE(MCP). This is typically an iPhone connected behind a CPE. We want that
> MPTCP communications issued from this host are forwarded as such to their
> destination without soliciting the downstream MCP.
> 
>  In the
> [Med] It will default to TCP because it has no means to distinguich an
> MPTCP connection initiated by an MPTCP host from an proxied MPTCP
> connection, because all these connections will have the same source IP
> address (CPE's WAN IP address).
> 
> >
> [Med]  The upstream MCP is on the default path (this is your CPE,
> typically). The upstream MCP does not touch to MPTCP signal. All what the
> CPE does is to translate the source IP address of the MCTCP client to an
> external IP address assigned to the CPE.
> 
> * The upstream MCP is instructed to not alter MPTCP connections issued
> from that MPTCP-capable client (native MPTCP connections)
> [Med] As per its normal operations, the CPE needs to keep track of NAT
> sessions. Native MPTCP connections will be handled as any of these NAT
> sessions. The connections the MCP proxies to MPTCP will be tracked in a
> dedicated binding tables.
> 
> >
> > >> 2) Why make MP_CONVERT appear to be be part of MPTCP, when it’s not?
> In
> > S4
> [Med] It does because we need a clear indicator to distinguish native
> MPTCP connections from proxied ones. We don't want to break e2d MPTCP,
> when it is possible.
> 
> >
> > - It does not signal any information which is specific to the operation
> of
> > MPTCP
>

---

From: Alan Ford <[email-masked]>
Date: Thu, 3 Nov 2016 14:16:03 +0000

Hi Med,

Comments inline...

> On 3 Nov 2016, at 09:26, [email-masked] wrote:
> 

You corrected this later to “No” … So what does it mean? In fact, “please proxy me” ?

> [Med] The scenario is about an MPTCP-capable host located behind a CPE(MCP). This is typically an iPhone connected behind a CPE. We want that MPTCP communications issued from this host are forwarde…

OK I understand this scenario now, it is special for implicit mode, since if the CPE was acting in explicit mode, it would address the packets to the MCP.

What are the benefits of using implicit mode in this scenario? This is so you can target it to an MCP without knowing the target address. Are there any other reasons why this would be used rather than explicit mode? And what sort of deployment would this scenario exist in?

> [Med] It does as the base protocol asumes a deployment model when both endpoints are MPTCP-cabaple.

Whilst the base protocol document may be written that way, there are no assumptions in the protocol that require it to be used that way, and indeed you are making use of this with your proposal.

> [Med] It does because we need a clear indicator to distinguish native MPTCP connections from proxied ones. We don't want to break e2d MPTCP, when it is possible.  

Why? This has no effect on entities which do not implement your proposal.

> [Med] It is specific to MPTCP because it is about MPTCP deployment. That's obvious for me!

I’m making a web page. It has some funny cat pictures on it. This is delivered over HTTP. Therefore I should make funny cat pictures an extension of HTTP.

That’s essentially what you’re saying here! 

My cat pictures are an application of HTTP, just like your proxying here is an application of MPTCP. It doesn’t need an extension of the protocol to be delivered.

> [Med] We are not in the e2e MPTCP model, but "network-assisted MPTCP". 

But again, this does not require an extension to MPTCP to work.

Your protocol, as specified, is a signal in the payload. There is no need to make this signal like an MPTCP option, since it does not need to be handled by the MPTCP implementation.

The one slightly more complex case here is implicit mode. However, if you want devices to be able to inspect this signal without inspecting the payload of every MPTCP SYN, IP already has a standard in the form of RAO for alerting devices - RFCs 2113 and 2711.

Regards,
Alan

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 4 Nov 2016 10:34:08 +0100

Hi all,

first, I agree with Alan that such a signal does not need to/should not be part of the MPTCP protocol. MPTCP (as TCP is) is an end2end protocol. If you have one (or two) proxies in the middle, you split up the connection into multiple new ‚end2end‘ connections. If you need additional signaling information on one of the new connections, that a question for a high-layer protocol that uses MPTCP (which is what you do, when you propose it to be part of the payload).

Second, I’m not a big fan of the a two side proxy scenario where one side simply assumes that the destination is not MPTCP-capable. This does not support MPTCP deployment but hinders native MPTCP deployment (basically ensuring that these proxies stay forever in the network and add additional complexity even if all endpoints are MPTCP-enabled one day). I guess a proxy should always first forward the MCTCP handshake and only if the reply does not support MPTCP, then termite the connection, reply the initiator accordingly and setup a new TCP to the destination. This might cause additional delay but it provides a big benefit if the destination is MPTCP-capable and supports native deployment.

Mirja

> Am 03.11.2016 um 15:16 schrieb Alan Ford <[email-masked]>:
> 
> Hi Med,
> 
> Comments inline...
>

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 09:43:26 +0000

Hi Alan,

Please see inline.

Cheers,
Med

Envoyé : jeudi 3 novembre 2016 15:16

Hi Med,

Comments inline...

On 3 Nov 2016, at 09:26, [email-masked]<mailto:[email-masked]> wrote:

-----Message d'origine-----
De : Alan Ford [mailto:[email-masked]]
Envoyé : mercredi 2 novembre 2016 15:07
À : BOUCADAIR Mohamed IMT/OLN
Cc : [email-masked]<mailto:[email-masked]>
Objet : Re: [multipathtcp] potential MPTCP proxy charter item
By the diagrams,

it needs to now that a destination is not MPTCP capable - how does it
have

this knowledge?

[Med] Yes.

You corrected this later to “No” … So what does it mean? In fact, “please proxy me” ?
[Med] The presence of the option is an indication to be proxied.

What are the benefits of using implicit mode in this scenario? This is so you can target it to an MCP without knowing the target address. Are there any other reasons why this would be used rather than explicit mode?
[Med] The document avoids on purpose to compare “implicit” vs. “explicit”; this is left to the taste of each operator. That said, the only difference I see compared to the explicit mode for the case we are discussing, is that the CPE does not need to modify the initial subflow to be directly sent to the downstream MCP. I don’t consider that to be a “benefit” because this rewriting will be done for subsequent flows.

And what sort of deployment would this scenario exist in?
[Med] A deployment scenario would be a cluster of MCPs at the operator’s network; the MCP instance (i.e., IP address of MCP) that will be solicited to handle a given connection will be determined based on the real-time load distribution, for instance. Once an instance is elected to process a new MPTCP connection, the corresponding IP address will be advertised to steer subsequent subflows. I’m not arguing in favor of this deployment case, but I’m just sketching something that may be considered.

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 10:12:43 +0000

Hi Mirja, 

Please see inline.

Cheers,
Med

> De : Mirja Kühlewind [mailto:[email-masked]]
> Envoyé : vendredi 4 novembre 2016 10:34
> À : BOUCADAIR Mohamed IMT/OLN; Alan Ford
> first, I agree with Alan that such a signal does not need to/should not be
> part of the MPTCP protocol.

[Med] Defining another channel to carry the signal "I want to be MPTCP-proxied" outside the MPTCP normal channel will import the same set of hindrances we had with MPTCP design for middlebloxes traversal. The use of MPTCP to convey the "I want to be MPTCP-proxied" leverages on base MPTCP to detect "unfriendly MPTCP" in-path nodes. That's a pragmatic engineering design. 

 MPTCP (as TCP is) is an end2end protocol.

[Med] That's the theory. The deployment reality is as follows: 
* The network can help in some cases to establish (good) TCP connections (https://datatracker.ietf.org/doc/rfc3135/). 
* TCP proxies are deployed in many networks (mainly mobile) for other reasons such as: avoid quick retrieval of (video) content that will consume a subscriber data volume while the user won't view that content.
* MPTCP support is null at the server side. 
* There are some MPTCP-capable client, but those are still rare compared to the installed base.  

 If
> you have one (or two) proxies in the middle, you split up the connection
> into multiple new ‚end2end‘ connections. If you need additional signaling
> information on one of the new connections, that a question for a high-
> layer protocol that uses MPTCP (which is what you do, when you propose it
> to be part of the payload).

[Med] hmm..but our application is "MPTCP proxy"!

> 
> Second, I’m not a big fan of the a two side proxy scenario where one side
> simply assumes that the destination is not MPTCP-capable.

[Med] The current proposal is as follows: 
* MPTCP connections issued from MPTCP-capable hosts will be forwarded as such to their destinations without involving any MPTCP proxy.
* TCP connections that are issued from non MPTCP-capable hosts can benefit from the "Network-Assisted MPTCP". 

Can you please clarify what is "bad" with such design?

 This does not
> support MPTCP deployment but hinders native MPTCP deployment (basically
> ensuring that these proxies stay forever in the network and add additional
> complexity even if all endpoints are MPTCP-enabled one day).

[Med] The current proposals calls the following design goal:

   o  Avoids interference with native MPTCP connections.

Native connections will go direct without being reverted to TCP! Please consider reading https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09#section-7.  

 I guess a
> proxy should always first forward the MCTCP handshake and only if the
> reply does not support MPTCP, then termite the connection, reply the
> initiator accordingly and setup a new TCP to the destination. 

[Med] As I replied to Alan, this is an optimization that can be considered. Waiting for the SYN/ACK is not the only trigger that can be considered, there is also a whitelist approach. Those details can be worked out better once we have (hopefully) a WG document ;-)

This might
> cause additional delay but it provides a big benefit if the destination is
> MPTCP-capable and supports native deployment.
> 
> Mirja
> 
> 
> > OK I understand this scenario now, it is special for implicit mode,
> since if the CPE was acting in explicit mode, it would address the packets
> to the MCP.
> >
> > What are the benefits of using implicit mode in this scenario? This is
> so you can target it to an MCP without knowing the target address. Are
> there any other reasons why this would be used rather than explicit mode?
> > Whilst the base protocol document may be written that way, there are no
> assumptions in the protocol that require it to be used that way, and
> indeed you are making use of this with your proposal.
> >
> >>>
> > I’m making a web page. It has some funny cat pictures on it. This is
> delivered over HTTP. Therefore I should make funny cat pictures an
> extension of HTTP.
> >
> > My cat pictures are an application of HTTP, just like your proxying here
> is an application of MPTCP. It doesn’t need an extension of the protocol
> to be delivered.
> >
> >>> I feel like a broken record here but this signal does not need to be
> an
> >>> MPTCP option because:
> >>> - It does not signal any information which is specific to the
> operation of
> >>> MPTCP
> >>
> > Your protocol, as specified, is a signal in the payload. There is no
> need to make this signal like an MPTCP option, since it does not need to
> be handled by the MPTCP implementation.
> >
> > The one slightly more complex case here is implicit mode. However, if
> you want devices to be able to inspect this signal without inspecting the
> payload of every MPTCP SYN, IP already has a standard in the form of RAO
> for alerting devices - RFCs 2113 and 2711.
> >
> > Regards,
> > Alan
> >
> >
> >

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 10:32:32 +0000

Mirja, 

I'm replying to this point separately because I do think it is an important discussion to have.

* I see "a big benefit" for some operators to offload the MTPCP proxies if the server is MPTCP-capable.

* I don’t see "a big benefit" for the client from a QoE perspective, because they already have the MPTCP experience with the network assistance (network aggregation, path resiliency).
* I don’t see "a big benefit" for the client if the MPTCP-capable server decides to use a given path to place data, while that path is subject to a data volume quota. This means that the client will be mono-path quickly if an MPTCP-capable server adopts some traffic distribution schemes.
 
Can you please elaborate on "a big benefit" you mentioned in your message? 

Thank you.

Cheers,
Med

> Objet : Re: [multipathtcp] potential MPTCP proxy charter item
[SNIP]

I guess a
> initiator accordingly and setup a new TCP to the destination. This might

---

From: Alan Ford <[email-masked]>
Date: Fri, 4 Nov 2016 11:02:02 +0000

Hi Med,

Inline…

> On 4 Nov 2016, at 10:12, <[email-masked]> <[email-masked]> wrote:
> 

The MP_CONVERT signal has multiple purposes. When signalled with a target IP address in explicit mode, it is a pure application-layer signal, and is separate from the MPTCP connection entering the MCP.

In implicit mode this is indeed at a slightly different layer, and yes, you are asking the network to do something. But it’s still not a necessary part of the MPTCP protocol to make this happen. This is similar to a transparent HTTP proxy, or to a NAT - neither requires any changes to HTTP or TCP/IP to work.

You could detect MPTCP SYNs and see if your signal is present. If you don’t want to be inspecting every MPTCP SYN in implicit mode, use an existing standard - I suggest RAO. Given this is a controlled environment, you shouldn’t have any deployment concerns here.

If RAO is not appropriate for some reason, I might be persuaded that a flag in the MP_CAPABLE that says “proxies should inspect this” could be added to the base protocol. However, the rest of the protocol - i.e. the MP_CONVERT signal - is application-layer and outside MPTCP.

Ultimately MPTCP Proxying is an application of MPTCP and should be defined as such.

Regards,
Alan

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 4 Nov 2016 12:07:33 +0100

Alan,
>
> Thank you for updating these drafts, I have been awaiting these with great interest!
...
> To be clear - again - I have no issue with this work being done in MPTCP WG, it is “how to use proxies to increase deployment of MPTCP”. But this particular document is not MPTCP-specific; whether it is published here or elsewhere I don’t care, but it really shouldn’t’ be written as if it’s extending the MPTCP spec when there is no reason for it to be.
>
> Finally, you earlier mentioned the possible effect on the base spec regarding a flag, but I see no mention of it in these docs.

Sorry for the late reply. The main problem with the current MPTCP draft 
is the length of the TCP options in the SYN segment. With network 
assisted MPTCP, we could need to encode one or two IP addresses inside 
TCP options in the SYN segment. With IPv4, this might be possible 
depending on the other options that are used. With IPv6, this becomes 
quickly a problem.

As we redefine MP_CAPABLE when moving to RFC6824bis, I think that we 
should try to design a generic solution to allow a SYN segment to carry 
long TCP options. This should be part of the Multipath TCP standard 
specification to ensure that it is future-proof and can be extended later.

Different designs are possible and some have been discussed in the TCPM 
working group. Instead of defining a specific option to allow longer 
options, I would suggest to include this feature directly inside 
Multipath TCP. This would be cleaner and Multipath TCP is anyway the TCP 
variant that mostly requires this feature. Here are a few possible 
designs to start the discussion :

- always force the TCP SYN segment containing MP_CAPABLE to contain 
options in the SYN payload. If all options can fit in the extended TCP 
header, then the payload only contains the TCP EOL option. If there are 
TCP options in the payload, then the last option is always the EOL option.

- add a flag to MP_CAPABLE option to indicate whether the SYN payload 
contains TCP options (terminated by EOL) or not. Same as above for 
encoding the options in the SYN payload

- add a length field to MP_CAPABLE option to indicate the number of 
bytes (or words) of the SYN payload that contain TCP options. In this 
case, we do not need to terminate the options in the payload with EOL.

If RFC6824bis could include a reasonable solution that allows to place 
TCP options in the SYN payload, we would make a huge step to ensure that 
MPTCP is future-proof and that it will be possible to extend it in the 
future. In the short term, the solution could be limited to networks 
where it is known that there are no middleboxes that interfere with TCP 
options, but this would be a strong message to middlebox vendors that 
they will need to support TCP options in the SYN payload in the future.

I won't unfortunately be able to attend the next IETF meeting in Seoul 
but I'd be happy to discuss this by email

Olivier

---

From: Alan Ford <[email-masked]>
Date: Fri, 4 Nov 2016 11:23:28 +0000

Thanks for the comments, Olivier. If the working group thinks it would be a good idea, I would not be opposed to embedding some extended options support in MPTCP - as you say it seems a good chance to do so. I haven’t thought deeply yet about the various options below, all have their merits, but the flag sounds possibly easiest. Whatever extension was used, it could be particularly useful in the future for improved cryptographic handshakes.

This does not change my view, however, that MPTCP options should not be used for application-layer purposes, and that’s my main issue with MP_CONVERT as written - it’s a non-MPTCP-specific application-layer signal masquerading as a MPTCP option.

Regards,
Alan

> On 4 Nov 2016, at 11:07, Olivier Bonaventure <[email-masked]> wrote:
> 
> Alan,
>> 
> Sorry for the late reply. The main problem with the current MPTCP draft is the length of the TCP options in the SYN segment. With network assisted MPTCP, we could need to encode one or two IP addresses inside TCP options in the SYN segment. With IPv4, this might be possible depending on the other options that are used. With IPv6, this becomes quickly a problem.
> 
> As we redefine MP_CAPABLE when moving to RFC6824bis, I think that we should try to design a generic solution to allow a SYN segment to carry long TCP options. This should be part of the Multipath TCP standard specification to ensure that it is future-proof and can be extended later.
> 
> Different designs are possible and some have been discussed in the TCPM working group. Instead of defining a specific option to allow longer options, I would suggest to include this feature directly inside Multipath TCP. This would be cleaner and Multipath TCP is anyway the TCP variant that mostly requires this feature. Here are a few possible designs to start the discussion :
> 
> - always force the TCP SYN segment containing MP_CAPABLE to contain options in the SYN payload. If all options can fit in the extended TCP header, then the payload only contains the TCP EOL option. If there are TCP options in the payload, then the last option is always the EOL option.
> 
> - add a flag to MP_CAPABLE option to indicate whether the SYN payload contains TCP options (terminated by EOL) or not. Same as above for encoding the options in the SYN payload
> 
> - add a length field to MP_CAPABLE option to indicate the number of bytes (or words) of the SYN payload that contain TCP options. In this case, we do not need to terminate the options in the payload with EOL.
> 
> If RFC6824bis could include a reasonable solution that allows to place TCP options in the SYN payload, we would make a huge step to ensure that MPTCP is future-proof and that it will be possible to extend it in the future. In the short term, the solution could be limited to networks where it is known that there are no middleboxes that interfere with TCP options, but this would be a strong message to middlebox vendors that they will need to support TCP options in the SYN payload in the future.
> 
> I won't unfortunately be able to attend the next IETF meeting in Seoul but I'd be happy to discuss this by email
> 
> 
> Olivier
>

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 4 Nov 2016 15:17:07 +0100

Alan,

> This does not change my view, however, that MPTCP options should not be used for application-layer purposes, and that’s my main issue with MP_CONVERT as written - it’s a non-MPTCP-specific applicat…

It's clear that a proxy can be developed entirely as an application. 
SOCKSv5 is one example. The main drawback with SOCKS is that this adds 
several round-trip-times to each connection establishment to setup the 
proxy. With an option in the SYN, a proxy to operate with zero-rtt 
overhead. Reducing latency is an objective that is shared by other 
working groups, see e.g. TLS 1.3, QUIC and others...

Olivier

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 4 Nov 2016 15:24:40 +0100

Hi Olivier, hi all,

I really don’t buy our argument why you should develop such a mechanism for MPTCP only compared to a generic solution that could be used by all (existing and future) TCP extensions. MPTCP is not the only extension that could make use of this (tcpinc is another example) and I’m sure there will more mechanisms in future that would benefit from additional option space in the SYN. 

Mirja

> Am 04.11.2016 um 12:23 schrieb Alan Ford <[email-masked]>:
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 4 Nov 2016 07:39:37 -0700

+1

> On Nov 4, 2016, at 7:24 AM, Mirja Kühlewind <[email-masked]> wrote:
> 
> Hi Olivier, hi all,
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 4 Nov 2016 07:52:46 -0700

Hi, all,

...
> >>> - add a flag to MP_CAPABLE option to indicate whether the SYN payload contains TCP options (terminated by EOL) or not. Same as above for encoding the options in the SYN payload

FWIW, this doesn't work and is the reason why TCPM has different
proposals for the SYN (and SYN-ACK) vs. other segments.

See the discussion in draft-ietf-tcpm-tcp-edo.
<https://datatracker.ietf.org/doc/draft-ietf-tcpm-tcp-edo/>

SYN proposals include draft-touch-tcpm-tcp-syn-ext-opt
<https://tools.ietf.org/id/draft-touch-tcpm-tcp-syn-ext-opt-05.txt>,
which includes a discussion of some of the alternatives that have been
proposed as well.

Joe

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 14:54:02 +0000

Re-,

There are already some generic proposals (e.g., SYN EOS), but IMHO those require an out of band SYN (second SYN) to extend the TCP option space, while the payload space of the original SYN can be used! Also, side effects of using two SYNs are to be yet evaluated. 

The use of a flag in the MP_CAPABLE does not mean that this will be specific to MPTCP. If we trust that MPTCP will be widely adopted in the future and all devices will be MPTCP-capable by default, then this feature will be used by tcpinc and long options that will be in the future. 

Cheers,
Med 

> De : multipathtcp [mailto:[email-masked]] De la part de Joe
> Touch
> Envoyé : vendredi 4 novembre 2016 15:40
> À : Mirja Kühlewind
> > I really don’t buy our argument why you should develop such a mechanism
> for MPTCP only compared to a generic solution that could be used by all
> (existing and future) TCP extensions. MPTCP is not the only extension that
> could make use of this (tcpinc is another example) and I’m sure there will
> more mechanisms in future that would benefit from additional option space
> in the SYN.
> >
> > Mirja
> >
> >
> >> Thanks for the comments, Olivier. If the working group thinks it would
> be a good idea, I would not be opposed to embedding some extended options
> support in MPTCP - as you say it seems a good chance to do so. I haven’t
> thought deeply yet about the various options below, all have their merits,
> but the flag sounds possibly easiest. Whatever extension was used, it
> could be particularly useful in the future for improved cryptographic
> handshakes.
> >>
> >> This does not change my view, however, that MPTCP options should not be
> used for application-layer purposes, and that’s my main issue with
> MP_CONVERT as written - it’s a non-MPTCP-specific application-layer signal
> masquerading as a MPTCP option.
> >>
> >> Regards,
> >> Alan
> >>
> >>> On 4 Nov 2016, at 11:07, Olivier Bonaventure
> >>>> To be clear - again - I have no issue with this work being done in
> MPTCP WG, it is “how to use proxies to increase deployment of MPTCP”. But
> this particular document is not MPTCP-specific; whether it is published
> here or elsewhere I don’t care, but it really shouldn’t’ be written as if
> it’s extending the MPTCP spec when there is no reason for it to be.
> >>>>
> >>> Sorry for the late reply. The main problem with the current MPTCP
> draft is the length of the TCP options in the SYN segment. With network
> assisted MPTCP, we could need to encode one or two IP addresses inside TCP
> options in the SYN segment. With IPv4, this might be possible depending on
> the other options that are used. With IPv6, this becomes quickly a
> problem.
> >>>
> >>> Different designs are possible and some have been discussed in the
> TCPM working group. Instead of defining a specific option to allow longer
> options, I would suggest to include this feature directly inside Multipath
> TCP. This would be cleaner and Multipath TCP is anyway the TCP variant
> that mostly requires this feature. Here are a few possible designs to
> start the discussion :
> >>>
> future. In the short term, the solution could be limited to networks where
> it is known that there are no middleboxes that interfere with TCP options,
> but this would be a strong message to middlebox vendors that they will
> need to support TCP options in the SYN payload in the future.
> >>>

---

From: Joe Touch <[email-masked]>
Date: Fri, 4 Nov 2016 07:56:41 -0700

On 11/4/2016 7:54 AM, [email-masked] wrote:
> Re-,
>
> There are already some generic proposals (e.g., SYN EOS), but IMHO those require an out of band SYN (second SYN) to extend the TCP option space, while the payload space of the original SYN can be used!

Please read the generic proposals. The explain why the SYN payload
cannot be used.

Joe

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 15:07:54 +0000

Re-,

I'm familiar with those documents, Joe. 

In addition to the proposal shared by Olivier, we are also considering this one: https://tools.ietf.org/html/draft-boucadair-tcpm-capability-option-00 for managed environments. 

Cheers,
Med

> De : Joe Touch [mailto:[email-masked]]
> Envoyé : vendredi 4 novembre 2016 15:57
> À : BOUCADAIR Mohamed IMT/OLN; Mirja Kühlewind
> > There are already some generic proposals (e.g., SYN EOS), but IMHO those
> require an out of band SYN (second SYN) to extend the TCP option space,
> while the payload space of the original SYN can be used!
>

---

From: Alan Ford <[email-masked]>
Date: Fri, 4 Nov 2016 15:17:51 +0000

Hi Olivier,

> On 4 Nov 2016, at 14:17, Olivier Bonaventure <[email-masked]> wrote:
> 
> It's clear that a proxy can be developed entirely as an application. SOCKSv5 is one example. The main drawback with SOCKS is that this adds several round-trip-times to each connection establishment to setup the proxy. With an option in the SYN, a proxy to operate with zero-rtt overhead. Reducing latency is an objective that is shared by other working groups, see e.g. TLS 1.3, QUIC and others...

One of us is clearly missing some key piece of information, since I still don’t know how we can be arguing something that seems so fundamentally clear to me.

This is a clear-cut application on top of MPTCP. In explicit mode, the application protocol is:

- a few bytes of signal that defines the real target address
- followed by the proxied payload

I’m not saying “use SOCKS” - you have a proposal for a 0-RTT proxy signalling protocol here. That’s fine. But it’s not MPTCP-specific. Why are you trying to make it a MPTCP option?

As I said in the reply to Med, the implicit mode is slightly more difficult since you are expecting interception of MPTCP traffic based on some logic. That logic could be a “proxy alert” flag, or it could be inspection of all MPTCP SYNs’ payloads (data-on-SYN issues are probably not such big deal in your relatively controlled environments), or it could be using RFC 2113 RAO. It does not need to be an MPTCP option specific to this extension.

Regards,
Alan

---

From: Joe Touch <[email-masked]>
Date: Fri, 4 Nov 2016 08:23:40 -0700

Hi, Med,

> I'm familiar with those documents, Joe. 
Given those docs explain why SYN data is not a viable way forward, can
you explain why you think it now is?

Further, EDO would be exactly the generic version of what Oliver
proposed. So if SYN data could be used, all we would need to do would be
to update the EDO WG doc to use the long ("extension") variant in the
SYN and SYN-ACK rather than the short ("supported") variant. But again,
that won't work.

> In addition to the proposal shared by Olivier, we are also considering this one: https://tools.ietf.org/html/draft-boucadair-tcpm-capability-option-00 for managed environments. 

That variant avoids 1 byte per kind, but only for 2-byte kinds. Those
options consume very little space in the SYN and even if they were
compressed to near zero the SYN would lack space to support a
combination of multibyte kinds. I.e., it adds complexity without
usefully addressing the problem of SYN space.

Joe

---

From: <[email-masked]>
Date: Fri, 4 Nov 2016 15:55:24 +0000

Re-,

If not an MPTCP option, which channel you would use to pass in-band some information between two MPTCP peers (client/proxy, proxy/proxy) while allowing for variable data to be exchanged, multiple instances to be included, no interference with TFO, etc. AND ensure interoperability between MPTCP peers?

Cheers,
Med

De : Alan Ford [mailto:[email-masked]]
Envoyé : vendredi 4 novembre 2016 16:18
À : [email-masked]
Cc : BOUCADAIR Mohamed IMT/OLN; [email-masked]
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

Hi Olivier,

On 4 Nov 2016, at 14:17, Olivier Bonaventure <[email-masked]<mailto:[email-masked]>> wrote:

Regards,
Alan

---

From: Alan Ford <[email-masked]>
Date: Fri, 4 Nov 2016 16:43:16 +0000

Err, exactly the way you’re proposing doing it today. In the TCP payload. Just don’t pretend it’s an MPTCP option when it’s not.

I was hoping that when these drafts were merged there would be a single desired deployment model, but it seems not. Explicit mode and Implicit mode are very different. Explicit mode is a proxy solution purely at the application layer and is not MPTCP-specific. You signal to a proxy - which is expecting it - information in the payload to define where the data is to be proxied on to.

The problem seems to be for implicit mode only. And more than that, it’s for implicit mode between two MCPs. I.e. scenario 3 in your document.

Looking again at the implicit mode scenario, in draft-nam-mptcp-deployment-considerations-00.txt, you have a case of a host (H1) which sends data to the far end; this is intercepted by the MCP, sent on as TCP-only, and the MCP inserts its own addresses. This does not need any additional signals to operate.

You later raised a scenario in an email which is not specifically stated in the document - one where you have TCP -> MCP1 -> MPTCP -> MCP2 -> TCP and you need to know if it’s come from another MCP or from the endpoint. Only data coming from MCP1 should be proxied. And MCP1 does not know about MCP2 so cannot address it directly.

But normal implicit mode operation would still work here, wouldn’t it? MCP2 could blindly add itself to the path in all cases. Does it actually matter if the MPTCP is coming from MCP1 or the end host?

Your email described a desire not to revert to TCP after MCP2 if the end host, rather than MCP1, had sent the MPTCP signalling. But surely that problem would go away if you decided to proxy based on the far end’s MPTCP capability - i.e. on the SYN/ACK, not the SYN?

I came into this email thinking I’d finally understood the scenario where a signal is required, but I’m now confused again. Please help me understand here!

Regards,
Alan

> On 4 Nov 2016, at 15:55, <[email-masked]> <[email-masked]> wrote:
> 
> Re-,
>  
> On 4 Nov 2016, at 14:17, Olivier Bonaventure <[email-masked] <mailto:[email-masked]>> wrote:
> 
>

---

From: Christoph Paasch <[email-masked]>
Date: Fri, 04 Nov 2016 11:16:15 -0700

Hello,

> Ultimately MPTCP Proxying is an application of MPTCP and should be defined as such.

I'm also late on this thread. But just want to reiterate what I already
mentioned before in July.

I also think that proxying between different transport-layer protocols is not
an MPTCP-specific operation and as such should not be tied to MPTCP.
Making it independent of MPTCP would be much better as it allows to proxy
across any kind of transport protocol. So, let's just put it (as already
suggested) in the payload, without linking it to MPTCP.

Christoph

---

From: Olivier Bonaventure <[email-masked]>
Date: Sun, 6 Nov 2016 22:06:49 +0100

Alan,

>> CPE(MCP). This is typically an iPhone connected behind a CPE. We want
>> that MPTCP communications issued from this host are forwarded as such
>> to their destination without soliciting the downstream MCP.
>
> since if the CPE was acting in explicit mode, it would address the
> packets to the MCP.
>
> there any other reasons why this would be used rather than explicit
> mode? And what sort of deployment would this scenario exist in?

The implicit mode was suggested by network operators. The MCP resides on 
the path between the CPE and the destination server. The main benefit of 
the implicit mode is that the MCP does not need to perform any address 
translation. Given all the complications with NAT, this is a huge 
benefit, in particular in IPv6 deployments. In IPv4 deployments, the CPE 
can use a single address and there is no need for a pool of IPv4 
addresses on the MCP.

In implicit mode, the MCP needs to distinguish between an MP_CAPABLE 
generated by the endhost and an MP_CAPABLE added by the CPE.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Sun, 6 Nov 2016 22:12:05 +0100

Mirja,
>
> first, I agree with Alan that such a signal does not need to/should not be part of the MPTCP protocol.

Adding the signal inside MPTCP reduces the end-to-end delay compared 
with a pure application layer solution likes SOCKS and this is really 
important.

> MPTCP (as TCP is) is an end2end protocol. If you have one (or two) proxies in the middle, you split up the connection into multiple new ‚end2end‘ connections. If you need additional signaling information on one of the new connections, that a question for a high-layer protocol that uses MPTCP (which is what you do, when you propose it to be part of the payload).

SOCKS has been tried in some deployments with MPTCP and it increases the 
end-to-end delay given the additional rtt that are required.
> Second, I’m not a big fan of the a two side proxy scenario where one side simply assumes that the destination is not MPTCP-capable. This does not support MPTCP deployment but hinders native MPTCP d…

It's clear that the MCP should send the MP_CAPABLE option in the SYN 
towards the destination server and fallback to TCP is this fails. Since 
there are some middleboxes that continue to silently drop the MP_CAPABLE 
option, the MCP should be able to maintain a list of destination servers 
where it had to fallback to regular TCP (e.g. after n retransmissions of 
the SYN with MP_CAPABLE as suggested in RFC6824) and no attempt to use 
MPTCP for these destinations. This cache should be reset on a regular 
basis to probe again the destination servers.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Sun, 6 Nov 2016 22:25:12 +0100

Alan,

> One of us is clearly missing some key piece of information, since I
> still don’t know how we can be arguing something that seems so
> fundamentally clear to me.
>
> This is a clear-cut application /on top of/ MPTCP. In explicit mode, the
> application protocol is:
>
> I’m not saying “use SOCKS” - you have a proposal for a 0-RTT proxy
> signalling protocol here. That’s fine. /But it’s not MPTCP-specific/.

Indeed, the same could apply for TCP, SCTP and probably any transport 
protocol.

> Why are you trying to make it a MPTCP option?

The need comes from MPTCP. It could be defined as a TCP option instead 
of an MPTCP option. I think that explicitly specifying a proxy node is a 
useful feature that should be part of the transport protocol and not 
tied to a specific application. We are less and less in a pure 
end-to-end deployment model for the Internet and making those transport 
proxies explicit would make sense.

> As I said in the reply to Med, the implicit mode is slightly more
> difficult since you are expecting interception of MPTCP traffic based on
> some logic. That logic could be a “proxy alert” flag, or it could be
> inspection of all MPTCP SYNs’ payloads (data-on-SYN issues are probably
> not such big deal in your relatively controlled environments), or it
> could be using RFC 2113 RAO. It does not need to be an MPTCP option
> specific to this extension.

RAO is not a solution because support for IP options on routers is far 
from optimal. The main reason why we proposed to use the option in 
implicit mode is because the working group asked to merge the plain-mode 
and transparent mode drafts in July.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Sun, 6 Nov 2016 22:46:53 +0100

Mirja,
> I really don’t buy our argument why you should develop such a mechanism for MPTCP only compared to a generic solution that could be used by all (existing and future) TCP extensions. MPTCP is not th…

I'd be very happy if TCPM came up with a solution to this problem that 
works for any TCP extension, but progress in this domain as been very 
slow. On the other hand, the deployment of MPTCP could be an opportunity 
to rethink the way options are negotiated in the SYN and improve that 
for the future without being forced to support 30 years of history of 
TCP stacks.

Tomorrow's hosts will very likely be multihomed and if not, they will 
probably use multiple IPv6 addresses. On such hosts, the ability to 
switch from one address to another will a common requirement. MPTCP 
handles those changes in a clean way.

As a working group, let us imagine that MPTCP becomes the default 
reliable transport protocol by year 202x and that all hosts use MPTCP. 
Having this dream in mind, what could we do today to ensure that MPTCP 
is an clean and evolvable protocol. Here are a few ideas :

- Ensure that the option space in the SYN is large enough to carry the 
options that are required, e.g. to exchange key material for tcpinc

- Why not consider that sending MP_CAPABLE implies other options that 
are negotiated independently today, e.g. :

    - WScale option (would it make sense in 202x to use a 64KB window ?)
    - TCP SACK (would it make sense in 202x to create a connection 
without SACK)
    - Although timestamp option is often negotiated today and required 
by RFC7323 for PAWS, MPTCP does not require PAWS since it uses 64bits DSNs
    - Could we include the TFO negotiation inside the MP_CAPABLE option ?

This would reduce option space in the SYN and simplify the MPTCP stack.

Olivier

---

From: Alan Ford <[email-masked]>
Date: Mon, 7 Nov 2016 06:22:34 +0000

Hi Olivier,

> On 6 Nov 2016, at 21:25, Olivier Bonaventure <[email-masked]> wrote:
>> 
> Indeed, the same could apply for TCP, SCTP and probably any transport protocol.

Good, I’m glad we agree here :-)

> The need comes from MPTCP. It could be defined as a TCP option instead of an MPTCP option. I think that explicitly specifying a proxy node is a useful feature that should be part of the transport protocol and not tied to a specific application. We are less and less in a pure end-to-end deployment model for the Internet and making those transport proxies explicit would make sense.

Why is it an option, however? When you are explicitly targeting a proxy node, surely the proxy is just as capable of pulling the data out of the payload here?

> RAO is not a solution because support for IP options on routers is far from optimal. The main reason why we proposed to use the option in implicit mode is because the working group asked to merge the plain-mode and transparent mode drafts in July.

I was under the impression you would be deploying this is controlled networks, or at least networks where operators had full choice of the equipment (CPEs and proxies) that would be deployed.

Regards,
Alan

---

From: Alan Ford <[email-masked]>
Date: Mon, 7 Nov 2016 06:22:36 +0000

Hi Olivier,

> On 6 Nov 2016, at 21:06, Olivier Bonaventure <[email-masked]> wrote:
> 
> The implicit mode was suggested by network operators. The MCP resides on the path between the CPE and the destination server. The main benefit of the implicit mode is that the MCP does not need to perform any address translation. Given all the complications with NAT, this is a huge benefit, in particular in IPv6 deployments. In IPv4 deployments, the CPE can use a single address and there is no need for a pool of IPv4 addresses on the MCP.
> 
> In implicit mode, the MCP needs to distinguish between an MP_CAPABLE generated by the endhost and an MP_CAPABLE added by the CPE.

Broad question - why?

If the answer is simply “to know when to terminate the MPTCP connection” (i.e. if the end host does it you want it to run all the way to destination, but if the CPE has done it you want to handle at the proxy), then thinking out loud here, could the desired behaviour not be achieved by deciding to proxy only based on the SYN/ACK (i.e. based on the far end’s capabilities) and otherwise be transparently adding MPTCP to everything? That way you get even greater MPTCP coverage whilst not interfering with destinations which have already deployed it.

Regards,
Alan

---

From: <[email-masked]>
Date: Mon, 7 Nov 2016 07:49:35 +0000

Hi Alan,

Please see inline.

Cheers,
Med

De : Alan Ford [mailto:[email-masked]]
Envoyé : vendredi 4 novembre 2016 17:43
À : BOUCADAIR Mohamed IMT/OLN
Cc : [email-masked]; [email-masked]
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

Err, exactly the way you’re proposing doing it today. In the TCP payload.
[Med] Good to know we agree on this.

Just don’t pretend it’s an MPTCP option when it’s not.
[Med] I think here is our disconnect. Let’s put this one aside for the moment.

[Med] The problem as discussed in draft-nam is that the MCP needs to demux native MPTCP connections from proxied ones. The extra signal is needed to avoid breaking MPTCP connections. It may also be needed for transporting other protocols inside the MPTCP connections, but this one is left out of scope.

[Med] FWIW, this is mentioned in Section 5.2.2.1 of the draft-nam and discussed with more details in the document defining the MP_CONVERT option.

But normal implicit mode operation would still work here, wouldn’t it? MCP2 could blindly add itself to the path in all cases. Does it actually matter if the MPTCP is coming from MCP1 or the end host?
[Med] It will revert to TCP those MPTCP connections that do not need the help of an MCP. We don’t want to interfere with those connections.

Your email described a desire not to revert to TCP after MCP2 if the end host, rather than MCP1, had sent the MPTCP signalling. But surely that problem would go away if you decided to proxy based on the far end’s MPTCP capability - i.e. on the SYN/ACK, not the SYN?
[Med] That’s an extra cost for the provider. The model is to avoid as soon as possible to involve MCPs when we know those are able to handle their MPTCP connections. There is no point to “MPTCP assist” MPTCP capable application. Having the signal has the advantage to be deterministic and optimized for the provider that deploys MCPs.

Regards,
Alan

On 4 Nov 2016, at 15:55, <[email-masked]<mailto:[email-masked]>> <[email-masked]<mailto:[email-masked]>> wrote:

Re-,

Cheers,
Med

De : Alan Ford [mailto:[email-masked]]
Envoyé : vendredi 4 novembre 2016 16:18
À : [email-masked]<mailto:[email-masked]>
Cc : BOUCADAIR Mohamed IMT/OLN; [email-masked]<mailto:[email-masked]>
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

Hi Olivier,

Regards,
Alan

---

From: Olivier Bonaventure <[email-masked]>
Date: Mon, 7 Nov 2016 08:50:17 +0100

Alan,

>> of an MPTCP option. I think that explicitly specifying a proxy node is
>> a useful feature that should be part of the transport protocol and not
>> end-to-end deployment model for the Internet and making those
>> transport proxies explicit would make sense.
>
> Why is it an /option/, however? When you are explicitly targeting a
> proxy node, surely the proxy is just as capable of pulling the data out
> of the payload here?
>

Because the addressing information that is used for the proxy is part of 
the control plane and not part of the bytestream. I think that it is 
very important from an architectural viewpoint to maintain a clean 
separation between the byte stream that is exchanged between the 
applications and all control information that should be placed inside 
options.

We explore the possibility of adding information in the bytestream for 
MPTCP and eventually decided to place all control information in 
options. If you look at HTTP, proxy-related information is part of the 
HTTP headers and not part of the payload. This is the same type of problem.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Mon, 7 Nov 2016 08:56:07 +0100

Alan,
>
>> On 6 Nov 2016, at 21:06, Olivier Bonaventure
>> <[email-masked]
>> <mailto:[email-masked]>> wrote:
>>
>> The implicit mode was suggested by network operators. The MCP resides
>> on the path between the CPE and the destination server. The main
>> benefit of the implicit mode is that the MCP does not need to perform
>> any address translation. Given all the complications with NAT, this is
>> a huge benefit, in particular in IPv6 deployments. In IPv4
>> deployments, the CPE can use a single address and there is no need for
>> a pool of IPv4 addresses on the MCP.
>>
> If the answer is simply “to know when to terminate the MPTCP connection”
> (i.e. if the end host does it you want it to run all the way to
> destination, but if the CPE has done it you want to handle at the
> proxy), then thinking out loud here, could the desired behaviour not be
> achieved by deciding to proxy only based on the SYN/ACK (i.e. based on
> the far end’s capabilities) and otherwise be transparently adding MPTCP
> to /everything/? That way you get even greater MPTCP coverage whilst not
> interfering with destinations which have already deployed it.

The problem is more complex than this because the connecitivity on the 
CPE is not the same as connectivity on the endhost. Consider the 
following scenario :

laptop <-----------+
                    |
smartphone <----> cpe <-------> mcp <------> server
     +              +
  cellular1     cellular2

The MCP resides on the DSL path between the CPE and the server.

The laptop is connected over WiFi and the CPE converts the TCP 
connection onto an MPTCP connection to bond DSL and cellular2.

The smartphone is connected over wifi to the cpe which is connected over 
DSL to the server via the MCP. When the smartphone uses MPTCP, it wants 
to use WiFi and cellular1. It could start over WiFi and then moves and 
want to switch to cellular1. The MCP has no idea of cellular1 and cannot 
terminate an MPTCP connection created by the smartphone.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Mon, 7 Nov 2016 09:00:17 +0100

Christoph,

> suggested) in the payload, without linking it to MPTCP.

 From an architectural viewpoint, providing addressing information for a 
proxy is clearly outside the bytestream exchanged by the application. It 
is cleaner to place this control plane information inside options 
instead of messing up the bytestream and changing the first few bytes of 
each connection. I think that we should have a clean separation between 
the control plane and the data plane in MPTCP. We already use options to 
carry addressing information related to the endhosts (ADD_ADDR, 
REMOVE_ADDR), the addresses required by the proxy are the same type of 
information and they should also be exchanged as TCP options. Encoding 
them in the payload would likely create more problems in the long term.

Olivier

---

From: <[email-masked]>
Date: Mon, 7 Nov 2016 08:15:52 +0000

Hi Christoph, 

The point is that we need to meet the following requirements to signal that information:
* avoid extra delay to establish the MPTCP connections
* allow for variable data to be exchanged
* multiple instances of the same data structure to be included
* no interference with TFO
* ensure interoperability between MPTCP host/proxy, or MPTCP proxy/proxy

Defining the data structure as an option allows up to deterministically treat them as part of the connection control, not part of the data to be delivered to an application.

Cheers,
Med

> De : [email-masked] [mailto:[email-masked]]
> Envoyé : vendredi 4 novembre 2016 19:16
> Cc : [email-masked]; Alan Ford
> > > [Med] Defining another channel to carry the signal "I want to be
> MPTCP-proxied" outside the MPTCP normal channel will import the same set
> of hindrances we had with MPTCP design for middlebloxes traversal. The use
> of MPTCP to convey the "I want to be MPTCP-proxied" leverages on base
> MPTCP to detect "unfriendly MPTCP" in-path nodes. That's a pragmatic
> engineering design.
> >
> > The MP_CONVERT signal has multiple purposes. When signalled with a
> target IP address in explicit mode, it is a pure application-layer signal,
> and is separate from the MPTCP connection entering the MCP.
> >
> > In implicit mode this is indeed at a slightly different layer, and yes,
> you are asking the network to do something. But it’s still not a necessary
> part of the MPTCP protocol to make this happen. This is similar to a
> transparent HTTP proxy, or to a NAT - neither requires any changes to HTTP
> or TCP/IP to work.
> >
> > You could detect MPTCP SYNs and see if your signal is present. If you
> don’t want to be inspecting every MPTCP SYN in implicit mode, use an
> existing standard - I suggest RAO. Given this is a controlled environment,
> you shouldn’t have any deployment concerns here.
> >
> > If RAO is not appropriate for some reason, I might be persuaded that a
> flag in the MP_CAPABLE that says “proxies should inspect this” could be
> added to the base protocol. However, the rest of the protocol - i.e. the
> MP_CONVERT signal - is application-layer and outside MPTCP.
> >

---

From: <[email-masked]>
Date: Mon, 7 Nov 2016 08:45:10 +0000

Re-,

In addition to what was mentioned by Olivier, and from where I sit, an answer to Alan's question is also the result of a tension between the e2e principle and deployment requirements:
1.	Favor e2e MPTCP connections (Goal 1)
2.	Optimize MCP resources usage (Goal 2)
3.	Allow for policies to determine flows eligible to the network-assisted MPTCP (Goal 3)

Requiring that every MPTCP host will be eligible to use MCP resources is not viable, IMHO. 

The solution that allows to distinguish in a deterministic manner native MPTCP connections from proxied one has the advantages:
.	to not interfere with native MPTCP connections issued from MPTCP capable host, by default (Goal 1).
.	solid signal to the MCP when it should be involved (or not) (Goal 2)
.	Native MPTCP connections can benefit from the NAM service **IF** policies provided to the CPE allow for that (Goal 3). 

Cheers,
Med

> De : Olivier Bonaventure [mailto:[email-masked]]
> Envoyé : lundi 7 novembre 2016 08:56
> À : Alan Ford
> > If the answer is simply "to know when to terminate the MPTCP connection"
> > the far end's capabilities) and otherwise be transparently adding MPTCP

---

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 7 Nov 2016 16:42:34 +0100

Hi Olivier,

a few comments/questions below.

> Am 06.11.2016 um 22:12 schrieb Olivier Bonaventure <[email-masked]>:
> 
> Mirja,
>> 
> Adding the signal inside MPTCP reduces the end-to-end delay compared with a pure application layer solution likes SOCKS and this is really important.
> 
> SOCKS has been tried in some deployments with MPTCP and it increases the end-to-end delay given the additional rtt that are required.

I didn’t propose SOCKS. For me the solution could more or less be the same than you propose: you use TCP payload in the SYN for signal the needed information. I’m just saying this does not need to be part of the MPTCP protocol. I can simple be an own thing.

> It's clear that the MCP should send the MP_CAPABLE option in the SYN towards the destination server and fallback to TCP is this fails. Since there are some middleboxes that continue to silently drop the MP_CAPABLE option, the MCP should be able to maintain a list of destination servers where it had to fallback to regular TCP (e.g. after n retransmissions of the SYN with MP_CAPABLE as suggested in RFC6824) and no attempt to use MPTCP for these destinations. This cache should be reset on a regular basis to probe again the destination servers.

Do you mean the MCP forwards the original SYN (and basically does nothing if the server supports MPTCP) or does the MCP terminate the TCP connection and start a new TCP connection with MP_CAPABLE towards the server?

Mirja

---

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 7 Nov 2016 16:53:08 +0100

Hi Olivier,

see inline.

> Am 06.11.2016 um 22:46 schrieb Olivier Bonaventure <[email-masked]>:
> 
> Mirja,
>> 
> I'd be very happy if TCPM came up with a solution to this problem that works for any TCP extension, but progress in this domain as been very slow. On the other hand, the deployment of MPTCP could be an opportunity to rethink the way options are negotiated in the SYN and improve that for the future without being forced to support 30 years of history of TCP stacks.

I don’t think supporting TCP stacks is the problem. Network deployment is. And MPTCP will have the same problem than plain TCP has. If there is an (industrial) need for tcp-edo this could drive the working in tcpm forward.

> 
> Tomorrow's hosts will very likely be multihomed and if not, they will probably use multiple IPv6 addresses. On such hosts, the ability to switch from one address to another will a common requirement. MPTCP handles those changes in a clean way.

multipathing can be done on multiple layers of the stack. MPTCP is not the only solution.

> 
> As a working group, let us imagine that MPTCP becomes the default reliable transport protocol by year 202x and that all hosts use MPTCP. Having this dream in mind, what could we do today to ensure that MPTCP is an clean and evolvable protocol. Here are a few ideas :
> 
> - Ensure that the option space in the SYN is large enough to carry the options that are required, e.g. to exchange key material for tcpinc
> 
> - Why not consider that sending MP_CAPABLE implies other options that are negotiated independently today, e.g. :

Because these are independent mechanisms and there is value in keeping them independent. If you want to negotiate all options at once and save option space, then you should specify one new TCP option instead of doing something like this implicit for MPTCP. That’s still work that would go into the tcpm working group. The main purpose of the mptcp working group currently is to specific the bis version of the protocol (and support documentation). However, in the long-term any tcp maintenance including mptcp maintenance should be done in tcpm because mptcp is not an own protocol; it’s a tcp extension.

Mirja

>   - TCP SACK (would it make sense in 202x to create a connection without SACK)
>   - Although timestamp option is often negotiated today and required by RFC7323 for PAWS, MPTCP does not require PAWS since it uses 64bits DSNs

---

From: Joe Touch <[email-masked]>
Date: Mon, 7 Nov 2016 07:56:20 -0800

> Do you mean the MCP forwards the original SYN (and basically does nothing if the server supports MPTCP) or does the MCP terminate the TCP connection and start a new TCP connection with MP_CAPABLE t…
If you're OK with needing to terminate a failed option exchange, then it
might be possible to use EDO in the SYN in its current form.

TCPM decided to prohibit that in the general case, but I could ask them
to allow that in very limited environments (but it could NEVER be
default on).

Note - the use cases I'm hearing appear to assume very strong knowledge
about the other end of the connection and the path. In that case, you
probably can skip most - if not all - of the 'negotiation' options and
just start using them during the SYN too. However, if you say "no, we
need to confirm", then you would not be able to use EDO inside the SYN.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 7 Nov 2016 08:07:56 -0800

Oh - a quick followup.

If you have an option that "poisons" the connection if not confirmed (as
would using long EDO inside the SYN), then you have to retry without
that option. That is *possible*, but there's another problem.

RFC793 defines options as optional. Making any option mandatory would
require updating RFC793. That sort of change is exactly why long EDO
(actually extending the option space, rather than declaring the
capability) is currently prohibited in EDO.

Joe

---

From: Christoph Paasch <[email-masked]>
Date: Mon, 07 Nov 2016 08:24:02 -0800

Hello,

On 07/11/16 - 08:56:07, Olivier Bonaventure wrote:
> The problem is more complex than this because the connecitivity on the CPE
> is not the same as connectivity on the endhost. Consider the following
> scenario :
> 
> laptop <-----------+
>                    |
> The laptop is connected over WiFi and the CPE converts the TCP connection
> onto an MPTCP connection to bond DSL and cellular2.
> 
> The smartphone is connected over wifi to the cpe which is connected over DSL
> to the server via the MCP. When the smartphone uses MPTCP, it wants to use
> WiFi and cellular1. It could start over WiFi and then moves and want to
> switch to cellular1. The MCP has no idea of cellular1 and cannot terminate
> an MPTCP connection created by the smartphone.

I'm not sure, why Alan's suggestion to only proxy based on the SYN/ACK
doesn't work here.

There are two scenarios here:

1. server supports MPTCP: Then, everything is fine. CPE and MCP will stay
   out of the business and native MPTCP will be done from smartphone to
   server.
2. server does not support MPTCP: In that case the MCP will announce its
   public IP to the smartphone and this one can then create an MPTCP-subflow
   via cellular1.

Christoph

---

From: Joe Touch <[email-masked]>
Date: Mon, 7 Nov 2016 08:33:55 -0800

More specifically - see section 4.2.2.5 of rfc1122.

Joe

> On Nov 7, 2016, at 8:07 AM, Joe Touch <[email-masked]> wrote:
> 
> Oh - a quick followup.
>

---

From: Alan Ford <[email-masked]>
Date: Mon, 7 Nov 2016 16:36:42 +0000

Hi Olivier,

> On 7 Nov 2016, at 07:50, Olivier Bonaventure <[email-masked]> wrote:
> 
>> 
> Because the addressing information that is used for the proxy is part of the control plane and not part of the bytestream. I think that it is very important from an architectural viewpoint to maintain a clean separation between the byte stream that is exchanged between the applications and all control information that should be placed inside options.
> 
> We explore the possibility of adding information in the bytestream for MPTCP and eventually decided to place all control information in options. If you look at HTTP, proxy-related information is part of the HTTP headers and not part of the payload. This is the same type of problem.

And this appears to be where we fundamentally disagree. I don’t see this as part of the control plane, I see proxying (at least in explicit mode) as an application of MPTCP, and should be at the application layer. It seems too specific an application to be embedded in the protocol. This does not seem equivalent to NAT, since it is requiring additional signalling.

Similarly I don’t see it compares like-for-like with HTTP headers and payload; HTTP is an application-layer protocol.

Having said that, data can now be tunnelled in HTTP e.g. by using WebSockets. I see this as an application-layer negotiation: negotiate the escalation to WebSockets, then you have a bidirectional socket in operation. This is closer to the tunnelling you propose; however, I see this very much as a negotiation at the first few bytes of the payload, followed by the data.

Regards,
Alan

---

From: Christoph Paasch <[email-masked]>
Date: Mon, 07 Nov 2016 08:48:31 -0800

Hello,

On 07/11/16 - 09:00:17, Olivier Bonaventure wrote:
> proxy is clearly outside the bytestream exchanged by the application. It is
> cleaner to place this control plane information inside options instead of
> messing up the bytestream and changing the first few bytes of each
> connection. I think that we should have a clean separation between the
> control plane and the data plane in MPTCP. We already use options to carry
> addressing information related to the endhosts (ADD_ADDR, REMOVE_ADDR), the
> addresses required by the proxy are the same type of information and they
> should also be exchanged as TCP options. Encoding them in the payload would
> likely create more problems in the long term.

Now I'm confused. I thought that the MCP-option already is in the SYN's payload.

Wrt to the layer-separation. I think that having a 0-RTT proxying protocol
in the payload, where the first few bytes of a bytestream indicate the
server's IP-address would be very beneficial. These few bytes would be
placed in the SYN, just like the MCP-option, but without the link to MPTCP.
This would address Med's concerns wrt to additional delay, not interfere
with TFO,...

This would allow for some interesting use-cases:

* I could do a SCTP-proxy where each stream goes to a different server
* I could do a Quic-proxy
* I could do a TCP-RACK proxy that allows me to use TCP-RACK as
  server-support is not there yet (besides Google)
* ...

Cheers,
Christoph

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 06:37:56 +0000

Hi Joe, 

It is encouraging to consider using EDO, but the current approach we adopted for the MP_CONVERT option have the same failure risk as for introducing MPTCP at the scale of the Internet. 

Using EDO and/or defining MP_CONVERT as a TCP option exacerbates the failure vectors. Below a table that summarizes the failures that will be experienced for the MP_CONVERT case as a function if the design approach: 

   +------------------+-----------+-----------+-----------+------------+
   |                  | MP_CONVERT| MP_CONVERT| MP_CONVERT| MP_CONVERT |
   |                  |   MPTCP   |    TCP    |   MPTCP   | TCP Option |
   |                  |   Option  |   Option  |  Option + |   + EDO    |
   |                  |           |           |    EDO    |            |
   +------------------+-----------+-----------+-----------+------------+
   | MPTCP-unfriendly |    FAIL   |    FAIL   |    FAIL   |    FAIL    |
   |             path |           |           |           |            |
   +------------------+-----------+-----------+-----------+------------+
   |   EDO-unfriendly |     NA    |     NA    |    FAIL   |    FAIL    |
   |             path |           |           |           |            |
   +------------------+-----------+-----------+-----------+------------+
   | "Unknown TCP opt |     NA    |    FAIL   |     NA    |    FAIL    |
   |  ion"-unfriendly |           |           |           |            |
   |             path |           |           |           |            |
   +------------------+-----------+-----------+-----------+------------+

Cheers,
Med

> Envoyé : lundi 7 novembre 2016 16:56
> À : Mirja Kühlewind; [email-masked]
> > Do you mean the MCP forwards the original SYN (and basically does
> nothing if the server supports MPTCP) or does the MCP terminate the TCP
> connection and start a new TCP connection with MP_CAPABLE towards the
> server?
> >
> > Mirja

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 07:31:21 +0000

Re-,

Please see inline.

Cheers,
Med

> Christoph Paasch
> Envoyé : lundi 7 novembre 2016 17:24
> À : Olivier Bonaventure
> CPE
> DSL

[Med] Existing network-assisted MPCP documents already cover this case. Native MPTCP connections will be established by that smartphone without even being intercepted by an MCP. The MCP has not to inspect the SYN/ACK. The problem is more on the laptop side. 

Let's consider a TCP connection from that laptop. This TCP connections is relayed to MPTCP via the fixed line, and the intercepted by the MCP. If the MCP withdraws itself because it receives an MP_CAPBLE in the SYN/ACK from the ultimate server, the secondary subflow (via cellular2) may not be established because only private IPv4 addresses may be assigned in that leg (e.g., dedicated APN is used). In this case, the withdrawal of the MCP will lead to more failures. 

This is exactly why I'm advocating for a configurable parameter.

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 08:08:26 +0000

Hi Alan,

Please see inline.

Cheers,
Med

Envoyé : lundi 7 novembre 2016 17:37

Hi Olivier,

On 7 Nov 2016, at 07:50, Olivier Bonaventure <[email-masked]<mailto:[email-masked]>> wrote:

And this appears to be where we fundamentally disagree. I don’t see this as part of the control plane, I see proxying (at least in explicit mode) as an application of MPTCP, and should be at the application layer.

[Med] I still don’t understand this subtlety between “application of MPTCP” (you claim) and “MPTCP-specific use case” (we claim). If we extrapolate your argument, one can object that multi-pathing is an “application of TCP” (not even specific to TCP), and should not be defined as a TCP option. But the reality is that there is a TCP option for multi-pathing.

It seems too specific an application to be embedded in the protocol. This does not seem equivalent to NAT, since it is requiring additional signalling.

[Med] FWIW, instructing NAT state with in-band signaling information is specified by the IETF and widely deployed:

·         https://tools.ietf.org/html/rfc6052

·         https://tools.ietf.org/html/rfc6146

·         https://tools.ietf.org/html/rfc6877

Regards,
Alan

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 10:07:07 +0000

Joe,

I'm inserting the full text of that section here: 

====
         4.2.2.5  TCP Options: RFC-793 Section 3.1

            A TCP MUST be able to receive a TCP option in any segment.
            A TCP MUST ignore without error any TCP option it does not
            implement, assuming that the option has a length field (all
            TCP options defined in the future will have length fields).
            TCP MUST be prepared to handle an illegal option length
            (e.g., zero) without crashing; a suggested procedure is to
            reset the connection and log the reason.
====

   The proposed design for the MP_CONVERT option does not violate that
   text, IMO.  I'm listing here the design approaches that we considered
   so far:

      (1) no change to MP_CAPABLE: MPTCP proxies are prepared by design
      to look at the payload of SYN to retrieve MP_CONVERT options.

      *  MP_CONVERT options are inserted in the payload of a SYN that
         includes MP_CAPABLE.
      *  If the remote peer does not support MPTCP, the connection will
         be reverted to TCP according to the base MPTCP specification.
      *  If the remote peer supports MPTCP, and it is prepared to
         receive options in the SYN payload, it will echo those options
         in the SYN/ACK.  The MPTCP connection will continue.
      *  If the remote peer supports MPTCP, but it is not prepared to
         receive options in the SYN payload, it will reply with a SYN/
         ACK that does not echo MP_CONVERT.  
      *  The capabilities of the MPTCP proxy may be stored locally to
         determine whether future connections can make use of the
         MP_CPNVERT or not.  Of course, this information will be flushed
         if a new MPTCP proxy is configured, or after the expiry of a
         timer.  This is particularly useful for the explicit mode where
         it is likely that the same MPTCP proxy will handle all
         connections from the same subscriber.

     (2) change MP_CAPABLE to explicitly indicate that options are
      present in the SYN payload:

      *  A flag in the MP_CAPABLE will be set to indicate to a remote
         peer that MP_CONVERT options are included in the payload.
      *  If the remote peer does not support MPTCP, the connection will
         be reverted to TCP as per base MPTCP specification.
      *  If the remote peer supports MPTCP, and is prepared to receive
         options in the payload, it will echo the flag in the SYN/ACK
         (and eventually the options).  The MPTCP connection will
         continue.
      *  If the remote peer supports MPTCP, but does not support that
         Flag, it won't echo the flag in the SYN/ACK.
      *  The capabilities of the MPTCP proxy may be stored locally to
         determine whether future connections can make use of the
         MP_CPNVERT or not.  Of course, this information will be flushed
         if a new MPTCP proxy is configured, or after the expiry of a
         timer.  This is particularly useful for the explicit mode where
         it is likely that the same MPTCP proxy will handle all
         connections from the same subscriber.

   CONCLUSION: The proposed design satisfy both requirements in
   Section 4.2.2.5.  In particular, this one: "A TCP MUST ignore without
   error any TCP option it does not implement"

Cheers,
Med

> Envoyé : lundi 7 novembre 2016 17:34

---

From: Alan Ford <[email-masked]>
Date: Tue, 8 Nov 2016 12:29:33 +0000

Hi Med,

Inline...

> [Med] Existing network-assisted MPCP documents already cover this case. Native MPTCP connections will be established by that smartphone without even being intercepted by an MCP. The MCP has not to…

But inspecting the SYN/ACK would allow the MCP to provide Multipath capabilities when the smartphone is MPTCP capable but the server is not. However, that’s a side-effect...

> Let's consider a TCP connection from that laptop. This TCP connections is relayed to MPTCP via the fixed line, and the intercepted by the MCP. If the MCP withdraws itself because it receives an MP_…

Sorry I don’t follow that scenario. If cellular2 tried to join, it would talk direct to the far end, and not to the MCP,  if the MCP was not in the path… Right? So what’s the problem?

If the MCP remained in the path, then what would be different for this scenario to make it work?

>>   public IP to the smartphone and this one can then create an MPTCP-

These two scenarios make sense to me (at the moment), I am confused as to why this would not work?

Regards,
Alan

---

From: Alan Ford <[email-masked]>
Date: Tue, 8 Nov 2016 12:41:29 +0000

Hi Med,

Inline...

> De : Alan Ford [mailto:[email-masked] <mailto:[email-masked]>] 
> À : [email-masked] <mailto:[email-masked]>
> Cc : BOUCADAIR Mohamed IMT/OLN; [email-masked] <mailto:[email-masked]>
> On 7 Nov 2016, at 07:50, Olivier Bonaventure <[email-masked] <mailto:[email-masked]>> wrote:
>  
> 
> [Med] I still don’t understand this subtlety between “application of MPTCP” (you claim) and “MPTCP-specific use case” (we claim). If we extrapolate your argument, one can object that multi-pathing is an “application of TCP” (not even specific to TCP), and should not be defined as a TCP option. But the reality is that there is a TCP option for multi-patting.

This is a layering issue. MPTCP provides an end-to-end transport. Proxying solutions are also end-to-end protocols - one end encapsulates in a different transport (from TCP to MPTC), and the other end decapsulates (from MPTCP to TCP). It does not make sense (to me) to apply an end-to-end application (proxying) at the transport layer.

Network translation is different and relies on changing fields in the same transport, not changing to a different one (I am coming at this from the perspective that MPTCP, although an extension to TCP, provides sufficiently different functionality that it it a different transport for the purposes of this discussion).

> ·         https://tools.ietf.org/html/rfc6052 <https://tools.ietf.org/html/rfc6052>
> ·         https://tools.ietf.org/html/rfc6146 <https://tools.ietf.org/html/rfc6146>
> ·         https://tools.ietf.org/html/rfc6877 <https://tools.ietf.org/html/rfc6877>  
>  

As far as I can see, none of those involve changes at the transport layer.

Regards,
Alan

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 14:24:08 +0000

Re-,

Please see inline.

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 13:30
> Cc : Christoph Paasch; Olivier Bonaventure; [email-masked]
> > [Med] Existing network-assisted MPCP documents already cover this case.
> Native MPTCP connections will be established by that smartphone without
> even being intercepted by an MCP. The MCP has not to inspect the SYN/ACK.
> The problem is more on the laptop side.
> 
> But inspecting the SYN/ACK would allow the MCP to provide Multipath
> capabilities when the smartphone is MPTCP capable but the server is not.
> However, that’s a side-effect...
> 
> >
> > Let's consider a TCP connection from that laptop. This TCP connections
> is relayed to MPTCP via the fixed line, and the intercepted by the MCP. If
> the MCP withdraws itself because it receives an MP_CAPBLE in the SYN/ACK
> from the ultimate server, the secondary subflow (via cellular2) may not be
> established because only private IPv4 addresses may be assigned in that
> leg (e.g., dedicated APN is used). In this case, the withdrawal of the MCP
> will lead to more failures.
> 
> Sorry I don’t follow that scenario. If cellular2 tried to join, it would
> talk direct to the far end, and not to the MCP,  if the MCP was not in the
> path… Right? So what’s the problem?

[Med] The problem arise when only private IPv4 are assigned on cellular 2.

> scenario to make it work?

[Med] It will use a globally routable IP address as source address (either the same IP address as the one used by the host to establish to first subflow) or an address that assigned locally by the MCP.

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 14:31:16 +0000

Re-,

Please see inline.

Cheers,
Med

Envoyé : mardi 8 novembre 2016 13:41

Hi Med,

Inline...

Hi Olivier,

This is a layering issue. MPTCP provides an end-to-end transport. Proxying solutions are also end-to-end protocols - one end encapsulates in a different transport (from TCP to MPTC), and the other end decapsulates (from MPTCP to TCP). It does not make sense (to me) to apply an end-to-end application (proxying) at the transport layer.
[Med] Yes, the MPTCP connection in the network-assisted case is managed between two MCPs or between an MPTCP-capable host and an MCP. I don’t see why defining an MPTCP option to help the gluing of MPTCP/TCP or avoid breaking an MPTCP connection is harmful.

[Med] FWIW, instructing NAT state with in-band signaling information is specified by the IETF and widely deployed:
•         https://tools.ietf.org/html/rfc6052
•         https://tools.ietf.org/html/rfc6146
•         https://tools.ietf.org/html/rfc6877

As far as I can see, none of those involve changes at the transport layer.
[Med] I didn’t claimed that. I was answering to this part of your answer “This does not seem equivalent to NAT, since it is requiring additional signaling.”. What I’m saying is that there are NATs that require (in-band) signaling.

Regards,
Alan

---

From: Joe Touch <[email-masked]>
Date: Tue, 8 Nov 2016 06:30:44 -0800

Hi, Med,

I cannot parse the note below. Can you please clarify? Or maybe it is
more useful to address my specific issue with SYN data as option, in
response to your other email today.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 8 Nov 2016 06:35:07 -0800

Hi, Med,

>          be reverted to TCP according to the base MPTCP specification.
...
Please explain. If there is option information in the SYN data, then a
legacy peer is allowed to keep that data (and ACK it) in the SYN ACK.

So now it has option info that it interprets as data. You can't undo
that unless you RST the connection, but that violates the cited text in
RFC1122 above, i.e., you have created an option you cannot ignore by
simply not ACKing it.

>          be reverted to TCP as per base MPTCP specification.
...
This has exactly the same problem.

It's not about "MUST ignore" - it's about "without error". Your approach
creates an error in the TCP connection.

Joe

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 14:47:05 +0000

Re-,
 
The point Joe, is that if we define the MP_CONVERT option as a TCP option or if we require EDO to be present to explicitly indicate the data offset, we are adding new failure vectors compared with the other design that leverages on MP_CAPABLE to achieve the same result.

The table cites the failure cases. 

Please let me know if I misunderstood your question.

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 15:31
> À : BOUCADAIR Mohamed IMT/OLN; Mirja Kühlewind;
> [email-masked]
> > It is encouraging to consider using EDO, but the current approach we
> adopted for the MP_CONVERT option have the same failure risk as for
> introducing MPTCP at the scale of the Internet.
> >
> > Using EDO and/or defining MP_CONVERT as a TCP option exacerbates the
> failure vectors. Below a table that summarizes the failures that will be
> experienced for the MP_CONVERT case as a function if the design approach:
> >

---

From: Joe Touch <[email-masked]>
Date: Tue, 8 Nov 2016 06:51:06 -0800

> The table cites the failure cases. 
I don't understand "failure case" - are you talking about connections
that the originator needs to abort?

That's not a "failure case" of a connection; that's a failure of the
design of the option.

Joe

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 15:06:28 +0000

Re-,

Please see inline.

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 15:35
> So now it has option info that it interprets as data.

[Med] But a legacy node is not allowed to pass it to the application till the 3WHS is completed. https://tools.ietf.org/html/rfc793#section-3.4 says the following:

  Several examples of connection initiation follow.  Although these
  examples do not show connection synchronization using data-carrying
  segments, this is perfectly legitimate, so long as the receiving TCP
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                   
  doesn't deliver the data to the user until it is clear the data is
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  valid (i.e., the data must be buffered at the receiver until the
  connection reaches the ESTABLISHED state).  The three-way handshake
  reduces the possibility of false connections.

 You can't undo
> RFC1122 above, i.e., you have created an option you cannot ignore by

[Med] No error is encountered by the receiver side; we are safe at the front. RSTing the connection would not violate RFC1122, though. Further, this is encountered only at the first connection to a new MPTCP proxy.

> creates an error in the TCP connection.

[Med] I agree that "without error" is an important aspect, but this text is about the receiver side. "it does not implement" does not make sense for the sender as it included the option in the first place! In the cases I mentioned above no error is declared by the receiver.

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 15:10:18 +0000

Re-,

Please see inline. 

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 15:51
> > The point Joe, is that if we define the MP_CONVERT option as a TCP
> option or if we require EDO to be present to explicitly indicate the data
> offset, we are adding new failure vectors compared with the other design
> that leverages on MP_CAPABLE to achieve the same result.
> >

[Med] By "failure cases", I meant the inability to establish an MPTCP connection with the remote MPTCP proxy.  

> That's not a "failure case" of a connection; that's a failure of the

[Med] I would say these are "failure cases" that show the failure of some design approaches.

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 8 Nov 2016 16:17:13 +0100

+1 because that is also the only solution that really supports native MPTCP 
deployment (trading off a little additional delay).

On 07.11.2016 07:22, Alan Ford wrote:
> (i.e. if the end host does it you want it to run all the way to destination,
> but if the CPE has done it you want to handle at the proxy), then thinking
> out loud here, could the desired behaviour not be achieved by deciding to
> proxy only based on the SYN/ACK (i.e. based on the far end’s capabilities)
> and otherwise be transparently adding MPTCP to /everything/? That way you get
> even greater MPTCP coverage whilst not interfering with destinations which
> have already deployed it.

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 8 Nov 2016 16:13:42 +0100

Hi all, hi Olivier,

see below.

> On 07/11/16 - 09:00:17, Olivier Bonaventure wrote:
[snip]
>> likely create more problems in the long term.

 From an architectural point of view it's not part of the application 
bytestream (as you call it; which is the data the application wants to get 
from one point to the other) but also not part of the end-to-end transport 
protocol (which makes sure that these data are delivered in some specified 
way). It should probably be part of the network layer or somewhere in 
between, however, that layer doesn't really exists (see PLUS problem statement).

If you put this information in the (MP)TCP payload in the SYN as proposed, 
you should see this as a shim layer and not part of the bytestream.

>   server-support is not there yet (besides Google)

Yes, given this is not a specific problem to MPTCP tunnels, that's why we 
have the BANANA BoF!

Mirja

---

From: <[email-masked]>
Date: Tue, 8 Nov 2016 15:32:50 +0000

Mirja,

I agree that we need to encourage native MPTCP connections. 

The problem is that if the MCP blindly withdraws itself based on the server capabilities, MPTCP connections may not be established in some cases, e.g., private IPv4 addresses are assigned for some secondary access networks. 

That would be unfortunate. 

BTW, relying on the SYN/ACK is not the **only** solution. One can consider configuring a whitelist to the proxy. 

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 16:17
> À : Alan Ford; [email-masked]
> destination,
> > proxy only based on the SYN/ACK (i.e. based on the far end's
> capabilities)

---

From: Alan Ford <[email-masked]>
Date: Tue, 8 Nov 2016 16:00:18 +0000

Med,

> On 8 Nov 2016, at 15:32, [email-masked] wrote:
> 

Could you walk me through this scenario? Despite Olivier’s and your diagrams I’m not following how this does not work. If the far end supports MPTCP, the secondary interface gets to the far end by its own route and joins the MPTCP connection. If the far end does not, then the MCP will have inserted its address in the MPTCP signalling, so the secondary interface could route to it directly too.

So in what scenario does this not work?

Regards,
Alan

---

From: Joe Touch <[email-masked]>
Date: Tue, 8 Nov 2016 08:43:44 -0800

> [Med] But a legacy node is not allowed to pass it to the application till the 3WHS is completed. https://tools.ietf.org/html/rfc793#section-3.4 says the following:
Agreed.

So now you have an option that violates 1122 because the receiver is
ignoring an option it doesn't understand but it would be an error to
continue with that connection.

Even if you assume that's true, you have another problem.

You've just opened a connection using a pair of ports. You sent a RST,
but RSTs are not reliable. So you can't reuse that port pair until
TIME-WAIT expires.

If you declare it "off" for an endpoint, do you ever check again?

Also, you can never use this option with TFO (which *would* try to pass
the data to the app) . I think you mentioned that, but then it's clearly
not generic.

Joe

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 06:41:21 +0000

Hi Alan, 

Below some examples to illustrate the problem I'm talking about. 

(1) Private IPv4 addresses or RFC6598 Shared Address space

Let's consider this configuration (reusing the same topology in this thread): 

laptop <-----------+
                   | (Public IPv4@)
smartphone <----> cpe <----FIXED------> mcp <------> server
     +             + 
  cellular1     cellular2
                (Private IPv4)

In this example, the multipath CPEs gets a public IPv4 address from the fixed network and a private IPv4 address (or an address from RFC6598 range) from the mobile network (cellular2).

Consider the server is MPTCP-capable. 

The first subflow can be placed using the fixed line. Fine.
The MCP will remove itself from the connection because the server is MPTCP-capable. Cool!
The second subflow that will use cellaulr2 cannot be established because of a basic forwarding problem: packets sourced with private IPv4 addresses cannot be forwarded over the public Internet. 

(2) IPv6-only access network

Lets' consider this configuration: 

laptop <-----------+
                   | (Public IPv4@              
smartphone <----> cpe <----FIXED------> mcp <------> server
     +              +                                IPv4-only
  cellular1     cellular2
                (IPv6-only prefix)

In this example, the multipath CPEs gets a public IPv4 address from the fixed network and an IPv6 prefix from the mobile network (cellular2). 

Consider the server is MPTCP-capable, but IPv4-only. 

The first subflow can be placed using the fixed line. Fine.
The MCP will remove itself from the connection because the server is MPTCP-capable. Cool!
The second subflow that will use cellaulr2 cannot be added because the remote server does not support IPv6.

I can share other examples where problems arise when the proxy is blindly removed from the connection based on the presence of MP_CAPABLE in SYN/ACK. 

Cheers,
Med

> Envoyé : mardi 8 novembre 2016 17:00
> Cc : Mirja Kühlewind; [email-masked];
> [email-masked]
> > The problem is that if the MCP blindly withdraws itself based on the
> server capabilities, MPTCP connections may not be established in some
> cases, e.g., private IPv4 addresses are assigned for some secondary access
> networks.
> 
> Could you walk me through this scenario? Despite Olivier’s and your
> diagrams I’m not following how this does not work. If the far end supports
> MPTCP, the secondary interface gets to the far end by its own route and
> joins the MPTCP connection. If the far end does not, then the MCP will
> have inserted its address in the MPTCP signalling, so the secondary
> interface could route to it directly too.
>

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 07:06:06 +0000

Hi Joe,

Please see inline.

Cheers,
Med

De : Joe Touch [mailto:[email-masked]]
Envoyé : mardi 8 novembre 2016 17:44
À : BOUCADAIR Mohamed IMT/OLN; Mirja Kühlewind; [email-masked]
Cc : [email-masked]
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

On 11/8/2016 7:06 AM, [email-masked]<mailto:[email-masked]> wrote:

So now you have an option that violates 1122 because the receiver is ignoring an option it doesn't understand
[Med] This is not violating RFC1122.
but it would be an error to continue with that connection.
[Med] The receiver does not crash or generate an error, all is handled at the sender side. The sender side can decide to continue with the connection or to fallback. For example, the sender can continue with the connection if the MP_CONVERT option was included as part of the implicit mode (that is say to MPTCP proxy: “this is an MPTCP connection, please don’t revert it to TCP”). The sender can also decide for this connection and only this connection, to RST the ongoing connection and fallback to TCP.

You've just opened a connection using a pair of ports. You sent a RST, but RSTs are not reliable. So you can't reuse that port pair until TIME-WAIT expires.
[Med] This a fair observation. Another source port number will be used for the fallback connection (if any).

[Med] Yes, e.g.,

·         When a new MPTCP proxy is discovered (e.g., https://tools.ietf.org/html/draft-boucadair-mptcp-dhc-05)

·         Upon reboot of the host

·         When attaching to a new network

·         When a timer expires

Also, you can never use this option with TFO (which *would* try to pass the data to the app) . I think you mentioned that, but then it's clearly not generic.
[Med] This is more subtle, Joe:

·         MP_CONVERT negotiation is done per MPTCP Proxy. In particular, the procedure is not repeated for each new connection with an ultimate server.

·         TFO is negotiated per server not per MPTCP proxy.

·         Using TFO requires an initial connection to get the cookie prior to making use of it.

·         If MP_CONVERT negotiation happened before TFO negotiation, there are no implications on TFO.

·         If the MP_CONVERT negotiation and TFO negotiation happen in the same connection, there are again no implications on TFO.

·         If TFO negotiation happens after MP_CONVERT negotiation, there are two cases:

o    If it is safe to use MP_CONVERT and TFO, then the TFO data in the payload is clearly delimited because we are using EOL as a marker. The proxy will unambiguously identify TFO data that it will forward in an upstream packet to the ultimate server.

o    If it is not safe to use MP_CONVERT, there are again no implications on TFO.

Joe

---

From: Alan Ford <[email-masked]>
Date: Wed, 9 Nov 2016 08:04:09 +0000

Hi Med,

Thanks for providing some scenarios. Comments inline…

> On 9 Nov 2016, at 06:41, [email-masked] wrote:
> 
> Hi Alan, 
> 

Presumably cellular2 goes through a NAT in order to talk to anything in the public Internet. So it can be addressed to the server, and work like any other MPTCP subflow behind a NAT would work today.

What am I missing? Why is this any different to today with a MPTCP end host with private IP addresses?

The smartphone does not have knowledge of cellular2, presumably… So the CPE would have to do something funky with the traffic to split it over fixed and cellular2 - but it could still establish the subflow towards the server on cellular2, without any knowledge of the smartphone or the network.

> The second subflow that will use cellaulr2 cannot be added because the remote server does not support IPv6.

So in this case, if the MCP was on-path, it could be dual-stack and could add its IPv6 address to the subflows.

If the MCP knew the connection was being proxied, would this help? Since as you’ve already defined, the MCP <—> server path would then be TCP. If the MCP stayed on-path, you could do multi path towards the MCP but then lose MPTCP capabilities to the server.

Or alternatively, the MCP could always stay on-path but only add IPv6 addresses if it somehow had knowledge that was required based on the source or destination address. Does it need to know by a flag? Would there actually be any harm by maintaining itself on the path anyway? It’s not like it’s doing anything other than packet forwarding on the initial subflow, and just adds its IPv6 addresses to the initial exchanges in case it would be useful - and if it receives traffic to the relevant address/port, it can start forwarding it to the far end too (possibly over a second IPv4 subflow). Again, no need or any explicit signalling to make this work.

> I can share other examples where problems arise when the proxy is blindly removed from the connection based on the presence of MP_CAPABLE in SYN/ACK. 

If it would help to clarify, please do! Since I don’t see a problem in (1), and whilst I can see a potential problem in (2) above, I am not sure how you are resolving it. Any other examples very welcome!

Best regards,
Alan

---

From: Joe Touch <[email-masked]>
Date: Wed, 9 Nov 2016 00:59:42 -0800

> [Med] This a fair observation. Another source port number will be used for the fallback connection (if any).  
That is no longer TCP.
Remember that it is not TCP that picks these ports. The OS does when the user didn't but the user can.

Joe

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 09:45:20 +0000

Re-,

Please see inline.

Cheers,
Med

> Envoyé : mercredi 9 novembre 2016 09:04
> > In this example, the multipath CPEs gets a public IPv4 address from the
> fixed network and a private IPv4 address (or an address from RFC6598
> range) from the mobile network (cellular2).
> >
> > The second subflow that will use cellaulr2 cannot be established because
> of a basic forwarding problem: packets sourced with private IPv4 addresses
> cannot be forwarded over the public Internet.
> 
> Presumably cellular2 goes through a NAT in order to talk to anything in
> the public Internet. So it can be addressed to the server, and work like
> any other MPTCP subflow behind a NAT would work today.

[Med] You are assuming that the same Internet APN will be used for the hybrid access. This is not a valid assumption. Some deployment are considering dedicated APN for this service to avoid additional public IPv4 address are assigned to this service and also to avoid overload the service function at the Gi interface with traffic that will be routed to the backbone of the same provider.

> host with private IP addresses?

[Med] The difference is that there is no NAT in the path before the MCP. The MCP has the role to use the IP address assigned by the fixed line when forwarding the secondary subflow. 

> 
> The smartphone does not have knowledge of cellular2, presumably… So the
> CPE would have to do something funky with the traffic to split it over
> fixed and cellular2

[Med] This is a proxy function (MCP) in the CPE! I didn't focused on that part of the flow because there are other problems there. If the CPE does not embed an MCP, an MPTCP capable host (laptop of smartphone) may not be able to aggregate the resources of multiple access links. Going native will lead to less QoE compared to involving the MCP located in the CPE. 

 - but it could still establish the subflow towards the
> server on cellular2, without any knowledge of the smartphone or the
> network.
> 
> >
> > In this example, the multipath CPEs gets a public IPv4 address from the
> fixed network and an IPv6 prefix from the mobile network (cellular2).
> >

[Med] Yes. 

> 
> If the MCP knew the connection was being proxied, would this help? Since
> as you’ve already defined, the MCP <—> server path would then be TCP. If
> the MCP stayed on-path, you could do multi path towards the MCP but then
> lose MPTCP capabilities to the server.

[Med] The MCP can do the following:
* It removes itself from the connection, then the connection will be e2e MPTCP... but only one access network will be used at the client side.
* The MCP may decide the stay in the connection to glue two adjacent MPTCP connections. Doing so, we benefit from both MPTCP capabilities at the client and servier side. 

The exact behavior should IMHO be configurable. 

> 
> Or alternatively, the MCP could always stay on-path but only add IPv6
> addresses if it somehow had knowledge that was required based on the
> source or destination address. Does it need to know by a flag? Would there
> actually be any harm by maintaining itself on the path anyway?

[Med] I don't see a harm in doing so (see above), but those who want to have an e2e MPTCP connection may see something bad in that design. I though they wanted to completely remove the MCP from the path when both the client and server are MPTCP-capable. 

 It’s not
> like it’s doing anything other than packet forwarding on the initial
> subflow, and just adds its IPv6 addresses to the initial exchanges in case

[Med] It needs to terminate the MPTCP connections, mange them, etc. This is exactly the MCP job. 

> it would be useful - and if it receives traffic to the relevant
> address/port, it can start forwarding it to the far end too (possibly over
> a second IPv4 subflow). Again, no need or any explicit signalling to make
> this work.

[Med] We are not discussing about any explicit signaling her, Alan. We are discussing the implications on removing the MCP from the path based on SYN/ACK that will carry an MP_CAPABLE option!

> 
> > I can share other examples where problems arise when the proxy is
> blindly removed from the connection based on the presence of MP_CAPABLE in
> SYN/ACK.
> 
> If it would help to clarify, please do! Since I don’t see a problem in
> (1), and whilst I can see a potential problem in (2) above, I am not sure
> how you are resolving it. Any other examples very welcome!

[Med] Happily. You can consider a homenet architecture https://tools.ietf.org/html/rfc7368 where MPTCP-capable hosts may not be aware of the presence of multiple paths (see this simple topology).  The use of a proxy will to deterministically make use of the path diversity.

           +-------+-------+    +-------+-------+          \
           |   Service     |    |   Service     |           \
           |  Provider A   |    |  Provider B   |            | Service
           |    Router     |    |    Router     |            | Provider
           +-------+-------+    +------+--------+            | Network
                   |                   |                     /
                   |     Customer      |                    /
                   |     Internet      |                   /
                   |    Connections    |
                 +-----------+-----------+                 \
                 |                       |                  \
                 |     Customer Edge     |                   \
                 |        Router         |                   /
                 +-----------+-----------+                  /
                             |                             /
                             |                            | End-User
     ---+------------+-------+--------+-------------+---  | Network(s)
        |            |                |             |      \
   +----+-----+ +----+-----+     +----+-----+ +-----+----+  \
   |IP   Host | |IP   Host |     | IP   Host| |IP   Host |  /
   |   H1     | |   H2     |     |    H3    | |   H4     | /
   +----------+ +----------+     +----------+ +----------+

Things may be complicated for the enterprise case: 

           +-------+-------+    +-------+-------+          \
           |   Service     |    |   Service     |           \
           |  Provider A   |    |  Provider B   |            | Service
           |    Router     |    |    Router     |            | Provider
           +-------+-------+    +------+--------+            | Network
                   |                   |                     /
                   |     Customer      |                    /
                   |     Internet      |                   /
                   |    Connections    |
                 +-----------+-----------+                 \
                 |                       |                  \
                 |     Customer Edge     |                   \
                 |        Router         |                   /
                 +-----------+-----------+                  /
          Network A        | |   |      Network B(E)         |
    ----+-------------+----+ |   +---+-------------+------+  |
        |             |      |       |             |      |  |
   +----+-----+ +-----+----+ |  +----+-----+ +-----+----+ |  |
   |IP   Host | |IP   Host | |  |  IP  Host| |IP   Host | |  |
   |    H1    | |    H2    | |  |    H3    | |    H4    | |  |
   +----------+ +----------+ |  +----------+ +----------+ |  |
                             |        |             |     |  |
                      Link F |     ---+------+------+-----+  |
                             |               | Network E(B)  |
                      +------+--------+      |               | End-User
                      |     IP        |      |               | Networks
                      |   Interior    +------+               |
                      |    Router     |                      |
                      +---+-------+-+-+                      |
          Network C       |       |   Network D              |
    ----+-------------+---+       +---+-------------+---     |
        |             |               |             |        |
   +----+-----+ +-----+----+     +----+-----+ +-----+----+   |
   |IP   Host | |IP   Host |     | IP   Host| |IP   Host |   |
   |   H5     | |   H6     |     |    H7    | |    H8    |   /
   +----------+ +----------+     +----------+ +----------+  /

Of course, we can propose solutions without involving an MCP but this will require major changes to how forwarding is achieved, further it make some assumptions about the internal addressing scheme.

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 10:07:46 +0000

Re-,

There are two cases :

·         UC#1 : MPTCP Client=================MPTCP Proxy----Server

·         UC#2 : Client---CPE(MPTCP Proxy)==== MPTCP Proxy----Server

For UC#1, I don’t see why the same port number will be returned by the OS given that TIME_WAIT is not expired.

Since the MPTCP Proxy in UC#2 is tracking TCP connections (TCP/MPTCP glue), it is the responsibility of the MPTCP Proxy to select the source port to use. This is similar to a NAT function. We are leveraging on the TCP BEHAVE recommendations.

Cheers,
Med

De : Joe Touch [mailto:[email-masked]]
Envoyé : mercredi 9 novembre 2016 10:00
À : BOUCADAIR Mohamed IMT/OLN
Cc : Mirja Kühlewind; [email-masked]; [email-masked]
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

Joe

---

From: Alan Ford <[email-masked]>
Date: Wed, 9 Nov 2016 13:38:51 +0000

Hi Med,

Inline...

> On 9 Nov 2016, at 09:45, [email-masked] wrote:
> 

Regrettably I’m not familiar with all these 3GPP terms here, however are you saying that cellular2 would not be able to have external, public, IP connectivity? If not, what’s the point of that interface? I was assuming that it would, and therefore would be able to talk direct to the server.

> [Med] The difference is that there is no NAT in the path before the MCP. The MCP has the role to use the IP address assigned by the fixed line when forwarding the secondary subflow. 

I wasn’t talking before the MCP; in this scenario we are talking that the MCP gets out of the path - so what cellular2 needs to do is reach the server to open the second subflow.

> The exact behavior should IMHO be configurable. 

So the desire is for the CPE to make this choice, based on some internal logic, and without having a MCP address to explicitly communicate with?

> [Med] I don't see a harm in doing so (see above), but those who want to have an e2e MPTCP connection may see something bad in that design. I though they wanted to completely remove the MCP from the…

That would certainly be desirable, yes. But if not possible, then there’s not much harm with always sitting on the path - it’s a bit like a NAT.

> [Med] We are not discussing about any explicit signaling her, Alan. We are discussing the implications on removing the MCP from the path based on SYN/ACK that will carry an MP_CAPABLE option!

My hypothesis here is that an explicit signal is not required if you choose to sit on the path or not based on the presence of MP_CAPABLE in the SYN/ACK.

> [Med] Happily. You can consider a homenet architecture https://tools.ietf.org/html/rfc7368 where MPTCP-capable hosts may not be aware of the presence of multiple paths (see this simple topology).…

Great! So the CPE can act as a MCP here, and optionally there could be a secondary MCP in the path to terminate paths for the far ends that do not support MPTCP (i.e. sitting on the path if the SYN/ACK does not contain MPTCP capability, and could advertise its own address in an ADD_ADDR). So why do you need an explicit signal in this case?

Regards,
Alan

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 14:12:32 +0000

Re-,

Please see inline.

Cheers,
Med

Envoyé : mercredi 9 novembre 2016 14:39

Hi Med,

Inline...

On 9 Nov 2016, at 09:45, [email-masked]<mailto:[email-masked]> wrote:

-----Message d'origine-----
De : Alan Ford [mailto:[email-masked]]
Envoyé : mercredi 9 novembre 2016 09:04
À : BOUCADAIR Mohamed IMT/OLN
Cc : Mirja Kühlewind; [email-masked]<mailto:[email-masked]>;
[email-masked]<mailto:[email-masked]>
Objet : Re: [multipathtcp] potential MPTCP proxy charter item

Hi Med,

On 9 Nov 2016, at 06:41, [email-masked]<mailto:[email-masked]> wrote:

Hi Alan,

Let's consider this configuration (reusing the same topology in this
thread):

laptop <-----------+
                 | (Public IPv4@)

The first subflow can be placed using the fixed line. Fine.
The MCP will remove itself from the connection because the server is
MPTCP-capable. Cool!

Regrettably I’m not familiar with all these 3GPP terms here, however are you saying that cellular2 would not be able to have external, public, IP connectivity?
[Med] Yes, because this is supposed to be a dedicated APN for hybrid access service. FWIW, you can refer to the concept of APN here https://tools.ietf.org/html/rfc6459#section-2.2.

If not, what’s the point of that interface? I was assuming that it would, and therefore would be able to talk direct to the server.
[Med] That’s may be another APN: Internet APN.

What am I missing? Why is this any different to today with a MPTCP end
host with private IP addresses?

I wasn’t talking before the MCP; in this scenario we are talking that the MCP gets out of the path
[Med] I got that.

- so what cellular2 needs to do is reach the server to open the second subflow.
[Med] Cellular2 needs nothing! But MPTCP connections need the flows to be intercepted but the MCP. That is avoid withdrawing the MCP even if the remote server is MPTCP-capable.  Please note that I’m not expressing my own preferences here, but I’m explaining what kind of problems may arise if we make invalid assumptions.

laptop <-----------+
                 | (Public IPv4@

The first subflow can be placed using the fixed line. Fine.
The MCP will remove itself from the connection because the server is
MPTCP-capable. Cool!

The second subflow that will use cellaulr2 cannot be added because the
remote server does not support IPv6.

So in this case, if the MCP was on-path, it could be dual-stack and could
add its IPv6 address to the subflows.

[Med] Yes.

So the desire is for the CPE to make this choice, based on some internal logic, and without having a MCP address to explicitly communicate with?
[Med] My take on this is to have configurable parameters that will be instructed by an operator. The MCP will behave according to those instructions. If an MCP is configured to be removed from a connection, it will coordinate its withdrawal accordingly.

It’s not

My hypothesis here is that an explicit signal is not required if you choose to sit on the path or not based on the presence of MP_CAPABLE in the SYN/ACK.
[Med] The explicit signal is useful for not intercepting the initial subflow. That is, we don’t even enter the phase of waiting a SYN/ACK as the MCP does not see those.

Great! So the CPE can act as a MCP here, and optionally there could be a secondary MCP in the path to terminate paths for the far ends that do not support MPTCP (i.e. sitting on the path if the SYN/ACK does not contain MPTCP capability, and could advertise its own address in an ADD_ADDR). So why do you need an explicit signal in this case?
[Med] It is useful to deterministically force a connection to be forwarded via a node that embeds the MCP. Classical dst-based forwarding may not always allow for that. In such case, the explicit mode can be enabled inside the user’s internal network. The signal may be useful for cases that do not require reverting MPTCP back to TCP in the providers networks (implicit mode). It is also useful if.…one decides to use MPTCP connection to carry other protocol’s payload (UDP, for example) (as captured in the “Protocol” field of the MP_CONVERT MPTCP option).

Regards,
Alan

---

From: Joe Touch <[email-masked]>
Date: Wed, 9 Nov 2016 06:51:06 -0800

> On Nov 9, 2016, at 2:07 AM, <[email-masked]> <[email-masked]> wrote:
> 
> Re-,
>  
> There are two cases :

TCP is asked to connect once. You seem to now expect it to try twice - TCP cannot (it can't pick ports by itself) and user land cannot be expected to or may not be able to.

> Since the MPTCP Proxy in UC#2 is tracking TCP connections (TCP/MPTCP glue), it is the responsibility of the MPTCP Proxy to select the source port to use. This is similar to a NAT function. We are l…

 It might not be able to use the source port it wants either.

Joe

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 15:37:09 +0000

Hi Joe,

Please see inline.

Cheers,
Med

Envoyé : mercredi 9 novembre 2016 15:51

On Nov 9, 2016, at 2:07 AM, <[email-masked]<mailto:[email-masked]>> <[email-masked]<mailto:[email-masked]>> wrote:
Re-,

There are two cases :

TCP is asked to connect once. You seem to now expect it to try twice - TCP cannot (it can't pick ports by itself) and user land cannot be expected to or may not be able to.
[Med] How that is different from establishing many TCP subflows between the same pair of IP address, using distinct source port numbers: https://tools.ietf.org/html/draft-ietf-mptcp-experience-07#section-3.4?

   This subflow manager creates N subflows between the same pair of IP
   addresses.  The N subflows are created by the client and differ only
   in the source port selected by the client.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 It might not be able to use the source port it wants either.
[Med] The only case I see is when all external ports are exhausted. This is a very very extremely case.

Joe

Cheers,
Med

Cc : Mirja Kühlewind; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 9 Nov 2016 07:49:42 -0800

On 11/9/2016 7:37 AM, [email-masked] wrote:
>
> TCP is asked to connect once. You seem to now expect it to try twice -
> TCP cannot (it can't pick ports by itself) and user land cannot be
> expected to or may not be able to.
>
> [Med] How that is different from establishing many TCP subflows
> between the same pair of IP address, using distinct source port
> numbers:
> https://tools.ietf.org/html/draft-ietf-mptcp-experience-07#section-3.4?
>

The difference is that the ports of the primary flow are no longer under
your control.

I don't know if there's a workaround for MPTCP, but this is exactly why
this technique cannot work for TCP in general. Further, if it doesn't
work for TCP, then MPTCP is no longer TCP in some cases (which means it
can't be used as such interchangeably).

> Since the MPTCP Proxy in UC#2 is tracking TCP connections (TCP/MPTCP
> glue), it is the responsibility of the MPTCP Proxy to select the
> source port to use. This is similar to a NAT function. We are
> leveraging on the TCP BEHAVE recommendations.
>
>  
>

I'm speaking of a case where you want/need to pick both port numbers.

Joe

---

From: <[email-masked]>
Date: Wed, 9 Nov 2016 15:58:39 +0000

Re-,

Thank you for the clarification.

Noted. As I don’t have concrete cases where there will be a need to pick a particular port, I’m not sure this is a big issue.

Cheers,
Med

Envoyé : mercredi 9 novembre 2016 16:50

On 11/9/2016 7:37 AM, [email-masked]<mailto:[email-masked]> wrote:

The difference is that the ports of the primary flow are no longer under your control.

I don't know if there's a workaround for MPTCP, but this is exactly why this technique cannot work for TCP in general. Further, if it doesn't work for TCP, then MPTCP is no longer TCP in some cases (which means it can't be used as such interchangeably).

[Med] Acked.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 9 Nov 2016 08:00:44 -0800

It might not be for MPTCP, but it's a huge issue for TCP.

Which begs the question of whether MPTCP would then be diverging so far
from TCP that it isn't TCP anymore.

Joe

On 11/9/2016 7:58 AM, [email-masked] wrote:
> Noted. As I don’t have concrete cases where there will be a need to
> pick a particular port, I’m not sure this is a big issue.
>
>  
>
> Cheers,
>
> Med
>
>  
>
> *De :*Joe Touch [mailto:[email-masked]]
> *Envoyé :* mercredi 9 novembre 2016 16:50
> *À :* BOUCADAIR Mohamed IMT/OLN
> *Cc :* Mirja Kühlewind; [email-masked];
> [email-masked]
> *Objet :* Re: [multipathtcp] potential MPTCP proxy charter item
>
>  
>
>  
>
>  
>
> On 11/9/2016 7:37 AM, [email-masked]
> <mailto:[email-masked]> wrote:
>
>      
>
>     TCP is asked to connect once. You seem to now expect it to try
>     twice - TCP cannot (it can't pick ports by itself) and user land
>     cannot be expected to or may not be able to.
>
> I don't know if there's a workaround for MPTCP, but this is exactly
> why this technique cannot work for TCP in general. Further, if it
> doesn't work for TCP, then MPTCP is no longer TCP in some cases (which
> means it can't be used as such interchangeably).
>
>
>  
>

---

From: Alan Ford <[email-masked]>
Date: Wed, 9 Nov 2016 16:48:48 +0000

OK got it - I think. I’ll summarise here and please correct me if my understanding is wrong.

Essentially, there are cases where the CPE’s secondary uplink does not have public IP access and thus needs to go via the MCP, but if the MCP has removed itself from the path, the secondary uplink won’t know about the MCP. You need a way to signal this information to a MCP over the primary path so it doesn’t remove itself.

Is this the only case where this signal is required?

Why can’t these scenarios be handled with explicit mode, for example with using the DHCP option to pass down the MCP address?

Regards,
Alan

> On 9 Nov 2016, at 14:12, [email-masked] wrote:
> 
> Re-,
>  
> Please see inline.
>  
> Cheers,
> Med
>  
> Cc : Mirja Kühlewind; [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> On 9 Nov 2016, at 09:45, [email-masked] <mailto:[email-masked]> wrote:
> 
> 
> Cc : Mirja Kühlewind; [email-masked] <mailto:[email-masked]>;
> [email-masked] <mailto:[email-masked]>
> On 9 Nov 2016, at 06:41, [email-masked] <mailto:[email-masked]> wrote:
> 
> Hi Alan,
> 
> [Med] Yes, because this is supposed to be a dedicated APN for hybrid access service. FWIW, you can refer to the concept of APN herehttps://tools.ietf.org/html/rfc6459#section-2.2 <https://tools.ietf.org/html/rfc6459#section-2.2>.
>  
> [Med] Happily. You can consider a homenet architecture https://tools.ietf.org/html/rfc7368 <https://tools.ietf.org/html/rfc7368> where MPTCP-capable hosts may not be aware of the presence of multiple paths (see this simple topology).  The use of a proxy will to deterministically make use of the path diversity.
>

---

From: <[email-masked]>
Date: Thu, 10 Nov 2016 06:48:29 +0000

Hi Alan,

Please see inline.

Cheers,
Med

Envoyé : mercredi 9 novembre 2016 17:49

Essentially, there are cases where the CPE’s secondary uplink does not have public IP access and thus needs to go via the MCP, but if the MCP has removed itself from the path, the secondary uplink won’t know about the MCP. You need a way to signal this information to a MCP over the primary path so it doesn’t remove itself.
[Med] The signal to be removed from the path can be at least in-band (like what was already discussed in the network-assisted MPTCP drafts) or be a local configuration parameter to an MCP.

Is this the only case where this signal is required?
[Med] Frankly speaking, I don’t know. We will need to figure out this once this work is endorsed by the WG.

Why can’t these scenarios be handled with explicit mode, for example with using the DHCP option to pass down the MCP address?
[Med] The same problem applies for the explicit mode. If an MCP decides to withdraw itself from a connection because it receives an MP_CAPABLE option from the server, subsequent subflows may not be established directly with the server for the same reasons we discussed earlier.

Regards,
Alan

On 9 Nov 2016, at 14:12, [email-masked]<mailto:[email-masked]> wrote:

Re-,

Please see inline.

Cheers,
Med

Hi Med,

Inline...

-----Message d'origine-----
De : Alan Ford [mailto:[email-masked]]
Envoyé : mercredi 9 novembre 2016 09:04
À : BOUCADAIR Mohamed IMT/OLN
Cc : Mirja Kühlewind; [email-masked]<mailto:[email-masked]>;

Hi Med,

Hi Alan,

laptop <-----------+
                 | (Public IPv4@)

Regrettably I’m not familiar with all these 3GPP terms here, however are you saying that cellular2 would not be able to have external, public, IP connectivity?
[Med] Yes, because this is supposed to be a dedicated APN for hybrid access service. FWIW, you can refer to the concept of APN herehttps://tools.ietf.org/html/rfc6459#section-2.2.

laptop <-----------+
                 | (Public IPv4@

[Med] Yes.

It’s not

Regards,
Alan

---

From: <[email-masked]>
Date: Thu, 10 Nov 2016 07:20:36 +0000

Hi Joe,

At least from where I sit, I don’t see this is different from how fallback to TCP is handled in MPTCP.

For TCP, it would be good to record this in EDO and EOS specifications.

Thank you.

Cheers,
Med

Envoyé : mercredi 9 novembre 2016 17:01

Which begs the question of whether MPTCP would then be diverging so far from TCP that it isn't TCP anymore.

Joe

On 11/9/2016 7:58 AM, [email-masked]<mailto:[email-masked]> wrote:
Re-,

Cheers,
Med

Joe