Subject: [multipathtcp] Questions on https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09

From: <[email-masked]>
Date: Thu, 10 Nov 2016 11:40:12 +0000

A few questions

Sections 4 & 5 discuss operation with a single-ended proxy, ie one end host is MPTCP-enabled and the other end host is 'legacy' TCP. The figures and discussion seem to assume the MPTCP-enabled end host initiates the connection. Is the case also covered where the TCP end host initiates the connection?

In Sections 4 & 5, the MPTCP-enabled end host is assumed to know the address of the TCP end host. I assume this is through existing techniques. It may mean that an operator has to be careful, for instance if they're doing caching at the edge of the network and the proxy is more central. from the point of view of the path packets take, this seems the same drawback as if tunnelling is present - not sure if there could be unpleasant mptcp protocol interactions?

In section 6, two-ended MPTCP proxy (neither end host is MPTCP-enabled)

<<  If the downstream MCP is deployed on-path, it only terminates MPTCP
   connections that carry an empty MP_CONVERT option inside their SYN
   (i.e., no address is conveyed).  If the MCP receives a SYN segment
   that contains the MP_CAPABLE option but no MP_CONVERT option, it MUST
   forward the SYN to its final destination without any modification.
Comment: I found the description confusing. Not sure what downstream and upstream mean, as the figure shows bi-directional communications. Off path and on path I also found confusing.

If I get it right, on path means that you guarantee the first subflow goes through the proxy - what happens if it doesn't?

what benefit does having this extra on-path (implicit mode) mechanism bring? (I know there's been some discussion on this, is there a summary of the conclusion?)

How does the first proxy know the address of the most appropriate second proxy (ie the proxy nearest the other end host)?

The mechanism mentions a token, is this a re-use of one of the existing tokens in https://datatracker.ietf.org/doc/draft-ietf-mptcp-rfc6824bis/ or a new one?

Thanks
phil

---

From: <[email-masked]>
Date: Thu, 10 Nov 2016 12:54:05 +0000

Hi Phil,

Please see inline.

Cheers,
Med

De : multipathtcp [mailto:[email-masked]] De la part de [email-masked]
Envoyé : jeudi 10 novembre 2016 12:40
À : [email-masked]
Objet : [multipathtcp] Questions on https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09

A few questions

Sections 4 & 5 discuss operation with a single-ended proxy, ie one end host is MPTCP-enabled and the other end host is 'legacy' TCP. The figures and discussion seem to assume the MPTCP-enabled end host initiates the connection. Is the case also covered where the TCP end host initiates the connection?
[Med] Yes. We already have those details in -08 of the draft and some of the details were presented in Buenos Aires meeting (see slide 14 of https://www.ietf.org/proceedings/95/slides/slides-95-mptcp-1.pdf). We removed these details because we wanted to make sure that the base behaviour is well understood.

In Sections 4 & 5, the MPTCP-enabled end host is assumed to know the address of the TCP end host. I assume this is through existing techniques.
[Med] Yes.

It may mean that an operator has to be careful, for instance if they're doing caching at the edge of the network and the proxy is more central. from the point of view of the path packets take, this seems the same drawback as if tunnelling is present - not sure if there could be unpleasant mptcp protocol interactions?
[Med] Two comments here:

·         This is an issue for MPTCP in general: https://tools.ietf.org/html/draft-ietf-mptcp-experience-07#section-3.9

·         The location of the MCP is deployment-specific. An operator that controls both access legs is likely to make sure that DNS service is appropriately configured. An example would be to collocate the MCP with the BNG function and mandate the use of the DNS server provisioned via the fixed line. Other approaches can be considered but this is really deployment-specific.

Comment: I found the description confusing. Not sure what downstream and upstream mean, as the figure shows bi-directional communications.
[Med] Section 6 clearly indicates what is meant by Upstream and Downstream MCP

             Upstream        Downstream
        C <---> MCP <===========> MCP <------------> S
                  +<=============>+

   Legend:
        <===>: MPTCP leg
        <--->: TCP leg

   Figure 7: A Client interacts with a Server through an upstream and a
                  downstream Multipath Conversion Points

and

             Upstream                     Downstream
   +--+      +-----+                        +-----+      +--+
   |C1|      | MCP |                        | MCP |      |S1|
   +--+      +-----+                        +-----+      +--+
    C@1   uMCP@1 uMCP@2                       dMCP@       S@
...

Further, the use case section states the following:

==

   In this use case, neither the Client nor the Server support MPTCP.

   Two MCPs are used as illustrated in Figure 2.  The upstream MCP is

   embedded in the CPE while the downstream MCP is located in the

   provider's network.  The CPE is attached to multiple access networks

   (e.g., xDSL and LTE).  The upstream MCP transparently terminates the

   TCP connections initiated by the Client and converts them into MPTCP

   connections.

                  Upstream                      Downstream

        +--+      +-----+                         +-----+      +--+

        |H1|      | MCP |                         | MCP |      |RM|

        +--+      +-----+                         +-----+      +--+

         |           |                              |           |

         |<---TCP--->|<========MPTCP Leg===========>|<---TCP--->|

         |           |                              |           |

            Figure 2: Network-assisted MPTCP (CPE-based Model)

==

Off path and on path I also found confusing.
[Med] Fair comment. We thought that the reader is familiar with the deployment models in: https://tools.ietf.org/html/draft-nam-mptcp-deployment-considerations-00.html#section-5.2

If I get it right, on path means that you guarantee the first subflow goes through the proxy
[Med] Yes. For example, the proxy is on the default forwarding path of a primary link.

- what happens if it doesn't?
[Med] Network-assisted MPTCP connections will fail. Note that if the primary link is unavailable for some reason, (1) the service cannot be provided through the secondary link in situations where only private IPv4 addresses are assigned on the secondary link or (2) some communications may not be established if only IPv6 prefixes are assigned in that link.

what benefit does having this extra on-path (implicit mode) mechanism bring? (I know there's been some discussion on this, is there a summary of the conclusion?)
[Med] It may allow for quick network-assisted MPTCP deployments if a provider does not want to wait for the MP_CONVERT to be fully endorsed. Native MPTCP connections will be broken because the MCP will systematically revert them to TCP. The support MP_CONVERT will solve that problem for the implicit mode. BTW, as an editor of the document, I'm trying to avoid mandating explicit or implicit as this should be the responsibility of each operator.

How does the first proxy know the address of the most appropriate second proxy (ie the proxy nearest the other end host)?
[Med] This is done by means of DHCP, for instance. Please refer to Section 6.2.2:

   In this section, we assume that the upstream MCP has been configured

   with one address of the downstream MCP.  This address can be

   configured statically, dynamically distributed by means of a DHCP

   option [I-D.boucadair-mptcp-dhc<https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09#ref-I-D.boucadair-mptcp-dhc>] or by any other means that are

   outside the scope of this document.

The mechanism mentions a token, is this a re-use of one of the existing tokens in https://datatracker.ietf.org/doc/draft-ietf-mptcp-rfc6824bis/ or a new one?
[Med] This is the token defined in https://tools.ietf.org/html/rfc6824:

   Token:  A locally unique identifier given to a multipath connection
      by a host.  May also be referred to as a "Connection ID".

Thanks
[Med] Thank you.
phil

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 10 Nov 2016 11:20:34 -0800

Hi Med,

Thanks for the clarification. So, in my understanding,
off-path/on-path concept is like the followings.  (please point out if
I miss something)

on-path (implicit mode) ... The client doesn't need to know the
presence of MCP. The adpp just specifies the server address for the
end point of mptcp connection.

off-path (explicit mode) ... The client needs to know the presence of
MCP. The app needs to use MCP's address for the destination. It also
need to get MCP's address (via DHCP or something else) and tells mptcp
stack to put it in MP_CONVERT option (via API?)

In this case, I think the off-path clients need to know that they are
off-path nodes and need to rely on MCP *before* communication starts.
But, I'm not very sure how they get such knowledge. Even if they get a
MCP address via DHCP, there might be possibility that the MCP is
actually on-path.

Thanks,
--
Yoshi

On Thu, Nov 10, 2016 at 4:54 AM,  <[email-masked]> wrote:
> Objet : [multipathtcp] Questions on
> https://tools.ietf.org/html/draft-boucadair-mptcp-plain-mode-09
>
>
>
> A few questions
>
>
>
> Sections 4 & 5 discuss operation with a single-ended proxy, ie one end host
> is MPTCP-enabled and the other end host is ‘legacy’ TCP. The figures and
> discussion seem to assume the MPTCP-enabled end host initiates the
> connection. Is the case also covered where the TCP end host initiates the
> connection?
>
> [Med] Yes. We already have those details in -08 of the draft and some of the
> details were presented in Buenos Aires meeting (see slide 14 of
> https://www.ietf.org/proceedings/95/slides/slides-95-mptcp-1.pdf). We
> removed these details because we wanted to make sure that the base behaviour
> is well understood.
>
>
>
> In Sections 4 & 5, the MPTCP-enabled end host is assumed to know the address
> of the TCP end host. I assume this is through existing techniques.
>
> [Med] Yes.
>
>
>
> It may mean that an operator has to be careful, for instance if they’re
> doing caching at the edge of the network and the proxy is more central. from
> the point of view of the path packets take, this seems the same drawback as
> if tunnelling is present – not sure if there could be unpleasant mptcp
> protocol interactions?
>
> [Med] Two comments here:
>
> ·         This is an issue for MPTCP in general:
> https://tools.ietf.org/html/draft-ietf-mptcp-experience-07#section-3.9
>
> ·         The location of the MCP is deployment-specific. An operator that
> controls both access legs is likely to make sure that DNS service is
> appropriately configured. An example would be to collocate the MCP with the
> BNG function and mandate the use of the DNS server provisioned via the fixed
> line. Other approaches can be considered but this is really
> deployment-specific.
>
>
>
> Comment: I found the description confusing. Not sure what downstream and
> upstream mean, as the figure shows bi-directional communications.
>
> …
> [Med] Fair comment. We thought that the reader is familiar with the
> deployment models in:
> https://tools.ietf.org/html/draft-nam-mptcp-deployment-considerations-00.html#section-5.2
>
>
>
> – what happens if it doesn’t?
>
> [Med] Network-assisted MPTCP connections will fail. Note that if the primary
> link is unavailable for some reason, (1) the service cannot be provided
> through the secondary link in situations where only private IPv4 addresses
> are assigned on the secondary link or (2) some communications may not be
> established if only IPv6 prefixes are assigned in that link.
>
>
>
> what benefit does having this extra on-path (implicit mode) mechanism bring?
> (I know there’s been some discussion on this, is there a summary of the
> conclusion?)
>
> [Med] It may allow for quick network-assisted MPTCP deployments if a
> provider does not want to wait for the MP_CONVERT to be fully endorsed.
> Native MPTCP connections will be broken because the MCP will systematically
> revert them to TCP. The support MP_CONVERT will solve that problem for the
> implicit mode. BTW, as an editor of the document, I’m trying to avoid
> mandating explicit or implicit as this should be the responsibility of each
> operator.
>
>
>
> How does the first proxy know the address of the most appropriate second
> proxy (ie the proxy nearest the other end host)?
>
>    option [I-D.boucadair-mptcp-dhc] or by any other means that are
>
> The mechanism mentions a token, is this a re-use of one of the existing
> tokens in https://datatracker.ietf.org/doc/draft-ietf-mptcp-rfc6824bis/ or a
> new one?
>
> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp
>

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 11 Nov 2016 16:04:56 +0100

Phil,

Let me complement some of the answers provided by Med

> If I get it right, on path means that you guarantee the first subflow
> goes through the proxy – what happens if it doesn’t?

yes, the underlying network guarantees this. Otherwise, there will be a 
fallback to regular TCP.

>
> what benefit does having this extra on-path (implicit mode) mechanism
> bring? (I know there’s been some discussion on this, is there a summary
> of the conclusion?)
>

The main benefit that network operator see with an on-path mechanism is 
that the addresses that are used by the clients are visible to the 
servers. This simplifies operations since addresses do not change inside 
the network. If the operator uses Netflow/IPFix, needs to perform LI, or 
debug problems, he sees packets with the correct source and destination 
addresses.

Olivier

---

From: <[email-masked]>
Date: Mon, 14 Nov 2016 07:20:20 +0000

Hi Yoshi, 

Please see inline. 

Cheers,
Med

> -----Message d'origine-----
> De : Yoshifumi Nishida [mailto:[email-masked]]
> Envoyé : jeudi 10 novembre 2016 20:21
> À : BOUCADAIR Mohamed IMT/OLN
> Cc : [email-masked]; [email-masked]
> Objet : Re: [multipathtcp] Questions on https://tools.ietf.org/html/draft-
> boucadair-mptcp-plain-mode-09
> 
> Hi Med,
> 

[Med] Yes, this is only for the initial subflow. Subsequent subflows will use an address of the proxy.

> stack to put it in MP_CONVERT option (via API?)

[Med] this may be one way to implement it, but this is not my favorite one. I suggest the following:
* The application does not need to know about the on-off/path modes.
* This is handled at the MPTCP service level: if an address is configured via DHCP (or other means), it will behave in the Explicit mode.
* MP_CONVERT option(s) are inserted by the MPTCP server accordingly. 

> off-path nodes and need to rely on MCP *before* communication starts.

[Med] This is typically an information that is provided during network attachment, etc. For example, 
* a cellular device can retrieve the MCP information during the PDP context activation by means of a dedicated PCO Information Element.
* a CPE will retrieve this information when it bootstraps.

Note that  the client needs DNS information and other important service elements before to make use of a service. That's not particular to MPTCP. 

> But, I'm not very sure how they get such knowledge.

[Med] Same as for DNS and other services (SBC, etc.). Do you foresee any particular issue?

 Even if they get a
> MCP address via DHCP, there might be possibility that the MCP is

[Med] this is exactly why https://tools.ietf.org/html/draft-nam-mptcp-deployment-considerations-00.html#section-5.2 uses Implicit/Explicit terminology.  

> > [Med] Section 6 clearly indicates what is meant by Upstream and
> Downstream
> > MCP
> >
> >
> >
> >              Upstream        Downstream
> >
> goes
> > through the proxy
> bring?
> > [Med] This is done by means of DHCP, for instance. Please refer to
> Section
> > 6.2.2:
> >
> >
> >

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 15 Nov 2016 06:35:23 -0800

Hi Med,
Sorry for my slow response. I put comments in lines.

> * MP_CONVERT option(s) are inserted by the MPTCP server accordingly.

Hmm. OK. But, DHCP is outside of mptcp. How the info from DHCP will be
delivered to mptcp without using API?

> [Med] Same as for DNS and other services (SBC, etc.). Do you foresee any particular issue?

i might miss something here. but, please let me explain a bit more.
BTW, I am looking at the figure 1 in the draft.
Let's say when H1 got address via DHCP, it found a MCP address in the
option. H1 doesn't know RM supports mptcp or not.
So, it use MCP as destination for mptcp connection. But, when RM
happens to be a mptcp node, this will be suboptimal.

To avoid this, I guess H1 needs to go into on-path mode (ignore MCP
info and use RM for destination) for this RM.
But, this might mean H1 will need to change modes based on
destinations and looks tricky.

--
Yoshi

---

From: <[email-masked]>
Date: Tue, 15 Nov 2016 15:10:52 +0000

Hi Yoshi, 

Please see inline. 

Cheers,
Med

> Envoyé : mardi 15 novembre 2016 15:35
> Cc : Yoshifumi Nishida; [email-masked]; [email-masked]
> > * This is handled at the MPTCP service level: if an address is
> configured via DHCP (or other means), it will behave in the Explicit mode.

[Med] This is OS specific. That said, the same methods used to provide a SIP stack with the address of an SBC, a DNS stub-resolver with the DNS Servers list, a DS-Lite B4 element with the address of the AFTR, etc. will be followed to present a list of MCP proxies to an MPTCP service/stack.

> > * a cellular device can retrieve the MCP information during the PDP
> context activation by means of a dedicated PCO Information Element.
> > Note that  the client needs DNS information and other important service
> elements before to make use of a service. That's not particular to MPTCP.
> >

[Med] Given the current state of support of MPTCP by servers, the default mode is to relay MPTCP connections (sent explicitly to the proxy) into TCP. A configurable parameter may be tweaked on the proxy so that the communication leg with an MPTCP-capable server relies on MPTCP. 

As discussed on this list, blindly withdrawing the proxy from the path may be sub-optimal, too. Take the example of a dual-stack client communicating via a dual-stack proxy with a remote IPv4-only server. If the client is assigned only an IPv6 prefix by one of its network attachment, it cannot use the resources of that network to communicate directly with a remote IPv4-only MPTCP-capable server. If the MCP is removed from the path, this will lead to an MPTCP connection but without the benefits of MPTCP since additional subflows cannot be placed over IPv6 (i.e., E2E MPTCP is suboptimal compared to the Network-assisted MPTCP)!

> info and use RM for destination) for this RM.

[Med] Only flows that are eligible to the network-assisted MPTCP server will be forwarded to the MCP. These flows are subject to policies that may be provisioned to a host/CPE (https://tools.ietf.org/html/draft-nam-mptcp-deployment-considerations-00.html#section-5.6.2). Flows that do not match those policies will be forwarded using legacy means; that is no insertion of the MP_CONVERT option. This traffic is used to be called the bypass traffic. 

> destinations and looks tricky.

[Med] The key point is to know that a remote server is MPTCP-capable at the first place. If that information is available, communications with native MPTCP servers can be classified as part of the bypass traffic.