Subject: [multipathtcp] Regarding rate control at a subflow level

From: Nagesh shamnur <[email-masked]>
Date: Fri, 17 May 2019 02:37:01 +0000

Dear MPTCP Group,
                Greetings. In case of Mobile deployments of MPTCP, though the data rates are getting cheaper, still it would be wise not to run the cellular path to full limit but to throttle to a certain extent considering cost in mind or if server wants to limit the client at subflow level, then I couldn't find the support for the same in the specification. So, while going through the discussion archives, could only find that, the peer(server) can throttle the speed for the entire connection by publishing a smaller receiver window rather than for a particular subflow. I feel, it would be a good idea if the peers can exchange this information using the control packets.
                Any thoughts?

Regards,
Nagesh S

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 17 May 2019 09:59:54 +0000

Dear Nagesh,
> 
>                  Greetings. In case of Mobile deployments of MPTCP, 
> though the data rates are getting cheaper, still it would be wise not to 
> run the cellular path to full limit but to throttle to a certain extent 
> considering cost in mind or if server wants to limit the client at 
> subflow level, then I couldn’t find the support for the same in the 
> specification. 

We have developed several prototypes that include this capability in the 
Linux kernel.

> So, while going through the discussion archives, could 
> only find that, the peer(server) can throttle the speed for the entire 
> connection by publishing a smaller receiver window rather than for a 
> particular subflow. I feel, it would be a good idea if the peers can 
> exchange this information using the control packets.

We could imagine an MPTCP option that provides the maximum rate on a 
per-subflow level, but I was wondering whether the use case is not to 
limit the bandwidth on the smartphone at the link level (i.e. multiple 
tcp connections or udp flows) and not at the subflow level. Could you 
precise your use case for the subflow level ?

Thanks

Olivier

---

From: Alexander Frömmgen <[email-masked]>
Date: Sat, 18 May 2019 21:16:32 +0200

Hi,

We experimented with different schedulers that implement related behaviour, e.g., to only use cellular data if a target throughput can't be provided otherwise and only transfer the fraction of data on the cellular path that is required to achieve the target throughput (which is effectively a kind of dynamic rate limiting). See https://progmp.net for further information.

Best,

Alexander 

Am 17. Mai 2019 11:59:54 MESZ schrieb Olivier Bonaventure <[email-masked]>:
>_______________________________________________
>multipathtcp mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Madhan Raj Kanagarathinam <[email-masked]>
Date: Mon, 20 May 2019 11:56:34 +0530

Dear Nagesh,

>> There are quite some research going on SBR and related areas. But if there has to options exchanged (or control), it has to be secured to protect from mangling. Another challenge is that estimating the bandwidth and frequent changing might also be overhead in wireless.

In my research work, to control the same, we throttle the UE RTT by introducing additional delay and controlling the path manager. There are few methods of ACK throttling as well which might help the same. So I would suggest indirect control instead of direct option exchanged.

Regards,

Madhan

--------- Original Message ---------

Sender : Alexander Frömmgen <[email-masked]>

Date : 2019-05-19 00:47 (GMT+5:30)

Title : Re: [multipathtcp] Regarding rate control at a subflow level

Hi,

Best,

Alexander

Dear Nagesh,

Thanks

Oliviermultipathtcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Nagesh shamnur <[email-masked]>
Date: Mon, 20 May 2019 09:09:24 +0000

Reply inline.

Regards,
Nagesh.

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]] 
Sent: 17 May 2019 15:30
To: Nagesh shamnur <[email-masked]>; [email-masked]
Cc: Ashutosh prakash <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Dear Nagesh,
> though the data rates are getting cheaper, still it would be wise not 
> to run the cellular path to full limit but to throttle to a certain 
> extent considering cost in mind or if server wants to limit the client 
> at subflow level, then I couldn't find the support for the same in the 
> specification.

We have developed several prototypes that include this capability in the Linux kernel.

> So, while going through the discussion archives, could only find that, 
> the peer(server) can throttle the speed for the entire connection by 
> publishing a smaller receiver window rather than for a particular 
> subflow. I feel, it would be a good idea if the peers can exchange 
> this information using the control packets.

We could imagine an MPTCP option that provides the maximum rate on a per-subflow level, but I was wondering whether the use case is not to limit the bandwidth on the smartphone at the link level (i.e. multiple tcp connections or udp flows) and not at the subflow level. Could you precise your use case for the subflow level ?
[Nagesh]: This is to ensure similar bandwidth availability to the server for all the clients.

Thanks

Olivier

---

From: Nagesh shamnur <[email-masked]>
Date: Mon, 20 May 2019 09:10:32 +0000

From: Alexander Frömmgen [mailto:[email-masked]]
Sent: 19 May 2019 00:47
To: Olivier Bonaventure <[email-masked]>; Nagesh shamnur <[email-masked]>; [email-masked]
Cc: Ashutosh prakash <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hi,

We experimented with different schedulers that implement related behaviour, e.g., to only use cellular data if a target throughput can't be provided otherwise and only transfer the fraction of data on the cellular path that is required to achieve the target throughput (which is effectively a kind of dynamic rate limiting). See https://progmp.net for further information.
                Sure, thanks for the update. Will look into it.

Best,

Alexander
Am 17. Mai 2019 11:59:54 MESZ schrieb Olivier Bonaventure <[email-masked]<mailto:[email-masked]>>:

Dear Nagesh,

Thanks

Olivier

________________________________

multipathtcp mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Nagesh shamnur <[email-masked]>
Date: Mon, 20 May 2019 09:19:00 +0000

From: Madhan Raj Kanagarathinam [mailto:[email-masked]]
Sent: 20 May 2019 11:57
To: Nagesh shamnur <[email-masked]>
Cc: Alexander Frömmgen <[email-masked]>; Olivier Bonaventure <[email-masked]>; [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE:(2) [multipathtcp] Regarding rate control at a subflow level

Dear Nagesh,

            Madhan, Thanks for taking note of the problem. Using control channel was one of the alternative which I thought and I agree it might not be the best one. But the issue is genuine and needs to be addressed. Regarding the protecting the header from maligning, it is true for any other as well which is part of MPTCP right?

            Any indirect control IMO would still be a patchy solution rather than directly addressing the problem, right?

Regards,

Madhan

--------- Original Message ---------

Sender : Alexander Frömmgen <[email-masked]<mailto:[email-masked]>>

Hi,

Best,

Alexander

Dear Nagesh,

Thanks

Olivier

[http://ext.samsung.net/mail/ext/v1/external/status/update?userid=madhan.raj&do=bWFpbElEPTIwMTkwNTIwMDYyNjM0ZXBjbXM1cDE1Nzg2MzJiZjhhNzkxZWVmZTk3MGJiNzY3OGU5NjYyNSZyZWNpcGllbnRBZGRyZXNzPW5hZ2VzaC5zaGFtbnVyQGh1YXdlaS5jb20_]

---

From: Christoph Paasch <[email-masked]>
Date: Mon, 20 May 2019 14:50:14 +0100

Hello,

> precise your use case for the subflow level ?

another use-case for rate-control I see is when a client wants to tell a sender
to gracefully close a subflow.

- Sending a TCP-RST results in packet-loss of the in-flight data.
- Reducing the window is going to stall the whole connection because the window is shared.
- Setting MP_PRIO also won't work because none might want to drain a secondary
  subflow even if the "primary" subflow is having severe packet-loss.

Thus, a "maximum-rate" option on a per-subflow level would allow to send the
rate to 0, which would drain the subflow.

Christoph

---

From: <[email-masked]>
Date: Thu, 30 May 2019 17:53:19 +0000

I think the use case a hybrid operator could be interested in is the following. Similar to Alexander's use case.

We want to provide a particular customer with broadband access that is faster than their DSL alone can provide, for instance to meet some minimum rate to meet a target of say 20Mbps. The rate can't quite be met by DSL, therefore cellular or even satellite is used as a top-up. DSL capacity is much cheaper than cellular /satellite, therefore the ideal scheduler would favour DSL. It would react reasonably quickly to increased total load above the DSL rate (but not so fast that the instantaneous rate from a variable rate source 'kicks in' the cellular when the rate over a slightly longer time can be met by DSL). Note that the DSL rate is not completely static.  It supports deployments with a proxy in the home gateway and in the network (under the control of the operator), and deployments where there's only the proxy in the network (ie MPTCP is in the multi-interface phone). Possible to ensure that an individual flow goes over only one access. Downward direction (from network to house) more important than upward.  In a long-running session using a lot of bandwidth, because the amount of other traffic varies, it may be that this session sometimes uses just the DSL and sometimes is spread over both accesses. On a minor point, ideally a speed test should measure the rate correctly (I don’t mean the scheduler identifies a speed test and favours it; rather, ideally the speed test shouldn’t eg measure just the DSL rate - alternatively re-define the speed test). 

So in terms of Olivier's question below, I think this means that the limit is about the total and not per subflow. 

In terms of mechanism, I'm open to whatever is most suitable. I'm very interested if the best method would mean that ideally some new functionality is added to the MPTCP standard (and by implication to MP-QUIC).

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

On 17/05/19 - 09:59:54, Olivier Bonaventure wrote:
> > though the data rates are getting cheaper, still it would be wise 
> > not to run the cellular path to full limit but to throttle to a 
> > certain extent considering cost in mind or if server wants to limit 
> > the client at subflow level, then I couldn’t find the support for 
> > the same in the specification.
> 
> > So, while going through the discussion archives, could only find 
> > that, the peer(server) can throttle the speed for the entire 

another use-case for rate-control I see is when a client wants to tell a sender to gracefully close a subflow.

Thus, a "maximum-rate" option on a per-subflow level would allow to send the rate to 0, which would drain the subflow.

Christoph

---

From: Nagesh shamnur <[email-masked]>
Date: Fri, 31 May 2019 05:19:23 +0000

Hi Phil,
	I am glad the problem is being discussed upon.  One more possible usecase is for load balancing to achieve one or a set of clients doesn't hog the entire bandwidth available on the server side. Though the links to server supports the bandwidth but still chooses to limit the bandwidth. I agree, this is left best to be dealt with congestion control algorithm to achieve the bandwidth sharing, but there can be cases where the application decides to limit the bandwidth to avoid possibly overload kind of scenarios.

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: Madhan Raj Kanagarathinam <[email-masked]>
Date: Fri, 31 May 2019 14:48:49 +0530

Hi,

There are multiple usecases and even for smartphones, the users and also the operators prefer using Wi-Fi (unlicensed) to LTE (licensed and costly).

How exactly are we going to achieve the same is the question. The problem is not introducing a signalling (or an option) but how/when do we set this value.

There can be multiple sub-cases

1) User wants to use the subflows only if max value in primary or preferred subflow doesn't reach.

Ex: If Wi-Fi can fetch 100Mbps speed, the use of LTE wouldn't even be required.

2) Assisted Round-Robin : The provided subflows can decide the ration and inform the RR scheduler. The Server can schedule accordingly.

Ex: The ratio of Wi-Fi(10Mbps) to LTE(1Mbps) speed as 10:1  and the scheduler can adapt based on this value.

This would be highly useful in cases where  RTT difference between subflows are high. When high, SPTCP performs better than MPTCP.

3) Hard control on the bandwidth: This case the user wants to force limit to bandwidth.

Ex: In 5G, the mobile data speed can be over 500Mbps and would tend to use only mobile data (not Wi-Fi). Here, one of the subflow alone can try to control the rate.

The main problem (at least in wireless) is that, we cannot predict the exact available bandwidth. There are predictions based on multiple values however it is very dynamic.

In my previous work, we targeted video streaming case alone. As we can predict the bandwidth required for video streaming (based on quality), we controlled in the UE side (my controlling RTT and subflow creation) to maximize the user over Wi-Fi.

How about reusing the SBR (https://tools.ietf.org/html/draft-flinck-mobile-throughput-guidance-04), as this already provides (subflow level or) per connection level control.

Regards,

Madhan

--------- Original Message ---------

Sender : Nagesh shamnur <[email-masked]>

Date : 2019-05-31 10:49 (GMT+5:30)

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph