Subject: Re: [multipathtcp] Regarding rate control at a subflow level

From: <[email-masked]>
Date: Mon, 3 Jun 2019 10:15:31 +0000

Nagesh,
Could you explain your scenario a bit more please? Not sure I get it right.  Is this about uplink? Is the application decision to limit the bandwidth made at the client or at the server, and does the network have a role (either in the decision making or supplying info)?
Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]] 
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	I am glad the problem is being discussed upon.  One more possible usecase is for load balancing to achieve one or a set of clients doesn't hog the entire bandwidth available on the server side. Though the links to server supports the bandwidth but still chooses to limit the bandwidth. I agree, this is left best to be dealt with congestion control algorithm to achieve the bandwidth sharing, but there can be cases where the application decides to limit the bandwidth to avoid possibly overload kind of scenarios.

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

I think the use case a hybrid operator could be interested in is the following. Similar to Alexander's use case.

We want to provide a particular customer with broadband access that is faster than their DSL alone can provide, for instance to meet some minimum rate to meet a target of say 20Mbps. The rate can't quite be met by DSL, therefore cellular or even satellite is used as a top-up. DSL capacity is much cheaper than cellular /satellite, therefore the ideal scheduler would favour DSL. It would react reasonably quickly to increased total load above the DSL rate (but not so fast that the instantaneous rate from a variable rate source 'kicks in' the cellular when the rate over a slightly longer time can be met by DSL). Note that the DSL rate is not completely static.  It supports deployments with a proxy in the home gateway and in the network (under the control of the operator), and deployments where there's only the proxy in the network (ie MPTCP is in the multi-interface phone). Possible to ensure that an individual flow goes over only one access. Downward direction (from network to house) more important than upward.  In a long-running session using a lot of bandwidth, because the amount of other traffic varies, it may be that this session sometimes uses just the DSL and sometimes is spread over both accesses. On a minor point, ideally a speed test should measure the rate correctly (I don’t mean the scheduler identifies a speed test and favours it; rather, ideally the speed test shouldn’t eg measure just the DSL rate - alternatively re-define the speed test). 

So in terms of Olivier's question below, I think this means that the limit is about the total and not per subflow. 

In terms of mechanism, I'm open to whatever is most suitable. I'm very interested if the best method would mean that ideally some new functionality is added to the MPTCP standard (and by implication to MP-QUIC).

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

On 17/05/19 - 09:59:54, Olivier Bonaventure wrote:
> Dear Nagesh,
> > 
> >                  Greetings. In case of Mobile deployments of MPTCP, 
> > though the data rates are getting cheaper, still it would be wise 
> > not to run the cellular path to full limit but to throttle to a 
> > certain extent considering cost in mind or if server wants to limit 
> > the client at subflow level, then I couldn’t find the support for 
> > the same in the specification.
> 
> We have developed several prototypes that include this capability in 
> the Linux kernel.
> 
> > So, while going through the discussion archives, could only find 
> > that, the peer(server) can throttle the speed for the entire 
> > connection by publishing a smaller receiver window rather than for a 
> > particular subflow. I feel, it would be a good idea if the peers can 
> > exchange this information using the control packets.
> 
> We could imagine an MPTCP option that provides the maximum rate on a 
> per-subflow level, but I was wondering whether the use case is not to 
> limit the bandwidth on the smartphone at the link level (i.e. multiple 
> tcp connections or udp flows) and not at the subflow level. Could you 
> precise your use case for the subflow level ?

another use-case for rate-control I see is when a client wants to tell a sender to gracefully close a subflow.

- Sending a TCP-RST results in packet-loss of the in-flight data.
- Reducing the window is going to stall the whole connection because the window is shared.
- Setting MP_PRIO also won't work because none might want to drain a secondary
  subflow even if the "primary" subflow is having severe packet-loss.

Thus, a "maximum-rate" option on a per-subflow level would allow to send the rate to 0, which would drain the subflow.

Christoph

_______________________________________________
multipathtcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Nagesh shamnur <[email-masked]>
Date: Mon, 3 Jun 2019 13:07:23 +0000

Hi Phil,
	The idea is about controlling the rate on the uplink rather than the downlink and the decision to limit the bandwidth is made at the server since, server can control the downlink traffic rate to all the clients connected to it, but the uplink limit needs to be pushed to the client. 	

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: <[email-masked]>
Date: Mon, 3 Jun 2019 13:26:50 +0000

Madhan,
Let me see if I can understand the scenarios further.
In 1 – presumably there is nothing to solve
In 2 – I’m quite intrigued by the suggestion of a fixed ratio between the subflows. Why would this be a good idea? It seems to imply ignoring real-time congestion information. Is the idea that (say) the Wifi is ~10Mbps and LTE ~1Mbps, so it’s considered simpler to divide on fixed ratio 10:1 rather than taking account of actual congestion & bandwidth available?
Also I don’t understand your comment about this approach being useful when there is a large difference in RTT between subflows – surely then it’s best to restrict each flow to using one of the subflows, otherwise all flows suffer the delay of the longest RTT and we need big buffers.
If I get it right, this ‘fixed ratio’ approach is also suggested in 3GPP TR23.793 - although I find the draft doc a bit unclear whether it’s talking about steering or splitting, ie choosing one access for a new flow vs dividing one flow across accesses.
[3GPP TR23.793 v16.0.0 (Dec 2018) – work in progress – “Study on access traffic steering, switch and splitting support in the 5G system architecture (Release 16)” https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3254 ]

In 3 – can we discuss this scenario a bit more. I don’t understand your example with 500Mbps 5G why a party wants to control the rate.
I suppose, continuing my example below where the operator wants to deliver a 20Mbps service (& the DSL capacity a bit less than this), then the operator may want a hard limit at 20Mbps. But in this case I guess the operator can configure the gateway at this limit, at least for downlink traffic, so no signalling is needed.
So I guess it’s some other scenario?
I can see that the flinck throughput guidance might be useful (end to end with the server or with an explicit proxy)

Thanks,
phil
From: Madhan Raj Kanagarathinam [mailto:[email-masked]]
Sent: 31 May 2019 10:19
To: Nagesh shamnur <[email-masked]>; Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE:(2) [multipathtcp] Regarding rate control at a subflow level

Hi,

There are multiple usecases and even for smartphones, the users and also the operators prefer using Wi-Fi (unlicensed) to LTE (licensed and costly).

How exactly are we going to achieve the same is the question. The problem is not introducing a signalling (or an option) but how/when do we set this value.

There can be multiple sub-cases

1) User wants to use the subflows only if max value in primary or preferred subflow doesn't reach.

Ex: If Wi-Fi can fetch 100Mbps speed, the use of LTE wouldn't even be required.

2) Assisted Round-Robin : The provided subflows can decide the ration and inform the RR scheduler. The Server can schedule accordingly.

Ex: The ratio of Wi-Fi(10Mbps) to LTE(1Mbps) speed as 10:1  and the scheduler can adapt based on this value.

This would be highly useful in cases where  RTT difference between subflows are high. When high, SPTCP performs better than MPTCP.

3) Hard control on the bandwidth: This case the user wants to force limit to bandwidth.

Ex: In 5G, the mobile data speed can be over 500Mbps and would tend to use only mobile data (not Wi-Fi). Here, one of the subflow alone can try to control the rate.

The main problem (at least in wireless) is that, we cannot predict the exact available bandwidth. There are predictions based on multiple values however it is very dynamic.

In my previous work<https://patents.google.com/patent/US20170339257A1/en>, we targeted video streaming case alone. As we can predict the bandwidth required for video streaming (based on quality), we controlled in the UE side (my controlling RTT and subflow creation) to maximize the user over Wi-Fi.

How about reusing the SBR (https://tools.ietf.org/html/draft-flinck-mobile-throughput-guidance-04), as this already provides (subflow level or) per connection level control.

Regards,

Madhan

--------- Original Message ---------

Sender : Nagesh shamnur <[email-masked]<mailto:[email-masked]>>

Date : 2019-05-31 10:49 (GMT+5:30)

Title : Re: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,

Nagesh S

-----Original Message-----

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]

Sent: 30 May 2019 23:23

To: cpaasch=[email-masked]<mailto:cpaasch=[email-masked]>; [email-masked]<mailto:[email-masked]>

Cc: [email-masked]<mailto:[email-masked]>; Ashutosh prakash <[email-masked]<mailto:[email-masked]>>; Nagesh shamnur <[email-masked]<mailto:[email-masked]>>

Best wishes,

phil

-----Original Message-----

Sent: 20 May 2019 14:50

To: Olivier Bonaventure <[email-masked]<mailto:[email-masked]>>

Cc: [email-masked]<mailto:[email-masked]>; Ashutosh prakash <[email-masked]<mailto:[email-masked]>>; Nagesh shamnur <[email-masked]<mailto:[email-masked]>>

Hello,

Christoph

[email-masked]<mailto:[email-masked]>

[email-masked]<mailto:[email-masked]>

[http://ext.samsung.net/mail/ext/v1/external/status/update?userid=madhan.raj&do=bWFpbElEPTIwMTkwNTMxMDkxODQ5ZXBjbXM1cDVjY2ViMzUxMzZhNzIzYTVmZjJmYzEwZTZiMjQ2ODQ3MyZyZWNpcGllbnRBZGRyZXNzPXBoaWxpcC5lYXJkbGV5QGJ0LmNvbQ__]

---

From: Madhan Raj Kanagarathinam <[email-masked]>
Date: Tue, 04 Jun 2019 13:24:37 +0530

Phil,

In 2 – I’m quite intrigued by the suggestion of a fixed ratio between the subflows

This is in specific to server that uses RR scheduler. The RR scheduler, can receive the feedback about the current UE (user equipment) parameters and schedule the packet accordingly. Here we send the ratio of scheduling rather than the actual bandwidth.

10:1 rather than taking account of actual congestion & bandwidth available?

Indeed, the RR scheduler doesn’t consider actual congestion and BDP. RR scheduler if not widely used though (in commercial level). But client assisted RR, can have significant enhancement.

This is very important case in mobile user perceptive.

-       When Mobile data is 500Mbps, Wi-Fi is 50Mbps. The ratio is around 10:1. Most of data would be consumed over 5G. Neither user not the network operator would prefer this.

-       Most of the countries has CAPPED the maximum data which can be consumed over mobile data with high speed (FUP). Exhausting the data over 5G, though Wi-Fi is available, might be overhead for users. As the operator doesn’t control, at the client (smartphones) we can control the mobile data rate i.e. reduce the mobile speed so that Wi-Fi data is consumed more.

Yes, I think this can be solution that can handle all the cases. But, we can incorporate SBR inside the MPTCP option, rather than introducing a completely new TCP option (which can be possibly stripped by middlebox).

Regards,

Madhan

--------- Original Message ---------

Sender : [email-masked] <[email-masked]>

Date : 2019-06-03 18:58 (GMT+5:30)

Title : RE: RE:(2) [multipathtcp] Regarding rate control at a subflow level

Madhan,

Thanks,

phil

Hi,

In my previous work, we targeted video streaming case alone. As we can predict the bandwidth required for video streaming (based on quality), we controlled in the UE side (my controlling RTT and subflow creation) to maximize the user over Wi-Fi.

Regards,

Madhan

--------- Original Message ---------

Sender : Nagesh shamnur <[email-masked]>

Hi Phil,

Regards,

Nagesh S

-----Original Message-----

Sent: 30 May 2019 23:23

Best wishes,

phil

-----Original Message-----

Sent: 20 May 2019 14:50

Hello,

Christoph

[email-masked]

[email-masked]

---

From: Gregory Detal <[email-masked]>
Date: Thu, 6 Jun 2019 12:24:06 +0200

Hi Phil,

On Thu, May 30, 2019 at 7:53 PM <[email-masked]> wrote:

> I think the use case a hybrid operator could be interested in is the
> following. Similar to Alexander's use case.
>
> We want to provide a particular customer with broadband access that is
> faster than their DSL alone can provide, for instance to meet some minimum
> rate to meet a target of say 20Mbps. The rate can't quite be met by DSL,
> therefore cellular or even satellite is used as a top-up. DSL capacity is
> much cheaper than cellular /satellite, therefore the ideal scheduler would
> favour DSL.

This is exactly the type of deployments that we have on the field.

> It would react reasonably quickly to increased total load above the DSL
> rate (but not so fast that the instantaneous rate from a variable rate
> source 'kicks in' the cellular when the rate over a slightly longer time
> can be met by DSL).

There are 2 possible levels: (1) macro: create a subflow when the load
reaches the DSL rate and (2) micro: schedule packets first on the DSL
subflow.
For (1) the CPE monitors the load on the DSL link  If the load is lower
than a minimum threshold, then no subflow is established over the LTE
network. If the load is above a maximum threshold, the CPE creates an
additional subflow over the LTE network. Note that (2) is always enabled as
as soon as the 2 subflows are available, the packet scheduler becomes
important. It first tries to fill the DSL link and once it is full, it
overflows over the LTE subflow.
We have deployments where (1) and (2) are enabled and others where only (2)
is enables. What is always important for a customer is that a Speedtest
always display the aggregated speed. In the first deployment mode, the
speed is increased in the middle of the test while with the second mode the
boost is observed directly.

> Note that the DSL rate is not completely static.  It supports deployments
> with a proxy in the home gateway and in the network (under the control of
> the operator),

This corresponds to the currently deployed Hybrid Access Networks.

> and deployments where there's only the proxy in the network (ie MPTCP is
> in the multi-interface phone).

This corresponds to the future ATSSS service in 5G networks that
will leverage the 0-rtt convert protocol that was started in the mptcp
working group and is now discussed within the tcpm working group.

> Possible to ensure that an individual flow goes over only one access.
> Downward direction (from network to house) more important than upward.  In
> a long-running session using a lot of bandwidth, because the amount of
> other traffic varies, it may be that this session sometimes uses just the
> DSL and sometimes is spread over both accesses.

Our experience shows that the scheduler that prioritises the DSL link is
sufficient to ensure that the traffic automatically moves to the DSL
network when the load decreases.

> On a minor point, ideally a speed test should measure the rate correctly
> (I don’t mean the scheduler identifies a speed test and favours it; rather,
> ideally the speed test shouldn’t eg measure just the DSL rate -
> alternatively re-define the speed test).
>

The Ookla speedtest is considered as the reference tool by customer and
typically opens n parallel TCP connections to saturate the available
network. Each of these connections typically creates and additional subflow
and both the DSL and the LTE link are used and tested. As said above, when
using the second deployment mode, both the DSL and the aggregated speed can
be seen but is is often tricky to tune properly. It could therefore be
interesting to think about a different and more reliable methodology to
measure hybrid access networks.

> is about the total and not per subflow.

Looking at the downstream traffic, it seems that you need to limit the rate
of the incoming traffic towards a specific IP address. This can be
implemented on the internet facing side of a HAG. This way, the total rate
of the DSL+LTE is limited because the split of the traffic is done in the
HAG (DSL is filled first). This can be done locally on the HAG and does not
require any MPTCP signalling. If IPv4 and IPv6 are used, then the rate
should apply to all the addresses assigned to the same CPE.

> In terms of mechanism, I'm open to whatever is most suitable. I'm very
> interested if the best method would mean that ideally some new
> functionality is added to the MPTCP standard (and by implication to
> MP-QUIC).
>

It is possible to add signaling to MPTCP and MPQUIC, but those signals
operate on a per-connection basis. It is difficult to have a signal that
affects a group of connections. Another point that needs to be addressed is
the trust of the device that sends the signal. In an hybrid access network,
should the CPE send inside an MPTCP option the maximum rate at which the
HAG should deliver data ? If you trust all CPEs, maybe, but if customers
can select their own CPE or modify them, there are some risks which such
signals...

> Cc: [email-masked]; Ashutosh prakash <[email-masked]>;
> Nagesh shamnur <[email-masked]>
> Thus, a "maximum-rate" option on a per-subflow level would allow to send
> the rate to 0, which would drain the subflow.
>
>
> Christoph
>

-- 
 <http://www.tessares.net> Gregory Detalº | Co-Founder
& CTO
[email-masked] | +32 486 83 50 51 <gregory.detal%40tessares.net>
Tessares SA | Hybrid Access Solutions
www.tessares.net
1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium
<https://www.google.com/maps?q=1+Avenue+Jean+Monnet,+1348+Ottignies-Louvain-la-Neuve,+Belgium>

ºTRAAL Consulting SPRL

-- 

Disclaimer: https://www.tessares.net/mail-disclaimer/ 
<https://www.tessares.net/mail-disclaimer/>

---

From: <[email-masked]>
Date: Tue, 18 Jun 2019 09:29:12 +0000

Sorry for the delay, been away.
Gregory – thanks, really interesting info.
In summary it sounds like my scenario works ok without any extra signalling or mods, with the following caveats:

-          Assumes DSL vs 4/5G vs satellite etc preference is fixed (basically: fill DSl first). I can see this is usually true, but question is whether a particular app the perference may be the other way round?

-          In the non-proxy mode the server somehow needs to know your preference – not sure how to indicate this?

-          Does anyone care about the upstream traffic?
Thanks
phil

From: Gregory Detal [mailto:[email-masked]]
Sent: 06 June 2019 11:24
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: cpaasch=[email-masked]; Olivier Bonaventure <[email-masked]>; multipathtcp <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

On Thu, May 30, 2019 at 7:53 PM <[email-masked]<mailto:[email-masked]>> wrote:
I think the use case a hybrid operator could be interested in is the following. Similar to Alexander's use case.

We want to provide a particular customer with broadband access that is faster than their DSL alone can provide, for instance to meet some minimum rate to meet a target of say 20Mbps. The rate can't quite be met by DSL, therefore cellular or even satellite is used as a top-up. DSL capacity is much cheaper than cellular /satellite, therefore the ideal scheduler would favour DSL.

It would react reasonably quickly to increased total load above the DSL rate (but not so fast that the instantaneous rate from a variable rate source 'kicks in' the cellular when the rate over a slightly longer time can be met by DSL).

There are 2 possible levels: (1) macro: create a subflow when the load reaches the DSL rate and (2) micro: schedule packets first on the DSL subflow.
For (1) the CPE monitors the load on the DSL link  If the load is lower than a minimum threshold, then no subflow is established over the LTE network. If the load is above a maximum threshold, the CPE creates an additional subflow over the LTE network. Note that (2) is always enabled as as soon as the 2 subflows are available, the packet scheduler becomes important. It first tries to fill the DSL link and once it is full, it overflows over the LTE subflow.
We have deployments where (1) and (2) are enabled and others where only (2) is enables. What is always important for a customer is that a Speedtest always display the aggregated speed. In the first deployment mode, the speed is increased in the middle of the test while with the second mode the boost is observed directly.

Note that the DSL rate is not completely static.  It supports deployments with a proxy in the home gateway and in the network (under the control of the operator),

and deployments where there's only the proxy in the network (ie MPTCP is in the multi-interface phone).

This corresponds to the future ATSSS service in 5G networks that will leverage the 0-rtt convert protocol that was started in the mptcp working group and is now discussed within the tcpm working group.

Possible to ensure that an individual flow goes over only one access. Downward direction (from network to house) more important than upward.  In a long-running session using a lot of bandwidth, because the amount of other traffic varies, it may be that this session sometimes uses just the DSL and sometimes is spread over both accesses.

Our experience shows that the scheduler that prioritises the DSL link is sufficient to ensure that the traffic automatically moves to the DSL network when the load decreases.

On a minor point, ideally a speed test should measure the rate correctly (I don’t mean the scheduler identifies a speed test and favours it; rather, ideally the speed test shouldn’t eg measure just the DSL rate - alternatively re-define the speed test).

The Ookla speedtest is considered as the reference tool by customer and typically opens n parallel TCP connections to saturate the available network. Each of these connections typically creates and additional subflow and both the DSL and the LTE link are used and tested. As said above, when using the second deployment mode, both the DSL and the aggregated speed can be seen but is is often tricky to tune properly. It could therefore be interesting to think about a different and more reliable methodology to measure hybrid access networks.

Looking at the downstream traffic, it seems that you need to limit the rate of the incoming traffic towards a specific IP address. This can be implemented on the internet facing side of a HAG. This way, the total rate of the DSL+LTE is limited because the split of the traffic is done in the HAG (DSL is filled first). This can be done locally on the HAG and does not require any MPTCP signalling. If IPv4 and IPv6 are used, then the rate should apply to all the addresses assigned to the same CPE.

It is possible to add signaling to MPTCP and MPQUIC, but those signals operate on a per-connection basis. It is difficult to have a signal that affects a group of connections. Another point that needs to be addressed is the trust of the device that sends the signal. In an hybrid access network, should the CPE send inside an MPTCP option the maximum rate at which the HAG should deliver data ? If you trust all CPEs, maybe, but if customers can select their own CPE or modify them, there are some risks which such signals...

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Ashutosh prakash <[email-masked]<mailto:[email-masked]>>; Nagesh shamnur <[email-masked]<mailto:[email-masked]>>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

--
[Tessares SA]<http://www.tessares.net>
Gregory Detalº | Co-Founder & CTO
[email-masked] | +32 486 83 50 51<mailto:gregory.detal%40tessares.net>
Tessares SA | Hybrid Access Solutions
www.tessares.net<http://www.tessares.net>
1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium<https://www.google.com/maps?q=1+Avenue+Jean+Monnet,+1348+Ottignies-Louvain-la-Neuve,+Belgium>

ºTRAAL Consulting SPRL

________________________________

---

From: <[email-masked]>
Date: Tue, 18 Jun 2019 09:29:22 +0000

Sorry for the delay in following up, been away. In-line below [phil2]

From: Madhan Raj Kanagarathinam [mailto:[email-masked]]
Sent: 04 June 2019 08:55
To: Eardley,PL,Philip,TUD1 R <[email-masked]>
Cc: [email-masked]; cpaasch=[email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: RE:(4) [multipathtcp] Regarding rate control at a subflow level

Phil,

[phil] So I think that (for downstream traffic) the server gets a hint from the client about the maximum capacities of each of the interfaces (I assume this is what you mean by “client assisted RR” – is that right?) (I assume this is a one-off hint, rather than an on-going signal that reflects changes in the spare capacity of each interface.).  I agree this is better than having no info, but presumably not (?nearly?) as good as getting info about the current congestion and BDP. It also creates the problem about how to signal this hint from the client to the server (whereas MP/TCP already provides a signal for congestion).

 So my question is why people would follow for this approach? I feel like I must be missing something, it seems to require a new signal and give worse performance – but perhaps it is simpler to implement?

[/phil2]

[phil] I can partly understand the scenario. But some things that I think should be taken into account:-

[1] surely a capped max data rate over 5G is a contractual /policy setting, so the operator already know the value and thus it doesn’t need to be signalled? ;

[2] the capping limit surely applies to all the mobile client’s traffic (and may include some non-MP/TCP traffic), which will be going to many different servers ;

[3] limiting the mobile data rate to a particular value solves the problem of using too much mobile data to some extent–  but I think ideally you’d like something more flexible which….

[4] can potentially reflect user preferences (eg for this app at this moment I’m prepared to use up more of my mobile data)

[5] and potentially reflect operator preferences /knowledge (eg at the moment the DSL has a low throughput so operator needs to allow more to be sent on the mobile interface, in order to hit a 20Mbps target contractual rate)

[6] and potentially reflect knowledge /preferences about relative costs (eg satellite is much more expensive than wired DSL. I guess this info is reasonably static)

Re [1] & [2] see Gregory’s email, which says to enforce this at the HAG

It would be great to discuss this list (essentially requirements / scenarios / assumptions) and delete /modify /add to it. I think it will help the next stage of Multipath at the IETF (MPTCP & MP-QUIC)

Thanks

Phil

[/phil2]

Regards,

Madhan

--------- Original Message ---------

Sender : [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>

Madhan,

Thanks,
phil
From: Madhan Raj Kanagarathinam [mailto:[email-masked]]
Sent: 31 May 2019 10:19
To: Nagesh shamnur <[email-masked]<mailto:[email-masked]>>; Eardley,PL,Philip,TUD1 R <[email-masked]<mailto:[email-masked]>>; cpaasch=[email-masked]<mailto:cpaasch=[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; Ashutosh prakash <[email-masked]<mailto:[email-masked]>>
Subject: RE:(2) [multipathtcp] Regarding rate control at a subflow level

Hi,

Regards,

Madhan

--------- Original Message ---------

Hi Phil,

Regards,

Nagesh S

-----Original Message-----

Sent: 30 May 2019 23:23

Best wishes,

phil

-----Original Message-----

Sent: 20 May 2019 14:50

Hello,

Christoph

[http://ext.samsung.net/mail/ext/v1/external/status/update?userid=madhan.raj&do=bWFpbElEPTIwMTkwNjA0MDc1NDM3ZXBjbXM1cDhiMWM4NTEyMzJiOWRiYzliMWY1MjIxMTRlZGE5N2FjMCZyZWNpcGllbnRBZGRyZXNzPXBoaWxpcC5lYXJkbGV5QGJ0LmNvbQ__]

---

From: <[email-masked]>
Date: Tue, 18 Jun 2019 09:45:04 +0000

Nagesh,

Thanks, interesting to hear about an upstream use case. Would be good to explore the scenario in a bit more detail. 
As you say, the congestion control algorithm can be used to stop one connection starving others (ie server / network-side drop packets to slow down a connection). I think you're hinting that this may not be good enough in all scenarios (<< but there can be cases where the application decides to limit the bandwidth to avoid possibly overload kind of scenarios.>>) - would you explain this a bit more please?
I'm trying to understand what benefit we might gain from some extra signalling (whether of preferences, a rate to limit to, or whatever) and therefore whether it's something we should think about doing. 

Thanks
phil

ps Sorry for delay, been away.
-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]] 
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: <[email-masked]>
Date: Tue, 18 Jun 2019 10:14:53 +0000

Christoph, 
Coming back to your comment:

<<

Christoph,

Can you explain the 3rd bullet please? I think "none" must be a typo (I assume for 'one' / you)
Not sure I get why the approach in the bis document doesn’t work? Surely if I set the B flag on a subflow then traffic is no longer sent on it, even if the other subflow is dropping a lot of packets? (but presumably this case would mean that the packet rate dropped a lot). 

<<
Another use of the MP_PRIO option is to set the 'B' flag on a subflow
   to cleanly retire its use before closing it and removing it with
   REMOVE_ADDR Section 3.4.2, for example to support make-before-break
   session continuity, where new subflows are added before the
   previously used ones are closed.

Thanks
phil

---

From: Nagesh shamnur <[email-masked]>
Date: Thu, 20 Jun 2019 05:28:28 +0000

Hi Phil,
	Generally for the downlink traffic, server tries to share the available bandwidth among all the available connections (flows and subflows) and this would be achieved by Congestion control algorithm. Any algorithm would try to scale up to pipe limit. If server needs to throttle the speed then it wouldn't be possible with congestion control algorithm alone and server can realize the same with many ways and one of them would be to ask application on the server side to slow down the packet sending rate.
	How will server achieve the same to tell client side to throttle speed? Dropping packets on server side implicitly or explicitly can achieve it, but IMO it is not the best way. 
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 18 June 2019 15:15
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: <[email-masked]>
Date: Thu, 20 Jun 2019 15:14:24 +0000

Nagesh,
For upstream, isn’t the most likely case that the client side knows the available bandwidth on the multiple paths? so it can control the rate at which it sends traffic upstream on the different paths. (ie the server doesn’t need to tell the client side to throttle speed, because the client side can do this on its own)

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]] 
Sent: 20 June 2019 06:28
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 18 June 2019 15:15
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: <[email-masked]>
Date: Thu, 20 Jun 2019 15:24:18 +0000

Christoph,
Do other people think it's an important requirement to have graceful close of a subflow? (ie something better than sending TCP_RST on the subflow)

(If we do, then I guess we can discuss variety of possible mechanisms)

Thanks
Phil

Ps Sorry, I missed this one.

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: Nagesh shamnur <[email-masked]>
Date: Fri, 21 Jun 2019 05:13:15 +0000

Dear Phil,
	Yes, client can control the rate at which data is pushed to server, but what if server what to throttle the bandwidth usage of the client targeted towards the server due to various server policies. One of them could be to reduce the load on the server.

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 20 June 2019 20:44
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 20 June 2019 06:28
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 18 June 2019 15:15
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 21 Jun 2019 08:13:35 +0000

Phil,

> Do other people think it's an important requirement to have graceful close of a subflow? (ie something better than sending TCP_RST on the subflow)

I believe that this would be an important feature for applications that 
understand MPTCP.

Olivier

---

From: Madhan Raj Kanagarathinam <[email-masked]>
Date: Fri, 21 Jun 2019 14:36:39 +0530

>> Its important for Smartphones for power saving in idle time (screen off).

--------- Original Message ---------

Date : 2019-06-20 20:54 (GMT+5:30)

Christoph,

Thanks
Phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: <[email-masked]>
Date: Tue, 25 Jun 2019 09:51:39 +0000

Nagesh,
Sure - but I don’t think this is related to the use of multiple paths (ie in your example, the server wants to throttle the client - independently of the client's use of multiple paths)
phil
-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]] 
Sent: 21 June 2019 06:13
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Dear Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 20 June 2019 20:44
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 20 June 2019 06:28
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 18 June 2019 15:15
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph

---

From: Nagesh shamnur <[email-masked]>
Date: Wed, 26 Jun 2019 06:15:29 +0000

Hi Philip,
	Thanks for the reply. I beg to differ. Since the problem I believe is related to all the multipath protocols like MPTCP, MPQUIC etc. I would like to quote a scenario, where server wants to throttle the rate to an extent that, it is more than the individual sub flows but much lesser than the sum of all the available sub flows. Existing schedulers can achieve this by loading the primary sub flow and once it reaches the limit, subsequent data would be sent on secondary sub flow. But the point here is, if the client pushes data more than the allotted limit, it can overrun the limit which server expects from the client.
	I would be interested to know, if it can be realized with any of the existing techniques.
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 25 June 2019 15:22
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,
Sure - but I don’t think this is related to the use of multiple paths (ie in your example, the server wants to throttle the client - independently of the client's use of multiple paths) phil -----Original Message-----

Dear Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 20 June 2019 20:44
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 20 June 2019 06:28
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	
Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 18 June 2019 15:15
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

Thanks
phil

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 03 June 2019 15:46
To: Nagesh shamnur <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Nagesh,

-----Original Message-----
From: Nagesh shamnur [mailto:[email-masked]]
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <[email-masked]>; cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,

Regards,
Nagesh S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 30 May 2019 23:23
To: cpaasch=[email-masked]; [email-masked]
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Best wishes,
phil

-----Original Message-----
From: multipathtcp [mailto:[email-masked]] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <[email-masked]>
Cc: [email-masked]; Ashutosh prakash <[email-masked]>; Nagesh shamnur <[email-masked]>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level

Hello,

Christoph