Subject: [iccrg] Follow-up to LEDBAT++ presentation

From: Michael Welzl <[email-masked]>
Date: Mon, 13 Nov 2017 16:54:52 +0800

Hi,

Just a small follow up on the LEDBAT++ presentation today, where you talked about issues that you found with LEDBAT:
David Ros and I published this paper documenting similar findings several years ago:

David Ros, Michael Welzl: "Assessing LEDBAT's Delay Impact", IEEE Communications Letters 17(5), pp. 1044-1047, 2013.
http://ieeexplore.ieee.org/document/6496997/?tp=&arnumber=6496997&queryText=assessing%20ledbat%27s%20delay%20impact

This also includes the “delay drift” problem.

Cheers,
Michael

---

From: Kuhn Nicolas <[email-masked]>
Date: Mon, 13 Nov 2017 10:24:19 +0000

Hi, 

On top of Michael's mail, I would like to share some comments.

- Multiple LEDBAT-like protocols
There are multiple "lower than best effort" protocols in the wild that seems relevant (e.g. CDG [1][2], FLOWER [3]) since they consider LEDBAT's issues in their state-of-the art.  Moreover, it is worth pointing out that CDG is in the Linux kernel. You may want to consider them in the evaluation of your proposal.

That being said, it is not clear to me how one can compare lower-than best effort protocols, since it depends on what you allow them to grab ( the "best" LBE protocol could just send nothing so that it does not disturb anyone :)) depending on their applicative-constraints. I think that the dead-line aware approach is relevant in this context.

- Interactions between LBE and AQM
You mentioned in your future work the interest for the interactions between AQM and LEDBAT++.
FYI authors of [4] and [5] have discussed this. 

- On the rationale of using fixed target delays
This point goes for either the LEDBAT++, TCP LoLa and BBR-related presentations. 
Using 'fixed' target delays does not seem to be a good option if protocols are to somehow be 'network independent' - unless the selected timers are relevant for data-centers and satellite corner cases ?

[1] David A. Hayes, Grenville Armitage (May 2011). Revisiting TCP congestion control using delay gradients. 10th International IFIP TC 6 Networking Conference (NETWORKING 2011)
[2] Tangenes, Tor Christian, David A. Hayes, Andreas Petlund and David Ros. “Evaluating CAIA Delay Gradient as a Candidate for Deadline-Aware Less-than-Best-Effort Transport.” (2017).
[3] Trang, Si Quoc Viet and Lochin, Emmanuel and Baudoin, Cédric and Dubois, Emmanuel and Gelard, Patrick FLOWER - Fuzzy Lower-than-Best-Effort Transport Protocol (2015) In: The 40th IEEE Conference on Local Computer Networks (LCN), 26 October 2015 - 29 October 2015
[4] Y. Gong, D. Rossi, C. Testa, S. Valenti and M. D. Täht, "Fighting the bufferbloat: On the coexistence of AQM and low priority congestion control," 2013 Proceedings IEEE INFOCOM, Turin, 2013, pp. 3291-3296.
[5] https://www.ietf.org/proceedings/93/slides/slides-93-iccrg-2.pdf 

Nicolas KUHN
CNES/DSO/NT/ST
(+33)5 61 27 32 13

-----Message d'origine-----
De : iccrg [mailto:[email-masked]] De la part de Michael Welzl
Envoyé : lundi 13 novembre 2017 09:55
À : Praveen Balasubramanian
Cc : [email-masked]
Objet : [iccrg] Follow-up to LEDBAT++ presentation

Hi,

Cheers,
Michael

_______________________________________________
iccrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/iccrg

---

From: Mario Hock <[email-masked]>
Date: Tue, 14 Nov 2017 10:24:04 +0100

Nicolas, thanks for your input regarding fixed target delays.

One of the major things we want to investigate with TCP LoLa is the 
dependence on queuing delay to achieve high throughput. High throughput 
and low delay are often presumed as conflicting goals. However, we 
believe that this is mostly due to experiences with existing congestion 
controls, especially Reno and Cubic, that actually need high queuing 
delays in order to achieve a high throughput.

Some networking scenarios definitely are tougher than others, e.g., 
links with strongly varying data rates in combination with long RTTs. 
That's why we want to look into the potential of explicit network 
feedback that supports the congestion control with information.

With TCP LoLa the target has to be equal for all flows that share a 
bottleneck. However, in a controlled environment a different value could 
be used than in the global Internet. For a Split TCP setup, like you 
showed in your presentation, the splitting middlebox could choose an 
arbitrary delay target.

Best regards, Mario

---

From: Emmanuel Lochin <[email-masked]>
Date: Tue, 14 Nov 2017 12:19:23 +0100

Dear Mario,

I saw this presentation at ICCRG and I think the approach is 
interesting. However, I feel the main problem is when route changes 
occur. I remember that during the talk there were 2 o 3 heuristics used 
within TCP Lola and one of them was to mitigate route change events. I 
think such heuristic could be a weakness and might strongly impact on 
the feasibility of such proposal. So I would be curious to get your 
paper to better understand Lola.

Regards

Emmanuel

On 14/11/2017 10:24, Mario Hock wrote:
> dependence on queuing delay to achieve high throughput. High 
> throughput and low delay are often presumed as conflicting goals. 
> However, we believe that this is mostly due to experiences with 
> existing congestion controls, especially Reno and Cubic, that actually 
> need high queuing delays in order to achieve a high throughput.
>
> bottleneck. However, in a controlled environment a different value 
> could be used than in the global Internet. For a Split TCP setup, like 
> you showed in your presentation, the splitting middlebox could choose 
> an arbitrary delay target.
>
> Best regards, Mario
>
>
>> Using 'fixed' target delays does not seem to be a good option if 
>> protocols are to somehow be 'network independent' - unless the 
>> selected timers are relevant for data-centers and satellite corner 
>> cases ?
>

-- 
Emmanuel LOCHIN
Professeur ISAE

ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE - http://www.isae-supaero.fr
Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 30

---

From: Kuhn Nicolas <[email-masked]>
Date: Wed, 15 Nov 2017 01:57:40 +0000

Dear Emmanuel, Mario, 

I agree with Emmanuel in the fact that heuristics based on fixed value of queue states might impact the feasibility of the proposal. 
I think that there are cases where TCP Lola may encounter issues (variable link rate, impact on other TCP traffic) when the state of the queue can hardly be estimated at the end point. 

Moreover: 
1- IRTF 'Common TCP Evaluation Suite' could help
The IRTF 'Common TCP Evaluation Suite' propose a list of scenarios under which TCP proposals could be assessed. 
As one example, the scenarios include wireless LAN where section 4.4.5.2 proposes to include variable delay and bitrate. 
This evaluation suite also proposes to assess the impact on standard TCP traffic (section 6) which could be interesting for your case. 

2- Delay-based TCP flavors and high throughput and low queue trade-off
Other delay-based protocols such as CDG target high throughput, low queue and coexistency with loss-based transport protocols. IMHO the main issue with delay based protocols is how they can compete with actually deployed loss-based protocols. You may be interested in considering CDG in your future work.

It is just complicated to fulfill these two goals when you consider loss-based transport protocols that do not exploit information from the network and/or variable capacity on the links. Loss-based TCP variants do not "need" high queuing delay - since CUBIC and RENO are loss-based, they just naturally fulfill buffers. Queuing is momentarily however relevant when it comes to cope up with variable capacity of burst of incoming data at the bottleneck. AQM/ECN can thus be exploited to avoid a permanent queue and allow loss-based protocols to actually provide both high throughput and low queuing delay.

3- TCP Split use-case
FYI, in the split TCP setup, we can actually gather much more information from the actually available capacity - knowing the network's characteristics help in setting up TCP internal parameters. 

4- Side note
Would it be relevant to update the 'common evaluation suite' ?
I think that what is required for evaluation suite is running code - such as it was done for the AQM evaluation suite recently. 
We are currently working on providing running code for the common TCP evaluation suite.

Kind regards,

Nicolas KUHN
CNES/DSO/NT/ST
(+33)5 61 27 32 13

-----Message d'origine-----
De : iccrg [mailto:[email-masked]] De la part de Emmanuel Lochin
Envoyé : mardi 14 novembre 2017 12:19
À : [email-masked]
Objet : Re: [iccrg] Follow-up to LEDBAT++ presentation

Dear Mario,

I saw this presentation at ICCRG and I think the approach is interesting. However, I feel the main problem is when route changes occur. I remember that during the talk there were 2 o 3 heuristics used within TCP Lola and one of them was to mitigate route change events. I think such heuristic could be a weakness and might strongly impact on the feasibility of such proposal. So I would be curious to get your paper to better understand Lola.

Regards

Emmanuel

On 14/11/2017 10:24, Mario Hock wrote:
>> - On the rationale of using fixed target delays This point goes for 
>> either the LEDBAT++, TCP LoLa and BBR-related presentations.

--
Emmanuel LOCHIN
Professeur ISAE

ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE - http://www.isae-supaero.fr Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 30

---

From: Roland Bless <[email-masked]>
Date: Wed, 15 Nov 2017 04:18:56 +0100

Hi Emmanuel,

> paper to better understand Lola.

The heuristic is mentioned on slide 16 in the backup slides.
The minimum filter is quite robust with respect to changes
toward lower values. However, it alone will never detect
when the new path RTT minimum gets _longer_ due to path changes.
The current heuristic checks whether there has been an RTT measured
"near" the current RTTmin within some time period.
So LoLa will react to such path changes, but on a coarser time scale.

As already mentioned during the talk, we did not thoroughly optimize the
parameters, so there is room for improvement. And yes, every heuristic
has a trade-off, in this case reaction time to path changes, lower
utilization, and stability.
However, LoLa aims to measure a standing queue and this makes sense for
periods in the order of several RTTs and the assumption is that path
changes will occur on even larger timescales (maybe except for mobile
hosts, but in this case you can locally provide interface change hints
etc.). I'll send a separate mail when the LoLa-Paper is available
for download.

Regards,
 Roland

---

From: Mario Hock <[email-masked]>
Date: Wed, 15 Nov 2017 11:13:31 +0100

Dear Emmanuel,

thanks for your interest in TCP LoLa, we just uploaded the paper (see my 
other mail). In addition to Roland's answer I want to add the following.

Route changes can, indeed, impact the performance of TCP LoLa. In a way, 
a change to a longer route is for LoLa what a random loss is for 
Cubic/Reno: Performance will be reduced for some time, due to a 
misinterpreted implicit congestion signal. But, as Roland wrote, TCP 
LoLa can recover from that after some time. A change to shorter route 
will be discovered quickly. Note that tailored decrease is performed 
quite often and designed to drain the queue completely. Although not 
explicitly stated in the paper, TCP LoLa further decreases its 
congestion window when it sees a new RTT_min after Tailored Decrease, 
which makes it very robust against being to aggressive due to a wrong 
RTT_min value..

That said, route changes are one of the reasons we're looking into 
explicit network feedback. Another goal is to quickly utilize available 
bandwidth without an "overshoot". This could be relevant for links with 
variable capacity and certainly is for the startup of short flows.

Best regards, Mario

---

From: Mario Hock <[email-masked]>
Date: Wed, 15 Nov 2017 13:10:38 +0100

Dear Nicolas,

I want to state our motivation behind the usage of queuing delay 
thresholds more clearly:

TCP LoLa tries to measure the "standing queue" at the bottleneck. 
Standing queue is meant as in the context of CoDel, a queue that 
persists longer than one RTT, i.e., data of the "last" congestion window 
is still in the queue when data from the subsequent congestion window 
arrives. (Of course it's a sliding window, but can be abstracted that way.)

The size of the standing queue directly correlates with the congestion 
windows (CWnd) of the senders. If the CWnds (in total) are too large, a 
standing queue builds up. The size of the standing queue indicates how 
much the CWnds have to be reduced in order to keep the throughput high 
but empty the queue (i.e., the CWnds correspond to the 
bandwidth-delay-product (BDP)).

We try to filter-out short increases of the queuing delay that might be 
caused by a bursty sender or some short term cross traffic in order to 
get size of the standing queue.

Loss-based TCP variants, especially Reno/Cubic, multiplicatively 
decrease the CWnd after a congestion signal. This means, in a large 
buffer (tail-drop, no random loss) they build up a large standing queue. 
After a multiplicative decrease the CWnds will still be larger than the 
BDP. Thus, after a decrease there will still be a standing queue and the 
throughput is constantly high (even after the loss).

In a small buffer, however, the standing queue is small. This means the 
CWnds nearly fit the BDP. After a multiplicative decrease of the CWnd, 
it is too small to fill the pipe (depending on the scenario). Therefore, 
Reno/Cubic need large buffers for high throughput. This can be mitigated 
to some extent with AQM (due to loss-desynchronization), especially in 
combination with ABE (alternative backoff with ECN). Complementary, with 
TCP LoLa we focus on the potential/possibilities a (new) congestion 
control has.

We do consider CDG. However, as explained in my other mail, we do not 
want to compete with loss-based congestion controls in (large) tail-drop 
buffers. Since this is only possible by giving up the low delay goal, 
which is our main goal. Therefore, we think the network has to be 
involved here and are working on that. Also CDG uses an additive 
increase like Reno and is therefore not scalable to high speed wide are 
links. Recently, CDG was mostly mentioned in the context of lower than 
best effort congestion controls.

Best, Mario

---

From: Roland Bless <[email-masked]>
Date: Wed, 15 Nov 2017 13:48:59 +0100

Hi,

On 15.11.2017 at 02:57 Kuhn Nicolas wrote:
> ....

> We are currently working on providing running code for the common TCP evaluation suite.

That would be helpful indeed. It is a lot of work
to do these tests.

Regards,
 Roland

---

From: David Hayes <[email-masked]>
Date: Thu, 16 Nov 2017 10:22:16 +0800

I'm a bit slow in the conversation, but....

In our work on DA-LBE we have a parameter \phi that was quickly glossed
over in the ICCRG talk. You will find details in the paper
(http://dl.ifip.org/db/conf/networking/networking2017/1570334752.pdf). This
parameter is the measured ratio of the probability of a congestion
indication weighted by the congestion responses of the used congestion
control mechanism with respect to a "standard" mechanism. In the paper
the "standard" mechanism used as an example is cubic. Its use is to
better enable congestion controls with different ideas of what
congestion is (ie loss and delay, etc or combinations) to compete more
fairly when necessary. It is only suitable for flows that are around
long enough to be able to collect the statistics (the scenario in the
DA-LBE use case), but may well be a useful way of considering congestion
control competition for the work you are doing.

Kind regards,

David

--
David Hayes