Subject: Re: [rmcat] [iccrg] draft-johansson-rmcat-scream-cc

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 3 Jul 2014 11:08:46 +0200

Hi Dave,

you definitely bring up a valid point and I believe you that this strategy 
helps to reduce your workload. Still I (personally and as chair) would like 
to comment on this. 

First of all, the draft has pseudo code attached which should provide an easy 
start to re-implement and re-produce the results. I really appreciate this. 
Further, this is new work with first results and I personally prefer to have 
published this early so other people might be aware of this work and can do 
an own implementation if they are interested. As a chair I think that it's 
actually good to have independent implementations from different people 
(only) based on the draft because that gives most confident that the draft is 
complete and understandable. In the current state of rmcat we are looking for 
independent evaluations to ensure that an algorithm is at least safe to 
experiment with in the wild. Of course that's easier if there are 
implementations already available. Anyway the author just started on this 
work and are planning to provide implementations in future, so that's 
absolutely the right way to do this.

Mirja

On Thursday 26 June 2014 19:06:29 Dave Taht wrote:
> On Thu, Jun 26, 2014 at 5:10 AM, Zaheduzzaman Sarker
>
> <[email-masked]> wrote:
> > Hi Luca,
> >
> > The simulation code is implemented in our proprietary simulator (I should
> > thank my MS thesis workers who contributed a lot). Thus cant really share
> > the code but what I can share is some simulation results.
>
> I have decided that I will no longer consider experimental results
> that cannot easily be reproduced independently, and will ignore future
> articles, internet drafts, and proposals that do not come with
> sufficient source code to reproduce independently.
>
> Nearly every published experiment that actually supplied sufficient
> information, raw data and code to reproduce it - that I've had the
> chance to examine closely and/or reproduce in more varied
> circumstances - has been flawed, and I have no reason whatsoever to
> believe that code I cannot see and run is not flawed in some respect,
> either.
>
> If I apply this criterion as a stage-1 filter, my workload drops
> enormously.
>
> Both chrome and firefox have mostly working webrtc implementations
> now, and whatever future algorithms proposed should be able to be
> easily hacked into those and tried in real life.
>
> open source ns2, ns3, and mininet simulators exist as well as a basis
> for future work.
>
> My policy is obviously not the ietf's nor the position of the chairs of
> this wg, but it is in the spirit of "rough consensus and running code".
>
> > Note that in our implementation we followed latest update for varying
> > threshold from the GCC draft
> >
> > If you want GCC related logs particularly for LTE simulations I cant not
> > provide that.We simulate lots of users and dont log all the GCC related
> > variables.
> >
> > However, I have attached simulation logs from two test cases-
> > 1. a staircase bandwidth test , that I found interesting in one of the
> > papers.
> > 2. and test case 5.1 from
> > http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01.
> >
> > I hope those plots will give you an idea about the implementation we have
> > and how it is behaving.
> >
> > I would suggest you run tests from
> > http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01 as well and
> > then we can compare the results.
> >
> > BR
> >
> > Zahed
> >
> > On 2014-06-25 17:32, Luca De Cicco wrote:
> >> Dear Zahed,
> >>
> >> can you share some information about the GCC simulation code?
> >>
> >> thanks,
> >> Luca
> >>
> >>
> >> On Wed, Jun 25, 2014 at 2:46 PM, Ingemar Johansson S
> >> <[email-masked]
> >> <mailto:[email-masked]>> wrote:
> >>
> >>     Hi____
> >>
> >>     __ __
> >>
> >>     Thanks for the interest in this work. ____
> >>
> >>     __ __
> >>
> >>     Crosspost to RMCAT as well, hope that is Ok____
> >>
> >>     Answers/comments inline____
> >>
> >>     __ __
> >>
> >>     /Ingemar____
> >>
> >>     __ __
> >>
> >>     *From:*Luca De Cicco [mailto:[email-masked]
> >>     <mailto:[email-masked]>]
> >>     *Sent:* den 25 juni 2014 13:23
> >>     *To:* Ingemar Johansson S
> >>     *Cc:* Dave Taht; [email-masked] <mailto:[email-masked]>
> >>     *Subject:* Re: [iccrg] draft-johansson-rmcat-scream-cc____
> >>
> >>     __ __
> >>
> >>     Dear Ingemar,____
> >>
> >>     __ __
> >>
> >>     I have some questions. ____
> >>
> >>     __ __
> >>
> >>
> >>     1) In the architecture figure (Figure 2) you show the case of two
> >>     videos produced by the ____
> >>
> >>
> >>     same source and going to the same destination (both videos are
> >>     encoded and fed to the same UDP socket),____
> >>
> >>
> >>     am I right? In the case SCReAM is used in a WebRTC context (in a
> >>     browser), what use case ____
> >>
> >>     are you imagining?____
> >>
> >>
> >>     [IJ] This was just to highlight how two sources are handled, I am
> >>     fully aware that it is probably not the most common use case, on the
> >>     other hand I don’t know how WebRTC will be used in the future. If
> >>     there are two destinations then one will need to congestion control
> >>     contexts (one for each UDP socket). The bandwidth distribution may
> >>     then be based on rate shaping but another, possibly more useful,
> >>     option is to target the congestion window functions directly, for
> >>     instance if the congetsion window for one flow is knocked down just
> >>     a little then the other flow will grab the free capacity
> >>     automatically. Another option is to adjust the gain factors for the
> >>     CWND calculations.____
> >>
> >>     __ __
> >>
> >>
> >>     2) How did you implement GCC algorithm? Do you have a paper where I
> >>     can see the dynamics of GCC (for instance____
> >>
> >>
> >>     the rate computed by the receiver side controller, the estimated one
> >>     way queuing delay variation "m")?____
> >>
> >>
> >>     [IJ] The GCC algorithm was implemented into our simulation platform
> >>     by Zahed, it is based in the availabe code and the drafts. We have
> >>     logging possibilities in our simulator but I don’t know of any paper
> >>     around this.____
> >>
> >>     __ __
> >>
> >>
> >>     3) You show average bitrate in the tables. There's an odd situation
> >>     where GCC gets a 50% packet loss ratio in the case of____
> >>
> >>
> >>     16 users/cell but with an average rate of 61 kbps. Can you elaborate
> >>     on this?____
> >>
> >>
> >>     [IJ] Could be an effect of a much too high load in combination with
> >>     a high mobility, could be an effect of large amount of packet losses
> >>     due to network queue overflow. I will BTW update the evaluation
> >>     results for GCC soon, it seems like it was tuned a but too
> >>     sensitive, with the new setting one get higher bitrates, but
> >>     unfortunately also the tail latency will increase more quickly with
> >>     increased load. ____
> >>
> >>     __ __
> >>
> >>     Cheers,____
> >>
> >>     Luca____
> >>
> >>     __ __
> >>
> >>
> >>     On Wed, Jun 18, 2014 at 11:37 AM, Ingemar Johansson S
> >>     <[email-masked]
> >>     <mailto:[email-masked]>> wrote:____
> >>
> >>
> >>     Hi
> >>
> >>     Besides the example code there is no ready to compile code available
> >>     (at least nothing that I know of). But on the other hand the
> >>     solution has similarities to TFWC (TCP Friendly Window-based
> >>     Congestion control) that was devised by Soo-Hyun Choi
> >>     (http://tfwc.hackerslab.eu/ ), perhaps that can be used as a basis ?
> >>
> >>     /Ingemar____
> >>
> >>      > -----Original Message-----
> >>      > From: Dave Taht [mailto:[email-masked]
> >>
> >>     <mailto:[email-masked]>]
> >>
> >>      > Sent: den 17 juni 2014 20:00
> >>      > To: Ingemar Johansson S
> >>      > Cc: [email-masked] <mailto:[email-masked]>
> >>      > Subject: Re: [iccrg] draft-johansson-rmcat-scream-cc
> >>      >
> >>      > My usual request: Is there running code somewhere?
> >>      >
> >>      > On Tue, Jun 17, 2014 at 10:35 AM, Ingemar Johansson S
> >>      > <[email-masked]
> >>
> >>     <mailto:[email-masked]>> wrote:
> >>      > > Hi
> >>      > >
> >>      > >
> >>      > >
> >>      > > This draft can be of interest for those of you interested in
> >>      > > congestion control for realtime media.
> >>
> >>    
> >> http://www.ietf.org/internet-drafts/draft-johansson-rmcat-scream-cc-00
> >>
> >>      > > .txt
> >>      > >
> >>      > >
> >>      > >
> >>      > > Abstract : “This memo describes a rate adaptation framework for
> >>      > > conversational
> >>      > >
> >>      > >    video services.  The solution conforms to the packet
> >>
> >>     conservation
> >>
> >>      > >    principle and uses a hybrid loss and delay based congestion
> >>
> >>     control
> >>
> >>      > >    algorithm.  The framework is evaluated over both simulated
> >>      > > bottleneck
> >>      > >
> >>      > >    scenarios as well as in a LTE (Long Term Evolution) system
> >>      > > simulator
> >>      > >
> >>      > >    and is shown to achieve both low latency and high video
> >>
> >>     throughput
> >>
> >>      > > in
> >>      > >
> >>      > >    these scenarios”
> >>      > >
> >>      > >
> >>      > >
> >>      > > /Ingemar
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > > =================================
> >>      > >
> >>      > > Ingemar Johansson  M.Sc.
> >>      > >
> >>      > > Senior Researcher
> >>      > >
> >>      > >
> >>      > >
> >>      > > Ericsson AB
> >>      > >
> >>      > > Wireless Access Networks
> >>      > >
> >>      > > Labratoriegränd 11
> >>      > >
> >>      > > 971 28, Luleå, Sweden
> >>      > >
> >>      > > Phone +46-1071 43042 <tel:%2B46-1071%2043042>
> >>      > >
> >>      > > SMS/MMS +46-73 078 3289 <tel:%2B46-73%20078%203289>
> >>      > >
> >>      > > [email-masked]
> >>
> >>     <mailto:[email-masked]>
> >>
> >>      > > www.ericsson.com <http://www.ericsson.com>
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > > “The difference between stupidity and
> >>      > >
> >>      > > genius is that genius has its limits."
> >>      > > Albert Einstein
> >>      > > =================================
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > >
> >>      > > _______________________________________________
> >>      > > iccrg mailing list
> >>      > > [email-masked] <mailto:[email-masked]>
> >>      > >
> >>      > > https://www.irtf.org/mailman/listinfo/iccrg
> >>      >
> >>      > --
> >>      > Dave Täht
> >>      >
> >>      > NSFW:
> >>
> >> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_ind
> >>ec
> >>
> >>      > ent.article
> >>
> >>     _______________________________________________
> >>     iccrg mailing list
> >>     [email-masked] <mailto:[email-masked]>
> >>     https://www.irtf.org/mailman/listinfo/iccrg____
> >>
> >>     __ __
> >
> > --
> >
> > Zahed
> >
> > ==================================================
> > ANM ZAHEDUZZAMAN SARKER
> >
> >
> > Ericsson AB
> > Services, Media and Network Features
> > Laboratoriegränd 11
> > 97128 Luleå, Sweden
> > Phone +46 10 717 37 43
> > Fax +46 920 996 21
> > SMS/MMS +46 76 115 37 43
> > [email-masked]
> > www.ericsson.com
> >
> > ==================================================

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Varun Singh <[email-masked]>
Date: Mon, 14 Jul 2014 18:01:22 +0300

Hi Luca,

My implementation of GCC was based on:
http://tools.ietf.org/html/draft-alvestrand-rtcweb-congestion-02
(spring/summer 2012) and earlier, which didn't yet contain the text
relating to processing multiple streams from a single endpoint. This
had me wondering for a moment on how to incorporate these streams into
GCC. With the addition of the RTP to NTP time conversion, the GCC
arrival filter can work across timestamps from repair, audio and video
packets.

Cheers,
Varun

On Thu, Jun 26, 2014 at 4:38 PM, Luca De Cicco <[email-masked]> wrote:
> Dear Varun,
>
> the way RTX or FEC are computed shouldn't impact GCC sending rate dynamics,
> but only
> the user's QoE.
>
> Cheers,
> Luca
>
> On Thu, Jun 26, 2014 at 3:11 PM, Varun Singh <[email-masked]> wrote:
>>
>> Hi Zahed,
>>
>> Does your GCC simulation code implement any FEC or Retx? Since the gcc
>> draft does not imply use of any repair mechanism even though the chrome
>> browser implements both.
>>
>> On 26 Jun 2014 14:11, "Zaheduzzaman Sarker"
>>> http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01 as well and then
>>> we can compare the results.
>>>
>>> BR
>>>
>>> Zahed
>>>
>>>
>>>> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indec
>>>>      > ent.article
>>> [email-masked]

-- 
http://www.netlab.tkk.fi/~varun/

---

From: Piers O'Hanlon <[email-masked]>
Date: Thu, 17 Jul 2014 12:24:14 +0100

Hi Ignemar and Zahed,

Great to see this work come to draft stage - I like the name. The approach looks interesting - I had a couple of questions:

- Since Scream is based upon LEDBAT I wondered if you had looked into some of the issues it suffers from like; intra-procotol fairness, late-comer advantage, delay-impact on other flows, and yielding behaviour to short of longer lived TCP flows etc.
- It is mentioned that there's an 'Adaptive delay target' - but in the code owdTarget does not change - Did you have strategies for choosing the TARGET delay on different links - or is it fixed? Or did you mean the adaptation based on off_target?
- What period do use to measure the Base Delay (or do use LEDBAT's 1 minute value) - the default seems quite high for mobile deployments...?
- What sort of filtering do you propose for queuing delay beyond MIN - Mobile provides for very noisy environments I wondered if you had any alternatives?
- How does it compete against a LEDBAT flow with a similar target?

Cheers,

Piers

On 17 Jun 2014, at 18:35, Ingemar Johansson S wrote:

> Hi
>  
> This draft can be of interest for those of you interested in congestion control for realtime media.
> http://www.ietf.org/internet-drafts/draft-johansson-rmcat-scream-cc-00.txt
>  
> Abstract : “This memo describes a rate adaptation framework for conversational
>    video services.  The solution conforms to the packet conservation
>    principle and uses a hybrid loss and delay based congestion control
>    algorithm.  The framework is evaluated over both simulated bottleneck
>    scenarios as well as in a LTE (Long Term Evolution) system simulator
>    and is shown to achieve both low latency and high video throughput in
>    these scenarios”
>  
> /Ingemar
>  
>  
> Phone +46-1071 43042
> SMS/MMS +46-73 078 3289

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 17 Jul 2014 18:53:21 +0000

Hi

Thanks for the interest in this contribution. Answers/comments inline.

Regards
Ingemar

From: Piers O'Hanlon [mailto:[email-masked]]
Sent: den 17 juli 2014 13:24
To: Ingemar Johansson S; Zaheduzzaman Sarker
Cc: [email-masked]; rmcat WG
Subject: Re: [iccrg] draft-johansson-rmcat-scream-cc

Hi Ingemar and Zahed,

- Since Scream is based upon LEDBAT I wondered if you had looked into some of the issues it suffers from like; intra-procotol fairness, late-comer advantage, delay-impact on other flows, and yielding behaviour to short of longer lived TCP flows etc.
[IJ] Zahed will present a few test cases at the RMCAT WG sessions. A few additional comments. Yes, I have see similar issues that are reported for LEDBAT in terms of late comers advantage etc. also in SCReAM, the current solution to this is to put media on hold for about one RTT at regular intervals if the one way delay exceeds a threshold and measure the OWD during that period.
This makes it possible to updated the base delay, and also competing LEDBAT based flows will update the base delay more accurately even though they don't use this solution. The behavior in the precense of competing TCP flows depend much on the type of AQM in the bottleneck, a few examples will be given in the presentation.

- It is mentioned that there's an 'Adaptive delay target' - but in the code owdTarget does not change - Did you have strategies for choosing the TARGET delay on different links - or is it fixed? Or did you mean the adaptation based on off_target?
[IJ] Correct, there is not yet any adaptive target delay functionality in the code, the adaptive delay target function use the same technique to put the media on hold and measure the OWD during that time and update the delay target. If there are no competing flows then the OWD will drop considerably, if there are competing flows then the OWD will not drop that much, and this is an indication that the delay target can be increased.
- What period do use to measure the Base Delay (or do use LEDBAT's 1 minute value) - the default seems quite high for mobile deployments...?
[IJ] Don't remember right now but it is definitely shorter than default LEDBAT, will look it up when I am back from the vacation for good in mid August
- What sort of filtering do you propose for queuing delay beyond MIN - Mobile provides for very noisy environments I wondered if you had any alternatives?
[IJ] Yes it is noisy but sofar I don't use any filtering at all, seems like it is sufficient with the filtering that is inherent in the CWND adjustment.
- How does it compete against a LEDBAT flow with a similar target?
[IJ] We don't have any example of it, I tried it out in the earlier stages but the results are lost and it is probably better to re run and see how it behaves.

Cheers,

Piers

Hi

Abstract : "This memo describes a rate adaptation framework for conversational
   these scenarios"

/Ingemar

Ericsson AB
Wireless Access Networks
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
[email-masked]<mailto:[email-masked]>
www.ericsson.com<x-msg://1596/www.ericsson.com>

"The difference between stupidity and
genius is that genius has its limits."
Albert Einstein
=================================

_______________________________________________
iccrg mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/iccrg

---

From: Piers O'Hanlon <[email-masked]>
Date: Wed, 23 Jul 2014 13:25:56 +0100

Hi Ingemar,

Thanks for your replies - more inline.

Regards,

Piers.

On 17 Jul 2014, at 19:53, Ingemar Johansson S wrote:

> This makes it possible to updated the base delay, and also competing LEDBAT based flows will update the base delay more accurately even though they don’t use this solution. The behavior in the precense of competing TCP flows depend much on the type of AQM in the bottleneck, a few examples will be given in the presentation.
>  
I guess this sort of technique may be more likely to work in LTE type of environments where each bearer channel only carries a few flows. But on wired links with more flows some research papers seem to show it may not be as effective? I wonder if there's an AQM that more suited to LEDBAT/Vegas style OWD measurement...? I suppose the 'low delay' AQMs would be beneficial but if you're running one of those then most protocols will benefit in terms of low delay.

And also I assume that it's behaviour would be similar to LEDBAT when competing with TCP type flows?

> [IJ] Correct, there is not yet any adaptive target delay functionality in the code, the adaptive delay target function use the same technique to put the media on hold and measure the OWD during tha…
Ah ok - Do you see any strange behaviour where different flows end up with different targets and suffer or do they correct themselves quickly enough?

> [IJ] Don’t remember right now but it is definitely shorter than default LEDBAT, will look it up when I am back from the vacation for good in mid August
It would be interesting to know how the actual RTT of on LTE can vary as a user moves, and I imagine handovers would lead to some complications.

> [IJ] Yes it is noisy but sofar I don’t use any filtering at all, seems like it is sufficient with the filtering that is inherent in the CWND adjustment.
I guess it depends on the variation of the noise - what sort of variation do you typically see?

> [IJ] We don’t have any example of it, I tried it out in the earlier stages but the results are lost and it is probably better to re run and see how it behaves.
Yes it would be interesting to see how it compares to LEDBAT and to aid in understanding the advantages Scream brings.

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Wed, 23 Jul 2014 18:31:22 +0000

Hi Piers,
Thanks for the comments.
Could you please send us pointers to the research papers you mentioned in your reply regarding performance issue with "media on hold" in the wireline network?.

BR

Zahed

From: Piers O'Hanlon [mailto:[email-masked]]
Sent: den 23 juli 2014 08:26
To: Ingemar Johansson S
Cc: Zaheduzzaman Sarker; [email-masked]; rmcat WG
Subject: Re: [iccrg] draft-johansson-rmcat-scream-cc

Hi Ingemar,

Regards,

Piers.

Hi

Regards
Ingemar

Cc: [email-masked]<mailto:[email-masked]>; rmcat WG

Hi Ingemar and Zahed,

Cheers,

Piers

Hi

/Ingemar

Ericsson AB
Wireless Access Networks
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289

---

From: Piers O'Hanlon <[email-masked]>
Date: Thu, 24 Jul 2014 10:03:05 +0100

Hi Zahed,

On 23 Jul 2014, at 19:31, Zaheduzzaman Sarker wrote:

> Hi Piers,
> Thanks for the comments.
> Could you please send us pointers to the research papers you mentioned in your reply regarding performance issue with “media on hold” in the wireline network?.

I was talking about the suggestions of related approaches such as the use of gaps (RFC6817 Sec 4.4) or 'Random window dropping' (section V.B) in "The quest for LEDBAT fairness, Giovanna Carofiglio, Luca Muscariello, Dario Rossi and Silvio Valenti, Globcom 2010. As mentioned in the paper the utility of media on hold type of approaches is inversely related to the number of flows.

Piers