Subject: [rmcat] Review of draft-ietf-rmcat-scream-cc-01

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 10 Jul 2015 14:06:11 +0200

Hi Zahed, hi Ingemar,

please find my review as an individual contributor below.

First of all I have a few general question on the algorithm(s):

1) When calculating cwnd you on the one hand calculate scl_i to basically 
quickly speed up to the previous maximum value cwnd_i and on the other hand try 
to maintain a low delay target. Aren't these two conflicting goals? Without 
having looked at your experimentation results now, I'd expect that you even when 
you are alone on the link induce more than owd_target delay (actually depending 
on the maximum buffer size as reflected by cwnd_i), and when competing with 
other traffic, you'd still cannot reach an equal sharing but will always get a 
lower share. Which is probably okay; however the part that maybe worries my a 
little is that the achieved delay and/or achieved sharing probably depends on 
the configured maximum buffer size (which might be large)...? Can you comment on 
this or provide more detail results on this?

2) I would have expected that the video rate controller takes cwnd or the actual 
sending window as an input parameter. However, your approach seems to have two 
more or less independent control loops for the sending rate and the coding rate. 
That seems quite weird to me (however the reasoning behind the design of the 
video coder is also not really clear to me; see below for further comments on 
this). I'd be worried that if those two loops are rather independent that they 
could actually make conflicting decision, e.g. video controller increases the 
rate and congestion controller decreases the rate. Also wouldn't it be useful to 
use the cwnd as an input parameter? Please comment!

And two more high-level/general comments on (the structure of) the document:

1) I think the algorithm is now well understandable with the code and respective 
explanation (expect the end of section 4.1.3 needs more explanation as just 
mentioned above). I would still like to propose some restructuring. I think the 
readability can be much improved by some rather simple restructuring using a 
model similar as in the LEDBAT draft: First give a high level summarize of all 
steps that have to be performed ("For each ACK/report the state on the delay and 
delay trend will be updated. If a loss is detected the congestion window cwnd 
will be decreased strongly, otherwise there are two operation mode: fast 
increase and regular operation. In fast increase the goal is to reach a previous 
target value quickly applying an multiplicative increase scheme, while otherwise 
a delay traget should be maintained..."). Then give all the pseudo code 
structured into functions and finally explain each function block in detail in 
an own paragraph. I already have a good idea how this would look like and if you 
want to we can sit down at the meeting and do this together.

2) Having reviewed NADA and Scream now basically in one go, I think the used 
framework is very similar. If you look at your Figure 1 and the Figure 1 in the 
NADA draft, I think I'm now basically able to do a 1:1 mapping. However, I think 
in your picture it is wrong to have multiple video coder/rate controller/queues 
shown because that indicates that there is only one transmission scheduler and 
congestion controller for all transmissions, which is not the case. If you 
remove that part, there is only one difference left between the two images and 
that is that there is an input from the congestion control to the video rate 
controller in NADA which you don't have (also see comment 2 above). Other than 
that the mapping would be the following way:

NADA
---
Reference Rate Calculator    -> Network congestion control
Video Target Rate Calculator -> Rate Control
Sending Rate Calculator      -> Transmission scheduler
Encoder                      -> Video Encoder
Rate Shaping Buffer          -> Queue RTP packets

I think it would be super helpful to use a common terminology here for everybody 
to better understand similarities and differences. I guess the same is true for 
variable and parameter names if there things that are actually the same. Note 
that you in some case already use the same names (I didn't check however if the 
things that have the same name actually are the same things...)

3) I think it is great to have a section to list all parameters and values. 
Thanks! However I do have to say I find tables here not the right choice because 
it just stretches the information over more pages than needed and therefore 
makes it actually harder to find the right bit. Further there are a few 
variables that are actually parameters and listed wrongly. Two examples I've 
found are: mss and min_cwnd (should be MSS and MIN_CWND which you even write 
this way sometimes). Further for variables I would not indicate the initial 
values here. I'd propose to have a init function instead in the later 
description where one can see all initial value at once (like in the LEDBAT 
draft). Further please review if all variables are actually needed. If there are 
only temporal and not variables that actually hold state in the pseudo code, I 
don't think they need t be listed here.

Further comments following the structure of the document:

Section 1: I've also made this comment regarding the NADA doc: I guess every 
algorithm doc should refer the requirements doc and to some extend discuss if 
and how these requirements have been addressed...

Section 1.1: not sure why this section is there; don't see a need to have this 
information in this document because it doesn't help me understand the algorithm 
any better.

Section 3:
- s/follows packet conservation principles/follows the packet conservation 
principle/
- I think think FACK is not the right reference for the packet conservation 
priniple, should be  "Congestion Avoidance and Control" by Van Jacobsen and 
Michael J. Karels, SIGCOM'88 (see reference Jac88 of FACK paper).
- s/The packet conservation principle is realized by/In Scream, the packet 
conservation principle is realized by/ -> otherwise it sounds like it's always 
realized this way, which is not true.
- Maybe give already a super brief description of what LEDBAT is doing:
   s/in a way similar to LEDBAT [RFC6817]./in a way similar to LEDBAT [RFC6817] 
by maintaining a given target delay./
- This text further say "a few steps to take to make the concept work with 
conversational media". Maybe already name these steps very briefly here: e.g. 
taking the delay trend into account and the previous maximum rate to compete 
with other non-delay based traffic...

Section 3.1
- You say there are similarities with newcwv but then never mention this again. 
Can you explain what the similarities are? I've just reviewed newcwv and this is 
absolutely not clear to me...
- Fast start: May I propose to name this Fast Increase instead (and also always 
write this with upper letters). Actually that's the same name I used for TCP 
SIAD where I also have this mechanism that is kind of similar to Slow Start but 
can be resumed later as well. I propose this change (not just because I want my 
name to be used ;-) but) because having 'start' in the name is confusing to my 
if you not only use this mechanism to actually start or re-start a transmission.
- Also maybe be slightly more concrete here (on a high level) about what fast 
start is doing: e.g "implements and multiplicative increase to quickly reach the 
maximum capacity where the increase factor is scaled based on a previous maximum 
rate (expect at transmission start). Fast start is resumed during a transmission 
if the delay trend goes below a certain threshold." Btw. I would also appreciate 
to introduce a parameter for this threshold and not only say "less than 0.2 for 
1.0 seconds or more" (see later in section 4.1.2.1).
- s/with FTP traffic/with bulk traffic/

Section 3.2.
- This section says packet pacing is used, however I don't think this is further 
mention in the rest of the text... please add something on this... or there 
might be a confusion between me and you what pacing means (see below).

Section 3.3.
- Maybe already include the relevant part of figure 1 here because a figure 
really helps. Or include the whole figure 1 in section 3 and structure the 
overview based on this image.

Section 4.1
- I guess you can remove the reference to I-D.ietf-rmcat-app-interaction because 
we are working on this draft anymore. However, it was anyway not clear what 
exactly you are referencing it for. From my point of view explaining the 
relationship to other docs (of the same wg) is often done best in the intro.
- On Figure 1: As said above I find it confusing that there are multiple video 
streams here because that indicates that there is only one congestion controller 
and one transmission scheduler for all transmissions. For the same reason I 
actually find the name transmission scheduler confusion because a scheduler 
would actually be the one who decides which packet to put on the link at which 
time (which often is done on the nic). However your transmission scheduler is 
'only' calculating the sending window, therefore I call it something like 
sending rate controller...

Section 4.1.2.1
- At the beginning of this section you define some terminology. Maybe put this 
in the terminology section instead?
- I've also asked this question for NADA: Don't you need synchronized clocks at 
the sender and receiver. Please further discuss this!
- Please also give more details how loss detection is done. I guess that's 
simply indicated in an RTCP report..? Please say which information are exactly used.
- Please give also some kind of pseudo code on how bytes_newly_acked is updated. 
That simply helps people to implement things correctly.
- As said above I would structure the pseudo code in functions where one 
function would be an init function here. That also helps to avoid implementation 
errors.
- Please give more details how the function R() is implemented.
- Why is scl_i called this way? What does the i stand for? Maybe take a slightly 
longer more meaningful name like scale_increase. This helps the reader to 
immediately understand what this is later used for!
- Why is scl_i limited to 0.2? Should this also a parameter?
- Please give some reasoning why scl_i is calculated this way! Is this similar 
to Cubic? If so please refer!
- I believe the calculation of off_target is only needed if not in fast start. 
If so please move it!
- cwnd_i is initialized to 1. That seems really weird to me. Does this work well 
for fast start at a beginning of a transmission? I guess the intention is to 
make fast start behave like slow start at the beginning of a connection. In this 
case scl_i should be 1. Why don't you just set scl_i to 1 if cwnd_i has not been 
set yet...?
- gain calculation: owd_trend is always a value between 0 and 1, therefore it 
can not be negative, therefore the max() in the gain calculation can be removed:
gain = GAIN*(1.0 + (1.0 - owd_trend/ 0.2))
- Where is this 0.2 in this gain calculation again coming from. Is there a good 
reasoning for this value? Should this be again a parameter? Is this the same 
parameter than the previous 0.2?
- I'd move the discussion what the difference to LEDBAT is into section 3.1 
because these point are very general and therefore no algorithm details are 
needed to be known and you mention those difference there already without saying 
what they actually are.
- However, you say you don't use ALLOWED_INCREASE but isn't 
MAX_BYTES_IN_FLIGHT_HEAD_ROOM effective the same...?
- The part on flow compensation is rather unclear! Suddenly you have two new 
parameters owd_norm_mean_sh and owd_norm_var: how are they calculated? When are 
they updated? I think there is something missing somewhere. Further where does 
this formula come from? I guess there is some previous working that proposed 
this approach? Can you give a reference and further explain why this approach is 
taken?

Section 4.1.2.2
- I'd move the initial part here with the two bullet points into section 3.2 
because this is rather some high level info.
- I don't think I would call the proposed approach pacing. Pacing is for me that 
if you can send you multiple packet at once (because the send/congestion window 
is large enough) that you spread the sending times of each packet over the time 
interval until the next ACK/update is expected.
- Further the calculation seems rather complicated. Is this really needed? Can 
you further explain in the doc in which case this approach helps which problem 
and why?!

Section 4.1.3
- Why is the estimation interval 200ms (shouldn't this be a parameter as well?) 
but the RATE_ADJUSTMENT_INTERVAL 100ms? If you only estimate every 200ms there 
will be no new information after 100ms to adjust something...?
- s/ if the congestion control is if fast start or not./ if the congestion 
control is in fast start or not./
- In general, as mention above, there is more explanation needed on the single 
steps of the video rate controller, especially on
   - What is RMAP_UP_TIME actually good for? Why is this 10s?
   - Please explain further why increment is calculated the way it is calculated!
   - Shouldn't the target_bitrate be only increased in steps that the encoder 
can handle?
   - Please you further explain the last part on rate_rtp_limit!!!

Section 4.2
- Here the table is good because there are less values and it actually provides 
a nice overview!

Section 5.
- This needs to go in an own draft at some point. Please also compare 
similarities with the information needed by NADA and GCC!

Section 7
- Actually a conclusion is not needed for an IETF Internet draft...

Section 8
- regarding the clock drift compensation, please see the appendix of the LEDBAT 
RFC. If you've implemented basically the same thing as described there, it 
probably enough to refer to that.

Mirja

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 14 Jul 2015 07:24:45 +0000

Hi Mirja and thanks for your comments, answers inline marked with [IJ]

Regards
/Ingemar

> -----Original Message-----
> From: Mirja Kühlewind [mailto:[email-masked]]
> Sent: den 10 juli 2015 14:06
> To: Zaheduzzaman Sarker; Ingemar Johansson S
> Cc: Karen E. E. Nielsen; [email-masked]
> Subject: Review of draft-ietf-rmcat-scream-cc-01
> 
> Hi Zahed, hi Ingemar,
> 
> quickly speed up to the previous maximum value cwnd_i and on the other
> hand try to maintain a low delay target. Aren't these two conflicting goals?
> Without having looked at your experimentation results now, I'd expect that
> you even when you are alone on the link induce more than owd_target delay
> (actually depending on the maximum buffer size as reflected by cwnd_i), and
> when competing with other traffic, you'd still cannot reach an equal sharing
> but will always get a lower share. Which is probably okay; however the part
> that maybe worries my a little is that the achieved delay and/or achieved
> sharing probably depends on the configured maximum buffer size (which
> might be large)...? Can you comment on this or provide more detail results
> on this?
[IJ] Actually the purpose of cwnd_i and scl_i is to slow down the increase on cwnd and target bitrate when close to the last known max value. This gives a chance for the algorithm to avoid a too large delay spike when the path capacity of a fixed BW link is reached, at the same time it gives a better opportunity to grab free bandwidth reasonably fast.
It may be an option to base the video target rate on something like the send window, not sure what it would like, you probably need to do something like cwnd/rtt. Like it is now there are actually two control loops as you correctly observe. Currently I don't see a way around this as the video coder can fluctuate quite a lot around the target bitrate. In the long run it is perhaps possible (and desirable) to make it one control loop, not sure that it will happens. Another thing that speaks in favor of two control loops is that SCReAM can congestion control multiple streams, for instance video and audio. This can be experimented with in the C++ code.
> 
> 2) I would have expected that the video rate controller takes cwnd or the
> actual sending window as an input parameter. However, your approach
> seems to have two more or less independent control loops for the sending
> rate and the coding rate.
> That seems quite weird to me (however the reasoning behind the design of
> the video coder is also not really clear to me; see below for further
> comments on this). I'd be worried that if those two loops are rather
> independent that they could actually make conflicting decision, e.g. video
> controller increases the rate and congestion controller decreases the rate.
> Also wouldn't it be useful to use the cwnd as an input parameter? Please
> comment!
> 
> 
> 1) I think the algorithm is now well understandable with the code and
> respective explanation (expect the end of section 4.1.3 needs more
> explanation as just mentioned above). I would still like to propose some
> restructuring. I think the readability can be much improved by some rather
> simple restructuring using a model similar as in the LEDBAT draft: First give a
> high level summarize of all steps that have to be performed ("For each
> ACK/report the state on the delay and delay trend will be updated. If a loss is
> detected the congestion window cwnd will be decreased strongly, otherwise
> there are two operation mode: fast increase and regular operation. In fast
> increase the goal is to reach a previous target value quickly applying an
> multiplicative increase scheme, while otherwise a delay traget should be
> maintained..."). Then give all the pseudo code structured into functions and
> finally explain each function block in detail in an own paragraph. I already
> have a good idea how this would look like and if you want to we can sit down
> at the meeting and do this together.
[IJ] I'd like to have some kind of sit together, unfortunately I cannot attend the IETF this time, perhaps some remote appear.in session or the like can be arranged in August.

> 
> 2) Having reviewed NADA and Scream now basically in one go, I think the
> used framework is very similar. If you look at your Figure 1 and the Figure 1 in
> the NADA draft, I think I'm now basically able to do a 1:1 mapping. However, I
> think in your picture it is wrong to have multiple video coder/rate
> controller/queues shown because that indicates that there is only one
> transmission scheduler and congestion controller for all transmissions, which
> is not the case. If you remove that part, there is only one difference left
> between the two images and that is that there is an input from the
> congestion control to the video rate controller in NADA which you don't have
> (also see comment 2 above). 
[IJ] We can probably remove the example with multiple videos if it makes things more simple, but fact of the matter is that SCReAM can handle multiple sources.

Other than that the mapping would be the
> following way:
> I think it would be super helpful to use a common terminology here for
> everybody to better understand similarities and differences. 
[IJ] Agree, if we can come up with some common terms, so much the better

>I guess the
> same is true for variable and parameter names if there things that are
> actually the same. Note that you in some case already use the same names (I
> didn't check however if the things that have the same name actually are the
> same things...)
> 
> Thanks! However I do have to say I find tables here not the right choice
> because it just stretches the information over more pages than needed and
> therefore makes it actually harder to find the right bit. Further there are a
> few variables that are actually parameters and listed wrongly. Two examples
> I've found are: mss and min_cwnd (should be MSS and MIN_CWND which
> you even write this way sometimes).
[IJ] I have to blame the use of tables on  my mediocre XML skills, is a list better ?. Will correct the MSS and MIN_CWND error, thanks

> Further for variables I would not
> indicate the initial values here. I'd propose to have a init function instead in
> the later description where one can see all initial value at once (like in the
> LEDBAT draft). Further please review if all variables are actually needed. If
> there are only temporal and not variables that actually hold state in the
> pseudo code, I don't think they need t be listed here.
[IJ] OK, sounds good.

> Section 1: I've also made this comment regarding the NADA doc: I guess
> every algorithm doc should refer the requirements doc and to some extend
> discuss if and how these requirements have been addressed...
> 
> information in this document because it doesn't help me understand the
> algorithm any better.
> 
> Section 3:
> priniple, should be  "Congestion Avoidance and Control" by Van Jacobsen
> and Michael J. Karels, SIGCOM'88 (see reference Jac88 of FACK paper).
[IJ] OK

>    s/in a way similar to LEDBAT [RFC6817]./in a way similar to LEDBAT
> [RFC6817] by maintaining a given target delay./
> taking the delay trend into account and the previous maximum rate to
> compete with other non-delay based traffic...
[IJ] OK
> Can you explain what the similarities are? I've just reviewed newcwv and this
> is absolutely not clear to me...
[IJ] OK, will need to clarify this.

> - Fast start: May I propose to name this Fast Increase instead (and also
> always write this with upper letters). Actually that's the same name I used for
> TCP SIAD where I also have this mechanism that is kind of similar to Slow Start
> but can be resumed later as well. I propose this change (not just because I
> want my name to be used ;-) but) because having 'start' in the name is
> confusing to my if you not only use this mechanism to actually start or re-
> start a transmission.
> start is doing: e.g "implements and multiplicative increase to quickly reach
> the maximum capacity where the increase factor is scaled based on a
> previous maximum rate (expect at transmission start). Fast start is resumed
> during a transmission if the delay trend goes below a certain threshold." Btw.
> I would also appreciate to introduce a parameter for this threshold and not
> only say "less than 0.2 for
[IJ] OK :-) , Fast Increase it is !, never felt comfortable with the "fast start" term. Do you have a reference to your TCP SIAD work. Would be nice to see it as I am looking at ways to recover from failed TCP Cubic HyStarts (related to issues we see with HyStart and LTE access)

> might be a confusion between me and you what pacing means (see below).
[IJ] Packet pacing was broken out to the appendix section A.1, the reason was that it did not seem to fit with the earlier consensus that the congestion control should compute a send window. But I guess the packet pacing section can be moved to an earlier section

> - I guess you can remove the reference to I-D.ietf-rmcat-app-interaction
> because we are working on this draft anymore. However, it was anyway not
> clear what exactly you are referencing it for. From my point of view
> explaining the relationship to other docs (of the same wg) is often done best
> in the intro.
[IJ] Perhaps Zahed can explain the ref to the app intercation as I have not been involved in that work

> streams here because that indicates that there is only one congestion
> controller and one transmission scheduler for all transmissions. For the same
> reason I actually find the name transmission scheduler confusion because a
> scheduler would actually be the one who decides which packet to put on the
> link at which time (which often is done on the nic). However your
> transmission scheduler is 'only' calculating the sending window, therefore I
> call it something like sending rate controller...
[IJ] Could be Ok to rename it to sending rate controller, I don't have any strong opinion. Note that SCReAM can in fact congestion control more than one stream, this is similar to how SCTP and QUIC works.   

> 
> Section 4.1.2.1
> - At the beginning of this section you define some terminology. Maybe put
> this in the terminology section instead?
> - I've also asked this question for NADA: Don't you need synchronized clocks
> at the sender and receiver. Please further discuss this!
[IJ] No. Synchronized clocks are not needed, I am a bit hesitant about the impact of clock skew. The "real" code resets the base delay history it the OWD takes on inreasonable values, this is not mentioned in the draft however. Will fix this

> simply indicated in an RTCP report..? Please say which information are exactly
[IJ] The n_loss counter is checked, if it has incremented by one or more steps in an RTT it is considered a loss event. Will clarify this

> - Please give also some kind of pseudo code on how bytes_newly_acked is
[IJ] OK

> function would be an init function here. That also helps to avoid
> implementation errors.
[IJ] It is challenging to write an autocorrelation function in ASCII art but I give it a(nother) try :-)

> - Why is scl_i limited to 0.2? Should this also a parameter?
[IJ] Good question , would probably be better to just use scl or tmp instead. 
0.2 is there to avoid that the congestion window growth is is set to zero.

> - Please give some reasoning why scl_i is calculated this way! Is this similar to
> Cubic? If so please refer!
[IJ] Yes, it is very similar to Cubic, perhaps I have only referred to this in emails, will clarify this

> - I believe the calculation of off_target is only needed if not in fast start.
[IJ] OK

> for fast start at a beginning of a transmission? I guess the intention is to make
> fast start behave like slow start at the beginning of a connection. In this case
> scl_i should be 1. Why don't you just set scl_i to 1 if cwnd_i has not been set
> yet...?
[IJ] Yes, you are right,  could be an alternative,

> can not be negative, therefore the max() in the gain calculation can be
[IJ] OK

> parameter than the previous 0.2?
[IJ] The 0.2 values (no relation between the two 0.2 values actually) are the results of experimentation. They are not that critical however, could have been other values like 0.3. Could be good to have them as parameters (actually I got the same remark from Daniel Lindström who helped me to get SCReAM into OpenWebRTC :-) .  

> needed to be known and you mention those difference there already
> without saying what they actually are.
[IJ] Hmm, will have to look into this.. need to come back

> - The part on flow compensation is rather unclear! Suddenly you have two
> new parameters owd_norm_mean_sh and owd_norm_var: how are they
> calculated? When are they updated? I think there is something missing
> somewhere. Further where does this formula come from? I guess there is
> some previous working that proposed this approach? Can you give a
> reference and further explain why this approach is taken?
[IJ] You beat me to the punch here. This is inspired by the work on shared bottleneck detection in RMCAT.  I implemented the variance and skew estimates an observed that they actually indicate quite well when competing flows enter and still the risk is quite small that it causes self-inflicted congestion. 

> if you can send you multiple packet at once (because the send/congestion
> window is large enough) that you spread the sending times of each packet
> over the time interval until the next ACK/update is expected.
[IJ] That is actually what the pacing function does in the code, perhaps I need to work the text here,
> you further explain in the doc in which case this approach helps which
> problem and why?!
[IJ] OK , will clarify
> 
> Section 4.1.3
> - Why is the estimation interval 200ms (shouldn't this be a parameter as
> well?) but the RATE_ADJUSTMENT_INTERVAL 100ms? If you only estimate
> every 200ms there will be no new information after 100ms to adjust
> something...?
[IJ] Probably a mistake, should be 200ms, according to the implementation

> - s/ if the congestion control is if fast start or not./ if the congestion control is
> in fast start or not./
> - In general, as mention above, there is more explanation needed on the
> single steps of the video rate controller, especially on
[IJ] This is actually a mistake, it is replaced by a RAMP_UP_SPEED parameter that indicates 
the max increase of the target rate in [bps/s]

>    - Please explain further why increment is calculated the way it is calculated!
[IJ] Will add an explanation

>    - Shouldn't the target_bitrate be only increased in steps that the encoder
[IJ] There are no actual steps that the encoder can handle. One typically just set a target rate and the rate control loop in the video encoder tries to adapt to it.

>    - Please you further explain the last part on rate_rtp_limit!!!
[IJ] Yes, this part probably needs a better explanation, it was added quite recently as a remedy to some real-life issues with video coding.

> - Here the table is good because there are less values and it actually provides
[IJ] OK, good !

> similarities with the information needed by NADA and GCC!
[IJ] OK

> - Actually a conclusion is not needed for an IETF Internet draft...
[IJ] OK, great!

> 
> Section 8
> - regarding the clock drift compensation, please see the appendix of the
> LEDBAT RFC. If you've implemented basically the same thing as described
> there, it probably enough to refer to that.
[IJ] No clock drift compensation implemented yet. Will have a look into the LEDBAT RFC.

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Tue, 14 Jul 2015 10:51:28 +0200

> [IJ] I'd like to have some kind of sit together, unfortunately I cannot attend the IETF this time, perhaps some remote appear.in session or the like can be arranged in August.
As I will be attending Prague we can sit together and if remote 
participant can join perhaps Ingemar can also join. We can at least 
start the process. And thanks for the this.

> [IJ] Agree, if we can come up with some common terms, so much the better
This (and the previous comment from Mirja and Ingemar on similarity 
between NADA and SCReAM) tell that common terminologies should be used 
in the different candidate drafts for ease of understanding if the terms 
are meaning the same things. We had a plan to have a framework document 
in RMCAT. Will that help?

-- 
Zahed

==================================================
ANM ZAHEDUZZAMAN SARKER

Ericsson AB
Services, Media and Network Features
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Fax +46 920 996 21
SMS/MMS +46 76 115 37 43
[email-masked]
www.ericsson.com

==================================================

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Tue, 14 Jul 2015 14:07:56 +0200

HI,

>>> I think it would be super helpful to use a common terminology here
>>> for everybody to better understand similarities and differences.
>are
>meaning the same things. We had a plan to have a framework document in
>RMCAT. Will that help?
>
[Karen ] Yes I think that can be a good idea, but I am not sure that the
above mapping of terms is right  - or is it  ?.
I very likely can be mistaken, but with screams multiple stream view then I
have thought that the following was the case:

Scream Transmission scheduler is in fact a true transmission scheduler as it
takes packets from the multiple RTP queues and schedule the packets
(following whatever multiple-stream scheduling policy/stream priority
relevant)
for consummation of the global sending rate/the congestion window calculated
and thus transmit the packets on the UDP socket (after which they eventually
will
go to the NIC for transmission). I further had assumed that pacing
(vis-à-vis cwnv) in this design is implemented as part of the transmission
scheduler and not as part of the RTP queue logic.
This model at least corresponds to SCTP's model for scheduling of the
multiple streams for consummation of CWND and transmission.

BR, Karen

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 14 Jul 2015 15:57:32 +0200

Hi Karen, hi all,

see below.

> multiple streams for consummation of CWND and transmission.

If I have red the current draft correctly, everything you've described here was 
moved to the appendix and all that is left in the main part is the calculation 
of the actual sending rate. In this case it would map (to NADA). Probably you 
want one image in the main part that is similar to the image in NADA and then 
have another image in the appendix where you show multiple sources and an 
actually transmission scheduler.

Mirja

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Tue, 14 Jul 2015 16:18:03 +0200

HI,
>>>
>> [Karen ] Yes I think that can be a good idea, but I am not sure that
>> the above mapping of terms is right  - or is it  ?.
>> I very likely can be mistaken, but with screams multiple stream view
>> then I have thought that the following was the case:
>>
>> Scream Transmission scheduler is in fact a true transmission scheduler
>> as it takes packets from the multiple RTP queues and schedule the
>> packets (following whatever multiple-stream scheduling policy/stream
>> priority
>> relevant)
>> for consummation of the global sending rate/the congestion window
>> calculated and thus transmit the packets on the UDP socket (after
>> which they eventually will go to the NIC for transmission). I further
>> had assumed that pacing (vis-à-vis cwnv) in this design is implemented
>> as part of the transmission scheduler and not as part of the RTP queue
>> logic.
>moved to the appendix and all that is left in the main part is the
>calculation of
>the actual sending rate. In this case it would map (to NADA). Probably you
[Karen ] Yes - I see your point. One note though is that in a multi-stream
model then pacing
logically is instantiated by the multi-stream transmission scheduler which
consumes the packets from the RTP queues
and not logically instantiated in the individual RTP queue objects.
I think that it is important that the terminology (and the definition of the
objects within the framework one can say)
accommodates for a both a single RTP stream and a multi-stream solution - Or
?

BR, Karen

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 14 Jul 2015 16:59:43 +0200

Hi Zahed,

see below.

> start the process. And thanks for the this.

Okay, I think I'll have some time on Tuesday. Let's discuss Sunday, when we are 
going to work on this!

> in RMCAT. Will that help?

For me personally it would be more important to have the same terminology and 
also the same variable names if things are really exactly the same thing in the 
drafts, than having a framework document. Also, if the drafts 'speak the same 
language', it should be easy to write a framework document.

As a chair I'd actually like to publish the current proposal as experimental 
RFCs as stand-alone documents. While if we end up to actually publish more than 
one scheme as standard track, I think it would be useful to have one framework 
document where we describe all similarities and potentially also discuss when to 
use which scheme... but that's for the working group to decide later on.

Mirja

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Tue, 14 Jul 2015 18:38:49 +0200

On 2015-07-14 16:59, Mirja Kühlewind wrote:
>>> [IJ] I'd like to have some kind of sit together, unfortunately I
>>> cannot attend the IETF this time, perhaps some remote appear.in
>>> session or the like can be arranged in August.
> Okay, I think I'll have some time on Tuesday. Let's discuss Sunday, when
> we are going to work on this!
Ok.
>>> [IJ] We can probably remove the example with multiple videos if it
>>> makes things more simple, but fact of the matter is that SCReAM can
>>> handle multiple sources.
>>>
>>>
> For me personally it would be more important to have the same
> terminology and also the same variable names if things are really
> exactly the same thing in the drafts, than having a framework document.
> Also, if the drafts 'speak the same language', it should be easy to
> write a framework document.
My point of bringing the framework document is to have a place to
discuss what are the correct/appropriate terminologies to use. I believe 
everyone is comfortable with their own terminologies thus discussions 
are required to change some of them to
streamline the docs . Personally, I dont have issues with changing
the terminologies as long as we agree on the definitions.

>
> As a chair I'd actually like to publish the current proposal as
> experimental RFCs as stand-alone documents. While if we end up to
> actually publish more than one scheme as standard track, I think it
> would be useful to have one framework document where we describe all
> similarities and potentially also discuss when to use which scheme...
> but that's for the working group to decide later on.
>
I agree with this view. Perhaps we can use the wiki to be a place
where we define the terminologies and then if required we can have them 
in a framework document.

BR
-- 
Zahed

Ericsson AB

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 14 Jul 2015 20:15:30 +0000

Hi Karen
Your interpretation of the SCReAM multistream handling and transmission scheduling is correct. And as such it resembles SCTP (with NDATA) or QUIC (with the addition of support for unreliable delivery) very much

Ingemar

Sent from Moxier Mail
(http://www.moxier.com)

----- Ursprungligt meddelande -----
Från: Karen Elisabeth Egede Nielsen <[email-masked]>
Till: Zaheduzzaman Sarker <[email-masked]>, Ingemar Johansson S <[email-masked]>, Mirja Kühlewind <[email-masked]>
Kopia: "[email-masked]" <[email-masked]>
Skickat: 2015-07-14 14:08
Ämne: RE: Review of draft-ietf-rmcat-scream-cc-01

HI,

BR, Karen

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Sun, 19 Jul 2015 08:09:04 +0200

HI Ingemar, Zahed

Reading the coupled congestion control draft and the app-interactions
draft I have come to wonder a
bit about the multi-stream capabilities of scream - or  the in-build
coupled congestion control of  scream, one can say.

The coupled congestion control of welzl-rmcat-couple-cc allows for
priority to be given to different flows managed by
one couple congestion control context (i.e. the flow identified as sharing
the same bottle neck) - and the app-interactions draft asks for the same
possibility
to be provided. The scream coupled cc also allow for such prioritization
to be done in that such can be achieved by scheduling priorities in
between
the multiple RTP streams that one CC context serves (here scream model is
the same model as sctp multi-stream cc follows).
HOWEVER what the  scream model does not directly allow for - as  far as I
can see - is prioritized coupling in between multiple RTP flows that do
not use the same UDP socket/connection tuple -- And then I start wonder if
the stream model is the right one.

Possibly one can see the scream model as introducing one more level in the
coupled cc model - i.e., prioritized coupling in between the flows using
the same UDP socket/UDP tuple is done by means of scream transmission
scheduling priorities and then coupled congestion control following the
welzl model on top of this, when more "advanced" coupling or SBD detection
is desired or relevant in between flows sharing the same bottleneck, but
not using the same UDP socket/UDP tuple.

I suppose that it should be up to implementations how they want to
implement coupled cc and if they want a leveled approach as the one just
described or a more "direct" approach where each RTP flows is handled
individually from a CC perspective and the coupled via coupled cc (wezlz).

On the other hand the one should understand what would be the motivation
for a leveled approach would be - also because it seems to give a more
complex model  - Is it so that such is motivated by - for example: ?

* It is more efficient (overhead or protocol, state, whatever) to manage
CC on "one" multi-stream RTP basis when such is possible -  as scream does
? and therefore the leveled approach has advantages from a general
perspective or possibly it is advantageous in the particular situations
where the default (build-in) shared bottle-neck detection of scream is the
only one relevant ?

I hope that this was somewhat understandable...

Thanks
BR, Karen

>From: Ingemar Johansson S [mailto:[email-masked]]
>Sent: Tuesday, July 14, 2015 10:16 PM
>To: Karen E. E. Nielsen
>Cc: Mirja Kühlewind; Zaheduzzaman Sarker; [email-masked]
>Subject: SV: Review of draft-ietf-rmcat-scream-cc-01
>
>Hi Karen
>Your interpretation of the SCReAM multistream handling and transmission
>scheduling is correct. And as such it resembles SCTP (with NDATA) or QUIC
>(with the addition of support for unreliable delivery) very much
>
>Ingemar
>
>Sent from Moxier Mail
>(http://www.moxier.com)
>
>
>Till: Zaheduzzaman Sarker <[email-masked]>, Ingemar
>Johansson S <[email-masked]>, Mirja Kühlewind
><[email-masked]>
>>in the different candidate drafts for ease of understanding if the
>>terms are meaning the same things. We had a plan to have a framework
>>document in RMCAT. Will that help?
>>
above
>mapping of terms is right  - or is it  ?.
>calculated and thus transmit the packets on the UDP socket (after which
they
>eventually will go to the NIC for transmission). I further had assumed
that
>pacing (vis-à-vis cwnv) in this design is implemented as part of the
>transmission scheduler and not as part of the RTP queue logic.
multiple
>streams for consummation of CWND and transmission.
>
>BR, Karen
>
>>--
>>Zahed
>>

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Sun, 19 Jul 2015 21:01:06 +0000

Hi Karen,

Please see inline.

BR

Zahed

> 
> Reading the coupled congestion control draft and the app-interactions draft I
> have come to wonder a bit about the multi-stream capabilities of scream - or
> the in-build coupled congestion control of  scream, one can say.
> 
> The coupled congestion control of welzl-rmcat-couple-cc allows for priority to
> be given to different flows managed by one couple congestion control context
> (i.e. the flow identified as sharing the same bottle neck) - and the app-
> interactions draft asks for the same possibility to be provided. The scream
> coupled cc also allow for such prioritization to be done in that such can be
> achieved by scheduling priorities in between the multiple RTP streams that one
> CC context serves (here scream model is the same model as sctp multi-stream
> cc follows).
> HOWEVER what the  scream model does not directly allow for - as  far as I can
> see - is prioritized coupling in between multiple RTP flows that do not use the
> same UDP socket/connection tuple -- And then I start wonder if the stream
> model is the right one.
[Zaheduzzaman Sarker] I believe your interpretation of a scream controller controlling multiple RTP flows only with same "UDP socket/connection tuple" is coming from a previous reply where it was said the controller controls flows only towards same destination. I think the correct thing to say is, the RTP packet passing through the a scream transmission scheduler more of less showing same characteristics, like sharing same bottleneck. Hence if there is a system like SBD already allows multiple RTP flows with different destination to be identified as sharing the same bottleneck then prioritizations can be applied to all those flows by same transmission scheduler.
In the implementation of scream there is a method to register a RTP flow to a scream controller which will be congestion controlled by the that particular scream controller (the transmission scheduler resides in the scream control). This also means the scream controller has to get the information from the RTCP feedbacks for the respective registered flows.
Hence, your interpretation of " the scream model does not directly allow for prioritized coupling in between multiple RTP flows that do not use the same UDP socket/connection tuple " is not completely true.

> coupled cc model - i.e., prioritized coupling in between the flows using the
> same UDP socket/UDP tuple is done by means of scream transmission
> scheduling priorities and then coupled congestion control following the welzl
> model on top of this, when more "advanced" coupling or SBD detection is
> desired or relevant in between flows sharing the same bottleneck, but not using
> the same UDP socket/UDP tuple.
[Zaheduzzaman Sarker] SCReAM model does not simply assumes the existence of coupled-cc by default. As SCReAM already uses a transmission scheduler it is easy to implement "prioritization coupling" there. 
> 
> I suppose that it should be up to implementations how they want to implement
> coupled cc and if they want a leveled approach as the one just described or a
> more "direct" approach where each RTP flows is handled individually from a CC
> perspective and the coupled via coupled cc (wezlz).
[Zaheduzzaman Sarker] I believe for implementers do have a choice. However, if they decided to implement SCReAM they have an easy alternative for coupled-cc as Michael W himself says.
> 
> On the other hand the one should understand what would be the motivation for
> a leveled approach would be - also because it seems to give a more complex
> model  - Is it so that such is motivated by - for example: ?
> 
> * It is more efficient (overhead or protocol, state, whatever) to manage CC on
> "one" multi-stream RTP basis when such is possible -  as scream does ? and
> therefore the leveled approach has advantages from a general perspective or
> possibly it is advantageous in the particular situations where the default (build-
> in) shared bottle-neck detection of scream is the only one relevant ?
> 
[Zaheduzzaman Sarker] I believe I have already answered those questions.

---

From: Michael Welzl <[email-masked]>
Date: Sun, 19 Jul 2015 23:11:37 +0200

Hi,

> [Zaheduzzaman Sarker] I believe for implementers do have a choice. However, if they decided to implement SCReAM they have an easy alternative for coupled-cc as Michael W himself says.

I don't think I said that (actually I'm not sure I even understand this sentence). What did you mean I said?

Cheers,
Michael

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Sun, 19 Jul 2015 21:28:47 +0000

Hi Michael,

If my memory supports me then I remember you saying prioritization can be easily applied with schedulers at some points in your presentation at RMCAT previous IETF meetings.

But if you think you haven't then please accept my apologies.

BR

Zahed

> From: Michael Welzl [mailto:[email-masked]]
> Sent: den 19 juli 2015 23:12
> To: Zaheduzzaman Sarker
> Cc: Karen E. E. Nielsen; Ingemar Johansson S; [email-masked]; Mirja Kühlewind
> Subject: Re: [rmcat] Review of draft-ietf-rmcat-scream-cc-01
> 
> Hi,
> 
> snip:
> 
> 
> >> implement coupled cc and if they want a leveled approach as the one
> >> just described or a more "direct" approach where each RTP flows is
> >> handled individually from a CC perspective and the coupled via coupled cc
> (wezlz).
> > [Zaheduzzaman Sarker] I believe for implementers do have a choice.
> However, if they decided to implement SCReAM they have an easy alternative
> for coupled-cc as Michael W himself says.
>

---

From: Michael Welzl <[email-masked]>
Date: Sun, 19 Jul 2015 23:56:17 +0200

> On 19. jul. 2015, at 23.28, Zaheduzzaman Sarker <[email-masked]> wrote:
> 
> 
> Hi Michael,
> 

Nono, I just didn't understand the sentence as you phrased it. Yes, I did say that. We also mention scheduling as another way to do it in the coupled-cc draft.
I think scheduling is fine if all trafic is multiplexed on the same 5-tuple and you therefore know it will always share the same bottleneck.

Our coupled-cc mechanism was made to make the code easier. This applies when you have situations with a shared bottleneck detection algorithm as input, to decide whether you're coupling or not. This is a dynamic decision, requiring the ability to turn the coupling on and off.

E.g., quickly going from:  one congestion controller, 3 flows are scheduled onto it   to: 3 congestion controllers, one per flow  ...
and back...
...is probably not an easy matter to implement. Turning our coupled-cc mechanism on and off is very easy.

But since this is all on one host only, I don't think we should strictly enforce any certain behavior.

Cheers,
Michael

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Mon, 20 Jul 2015 07:32:02 +0200

HI,

Yes - there is the possibility to implement coupled congestion control with
scheduling priorities,
There is no doubt about that :-), Indeed SCTP does it and scream does it.
But it is not so well-described in the
scream draft how exactly the prioritization works and which means an
application has to define different priorities.
(For information then for SCTP a number of different scheduling
possibilities is described in
http://datatracker.ietf.org/doc/draft-ietf-tsvwg-sctp-ndata/.)

Then for RMCAT coupled congestion control  we have the Wezlz, Safiqul
proposal where the coupled is done
in-between per flow-CC contexts by prioritization on manipulation of the
cwnd.
And the we also have, a not so detailed proposal/description, on how the
same can be achieved via scheduling (scream).
It should be ok to allow for different ways to implement coupling, but we
should then eventually have one document that describes both
solutions. Shouldn't we ?

BR, Karen

>From: Zaheduzzaman Sarker [mailto:[email-masked]]
>Sent: Sunday, July 19, 2015 11:01 PM
>To: Karen E. E. Nielsen; Ingemar Johansson S
>Cc: Mirja Kühlewind; [email-masked]
>Subject: RE: Review of draft-ietf-rmcat-scream-cc-01
>
>Hi Karen,
>
>Please see inline.
>
>BR
>
>Zahed
>
>>
>> draft I have come to wonder a bit about the multi-stream capabilities
>> of scream - or the in-build coupled congestion control of  scream, one
>> can
>say.
>>
>> priority to be given to different flows managed by one couple
>> congestion control context (i.e. the flow identified as sharing the
>> same bottle neck) - and the app- interactions draft asks for the same
>> possibility to be provided. The scream coupled cc also allow for such
>> prioritization to be done in that such can be achieved by scheduling
>> priorities in between the multiple RTP streams that one CC context
>> serves (here scream model is the same model as sctp multi-stream cc
>follows).
>> HOWEVER what the  scream model does not directly allow for - as  far
>> as I can see - is prioritized coupling in between multiple RTP flows
>> that do not use the same UDP socket/connection tuple -- And then I
>> start wonder if the stream model is the right one.
>[Zaheduzzaman Sarker] I believe your interpretation of a scream controller
>controlling multiple RTP flows only with same "UDP socket/connection tuple"
>is coming from a previous reply where it was said the controller controls
>flows
>only towards same destination. I think the correct thing to say is, the RTP
>packet passing through the a scream transmission scheduler more of less
>showing same characteristics, like sharing same bottleneck. Hence if there
>is a
>system like SBD already allows multiple RTP flows with different
>destination to
>be identified as sharing the same bottleneck then prioritizations can be
>applied to all those flows by same transmission scheduler.
>In the implementation of scream there is a method to register a RTP flow to
>a
>scream controller which will be congestion controlled by the that
>particular
>scream controller (the transmission scheduler resides in the scream
>control).
>This also means the scream controller has to get the information from the
>RTCP feedbacks for the respective registered flows.
>Hence, your interpretation of " the scream model does not directly allow
>for
>prioritized coupling in between multiple RTP flows that do not use the same
>UDP socket/connection tuple " is not completely true.
>
>> Possibly one can see the scream model as introducing one more level in
>> the coupled cc model - i.e., prioritized coupling in between the flows
>> using the same UDP socket/UDP tuple is done by means of scream
>> transmission scheduling priorities and then coupled congestion control
>> following the welzl model on top of this, when more "advanced"
>> coupling or SBD detection is desired or relevant in between flows
>> sharing the same bottleneck, but not using the same UDP socket/UDP tuple.
>[Zaheduzzaman Sarker] SCReAM model does not simply assumes the
>existence of coupled-cc by default. As SCReAM already uses a transmission
>scheduler it is easy to implement "prioritization coupling" there.
>>
>However, if they decided to implement SCReAM they have an easy
>alternative for coupled-cc as Michael W himself says.
>>
>> On the other hand the one should understand what would be the
>> motivation for a leveled approach would be - also because it seems to
>> give a more complex model  - Is it so that such is motivated by - for
>> example:
>?
>>
>> * It is more efficient (overhead or protocol, state, whatever) to
>> manage CC on "one" multi-stream RTP basis when such is possible -  as
>> scream does ? and therefore the leveled approach has advantages from a
>> general perspective or possibly it is advantageous in the particular
>> situations where the default (build-

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Mon, 20 Jul 2015 07:49:29 +0000

Hi,

It was/is not our prime intention to go through different  scheduling can be used. We described a simple way of doing prioritization in the scheduler.

Regarding moving into a common draft with other solutions, I don’t have any strong opinion about it but seems like a reasonable thing to do if RMCAT is going to produce a single document describing alternatives. The stream prioritization is already mentioned as an "additional feature" so separating that from SCReAM draft and just pointing to that the other doc should not be an issue.

BR

Zahed

> From: Karen Elisabeth Egede Nielsen [mailto:[email-masked]]
> Sent: den 20 juli 2015 07:32
> scheduling priorities, There is no doubt about that :-), Indeed SCTP does it and
> scream does it.
> (For information then for SCTP a number of different scheduling possibilities is
> described in
> proposal where the coupled is done in-between per flow-CC contexts by
> prioritization on manipulation of the cwnd.
> And the we also have, a not so detailed proposal/description, on how the same
> can be achieved via scheduling (scream).
> It should be ok to allow for different ways to implement coupling, but we should
> then eventually have one document that describes both solutions. Shouldn't we
> ?
> 
> BR, Karen
> 
> 
> 
> >> of scream - or the in-build coupled congestion control of  scream,
> >> one can
> >say.
> >>
> >[Zaheduzzaman Sarker] I believe your interpretation of a scream
> >controller controlling multiple RTP flows only with same "UDP
> socket/connection tuple"
> >is coming from a previous reply where it was said the controller
> >controls flows only towards same destination. I think the correct thing
> >to say is, the RTP packet passing through the a scream transmission
> >scheduler more of less showing same characteristics, like sharing same
> >bottleneck. Hence if there is a system like SBD already allows multiple
> >RTP flows with different destination to be identified as sharing the
> >same bottleneck then prioritizations can be applied to all those flows
> >by same transmission scheduler.
> >In the implementation of scream there is a method to register a RTP
> >flow to a scream controller which will be congestion controlled by the
> >that particular scream controller (the transmission scheduler resides
> >in the scream control).
> >This also means the scream controller has to get the information from
> >the RTCP feedbacks for the respective registered flows.
> >Hence, your interpretation of " the scream model does not directly
> >allow for prioritized coupling in between multiple RTP flows that do
> >not use the same UDP socket/connection tuple " is not completely true.
> >
> >> Possibly one can see the scream model as introducing one more level
> >> in the coupled cc model - i.e., prioritized coupling in between the
> >> flows using the same UDP socket/UDP tuple is done by means of scream
> >> transmission scheduling priorities and then coupled congestion
> >> control following the welzl model on top of this, when more "advanced"
> >existence of coupled-cc by default. As SCReAM already uses a
> >transmission scheduler it is easy to implement "prioritization coupling" there.
> >>
> >> scream does ? and therefore the leveled approach has advantages from
> >> a general perspective or possibly it is advantageous in the
> >> particular situations where the default (build-

---

From: Michael Welzl <[email-masked]>
Date: Mon, 20 Jul 2015 10:02:59 +0200

> On 20. jul. 2015, at 07.32, Karen Elisabeth Egede Nielsen <[email-masked]> wrote:
> 
> HI,
> 

I think so. There is already (short, because it's so simple) text about scheduling in coupled-cc and we can extend that and make it stand out more clearly (now it's in the intro I think).

Cheers,
Michael

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 22 Jul 2015 09:06:47 +0000

Hi

Still waiting for the summer to show up so I'll try to chime in while this subject is at least luke-warm. I may be slow with follow up on this, sorry for that in advance.

Stream prio in SCReAM is outlined in section A.2 in the draft (https://tools.ietf.org/html/draft-ietf-rmcat-scream-cc-01#page-28 ). In its function it resembles fq-codel quite a lot in the sense that streams that do not get scheduled at a given scheduling instant gets a credit that is used to increase the chance of being scheduled at the next scheduling instant. A configurable scheduling priority determines how much credit  that is given. Looking in the https://tools.ietf.org/pdf/draft-ietf-tsvwg-sctp-ndata-04.pdf draft I would say that this resembles the "Weighted Fair Queueing Scheduler"  the most. Some additional means are needed to make this work well in reality though (see below). 

The benefit with doing scheduling priority and a common congestion control is an increased probability that the path capacity is fully utilized, if a stream becomes idle (for instance no or little activity in one video source) , more resources are allocated to the other sources. And this is achieved with no or very little additional delay buildup on the sender side.
Another benefit with stream prioritization is that it is easier to make one congestion control behave well (low delay /packet loss) esp. on a wireless link. We have simulation results that supports this statement albeit in the different context (QUIC congestion control evolution).
The stream prioritization and one congestion control of course require that all the streams share the same bottleneck characteristics

With that said, the biggest challenge I have seen this far is the large variability of esp. video sources. Sit still in front of the camera and the output rate becomes very low regardless the target rate, move yourself closer to the camera and bang!!.. you get 2Mbps video rate all of a sudden. This variability causes issues with stream prioritization in the sense that less variable streams may be pushed back. Additional code is therefore added in SCReAM to correct this anomaly and ensure a (weighted) fair distribution  over a longer time scale. Still as mentioned in the SCReAM draft, this needs to be tried out some more. I have sofar only simulated video sources with variability to see that the current implementation is reasonably robust.
The bottom line here is that, while scheduling of streams with static properties may be considered simple, real video may be a taller order. 

/Ingemar

> Sent: den 20 juli 2015 10:03
> To: Karen E. E. Nielsen
> Cc: Zaheduzzaman Sarker; Ingemar Johansson S; [email-masked]; Mirja
> Kühlewind
> > Yes - there is the possibility to implement coupled congestion control
> > with scheduling priorities, There is no doubt about that :-), Indeed
> > SCTP does it and scream does it.
> > But it is not so well-described in the scream draft how exactly the
> > prioritization works and which means an application has to define
> > different priorities.
> > And the we also have, a not so detailed proposal/description, on how
> > the same can be achieved via scheduling (scream).
> > It should be ok to allow for different ways to implement coupling, but
> > we should then eventually have one document that describes both
> I think so. There is already (short, because it's so simple) text about
> scheduling in coupled-cc and we can extend that and make it stand out more
> clearly (now it's in the intro I think).
> 
> Cheers,
> Michael

---

From: Michael Welzl <[email-masked]>
Date: Wed, 22 Jul 2015 11:16:21 +0200

understood; i think these details are beyond the scope of our coupling draft which focuses on a different method.

cheers
michael

Sent from my iPhone

> On 22. juli 2015, at 11:06, Ingemar Johansson S <[email-masked]> wrote:
> 
> Hi
>

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 24 Jul 2015 08:42:17 +0000

Hi

Not sure I get it. 
Do you mean that coupling algo is immune to video coder output rate variation ?

/Ingemar

> Sent: den 22 juli 2015 11:16
> To: Ingemar Johansson S
> Cc: Karen E. E. Nielsen; Zaheduzzaman Sarker; [email-masked]; Mirja
> Kühlewind
> > On 22. juli 2015, at 11:06, Ingemar Johansson S
> <[email-masked]> wrote:
> >
> > Hi
> >
> > Still waiting for the summer to show up so I'll try to chime in while this
> subject is at least luke-warm. I may be slow with follow up on this, sorry for
> that in advance.
> >
> > Stream prio in SCReAM is outlined in section A.2 in the draft
> (https://tools.ietf.org/html/draft-ietf-rmcat-scream-cc-01#page-28 ). In its
> function it resembles fq-codel quite a lot in the sense that streams that do
> not get scheduled at a given scheduling instant gets a credit that is used to
> increase the chance of being scheduled at the next scheduling instant. A
> configurable scheduling priority determines how much credit  that is given.
> Looking in the https://tools.ietf.org/pdf/draft-ietf-tsvwg-sctp-ndata-04.pdf
> draft I would say that this resembles the "Weighted Fair Queueing
> Scheduler"  the most. Some additional means are needed to make this work
> well in reality though (see below).
> >
> > The benefit with doing scheduling priority and a common congestion
> control is an increased probability that the path capacity is fully utilized, if a
> stream becomes idle (for instance no or little activity in one video source) ,
> more resources are allocated to the other sources. And this is achieved with
> no or very little additional delay buildup on the sender side.
> > Another benefit with stream prioritization is that it is easier to make one
> congestion control behave well (low delay /packet loss) esp. on a wireless
> link. We have simulation results that supports this statement albeit in the
> different context (QUIC congestion control evolution).
> > The stream prioritization and one congestion control of course require
> > that all the streams share the same bottleneck characteristics
> >
> > With that said, the biggest challenge I have seen this far is the large
> variability of esp. video sources. Sit still in front of the camera and the output
> rate becomes very low regardless the target rate, move yourself closer to
> the camera and bang!!.. you get 2Mbps video rate all of a sudden. This
> variability causes issues with stream prioritization in the sense that less
> variable streams may be pushed back. Additional code is therefore added in
> SCReAM to correct this anomaly and ensure a (weighted) fair distribution
> over a longer time scale. Still as mentioned in the SCReAM draft, this needs
> to be tried out some more. I have sofar only simulated video sources with
> variability to see that the current implementation is reasonably robust.
> > The bottom line here is that, while scheduling of streams with static
> properties may be considered simple, real video may be a taller order.
> >
> > /Ingemar
> >
> >
> >
> >
> >>> Yes - there is the possibility to implement coupled congestion
> >>> control with scheduling priorities, There is no doubt about that
> >>> :-), Indeed SCTP does it and scream does it.
> >>> Then for RMCAT coupled congestion control  we have the Wezlz,
> >>> Safiqul proposal where the coupled is done in-between per flow-CC
> >>> contexts by prioritization on manipulation of the cwnd.
> >>> It should be ok to allow for different ways to implement coupling,
> >>> but we should then eventually have one document that describes both
> >> I think so. There is already (short, because it's so simple) text
> >> about scheduling in coupled-cc and we can extend that and make it
> >> stand out more clearly (now it's in the intro I think).
> >>
> >> Cheers,
> >> Michael
> >

---

From: Michael Welzl <[email-masked]>
Date: Fri, 24 Jul 2015 10:57:18 +0200

No.

Our algo lets you assign priorities, and they are valid at the time you set them, they just give you a portion of the overall rate and it applies as soon as you set it.
If  you want to adjust priorities based on the coder output rate variation, you can implement this using our algo or with a scheduler. A scheduler operates per packet, we operate per time, so implementation-wise it's a bit different.

My point is only: whether you do it with our algorithm or without a scheduler, I think this should not be in our draft  (and I'm neutral about: should it be in the SCReAM draft or separate from it).

Cheers,
Michael

> On 24. jul. 2015, at 10.42, Ingemar Johansson S <[email-masked]> wrote:
> 
> Hi
> 
> Not sure I get it.

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 24 Jul 2015 09:14:31 +0000

OK, understand. 

No hard feelings from my side if the scheduling details are removed from the SCReAM draft (and possibly put elsewhere), it is currently only an appendix section anyway. 

I think however the scheduling it will remain in the running code (C++ code and OWR) at least until a good alternative is available.

/Ingemar

> Sent: den 24 juli 2015 10:57
> To: Ingemar Johansson S
> Our algo lets you assign priorities, and they are valid at the time you set
> them, they just give you a portion of the overall rate and it applies as soon as
> you set it.
> If  you want to adjust priorities based on the coder output rate variation, you
> can implement this using our algo or with a scheduler. A scheduler operates
> per packet, we operate per time, so implementation-wise it's a bit different.
> 
> My point is only: whether you do it with our algorithm or without a
> scheduler, I think this should not be in our draft  (and I'm neutral about:
> should it be in the SCReAM draft or separate from it).
> 
> Cheers,
> Michael
> 
> 
> > On 24. jul. 2015, at 10.42, Ingemar Johansson S
> >> understood; i think these details are beyond the scope of our
> >> coupling draft which focuses on a different method.
> >>
> >> cheers
> >> michael
> >>
> >> Sent from my iPhone
> >>
> >> subject is at least luke-warm. I may be slow with follow up on this,
> >> sorry for that in advance.
> >>>
> >> (https://tools.ietf.org/html/draft-ietf-rmcat-scream-cc-01#page-28 ).
> >> In its function it resembles fq-codel quite a lot in the sense that
> >> streams that do not get scheduled at a given scheduling instant gets
> >> a credit that is used to increase the chance of being scheduled at
> >> the next scheduling instant. A configurable scheduling priority determines
> how much credit  that is given.
> >> Looking in the
> >> https://tools.ietf.org/pdf/draft-ietf-tsvwg-sctp-ndata-04.pdf
> >> Scheduler"  the most. Some additional means are needed to make this
> >> work well in reality though (see below).
> >>>
> >> control is an increased probability that the path capacity is fully
> >> utilized, if a stream becomes idle (for instance no or little
> >> activity in one video source) , more resources are allocated to the
> >> other sources. And this is achieved with no or very little additional delay
> buildup on the sender side.
> >> congestion control behave well (low delay /packet loss) esp. on a
> >> wireless link. We have simulation results that supports this
> >> statement albeit in the different context (QUIC congestion control
> evolution).
> >>> The stream prioritization and one congestion control of course
> >>> require that all the streams share the same bottleneck
> >>> characteristics
> >>>
> >> variability of esp. video sources. Sit still in front of the camera
> >> and the output rate becomes very low regardless the target rate, move
> >> yourself closer to the camera and bang!!.. you get 2Mbps video rate
> >> all of a sudden. This variability causes issues with stream
> >> prioritization in the sense that less variable streams may be pushed
> >> back. Additional code is therefore added in SCReAM to correct this
> >> anomaly and ensure a (weighted) fair distribution over a longer time
> >> scale. Still as mentioned in the SCReAM draft, this needs to be tried
> >> out some more. I have sofar only simulated video sources with variability
> to see that the current implementation is reasonably robust.
> >>>>> But it is not so well-described in the scream draft how exactly
> >>>>> the prioritization works and which means an application has to
> >>>>> define different priorities.
> >>>>> And the we also have, a not so detailed proposal/description, on
> >>>>> how the same can be achieved via scheduling (scream).
> >>>>> but we should then eventually have one document that describes
> >>>>> both solutions. Shouldn't we ?
> >>>>

---

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 30 Jul 2015 12:31:33 +0200

Hi Ingemar,

> I think however the scheduling it will remain in the running code (C++ code and OWR) at least until a good alternative is available.

If the scheduling stays in the appendix, we definitely need to see evaluation 
results for Scream without scheduling. If scheduling is an essential part of 
Scream, you also need a shared bottleneck detection! Actually I guess we need to 
see evaluation results for Scream without scheduling in any case, because there 
might always be the case where the sbd does not detect a shared bottleneck 
correctly.

Mirja

-- 
------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Communication Systems Group
Institute TIK, ETH Zürich
Gloriastrasse 35, 8092 Zürich, Switzerland

Room ETZ G93
phone: +41 44 63 26932
email: [email-masked]
------------------------------------------

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 30 Jul 2015 11:22:40 +0000

Hi

Probably needs some clarification. The transmission scheduler that decides when to transmit RTP packets is an integral part of SCReAM and described in section 4.1.2.2 in the draft. 
Stream prioritization is applied when two or more streams share a congestion control and is described in section A.2.

The former has been evaluated earlier , the latter is only exemplified in 
https://github.com/EricssonResearch/scream/blob/master/SCReAM-description.pdf 

The evaluation of stream prioritization leads to a set of follow-up questions. What kind of source behavior do be assume, is it CBR  or VBR. Stream prioritization of CBR sources is quite trivial to solve IMO, VBR is more complicated.

/Ingemar

> Sent: den 30 juli 2015 12:32
> To: Ingemar Johansson S; Michael Welzl
> Cc: Karen E. E. Nielsen; Zaheduzzaman Sarker; [email-masked]
> > I think however the scheduling it will remain in the running code (C++ code
> and OWR) at least until a good alternative is available.
> 
> Scream, you also need a shared bottleneck detection! Actually I guess we
> need to see evaluation results for Scream without scheduling in any case,
> because there might always be the case where the sbd does not detect a
> shared bottleneck correctly.
> 
> Mirja
> 
> 
> 
> >
> > /Ingemar
> >
> >
> >> Our algo lets you assign priorities, and they are valid at the time
> >> you set them, they just give you a portion of the overall rate and it
> >> applies as soon as you set it.
> >> If  you want to adjust priorities based on the coder output rate
> >> variation, you can implement this using our algo or with a scheduler.
> >> A scheduler operates per packet, we operate per time, so
> implementation-wise it's a bit different.
> >>
> >>>> subject is at least luke-warm. I may be slow with follow up on
> >>>> this, sorry for that in advance.
> >>>>>
> >>>> streams that do not get scheduled at a given scheduling instant
> >>>> gets a credit that is used to increase the chance of being
> >>>> scheduled at the next scheduling instant. A configurable scheduling
> >>>> priority determines
> >>>> and the output rate becomes very low regardless the target rate,
> >>>> move yourself closer to the camera and bang!!.. you get 2Mbps video
> >>>> rate all of a sudden. This variability causes issues with stream
> >>>> prioritization in the sense that less variable streams may be
> >>>> pushed back. Additional code is therefore added in SCReAM to
> >>>> correct this anomaly and ensure a (weighted) fair distribution over
> >>>> a longer time scale. Still as mentioned in the SCReAM draft, this
> >>>> needs to be tried out some more. I have sofar only simulated video
> >>>> sources with variability
> >>>>>> Cc: Zaheduzzaman Sarker; Ingemar Johansson S; [email-masked];
> >>>>>> Mirja Kühlewind
> >>>>>>> Safiqul proposal where the coupled is done in-between per
> >>>>>>> flow-CC contexts by prioritization on manipulation of the cwnd.
> >>>>>>> It should be ok to allow for different ways to implement
> >>>>>>> coupling, but we should then eventually have one document that
> >>>>>>> describes both solutions. Shouldn't we ?
> >>>>>>

---

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 30 Jul 2015 14:21:19 +0200

Hi Ingemar,

okay thanks for the clarification. I discussed this with Zahed and the scheduler 
as described in section 4.1.2.2. should probably be renamed to something like 
sending rate controller because without the prioritization it's only handling 
one flow at a time, right?

Mirja

Room ETZ G93
phone: +41 44 63 26932

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 30 Jul 2015 22:30:27 +0000

Hi

The term transmission scheduler was taken from a discussion on the TCPM WG (thread starts at http://www.ietf.org/mail-archive/web/tcpm/current/msg08768.html ), the definition was borrowed from TCP laminar. This means that I am currently convinced about the name change. 
The RTP packets from two or more streams (RTP queues) are in its simplest form picked in a round-robin fashion. This is described in section 4.1. If prioritization is desired then a credit based system is used, described in appendix A.2. In reality the credit based approach is always enabled even though no prioritization between streams is applied, in this case all streams get the same priority.
It is imperative however that all streams are able to transmit RTP packets frequently. This to avoid extra queueing delay on the sender side. With a 2:1 prioritization allocation, one stream gets to send 2 RTP packets, then the other stream gets to send 1 RTP packet.
It is for instance not too practical to allow one stream to send for 100ms and then let the other stream transmit for 50ms, this will only increase latency and risk making the video choppy. This is the main reason why SCReAMs transmit scheduling and prioritization operates is RTP packet transmission clocked instead of timer based. 
But...of course one can implement timer based prioritization with shorter intervals, in which case the differences would be smaller, but for SCReAM it is more practical to be RTP packet transmission based.

PS, deliberately omitted the fact than one (or even all streams) may have nothing to transmit, this is handled in the code however.  For those interested, it is possible to experiment with the C++ code to see how the stream prioritization performs.

/Ingemar  

> Sent: den 30 juli 2015 14:21
> okay thanks for the clarification. I discussed this with Zahed and the
> scheduler as described in section 4.1.2.2. should probably be renamed to
> something like sending rate controller because without the prioritization it's
> only handling one flow at a time, right?
> 
> Mirja
> 
> 
> > Probably needs some clarification. The transmission scheduler that decides
> when to transmit RTP packets is an integral part of SCReAM and described in
> section 4.1.2.2 in the draft.
> > Stream prioritization is applied when two or more streams share a
> congestion control and is described in section A.2.
> >
> > https://github.com/EricssonResearch/scream/blob/master/SCReAM-
> descript
> > ion.pdf
> >
> > The evaluation of stream prioritization leads to a set of follow-up
> questions. What kind of source behavior do be assume, is it CBR  or VBR.
> Stream prioritization of CBR sources is quite trivial to solve IMO, VBR is more
> complicated.
> >
> > /Ingemar
> >
> >
> >
> >
> >> If the scheduling stays in the appendix, we definitely need to see
> >> evaluation results for Scream without scheduling. If scheduling is an
> >> essential part of Scream, you also need a shared bottleneck
> >> detection! Actually I guess we need to see evaluation results for
> >> Scream without scheduling in any case, because there might always be
> >> the case where the sbd does not detect a shared bottleneck correctly.
> >>
> >> Mirja
> >>
> >>
> >>
> >>>
> >>> /Ingemar
> >>>
> >>>
> >>>> you set them, they just give you a portion of the overall rate and
> >>>> it applies as soon as you set it.
> >>>>> Do you mean that coupling algo is immune to video coder output
> >>>>> rate
> >>>> variation ?
> >>>>>
> >>>>> /Ingemar
> >>>>>
> >>>>>> Cc: Karen E. E. Nielsen; Zaheduzzaman Sarker; [email-masked];
> >>>>>> Mirja Kühlewind
> >>>>>> In its function it resembles fq-codel quite a lot in the sense
> >>>>>> that streams that do not get scheduled at a given scheduling
> >>>>>> instant gets a credit that is used to increase the chance of
> >>>>>> being scheduled at the next scheduling instant. A configurable
> >>>>>> scheduling priority determines
> >>>>>> Scheduler"  the most. Some additional means are needed to make
> >>>>>> this work well in reality though (see below).
> >>>>>>>
> >>>>>> control is an increased probability that the path capacity is
> >>>>>> fully utilized, if a stream becomes idle (for instance no or
> >>>>>> little activity in one video source) , more resources are
> >>>>>> allocated to the other sources. And this is achieved with no or
> >>>>>> very little additional delay
> >>>>>> variability of esp. video sources. Sit still in front of the
> >>>>>> camera and the output rate becomes very low regardless the target
> >>>>>> rate, move yourself closer to the camera and bang!!.. you get
> >>>>>> 2Mbps video rate all of a sudden. This variability causes issues
> >>>>>> with stream prioritization in the sense that less variable
> >>>>>> streams may be pushed back. Additional code is therefore added in
> >>>>>> SCReAM to correct this anomaly and ensure a (weighted) fair
> >>>>>> distribution over a longer time scale. Still as mentioned in the
> >>>>>> SCReAM draft, this needs to be tried out some more. I have sofar
> >>>>>> only simulated video sources with variability
> >>>>>>>>> control with scheduling priorities, There is no doubt about
> >>>>>>>>> that :-), Indeed SCTP does it and scream does it.
> >>>>>>>>> But it is not so well-described in the scream draft how
> >>>>>>>>> exactly the prioritization works and which means an
> >>>>>>>>> (For information then for SCTP a number of different
> >>>>>>>>> scheduling possibilities is described in
> >>>>>>>>> And the we also have, a not so detailed proposal/description,
> >>>>>>>>> on how the same can be achieved via scheduling (scream).
> >>>>>>>> I think so. There is already (short, because it's so simple)
> >>>>>>>> text about scheduling in coupled-cc and we can extend that and
> >>>>>>>> make it stand out more clearly (now it's in the intro I think).
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Michael
> >>>>>>>
> >>>
> >>
> >> --

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 31 Jul 2015 07:31:25 +0000

Errata... Should be "not convinced about the name change", thanks for the heads up Zahed :-)

...But if it is needed in order to get a common set of terms

Sent from Moxier Mail
(http://www.moxier.com)

----- Ursprungligt meddelande -----
Från: Ingemar Johansson S <[email-masked]>
Till: Mirja Kühlewind <[email-masked]>, Michael Welzl <[email-masked]>
Kopia: "Karen E. E. Nielsen" <[email-masked]>, Zaheduzzaman Sarker <[email-masked]>, "[email-masked]" <[email-masked]>, Ingemar Johansson S <[email-masked]>
Skickat: 2015-07-31 0:30
Ämne: RE: [rmcat] Review of draft-ietf-rmcat-scream-cc-01

Hi

/Ingemar

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 31 Jul 2015 11:13:07 +0200

Hi Ingemar,

the name scheduler is correct for what you describe below. However in Section 
4.1.2.2 it is only described how to calculate send_wnd. There is one sentence 
saying "Packets are transmitted at a pace given by the send window, computed 
below", however, no details about pacing are given in this section. As this 
section is only about send_wnd calculation, this section should be named e.g. 
'sending rate controller".

Anything that described the scheduling of multiple streams is currently in the 
appendix; also what you describe below is, I believe, not documented at all...? 
Whenever you talk about multiplexing stream, scheduling is the right term, 
however please try to separate send rate calculation and scheduling and keep the 
scheduling part including pacing in the appendix for now (until we got one step 
further with the framework doc)!

Moreover, if there are things in the implementation that are not documented in 
the draft (and influence the performance), please update the draft!

Mirja

Room ETZ G93
phone: +41 44 63 26932