Subject: [rmcat] Review of draft-ietf-rmcat-nada-00

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 08 Jul 2015 19:53:17 +0200

Hi Xiaoqing, hi all,

this is a review as an individual contributor and not as a wg chair.

I think the document is well written but I had some problem to easily follow 
some of the details. I think this could be easily improved a lot by renaming 
(nearly all) variables; I'll give further suggestions on this below. My main 
problem, however, was on understanding the formula(s) in section 5.1.2. This is 
partly because of the excessive use of variables with not super helpful names, 
but mostly because there is no reasoning giving why the formula looks the way it 
looks! As not currently done in the draft, could you maybe explain on the 
mailing list now, how you came up with this formula? What are the different 
components good for? How did you decided to add all these calculations?

Further there are some parts where a more detailed description is needed (see 
also further comments below). This document should allow people to re-implement 
NADA without knowing your code. I think there are a number of details missing in 
the description and other parts are not specified well enough as probably 
considered as implementation-specific details. However, I think there are things 
that actually will influence NADA performance if implemented differently and 
should therefore be further specified as well (at least one default approach), 
e.g. the filtering of delay values.

See more detailed comment in the order of the document structure below:

1) This is a rather general comment on the first paragraph in the introduction 
and may apply to other schemes as well: You mention challenges here. However, 
you don't refer to the rmcat requirements doc. Maybe you should refer this 
somewhere and potentially even discuss if and how the requirements have been 
address, or at least how the challenges you mention here are reflected in the 
requirements doc.

2) I would recommend to rename section 2 to 'Overview' instead of 'System Model' 
because that is what is actually is; it gives a summary description of what NADA 
is doing. As this text already introduced a number of variables, I would highly 
recommend to have Figure 1 already here because I needed to look at the Figure 
(knowing that there is one) to actually understand the text. Further I don't 
think the first point mentioning the source media stream is needed as this is 
also not shown in the figure.

3) Further I would recommend one new, own section that lists all variables and 
parameters! That would help really a lot!
Please also note that there is a difference between variables and parameters. 
Parameters are things that have a predefined value that you set once, while 
variables change over time. Usually parameter are noted in all upper case 
letters. In your currently description, parameters and variables look the same 
which makes things super difficult to read. E.g. d_th is a parameter and should 
be D_TH, or even better DELAY_THRES. And that's the second recommendation: I 
would recommend use longer, more descriptive names in many case because it helps 
the reader to follow. Further if you are going to have an own variables and 
parameter section, please indicate all default values for parameters.

4) On section 4.1
- t_r,n is hard to read, use t_{r,n} or t_r_n or even better ts_rec_n
- I'd also recommend to following variable names:
   z_n -> owd_n
   d_n -> qdelay_n
   d_f -> delay_base or owd_base
   d_hat_n -> qdelay_n_filter (or what does 'hat' stand for? or don't use 
d_hat_n at all and just have one more formula: d_n >- filter(d_n)...
- Please describe the estimation of the base delay in more detail because I 
think that's a case where there can be an influence on performance. E.g. in 
LEDBAT the minimum of 1-minutes minutes of the last 10 minutes is used.
- Did you also try a 2-point minimum filter? That's even easier and I'd assume 
that would be sufficient in most cases? Or what's the reason for the 5-point 
median filter?
- Do the clocks have to be synchronized for this approach? Please discuss this 
somewhere!

5) On section 4.2
- This section introduces a parameter without naming it (500ms). This is 
actually the same 500ms as introduces as delta in equation (2), right? Further I 
don' think 500ms is a good recommendation here. This value MUST be larger than 
one RTT and therefore there is a dependency on the RTT. Therefore for my point 
of view the recommendation should be something like 3*RTT and not a fixed value.
- Please specify more concretely the exponential smoothing filter that is used. 
Especially which weight is used!?!

6) I've detected a number of case, where the documents basically says something 
like 'further work is need here'. I would find it very helpful (also as a chair) 
to list all these issues in a open issues section. There are three issues I've 
found so far in section 4.2, 4.7 and 5.1:
   - Alternative approach to calculate the loss ratio (as e.g. TFRC)
   - OWD vs. RTT
   - Choice of start value
+ of course all discussion on parameters (which would include the question if a 
certain value is a MUST e.g. because of fairness reason, or if there is range to 
choose from)

7) Section 4.3
- I think this section should be a subsection of 4.1 because it's 'just' further 
steps to get the delay estimate.
- The formula was super hard to parse for me because there usually should be a 
large bracket. Maybe make it like this (as a graphic):

                      /  (d_hat_n),                  if d_hat_n < d_th;
                      |
                      |       (d_max - d_hat_n)^4
         d_tilde_n = <   d_th --------------------,  if  d_th<d_hat_n<d_max;
                      |         (d_max - d_th)^4
                      |
                      \   0,                         if d_hat_n > d_max.

- Parameters: d_max -> DELAY_MAX
               d_th -> DELAY_THRES

8) On section 4.4
- Why is this done on receiver side? Isn't there already some loss feedback in 
RTCP that could be used on sender side instead of sending a new report?
- Does the values of d_M=200ms and d_L=1s assume that the loss and marking 
probabilities are configured differently at the network node; lower marking 
threshold? Don't those values strongly depend on the configuration in the 
network node? Is it important that all competing NADA flows use the same value 
(for fairness)?
- I'm wondering if you really need to aggregated congestion anymore as you 
already have the scheme in section 4.3...? Wouldn't it be sufficient to use the 
estimated delay if there is no loss and use the other signals differently. E.g. 
if there is loss, halve as traditional cc would do?
- I think tuning these value is also an open issue and an important one!

9) Section 4.6
- Can't this part be done at the sender as well and then you don't have to 
transmit x_n'?
- Why do you need a feedback interval of 100ms if you only update any 
information every every 500ms?
- And if 250ms is sufficient, why do you use 100ms?

10) I'd recommend to move section 4.7 into an own section (7) at the end of the 
doc because it is not part of the algorithm description but 'only' some discussion.

11) "The sender initializes the reference rate R_n as R-min by default, or to
a value specified by the upper-layer application."
-> I think this is a big open issue. I don't think the application should 
specify this value because it could be misused, but the initial rate must fit to 
the initial coding I guess. So 10 packet/RTT might be to low...? Which rate are 
you using currently?

12) Section 5.1 say "if the reported queuing delay is close to zero". What does 
'close to zero' mean? Do you have another threshold here? How did you implement 
this? Please give a more detailed description here!

13) Section 5.1.1:
- "A typical value of T_th is 50ms." Does all NADA flows need to have the same 
value to be fair? Than this is a MUST value!
- "The rate R_n is updated as follows:" This does not specify when the rate is 
updated. On arrival of each report?
- This implements a multiplicative increase therefore I would denote the 
equation the following way:
   R_n = R_r + alpha * R_r
   I would also call the increase factor alpha as this is usually used in 
congestion control.
- I'm not sure if it is right that the increase rate should the large the 
smaller the base delay is. I'd say it should be the other was around. Please 
provide some reasoning here! Further is delta actually needed here? Or please 
explain further what the target ACK interval actually is!
- gamma_0 is also a confusing naming, should be gamma_max or alpha_max as 
suggested above.

14) Section 5.1.2:
- As I said, I've been starring at this for at least 15 minutes and could not 
figure out why you came up with this. This is my main issue. Please explain!
- Isn't delta_s the same than delta? Why don't you use the same parameter name? 
Again, should be DELTA or DELTA_REPORT.

15) Section 5.2 could be moved into section 2 because it again does not describe 
the algorithm.

16) The equation given in section 5.3 is really complicated where you actually 
just want to have the current number of bytes in the buffer. I think this is 
over-specified and would not need a formula. Further the use of the word size is 
confusing. A buffer size for me is the maximum amount of bytes a buffer could 
hold. I think you mean the current buffer occupancy (in bytes)...?

17) Please give values for beta_v and beta_s (as used in your implementation). 
Further I'm not sure if it is a good idea to adjust both R_v and R_n at once. 
Wouldn't it be sufficient to only adjust R_v?

18) One last comment on PCN (see last paragraph section 6): PCN was designed for 
admission control in one DiffServ domain and not the Internet, therefore this is 
not applicable here!

As a general recommendation (what I learn from my own work on congestion 
control), simplification is always a goal even if performance might be slightly 
worse: a simpler algorithm is easier to understand and therefore easier to 
evaluate, and people will be more likely to use it and make less implementation 
errors. So maybe at this stage you might want think about where you potentially 
could simplify NADA. Because I think up to now you only added stuff to make it 
work, but maybe there is now also something that could be removed instead...? 
Not sure if that's possible, just an idea...

Find other nits/proposals below:

s/The RTP sender also generates RTP timestamp in outgoing packets./The RTP 
sender also provides an RTP timestamp within each outgoing packets./

s/it is discounted following a non-linear curve/it is scaled down following a 
non-linear curve/

s/see Sec. 6.1.1 Accelerated ramp-up/see Sec. 5.1.1 Accelerated ramp-up/

s/R-min/R_min/

s/its rate increase after at least one round-trip time/its rate increase after 
at minimum one round-trip time/

s/ the rate increase multiplier should decrease with the delay  in the feedback 
control loop,/ the rate increase multiplier should smaller the larger the base 
delay in the feedback control loop is,/

Mirja

---

From: "Xiaoqing Zhu (xiaoqzhu)" <[email-masked]>
Date: Fri, 17 Jul 2015 15:23:07 +0000

Hi Mirja, 

Thanks much for your detailed review on the draft. Your input is very
helpful. 

Some high-level changes we plan to make on the draft after reading through
your comments:
- will make some structural changes to separate out the core congestion
control algorithm (e.g., accelerated ramp-up, gradual update) from
descriptions on recommended implementations (e.g., rate shaping buffer);
- will group mention of all open issues into one section instead of
spreading them across different parts of the draft;
- will add a table on list of parameters and default values (similar to
Fig. 2 in the updated video-traffic-source draft)
- regarding formulas in sec. 5.1.2: it was derived out of a
network-utility-maximization framework and we weren¹t sure how much of the
intermediate math should show up in the draft, since this is not a
research paper. Maybe I can add some citations to the theoretical
background of this work for folks who are potentially interested. Also, we
attempted to give some intuitive explanations on that in our previous
presentations. Maybe I can fold some of the text into the draft too.

Some more responses below inline (marked by [XZ]).

Best,
Xiaoqing

>e.g. the filtering of delay values.

[XZ] Yes, we learned about the need for further clarification of
implementation details after exchanging notes with Google colleagues for
their implementation of NADA. I was hoping to learn from their experience
during the Prague meeting, before coming back to revise the draft.
 
>address, or at least how the challenges you mention here are reflected in

[XZ]  Will refer to the requirement doc. Good suggestion.

>also not shown in the figure.

[XZ] Will do. 

>parameter section, please indicate all default values for parameters.

[XZ]  Will add a section on notations.  I¹m not so sure about denoting
parameters in all upper case and denoting variables in lower case: is that
a convention of IETF drafts? If so will be happy to follow suit.

>that would be sufficient in most cases? Or what's the reason for the

[XZ] will describe in further detail on how to estimate base delay, etc.
Have not tried With 2-point minimum filter. Will look into that. The
reason for the 5-pt median filter is that it worked well for outlier
rejection without adding excessive delay.

>- Do the clocks have to be synchronized for this approach? Please discuss

[XZ] Sec. 4.7 covers some of that. Since we use relative OWD, difference
between sender/receiver clocks is cancelled out. However, drift between
the two clocks over time may still affect performance, so we need some
external mechanism to help bound clock drifts.

>Especially which weight is used!?!

[XZ] The time window for packet loss ratio estimation (currently at 500ms)
is independent from the interval where derivative of queuing delay is
calculated.  Why MUST this interval be greater than RTT? Don¹t yet see the
reasoning ...

[XZ] In general we¹ll describe more clearly how packet loss ratio should
be estimated. 

>certain value is a MUST e.g. because of fairness reason, or if there is

[XZ] Will group all discussion on open issues into one section.

>- Parameters: d_max -> DELAY_MAX

[XZ]  I would like to clarify that non-linear warping of delay is not part
of delay estimation, but rather a mechanism for making the RMCAT flow more
robust against competing loss-based flow.  Also this step occurs after
estimation of loss (invoking of delay warping may also depend on detection
of loss, we are still exploring that variant). So keeping it separate will
be more clear. 

[XZ] Will work to make the equations/parameters/variables more readily
accessible. 

>RTCP that could be used on sender side instead of sending a new report?

[XZ] in fact some of the operations can be moved to sender instead of
receiver side. At this stage we don¹t have a strong preference. I can
mention in the draft the possibility of moving some operations to the
sender instead.  

>network node? Is it important that all competing NADA flows use the same

[XZ] The value of d_L and d_M does not depend on configurations at the
network node, but does assume that ECN markings, when available, will show
up first before losses. For fairness, yes, all NADA flows need to go with
the same penalty values.

>- I think tuning these value is also an open issue and an important one!

[XZ]  Yes, we do need the aggregation so that the algorithm will
seamlessly switch to reacting to loss-based congestion when competing
against loss-based flows that drive the queue to full.

>- And if 250ms is sufficient, why do you use 100ms?

[XZ] To clarify information is updated on an ongoing basis, just that they
are smoothed out over the past observation window. Will clarify that in
future draft. More frequent feedback gives the sender an opportunity to
react fast in face of sudden congestion.  It¹s a trade-off between
feedback overhead vs. feedback control loop responsiveness.

>doc because it is not part of the algorithm description but 'only' some

[XZ] Good point. Will merge that to the section on open issues.

>the initial coding I guess. So 10 packet/RTT might be to low...? Which

[XZ] The sender initializes the reference rate as R_min.  Discussion on
whether the sender can start at an arbitrarily specified rate by the
application belongs to the API/framework document, I think.

>this? Please give a more detailed description here!

[XZ]  Good catch. We can report on the threshold we use in considering
queuing delay as ³close to zero².

>- gamma_0 is also a confusing naming, should be gamma_max or alpha_max as

[XZ]  All rates are updated upon arrival of feedback report.  Thanks for
catching this and will clarify in draft.  I think we already used alpha
for the exponential smoothing parameter.  Will reconsider all notation
choices. 

[XZ]  We explained how the increase ratio was derived in our previous
presentation. Again, I need some guidance on how much of that derivation
should show up in the draftŠ

>15) Section 5.2 could be moved into section 2 because it again does not

[XZ] Good suggestion. Will do.

>hold. I think you mean the current buffer occupancy (in bytes)...?

[XZ] I think we can remove this one, since it¹s more of a description of
how the rate shaping buffer evolves than what the algorithm does.  Your
suggestion of describing in text sounds good.

>Wouldn't it be sufficient to only adjust R_v?

[XZ] Will add all default values of parameters to the table in the new
draft. 

[XZ] We do want to adjust both the outgoing rate to the network (R_s) and
the target rate for the encoder rate control (R_v) since it helps to drain
the local rate shaping buffer faster if the encoder reacts slowly.
  
>admission control in one DiffServ domain and not the Internet, therefore

[XZ] You are right that we did not use the term PCN in its original
intended context.  Instead we were really talking about an advanced
virtue-queu-based marking scheme that leveraged  the marking capability of
PCN. I¹ll change the wording here to make it less confusion.

>Not sure if that's possible, just an idea...

[XZ]  I too prefer simplicity whenever possible. The proposed NADA
algorithm is actually simpler than it appears to be from the current
draft. Some additional complexity was added to accommodate loss-based
competing flows. I hope after separating the core congestion control
algorithm from detailed implementation recommendations, this will become
more clear. 

>delay in the feedback control loop is,/

[XZ] Will keep these in mind for the next revision.  Thanks again for all
detailed comments and suggestions!