Subject: Re: [rmcat] Several comments about RRTCC

From: Stefan Holmer <[email-masked]>
Date: Mon, 7 Jan 2013 13:56:36 +0100

Some comments.

On Sat, Dec 29, 2012 at 2:30 PM, Michael Welzl <[email-masked]> wrote:

> Dear all,
>
> I / we have found a number of potential issues with
> draft-alvestrand-rtcweb-congestion-03, and would like to share them with
> the group. I have shared some but not all of these thoughts directly with
> the authors in the past, but I think it's useful to have them put together
> in a list.
>
> 1)
> The RRTCC acronym. The draft luckily doesn't contain it, but I think the
> mechanism should have an acronym anyway. I have to search through my
> mailbox every time to see if RRTCC is really what it was called… may I
> propose "DemiResi" ?  (Delay minimization with reduced signaling)
> "Resi" also gives you plenty of nice associations to mountains… cows…
> yodeling… red-white plaid skirts and Lederhosen!  Easier to remember than
> RRTCC, no doubt! To get you guys used to the idea, I will call it DemiResi
> in my text below  :-)
>

Fine with me, although I haven't had any problems remembering RRTCC (very
similar to RTC). :)

>
>
> 2)
> Given that minimizing the amount of feedback seems to be a goal, this
> feedback becomes very important. Losing a TCP ACK isn't a big deal, but
> losing a DemiResi feedback message is. Given the minimal space needed
> (essentially, one bit … plus a little more, to avoid ambiguity etc.), I
> think we should introduce "acks-of-acks", i.e. a message that says "I got
> your feedback". Yes, that message would only appear at the receiver within
> one RTT, but if the sender's timeout (2x t_max_fb_interval)  is larger than
> an RTT, it could help to make the receiver retransmit its feedback early
> when that message is missing. Personally I'm not convinced that it's a good
> idea to send less than a message / RTT anyway, in which case such
> ACKs-of-ACKs could still be useful, as they would disambiguate forward from
> backward loss. For instance, the next message could then tell the sender
> "don't reduce your rate because of the loss that you see, it's only the
> backward path".
>

Sounds like a reasonable approach to me.

>
>
> 3)
> The TCP steady-state throughput equation from RFC3448 (in what follows, I
> will call it "TFRC equation") is rather sensitive to the correct usage of
> p, the packet loss event ratio. The way it is used in the draft, p is
> packet loss but not the loss event ratio. Correct usage would require
> sending loss events from the receiver to the sender (DCCP has a way to do
> this), or, which would strike me as being much simpler, calculating this
> equation on the receiver side. The receiver has all the information about
> packet loss available. This would require the receiver to have some notion
> of the RTT, which it could get e.g. if item 2) above is implemented.
>

I agree that such a mechanism makes sense to have on the receiving side. We
initially put it on the sending side to have some type of compatibility
with receivers which did not implement the delay-based algorithm, but the
plan has been to also take the loss signal into account at the receiver.
Haven't gotten there yet though...

>
>
> 4)
> The text on page 12 of the draft reads as if it would be impossible for
> the receive-side estimate of the available bandwidth A(i) to go below the
> value obtained with the TFRC equation. If it does, however, DemiResi
> becomes like some form of TFRC, even in the absence of other TCPs. Can we
> be sure that this won't happen? It could be an idea to fall back to such
> TFRC-like behavior only if loss is large enough to indicate that we're
> competing with TCP… we might not always, and when we're not, we should keep
> delay low.
>

It should be interpreted as "As(i) can't go above A(i), and As(i) will only
go below what TFRC ( let's call it TFRC(i)) says if A(i) is less than
TFRC(i)."

Therefore we shouldn't have the problem you mention, but we will also not
be able to use this mechanism to compete with TCP. It should however be
useful in a scenario where we don't get a reliable delay-signal due to AQM.

>
> 5)
> The thresholds on page 12 seem troublesome to me: the behavior stays
> unchanged in the 2-10% loss regime (this is high loss!), it is reduced in
> proportion to the loss ratio in case of above 10% (seems reasonable, except
> that 10% is already quite extreme), BUT: "As long as less than 2% of the
> packets have been lost As(i) will be increased as As(i)=1.05(As(i-1)+1000)"
> …. this latter item means that, as long as we're below 2% loss, the
> mechanism increases its rate, making 2% its loss target?!  This strikes me
> as rather odd for a mechanism that's supposed to avoid loss and delay and
> keep queues low. Again, if this is done for interoperation with TCP, it
> could be an idea to do it only when loss is above a certain percentage (as
> it would be caused by TCP).
>

I agree that those thresholds may have to be changed. The 2% threshold
could indeed be interpreted as some kind of target for the AQM case, which
I agree is a bad idea. I think scaling down all of these thresholds makes a
lot of sense.

>
>
> 6)
> Rather obviously, ECN should be incorporated...
>
>
> Surely there's more stuff to find. I guess my main criticism is that we
> still don't have a paper, a technical report, simulation code or a simple
> standalone implementation to play with. Simply not enough data.
>
> I hope this was constructive...
>
> Cheers,
> Michael
>
>

---

From: Stefan Holmer <[email-masked]>
Date: Mon, 7 Jan 2013 14:07:17 +0100

On Mon, Dec 31, 2012 at 12:06 PM, Michael Welzl <[email-masked]> wrote:

>
> On Dec 31, 2012, at 10:58 AM, Sergio Garcia Murillo <
> [email-masked]> wrote:
>
> > El 29/12/2012 14:30, Michael Welzl escribió:
> >> [..]
> >> 5)
> > If i understood the paper correctly, if both sender and receiver
> implements it, the overuse detector will be triggered before the 2-10%
> ratio is reached, so in theory it is safe to increase the rate as the
> packets lost would not be associated to congestion. The problem comes when
> the receiver doesn't not implement its part, and then I agree with you that
> a 2-10% loss without any extra mechanism would provide a very bad
> experience. If FEC/RED are used, then maybe the loss is not noticeable, but
> haven't tried that myself yet. Also, there may be some other codec specific
> solutions that could make the  2-10% loss to not be a problem, like
> periodic intra refresh, disposable frames, etc..
>
> Assuming that up to 2% loss isn't necessarily due to congestion also means
> that the mechanism rather heavily relies on delay. Whether we use OWD, IPG…
> either way, it's a pretty unreliable signal, e.g. noise can be introduced
> by multiple bottlenecked links, and it loses granularity when AQM is used
> or queues are short. I'm not arguing against the general use of a delay
> measure - but I'm not sure I like the idea of assuming: unless a delay
> based over-use detector fires, losses are not due to congestion.
>
>
> >> [..]
> >>
> > I have been trying to implement the algorithm myself in my MCU and found
> quite some problems, especially with the initial values of the parameters,
> but after checking the google webrtc implementation I have quite positive
> results with the overuse detector.  Haven't got much luck yet with the
> bitrate estimator. I am also starting to implement some testing tools to
> help me in the implementation and troubleshooting, I will keep you posted
> when they are stable enough for playing.
>
> Cool!
>
> I'll leave it to the authors to answer your notes below (my own answer
> would be "not sure, would have to dig deeper into the scheme again").
>
> Cheers,
> Michael
>
>
> > Maybe it is stupid, but on the bitrate estimator, given that "Not until
> over-use has been detected for at least gamma_2 milliseconds and at least
> gamma_3 frames, a definitive over-use will be signaled.", couldn't we use
> the bitrate received during that period as a valid bitrate estimation of
> the available bandwidth?
>

Not sure what you mean, but it sounds very similar to what we are actually
doing. If we detect an over-use, the actual throughput calculated over some
window (0.5 to 1 second) is used as an estimate of the available bandwidth,
as described on page 10: A_hat(i) = alpha*R_hat(i). The factor alpha (< 1)
is used to make sure that we decrease the estimate to a rate somewhat lower
than what we believe the available bandwidth is so that the queues will
have a chance to drain.

> >
> > On a last note, I believe that the algorithm will work much better if
> the codecs are able to produce output bitrates as closed as the target
> bitrate as possible (i.e. almost CBR). Which seems to be confirmed by a
> comment in google's code:
>

Indeed it's better to have a codec which produces what you are requesting
from it. You can also have a smoother/pacer after the encoder to make sure
packets are paced out according to the estimated available bandwidth (with
the penalty of increased delay).

> >
> >  // We try to filter out very late frames. For instance periodic key
> >  // frames doesn't fit the Gaussian model well.
> >
> > Best regards
> > Sergio
>
>

---

From: Michael Welzl <[email-masked]>
Date: Mon, 7 Jan 2013 14:40:52 +0100

Hi,

In line:

On 7. jan. 2013, at 13:56, Stefan Holmer wrote:

> I / we have found a number of potential issues with draft-alvestrand-rtcweb-congestion-03, and would like to share them with the group. I have shared some but not all of these thoughts directly with the authors in the past, but I think it's useful to have them put together in a list.
> 
> 1)
> The RRTCC acronym. The draft luckily doesn't contain it, but I think the mechanism should have an acronym anyway. I have to search through my mailbox every time to see if RRTCC is really what it was called… may I propose "DemiResi" ?  (Delay minimization with reduced signaling)     "Resi" also gives you plenty of nice associations to mountains… cows… yodeling… red-white plaid skirts and Lederhosen!  Easier to remember than RRTCC, no doubt! To get you guys used to the idea, I will call it DemiResi in my text below  :-)
> 
> Fine with me, although I haven't had any problems remembering RRTCC (very similar to RTC). :)

Ohhhh then PLEASE call it DemiResi! I'd love that name!  :-)
(I'll admit that that's because I'll have to laugh every time I hear it  :-)   )

>  
> 2)
> Given that minimizing the amount of feedback seems to be a goal, this feedback becomes very important. Losing a TCP ACK isn't a big deal, but losing a DemiResi feedback message is. Given the minimal space needed (essentially, one bit … plus a little more, to avoid ambiguity etc.), I think we should introduce "acks-of-acks", i.e. a message that says "I got your feedback". Yes, that message would only appear at the receiver within one RTT, but if the sender's timeout (2x t_max_fb_interval)  is larger than an RTT, it could help to make the receiver retransmit its feedback early when that message is missing. Personally I'm not convinced that it's a good idea to send less than a message / RTT anyway, in which case such ACKs-of-ACKs could still be useful, as they would disambiguate forward from backward loss. For instance, the next message could then tell the sender "don't reduce your rate because of the loss that you see, it's only the backward path".
> 

Cool! So then we need to define that "I got your feedback message" somewhere...

>  
> 3)
> The TCP steady-state throughput equation from RFC3448 (in what follows, I will call it "TFRC equation") is rather sensitive to the correct usage of p, the packet loss event ratio. The way it is used in the draft, p is packet loss but not the loss event ratio. Correct usage would require sending loss events from the receiver to the sender (DCCP has a way to do this), or, which would strike me as being much simpler, calculating this equation on the receiver side. The receiver has all the information about packet loss available. This would require the receiver to have some notion of the RTT, which it could get e.g. if item 2) above is implemented.
> 
> I agree that such a mechanism makes sense to have on the receiving side. We initially put it on the sending side to have some type of compatibility with receivers which did not implement the delay-based algorithm, but the plan has been to also take the loss signal into account at the receiver. Haven't gotten there yet though...
>  
> 
> 
> 4)
> The text on page 12 of the draft reads as if it would be impossible for the receive-side estimate of the available bandwidth A(i) to go below the value obtained with the TFRC equation. If it does, however, DemiResi becomes like some form of TFRC, even in the absence of other TCPs. Can we be sure that this won't happen? It could be an idea to fall back to such TFRC-like behavior only if loss is large enough to indicate that we're competing with TCP… we might not always, and when we're not, we should keep delay low.
> 
> It should be interpreted as "As(i) can't go above A(i), and As(i) will only go below what TFRC ( let's call it TFRC(i)) says if A(i) is less than TFRC(i)."
> 
> Therefore we shouldn't have the problem you mention, but we will also not be able to use this mechanism to compete with TCP. It should however be useful in a scenario where we don't get a reliable delay-signal due to AQM.

I don't get that. I had thought that the idea is to at least be as aggressive as TFRC in the absence of a reliable delay-signal? I thought that this, in principle (minus the problem I mention above), is a nice idea, and you'd automatically get that if As(i) >= TFRC(i), and this is actually what the text on page 12 says. But what you say above is: As(i) <= TFRC(i) ?!

> 
> 5)
> The thresholds on page 12 seem troublesome to me: the behavior stays unchanged in the 2-10% loss regime (this is high loss!), it is reduced in proportion to the loss ratio in case of above 10% (seems reasonable, except that 10% is already quite extreme), BUT: "As long as less than 2% of the packets have been lost As(i) will be increased as As(i)=1.05(As(i-1)+1000)"
> …. this latter item means that, as long as we're below 2% loss, the mechanism increases its rate, making 2% its loss target?!  This strikes me as rather odd for a mechanism that's supposed to avoid loss and delay and keep queues low. Again, if this is done for interoperation with TCP, it could be an idea to do it only when loss is above a certain percentage (as it would be caused by TCP).
> 
> I agree that those thresholds may have to be changed. The 2% threshold could indeed be interpreted as some kind of target for the AQM case, which I agree is a bad idea. I think scaling down all of these thresholds makes a lot of sense.
>  
> 
> 
> 6)
> Surely there's more stuff to find. I guess my main criticism is that we still don't have a paper, a technical report, simulation code or a simple standalone implementation to play with. Simply not enough data.
> 

Cheers,
Michael

---

From: Stefan Holmer <[email-masked]>
Date: Mon, 7 Jan 2013 16:13:23 +0100

> I agree that such a mechanism makes sense to have on the receiving side.
> We initially put it on the sending side to have some type of compatibility
> It should be interpreted as "As(i) can't go above A(i), and As(i) will
> only go below what TFRC ( let's call it TFRC(i)) says if A(i) is less than
> TFRC(i)."
>
> I don't get that. I had thought that the idea is to at least be as
> aggressive as TFRC in the absence of a reliable delay-signal? I thought
> that this, in principle (minus the problem I mention above), is a nice
> idea, and you'd automatically get that if As(i) >= TFRC(i), and this is
> actually what the text on page 12 says. But what you say above is: As(i) <=
> TFRC(i) ?!
>

I can see how that text can be misinterpreted... It's not clear in the
draft what we do when TFRC(i) > A(i). In our implementation the last thing
we do is to limit As(i) to be less than or equal to A(i), so therefore we
_can_ choose an As(i) < TFRC(i) if A(i) < TFRC(i).

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 08 Jan 2013 14:44:28 +0100

Some inlines...

On 01/07/2013 01:56 PM, Stefan Holmer wrote:
> Some comments.
>
> On Sat, Dec 29, 2012 at 2:30 PM, Michael Welzl <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Dear all,
>
>     draft-alvestrand-rtcweb-congestion-03, and would like to share
>     them with the group. I have shared some but not all of these
>     thoughts directly with the authors in the past, but I think it's
>     useful to have them put together in a list.
>
>     1)
>     The RRTCC acronym. The draft luckily doesn't contain it, but I
>     think the mechanism should have an acronym anyway. I have to
>     search through my mailbox every time to see if RRTCC is really
>     what it was called… may I propose "DemiResi" ?  (Delay
>     minimization with reduced signaling)     "Resi" also gives you
>     plenty of nice associations to mountains… cows… yodeling…
>     red-white plaid skirts and Lederhosen!  Easier to remember than
>     RRTCC, no doubt! To get you guys used to the idea, I will call it
>     DemiResi in my text below  :-)
>
>

I frankly can't bring myself to care, but if you want to make a new 
label, shouldn't we attach it to the end product rather to the input 
proposal?
>
>
>
>     2)
>     Given that minimizing the amount of feedback seems to be a goal,
>     this feedback becomes very important. Losing a TCP ACK isn't a big
>     deal, but losing a DemiResi feedback message is. Given the minimal
>     space needed (essentially, one bit … plus a little more, to avoid
>     ambiguity etc.), I think we should introduce "acks-of-acks", i.e.
>     a message that says "I got your feedback". Yes, that message would
>     only appear at the receiver within one RTT, but if the sender's
>     timeout (2x t_max_fb_interval)  is larger than an RTT, it could
>     help to make the receiver retransmit its feedback early when that
>     message is missing. Personally I'm not convinced that it's a good
>     ACKs-of-ACKs could still be useful, as they would disambiguate
>     forward from backward loss. For instance, the next message could
>     then tell the sender "don't reduce your rate because of the loss
>     that you see, it's only the backward path".
>
>

As I keep reminding people .... a message/RTT devolves to a 
message/message for short-RTT scenarios. While many short-RTT scenarios 
are also high-bandwidth, not all are.

>
>
>     3)
>     The TCP steady-state throughput equation from RFC3448 (in what
>     follows, I will call it "TFRC equation") is rather sensitive to
>     the correct usage of p, the packet loss event ratio. The way it is
>     used in the draft, p is packet loss but not the loss event ratio.
>     Correct usage would require sending loss events from the receiver
>     to the sender (DCCP has a way to do this), or, which would strike
>     me as being much simpler, calculating this equation on the
>     receiver side. The receiver has all the information about packet
>     loss available. This would require the receiver to have some
>     notion of the RTT, which it could get e.g. if item 2) above is
>     implemented.
>
>
> I agree that such a mechanism makes sense to have on the receiving 
> side. We initially put it on the sending side to have some type of 
> compatibility with receivers which did not implement the delay-based 
> algorithm, but the plan has been to also take the loss signal into 
> account at the receiver. Haven't gotten there yet though...

Having the sender do something reasonable when the receiver just stops 
sending feedback is a required feature, I think.
>
>
>
>     4)
>     The text on page 12 of the draft reads as if it would be
>     impossible for the receive-side estimate of the available
>     bandwidth A(i) to go below the value obtained with the TFRC
>     equation. If it does, however, DemiResi becomes like some form of
>     TFRC, even in the absence of other TCPs. Can we be sure that this
>     won't happen? It could be an idea to fall back to such TFRC-like
>     behavior only if loss is large enough to indicate that we're
>     competing with TCP… we might not always, and when we're not, we
>     should keep delay low.
>
>
> only go below what TFRC ( let's call it TFRC(i)) says if A(i) is less 
> than TFRC(i)."
>
> Therefore we shouldn't have the problem you mention, but we will also 
> not be able to use this mechanism to compete with TCP. It should 
> however be useful in a scenario where we don't get a reliable 
> delay-signal due to AQM.
>
>
>
>     5)
>     The thresholds on page 12 seem troublesome to me: the behavior
>     stays unchanged in the 2-10% loss regime (this is high loss!), it
>     is reduced in proportion to the loss ratio in case of above 10%
>     (seems reasonable, except that 10% is already quite extreme), BUT:
>     "As long as less than 2% of the packets have been lost As(i) will
>     be increased as As(i)=1.05(As(i-1)+1000)"
>     …. this latter item means that, as long as we're below 2% loss,
>     the mechanism increases its rate, making 2% its loss target?!
>      This strikes me as rather odd for a mechanism that's supposed to
>     avoid loss and delay and keep queues low. Again, if this is done
>     for interoperation with TCP, it could be an idea to do it only
>     when loss is above a certain percentage (as it would be caused by
>     TCP).
>
>
> could indeed be interpreted as some kind of target for the AQM case, 
> which I agree is a bad idea. I think scaling down all of these 
> thresholds makes a lot of sense.

Seems like an area for running experiments to me.

>     Surely there's more stuff to find. I guess my main criticism is
>     that we still don't have a paper, a technical report, simulation
>     code or a simple standalone implementation to play with. Simply
>     not enough data.
>

---

From: Michael Welzl <[email-masked]>
Date: Tue, 8 Jan 2013 15:28:14 +0100

On 8. jan. 2013, at 14:44, Harald Alvestrand wrote:

> I frankly can't bring myself to care, but if you want to make a new label, shouldn't we attach it to the end product rather to the input proposal?

Well honestly that wasn't so serious, and I don't really care either. It's nice to have an acronym when talking about it.

> As I keep reminding people .... a message/RTT devolves to a message/message for short-RTT scenarios. While many short-RTT scenarios are also high-bandwidth, not all are.

Just to be sure we're not losing the context: there are two issues in my text above - 1) we shouldn't generally send less feedback than 1 message per RTT;  2) if we do, we should at least have some sort of "ACK-of-ACK". #1 was a sort of side comment, and item #2 is what this was really about, and what Stefan replied to. You're replying to #1 here.

Here's my answer to what you say:
My concern is the notion of a DesiResi  ;-)   sender that transmits a lot and competes with TCP connections on the same path, which, in the absence of extreme RTT variation, will have a timeout a bit  above an RTT. DesiResi's sender might not react for several RTTs and just keep sending, while the TCPs will all experience a timeout. The resulting behavior could be a level of unfairness that is user-perceivable and just not good.

Now, even TCP doesn't HAVE to send data every RTT, and when it doesn't, there's also no feedback... nothing wrong with that, neither for TCP nor for DesiResi. So my statement "Personally I'm not convinced that it's a good idea to send less than a message / RTT anyway" is too strict, it should be limited to the case of: sender's rate >= 1 message / RTT.

> Having the sender do something reasonable when the receiver just stops sending feedback is a required feature, I think.

I think we all agree on that, but anyway, it seems to me that what you say here is unrelated to the text just above it. This is about how, and where, to calculate the TFRC equation.

> Seems like an area for running experiments to me.

I agree