Subject: Re: [rmcat] clean up of CC drafts and app-interaction

From: "Van Der Auwera, Geert" <[email-masked]>
Date: Wed, 3 Dec 2014 23:02:02 +0000

Hi Michael,

I have a comment/question below marked as [GV].

Geert

From: rmcat [mailto:[email-masked]] On Behalf Of Michael Ramalho (mramalho)
Sent: Wednesday, November 19, 2014 8:06 AM
To: Ingemar Johansson S; Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

Randell: This is how we fixed the “impedance mismatch” from your previous email (thread is below).

Ingemar,

You have the basic point correct – I look forward to working with you and others on the details.

The TCP send window analogy may be better than your “releaseRtpPacket()” proposal. That is, the TCP send window is usually specified in bytes and if the send window is X bytes, it is basically saying that you MAY send UP TO X bytes now.

If the RTP payload you want to send is X+1 bytes long, you need to wait until the send window “opens” from X bytes to something larger in order for you to send your RTP packet payload.

In this way the RMCAT layer need not know the size (or nominal size) of the RTP payloads that your encoder is producing. In normal operation the RMCAT layer will “allow” more data to get to in the upcoming moments and it will signal this by opening up the send window to a value greater than X (but <= RTP payload MTU).

I mentioned during our meeting that under normal operation (i.e., no squelching) this interface is essentially a SHAPER in that if the coder produces data at the requested rate (or below) and that ALL of it should be allowed into the RMCAT layer – with the small exception that a shaper brings (try to send me too many bits at once and I will shape you to the rate of the shaper via the send window mechanism).

When exceptional transport conditions require “squelching”, this interface is similar to a POLICER (which normally drops a portion of the non-conformant traffic). The exception is that the application (the layer above the RMCAT layer) decides if IT WANTS to drop frames/packets based on the backlog(my word)/backpressure(Varun’s word)/impedance-mismatch(Randell’s word) prior to the RMCAT layer.

So, I expect that during those “squelching events” that the requested rate to the encoder should also drop (a la Geert’s presentation) … and if the encoder is quick to act on this new rate reduction request … that not many of its frames/packets will “back up” at that “media socket” interface. This should satisfy your “tuned more aggressively” issue and Geert’s points.

[GV] If my understanding of the proposed interface is correct, then the application layer would be responsible for estimating the ‘backlog’ size and taking appropriate action. However, the ‘backlog’ has many contributing parts and one of them is the rate controller’s reaction delay, which is the only part known at the application layer. Therefore, I think that the estimation of the ‘backlog’ (excluding rate control part) fits into the RMCAT layer, while the interface can pass the backlog estimate to the application layer, which can then take appropriate action. So, in this case the interface would consist of ‘requested rate’, ‘backlog size’, and the ‘media socket’.

I envision that partitioning the interface in this way (a requested rate interface and a “media socket” with a send window) will allow the encoder to be smart/intelligent/fast (e.g. SCReAM “frame skipping”) OR dumb/slow-to-change-rate (at the cost of badly delayed or badly encoded video because it didn’t self-police by dropping frames).

And, more importantly, the interface allows for shaping (the X or X+1 example above) to the RMCAT layer … and OPTIONAL self-policing by the layer above it (e.g., frame skipping/dropping for lowest-possible encoder-to-RMCAT layer delay). This allows for joint optimization of both the RMCAT and video encoding layers (I think this is Ericsson’s main point).

In my opinion, both of these properties - shaping and need-based policing - should be basic hygiene issues for the RMCAT layer (to satisfy Lars’/Mirja’s/Karen’s stated “safe-for-the-Internet” requirement and low-latency objectives).

I hope this makes the proposed mechanism – and the rationale behind it – a little more clear.

Michael Ramalho

From: Ingemar Johansson S [mailto:[email-masked]]
Sent: Wednesday, November 19, 2014 4:50 AM
To: Michael Ramalho (mramalho); Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG; Ingemar Johansson S
Subject: RE: [rmcat] clean up of CC drafts and app-interaction

Hi

The description sounds comprehensible.
While SCReAM and NADA has a slightly different terminology there are some similarities. While SCReAM talks about an RTP packet queue and a transmission scheduler, NADA use a rate shaper. They both try to achieve the same goal albeit in slighly different ways. But there are indeed commonalities here.
One suggestion is that one take the transmission scheduler, a simple interface towards the transmission scheduler is a instruction called releaseRtpPacket(), the function takes the next RTP packet from the queue (if there is any) and transmits it
How often and what decides when this function is called is up to the RMCAT (network congestion control) algorithm, this should work with both SCReAM and NADA, and GCC as well I guess ?

As regards to the actual video rate control. Unclear about the intention here. I guess one need some minimal rate control behavior here. The main difference between conversational video and e.g a file transfer is that the source bitrate is limited to a value somewhere around the actual transmit bitrate, and this source limitation will affect the behavior of the RMCAT network congestion control algorith. My experience is for instance that the network congetsion control is easier if the rate control is tuned more agressively, this however at the expense of possibly more queuing in the RTP packet queue.

/Ingemar

From: Michael Ramalho (mramalho) [mailto:[email-masked]]
Sent: den 13 november 2014 20:41
To: Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

All,

I think that Karen more-or-less got the intent of the consensus right. We desire a flow control of data from the codec/application to the RMCAT layer.

The “less” of the “more-or-less” part was that she used an explicit reference to cwnd – which assumes some form of clocking is used at the RMCAT congestion control layer. I don’t think that was the focus of the discussion (i.e., we didn’t talk about clocked or rate based rmcat control).

The “more” of the “more-or-less” part is that the consensus was to use a flow control mechanism – similar to the SEND WINDOW of TCP – to gate/admit application data (here app means the layer above RMCAT) into the RMCAT “media socket” (if you will).

That is, the RMCAT layer will ask for data – via a rate interface – from the application. However, based on transport conditions, the RMCAT layer may need to squelch/stop new data from being put on the wire faster than the application/codec can be expected to stop. When it needs to do this, we propose that it does something similar to setting the send window to zero bytes in TCP – thus stopping acceptance of new data to the RMCAT layer. Under nominal operation (no squelching desired) the send window effectively will allow all data (conformant to the rate) to be gated to the RMCAT layer at the media socket interface.

The application (here the codec) may keep piling up frames at “its side” of the “media socket” for later transmission. Or it could do something more intelligent. As an example, it can “forget/drop” those frames it intended to give the RMCAT layer and code a new frame to be presented to the socket when it clears (an I frame, a P frame using a previously sent reference, or anything else it desires!).

This allows the codec to do something “more intelligent” under these exceptional transport circumstances while still nominally producing frames at the requested rate when the flow control to the RMCAT doesn’t gate/stop the data flow. And since the proposed flow control is similar to TCP, it is hoped that coders might be more familiar with the mechanism. It also allows codecs to be dumb – at the cost of piling up frames that will incur an additional delay.

I hope this is clear.

Michael Ramalho

From: rmcat [mailto:[email-masked]] On Behalf Of Karen Elisabeth Egede Nielsen
Sent: Wednesday, November 12, 2014 5:38 PM
To: Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

Hi,

I confirm that there is no intention to reuse terminology from TCP. Perhaps the wording was bad.

BR, Karen

From: rmcat [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Varun Singh
Sent: Thursday, November 13, 2014 3:52 AM
To: Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

On 12 Nov 2014 16:48, "Randell Jesup" <[email-masked]<mailto:[email-masked]>> wrote:
>
> On 11/12/2014 6:58 PM, Karen Elisabeth Egede Nielsen wrote:
>>
>>
>>
>> The conclusion from our lunch meeting about clean-up of the CC candidate documents and the relation to the app-interaction draft was
>>
>> (to my understanding):
>>
>>
>>
>> The interaction in between the CC solutions and the application should be exposed via an  “flow control” api, analogy
>>
>> here would be the flow control function instantiated by a TCP SND buffer. Guidance towards the application/codec about the congestion control situation,  e.g., guidance than would make one codec drop a frame, another codex do something else it thinks is smarter,
>>
>> shall be modelled in the context of such a flow control api.
>>
>>
>>
>> We need to have this flow control api be exposed in the application-interaction draft.
>
>
> Looking forward to seeing it; please also take into account the discussion on the list recently about APIs.
>
> I still feel there's something of a impedance mismatch (if you will) between TCP-oriented terminology & APIs (CWND, TCP flow control) and what makes most sense for a delay-oriented transport.  However, I'll wait to see the updates, and in many cases it may just be an issue of potentially confusing a reader who makes assumptions based on existing TCP/etc knowledge.
>

I don't think the intention is to reuse the terminology in TCP.

> --
> Randell Jesup -- rjesup a t mozilla d o t com

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Thu, 4 Dec 2014 19:41:02 +0000

Hi Geert,

Let me repeat your question here and then comment on what you have said. I am only one opinion – so others are free to chime in!

[GV] If my understanding of the proposed interface is correct, then the application layer would be responsible for estimating the ‘backlog’ size and taking appropriate action. However, the ‘backlog’ has many contributing parts and one of them is the rate controller’s reaction delay, which is the only part known at the application layer.

[MR] Huh? Did you mean the rate controller output is only known at the transport/RMCAT layer (i.e., not the application layer)? Or did you imply the rate controller of the application (media encoder)? I answer without knowing which you intended.

[GV]Therefore, I think that the estimation of the ‘backlog’ (excluding rate control part) fits into the RMCAT layer, while the interface can pass the backlog estimate to the application layer, which can then take appropriate action. So, in this case the interface would consist of ‘requested rate’, ‘backlog size’, and the ‘media socket’.

As I stated, assuming no “squelching event” all the (conformant) data produced by the application layer will be “accepted” by the transport layer. No estimation of any “backlog” is needed by the application as there isn’t any from its perspective.

If there is a “squelching event” I also assume that a new lower rate will soon be requested at the API by the RMCAT layer … but I also assume that the application layer may still be producing data/frames at the PRIOR rate for some time (with time unknown to the transport layer). For this scenario there will be a “backlog of data” produced by the application that is not being “accepted” by the “media socket” – this data is “stuck” on the application side of the interface (just like normal TCP socket operations). As you state, the network (RMCAT) layer cannot know how responsive the application layer will be to any rate reduction request at the API. But the application layer sees the backlog (in terms of packets or frames) on its side of the socket and can do something about it (immediately lower its near-term rate, drop frames, encode differently, etc.).

If one ASSUMES the bottleneck is a WIRELESS (Cellular/LTE-like) link – and that the only observable measurement of the decrease in available capacity (before loss) is an increasing delay measurement – then I might agree with the “pass some form of backlog estimate to the application layer” (e.g., a queueing delay estimate, Randell’s bits-behind-the-8-ball, BB8B, estimate) as this may be useful information. But the very fact that not all data produced at the application layer is being accepted at the media socket is a strong hint to the application layer that some squelching was needed and it could generate a rough BB8B/backlog estimate based on that.

That is, the wireless link assumption above presumes that your flow is “self-loading” the bottleneck and that the available channel capacity has dropped (e.g., SPROUT: http://nms.csail.mit.edu/papers/nsdi13-sprout.pdf ).

Here is the problem: We are NOT JUST designing for a “self-loading/wireless” case.

For a WIRED BOTTLNECK the increase in delay (again the only observable congestion event prior to loss) MAY be caused by other aggressive non-delay adaptive sources! In this case the “backlog” (excess bits on the queue) is NOT ONLY DUE TO YOUR FLOW but the contribution of other sources sharing the queue. When viewed in this context one has to ask: “How much of the backlog is due to my flow?”. The RMCAT layer effectively makes that decision by updating/modulating the rate request (based on delay/loss observations) to the application layer – NOT the other way around! In my view, that is our RMCAT job – to calculate the “best” rate estimate possible (given the measurements) and to take “drastic action” (e.g., squelching) only when the RMCAT layer sees potential harm to the network if it didn’t.

For this reason I do not think we need to send “an indication of the backlog” (BB8B estimate assuming our flow are self-loading the channel) to the application layer under the belief that it knows better how to deal with the backlog estimate – how would it know any better than the transport layer about the CAUSE of the increased delay (answer: It can’t without other side information)?

Now … with all that being said … I would not object to a “bottleneck queue delay estimate” of some sort being provided to the application layer at the API as it MIGHT have other side information that the transport layer doesn’t have (e.g., session or other topology information). Then it could then CHOOSE to send less data to the media socket than the socket is prepared to accept (i.e., the RMCAT layer was prepared to gate more data onto the wire) if the application believed that it knew better than the transport layer how it should “feed the channel”.

Summary: I would not object to the availability of a “bottleneck queue delay estimate” to the application layer … I just don’t want ANY A PRIORI ASSUMPTION that the application is required to do something with it for the transport layer to operate correctly.

Does that make sense?

Michael

PS1 – If I understand Geert correctly, I think he would want a raw “instantaneous” (not very low-pass filtered) queue delay estimate at the API to allow his application to react quickly.
PS2 – The only indication that TCP provides to an application wanting to send more data is it’s send window (i.e., the media socket mechanism proposed). All rate based applications built using TCP (e.g., ABR services) estimate such “backlogs” based on send window (and corresponding buffer) dynamics.

From: Van Der Auwera, Geert [mailto:[email-masked]]
Sent: Wednesday, December 03, 2014 6:02 PM
To: Michael Ramalho (mramalho); Ingemar Johansson S; Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: RE: [rmcat] clean up of CC drafts and app-interaction

Hi Michael,

Geert

Ingemar,

Michael Ramalho

Hi

/Ingemar

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen

---

From: "Van Der Auwera, Geert" <[email-masked]>
Date: Fri, 5 Dec 2014 20:06:03 +0000

Please see below [GV2] for responses.

Best regards,

Geert

From: rmcat [mailto:[email-masked]] On Behalf Of Michael Ramalho (mramalho)
Sent: Thursday, December 04, 2014 11:41 AM
To: Van Der Auwera, Geert; Ingemar Johansson S; Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

Hi Geert,

[GV2] I meant the video encoder’s rate controller reaction delay, which is implementation dependent.

If there is a “squelching event” I also assume that a new lower rate will soon be requested at the API by the RMCAT layer … but I also assume that the application layer may still be producing data/frames at the PRIOR rate for some time (with time unknown to the transport layer). For this scenario there will be a “backlog of data” produced by the application that is not being “accepted” by the “media socket” – this data is “stuck” on the application side of the interface (just like normal TCP socket operations).

As you state, the network (RMCAT) layer cannot know how responsive the application layer will be to any rate reduction request at the API. But the application layer sees the backlog (in terms of packets or frames) on its side of the socket and can do something about it (immediately lower its near-term rate, drop frames, encode differently, etc.).

[GV2] I would like to clarify this a bit more. There is a delay before the decrease in channel capacity (channel rate drop) is detected and the sending window is adapted by RMCAT. Therefore, part of the ‘backlog’ will have made it through the interface’s ‘media socket’ before its sending window is reduced by RMCAT, so not all backlog data is stuck on the application side, some of it is in the network (I refer to it with ‘backlog in the network’ in a comment below).

[GV2] If my understanding is correct, when congestion (or onset of congestion) is detected by RMCAT both a reduced target rate and reduced sending window size are passed through the interface to the application layer, and the reduced sending window is applied to the ‘media socket’. The application then decides, for example, to skip frames for a while (rate is reduced to 0) or quickly reduce its rate to some value below requested rate (rate undershoot), to deal with the backlog on the application side (only). IF the RMCAT layer deals with the backlog in the network by configuring the interface’s parameters, then I agree that an indication of the size of the backlog in the network through the interface is not very useful to the application. However, this establishes that the RMCAT layer is responsible to estimate and ALSO deal with the backlog in the network through the interface mechanism.

Does that make sense?

[GV2] I think this enables some interesting options.

Michael

Hi Michael,

Geert

Ingemar,

Michael Ramalho

Hi

/Ingemar

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 12 Dec 2014 18:55:22 +0000

Hi

Trying to catch up on this thread. Mapping the below discussion to SCReAM (I need to do it in order to not get lost).
SCReAM in its current implementation does actually convey some information about the  network queue build-up (owd_trend) to the application layer, this information is used to reduce the target bitrate when it senses that the network queuing delay increases.
This helps to keep delay variations low. The feature is particularly  good for the self-loading cases but it can cause the algorithm to yield too much to other competing traffic, so it needs to be used with care.
With that said, the video target rate computation can actually work quite well without any extra network queuing delay information, this needs to be compensated for with an increased sensitivity to the RTP queue size (or rate-shaper queue size as some prefer to call it). Again.. making the sensitivity to the RTP queue size may cause it to yield too much to competing traffic. It is necessary to be greedy to survive, but not too greedy ☺

/Ingemar

From: Van Der Auwera, Geert [mailto:[email-masked]]
Sent: den 5 december 2014 21:06
To: Michael Ramalho (mramalho); Ingemar Johansson S; Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: RE: [rmcat] clean up of CC drafts and app-interaction

Best regards,

Geert

Hi Geert,

Does that make sense?

Michael

Hi Michael,

Geert

Ingemar,

Michael Ramalho

Hi

/Ingemar

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen