Subject: [rmcat] clean up of CC drafts and app-interaction

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 13 Nov 2014 00:58:56 +0100

The conclusion from our lunch meeting about clean-up of the CC candidate
documents and the relation to the app-interaction draft was

(to my understanding):

The interaction in between the CC solutions and the application should be
exposed via an  “flow control” api, analogy

here would be the flow control function instantiated by a TCP SND buffer.
Guidance towards the application/codec about the congestion control
situation,  e.g., guidance than would make one codec drop a frame, another
codex do something else it thinks is smarter,

shall be modelled in the context of such a flow control api.

We need to have this flow control api be exposed in the
application-interaction draft.

The  individual CC solutions, like Nada and Scream (and GCC) implement
various components such as (always)

a CC algorithm that calculates the CWND at any given time as well as other
logical components – like rate shaping – that takes actions

based on the calculated CWND.

The CC algorithm calculating the CWND need to be very clearly described as
a separate logical entity in the CC solution document.

Components that are an inherent part of the CC solution, i.e., if those
components (e.g. , rate shaping, FEC perhaps ..)

were taken away the CC solution would not be able

to meet the requirements, i.e., the solution should be evaluated with the
mentioned components and actions in place,

need be described as an internal part of the CC solution.

When applicable  different CC solutions should aim to use the same
terminology for the same CC internal component, like rate shaper.

Interactions with the applications or other in terms of what is outside of
the above,  need be split away and go outside of

the CC solution documents, though could perhaps go into the appendix (or
usecase example section) of the docs..

BR, Karen

---

From: Randell Jesup <[email-masked]>
Date: Wed, 12 Nov 2014 21:47:45 -0500

On 11/12/2014 6:58 PM, Karen Elisabeth Egede Nielsen wrote:
>
> The conclusion from our lunch meeting about clean-up of the CC 
> candidate documents and the relation to the app-interaction draft was
>
> (to my understanding):
>
> The interaction in between the CC solutions and the application should 
> be exposed via an  “flow control” api, analogy
>
> here would be the flow control function instantiated by a TCP SND 
> buffer. Guidance towards the application/codec about the congestion 
> control situation,  e.g., guidance than would make one codec drop a 
> frame, another codex do something else it thinks is smarter,
>

Looking forward to seeing it; please also take into account the 
discussion on the list recently about APIs.

I still feel there's something of a impedance mismatch (if you will) 
between TCP-oriented terminology & APIs (CWND, TCP flow control) and 
what makes most sense for a delay-oriented transport.  However, I'll 
wait to see the updates, and in many cases it may just be an issue of 
potentially confusing a reader who makes assumptions based on existing 
TCP/etc knowledge.

-- 
Randell Jesup -- rjesup a t mozilla d o t com

---

From: Varun Singh <[email-masked]>
Date: Wed, 12 Nov 2014 16:52:29 -1000

>> here would be the flow control function instantiated by a TCP SND
buffer. Guidance towards the application/codec about the congestion control
situation,  e.g., guidance than would make one codec drop a frame, another
codex do something else it thinks is smarter,
> I still feel there's something of a impedance mismatch (if you will)
between TCP-oriented terminology & APIs (CWND, TCP flow control) and what
makes most sense for a delay-oriented transport.  However, I'll wait to see
the updates, and in many cases it may just be an issue of potentially
confusing a reader who makes assumptions based on existing TCP/etc
knowledge.

I don't think the intention is to reuse the terminology in TCP.

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 13 Nov 2014 04:37:33 +0100

Hi,

I confirm that there is no intention to reuse terminology from TCP. Perhaps
the wording was bad.

BR, Karen

*From:* rmcat [mailto:[email-masked]] *On Behalf Of *Varun Singh
*Sent:* Thursday, November 13, 2014 3:52 AM
*To:* Randell Jesup
*Cc:* rmcat WG
*Subject:* Re: [rmcat] clean up of CC drafts and app-interaction

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Thu, 13 Nov 2014 19:41:03 +0000

All,

I think that Karen more-or-less got the intent of the consensus right. We desire a flow control of data from the codec/application to the RMCAT layer.

The “less” of the “more-or-less” part was that she used an explicit reference to cwnd – which assumes some form of clocking is used at the RMCAT congestion control layer. I don’t think that was the focus of the discussion (i.e., we didn’t talk about clocked or rate based rmcat control).

The “more” of the “more-or-less” part is that the consensus was to use a flow control mechanism – similar to the SEND WINDOW of TCP – to gate/admit application data (here app means the layer above RMCAT) into the RMCAT “media socket” (if you will).

That is, the RMCAT layer will ask for data – via a rate interface – from the application. However, based on transport conditions, the RMCAT layer may need to squelch/stop new data from being put on the wire faster than the application/codec can be expected to stop. When it needs to do this, we propose that it does something similar to setting the send window to zero bytes in TCP – thus stopping acceptance of new data to the RMCAT layer. Under nominal operation (no squelching desired) the send window effectively will allow all data (conformant to the rate) to be gated to the RMCAT layer at the media socket interface.

The application (here the codec) may keep piling up frames at “its side” of the “media socket” for later transmission. Or it could do something more intelligent. As an example, it can “forget/drop” those frames it intended to give the RMCAT layer and code a new frame to be presented to the socket when it clears (an I frame, a P frame using a previously sent reference, or anything else it desires!).

This allows the codec to do something “more intelligent” under these exceptional transport circumstances while still nominally producing frames at the requested rate when the flow control to the RMCAT doesn’t gate/stop the data flow. And since the proposed flow control is similar to TCP, it is hoped that coders might be more familiar with the mechanism. It also allows codecs to be dumb – at the cost of piling up frames that will incur an additional delay.

I hope this is clear.

Michael Ramalho

From: rmcat [mailto:[email-masked]] On Behalf Of Karen Elisabeth Egede Nielsen
Sent: Wednesday, November 12, 2014 5:38 PM
To: Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

Hi,

I confirm that there is no intention to reuse terminology from TCP. Perhaps the wording was bad.

BR, Karen

From: rmcat [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Varun Singh
Sent: Thursday, November 13, 2014 3:52 AM
To: Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

On 12 Nov 2014 16:48, "Randell Jesup" <[email-masked]<mailto:[email-masked]>> wrote:
>> The conclusion from our lunch meeting about clean-up of the CC candidate documents and the relation to the app-interaction draft was
>>
>> (to my understanding):
>>
>>
>>
>> The interaction in between the CC solutions and the application should be exposed via an  “flow control” api, analogy
>>
>> here would be the flow control function instantiated by a TCP SND buffer. Guidance towards the application/codec about the congestion control situation,  e.g., guidance than would make one codec drop a frame, another codex do something else it thinks is smarter,
>>
>> We need to have this flow control api be exposed in the application-interaction draft.
>
>
> Looking forward to seeing it; please also take into account the discussion on the list recently about APIs.
>
> I still feel there's something of a impedance mismatch (if you will) between TCP-oriented terminology & APIs (CWND, TCP flow control) and what makes most sense for a delay-oriented transport.  However, I'll wait to see the updates, and in many cases it may just be an issue of potentially confusing a reader who makes assumptions based on existing TCP/etc knowledge.
>

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 19 Nov 2014 09:50:16 +0000

Hi

The description sounds comprehensible.
While SCReAM and NADA has a slightly different terminology there are some similarities. While SCReAM talks about an RTP packet queue and a transmission scheduler, NADA use a rate shaper. They both try to achieve the same goal albeit in slighly different ways. But there are indeed commonalities here.
One suggestion is that one take the transmission scheduler, a simple interface towards the transmission scheduler is a instruction called releaseRtpPacket(), the function takes the next RTP packet from the queue (if there is any) and transmits it
How often and what decides when this function is called is up to the RMCAT (network congestion control) algorithm, this should work with both SCReAM and NADA, and GCC as well I guess ?

As regards to the actual video rate control. Unclear about the intention here. I guess one need some minimal rate control behavior here. The main difference between conversational video and e.g a file transfer is that the source bitrate is limited to a value somewhere around the actual transmit bitrate, and this source limitation will affect the behavior of the RMCAT network congestion control algorith. My experience is for instance that the network congetsion control is easier if the rate control is tuned more agressively, this however at the expense of possibly more queuing in the RTP packet queue.

/Ingemar

From: Michael Ramalho (mramalho) [mailto:[email-masked]]
Sent: den 13 november 2014 20:41
To: Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: Re: [rmcat] clean up of CC drafts and app-interaction

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Wed, 19 Nov 2014 16:05:54 +0000

Randell: This is how we fixed the “impedance mismatch” from your previous email (thread is below).

Ingemar,

You have the basic point correct – I look forward to working with you and others on the details.

The TCP send window analogy may be better than your “releaseRtpPacket()” proposal. That is, the TCP send window is usually specified in bytes and if the send window is X bytes, it is basically saying that you MAY send UP TO X bytes now.

If the RTP payload you want to send is X+1 bytes long, you need to wait until the send window “opens” from X bytes to something larger in order for you to send your RTP packet payload.

In this way the RMCAT layer need not know the size (or nominal size) of the RTP payloads that your encoder is producing. In normal operation the RMCAT layer will “allow” more data to get to in the upcoming moments and it will signal this by opening up the send window to a value greater than X (but <= RTP payload MTU).

I mentioned during our meeting that under normal operation (i.e., no squelching) this interface is essentially a SHAPER in that if the coder produces data at the requested rate (or below) and that ALL of it should be allowed into the RMCAT layer – with the small exception that a shaper brings (try to send me too many bits at once and I will shape you to the rate of the shaper via the send window mechanism).

When exceptional transport conditions require “squelching”, this interface is similar to a POLICER (which normally drops a portion of the non-conformant traffic). The exception is that the application (the layer above the RMCAT layer) decides if IT WANTS to drop frames/packets based on the backlog(my word)/backpressure(Varun’s word)/impedance-mismatch(Randell’s word) prior to the RMCAT layer.

So, I expect that during those “squelching events” that the requested rate to the encoder should also drop (a la Geert’s presentation) … and if the encoder is quick to act on this new rate reduction request … that not many of its frames/packets will “back up” at that “media socket” interface. This should satisfy your “tuned more aggressively” issue and Geert’s points.

I envision that partitioning the interface in this way (a requested rate interface and a “media socket” with a send window) will allow the encoder to be smart/intelligent/fast (e.g. SCReAM “frame skipping”) OR dumb/slow-to-change-rate (at the cost of badly delayed or badly encoded video because it didn’t self-police by dropping frames).

And, more importantly, the interface allows for shaping (the X or X+1 example above) to the RMCAT layer … and OPTIONAL self-policing by the layer above it (e.g., frame skipping/dropping for lowest-possible encoder-to-RMCAT layer delay). This allows for joint optimization of both the RMCAT and video encoding layers (I think this is Ericsson’s main point).

In my opinion, both of these properties - shaping and need-based policing - should be basic hygiene issues for the RMCAT layer (to satisfy Lars’/Mirja’s/Karen’s stated “safe-for-the-Internet” requirement and low-latency objectives).

I hope this makes the proposed mechanism – and the rationale behind it – a little more clear.

Michael Ramalho

From: Ingemar Johansson S [mailto:[email-masked]]
Sent: Wednesday, November 19, 2014 4:50 AM
To: Michael Ramalho (mramalho); Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG; Ingemar Johansson S
Subject: RE: [rmcat] clean up of CC drafts and app-interaction

Hi

/Ingemar

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 26 Nov 2014 07:00:25 +0000

Hi

Sorry for the slow response, my comments marked with [IJ] inline

/Ingemar

From: Michael Ramalho (mramalho) [mailto:[email-masked]]
Sent: den 19 november 2014 17:06
To: Ingemar Johansson S; Karen E. E. Nielsen; Varun Singh; Randell Jesup
Cc: rmcat WG
Subject: RE: [rmcat] clean up of CC drafts and app-interaction

Ingemar,

You have the basic point correct – I look forward to working with you and others on the details.
[IJ] Yes, should be possible to find some common ground here, I guess however that it may take one or two iterations until the drafts speak a similar language. For instance SCReAM use the term “RTP queue” while NADA use “rate shaper”. It can here be good (but not required) if we can use a common term, after all they serve the same purpose.

In this way the RMCAT layer need not know the size (or nominal size) of the RTP payloads that your encoder is producing. In normal operation the RMCAT layer will “allow” more data to get to in the upcoming moments and it will signal this by opening up the send window to a value greater than X (but <= RTP payload MTU).
[IJ] That sounds acceptable to me. Packet pacing complicates things and some releaseRtpPacket() function call would have made things easier but in the draft it should be sufficient describe the priciples of packet pacing so I am not worried on that point.

[IJ] Sounds OK

And, more importantly, the interface allows for shaping (the X or X+1 example above) to the RMCAT layer … and OPTIONAL self-policing by the layer above it (e.g., frame skipping/dropping for lowest-possible encoder-to-RMCAT layer delay). This allows for joint optimization of both the RMCAT and video encoding layers (I think this is Ericsson’s main point).
[IJ] Sounds OK, one remaining question though. As I see it it becomes necessary to provide with some level of detail on the video rate control. This because there is a relationship between how well the video encoder manages to provide with RTP packets and how well the network congestion control can utilize the link, a balance need to be struck here however to avoid excessive queue build-up in the RTP queue or rate shaper. The current plan is that the SCReAM draft will have a section that describes the video rate control, frame skipping will only be optional.

Michael Ramalho

Hi

/Ingemar

All,

I hope this is clear.

Michael Ramalho

Hi,

BR, Karen