Subject: [rmcat] application interface

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 10 Oct 2013 15:23:42 +0200

Hi group,

I have a question as an individual contributor. Looking at the current 
proposals which are all rate-based (right?), I was wondering if we maybe need 
a higher layer interface to announce the current rate to the application?

Moreover, if the sending rate is application limited, don't we need a way to 
tell the application that there is more space on the link and e.g. the 
quality could be increased? Therefore we would need to know how much more 
capacity would be available. Normally congestion control does this by 
probing/slowly increasing the sending rate, but if the traffic is application 
limited that's not possible. Thus we maybe need some kind of 
measurement/testing method, that maintains an average rate as provided by the 
application, but can send for very short intervals with a higher and 
respective lower rate, like chirping (see PathChirp or TCP Rapid).

Any opinions on this?

Mirja

---

From: Piers O'Hanlon <[email-masked]>
Date: Thu, 10 Oct 2013 15:28:54 +0100

Hi Mirja,

You bring up two points:

- I'd agree we do need to have some sort of communication between the congestion control and media/codec delivery - probably some API is needed to communicate current parameters such as operating rate, latency, and possibly some time limited projections (or confidence) of such parameters.

- The issue in your second point as I see is the issue of how deal with application limited and/or silence periods - which are a tricky area. I guess the corresponding concept for TCP is Congestion Window Validation, and the changes in TFRC from RFC3448 to  RFC5348 were largely down to such data-limited behaviours. For general info Gorry Fairhurst has a couple of recent papers on the subject:
"TCP-Friendly Rate Control (TFRC) for bursty media flows", ComCom, 2011
And for TCP:
"Enhancing TCP to Support Rate-Limited Traffic", CSWS, 2012

But you are suggesting a further addition to this - maybe when CWV limits are exceed - by adding a probe phase. Although I think it's possible to monitor the 'available' bandwidth on the link - the problem maybe that other other flows will rapidly take up their 'fair share' of the that available capacity. So probably the only way to maintain one's capacity is send some sort of dummy traffic - or if we're WebRTC mode - send some HTML/JS/CSS etc in between the lulls of media.

I guess this mix could be handled via some coupled congestion service or suitable multi-substream low latency transport with optional reliability per substream...

Cheers,

Piers

---

From: Kevin Gross <[email-masked]>
Date: Thu, 10 Oct 2013 09:05:19 -0600

I thought the scope of the work for this group was to develop techniques
applications can use to avoid congestion. Since this is all being done
internal to the application, there's no need for an API to an application.
We may need to have interfaces to the network for ECN or whatnot.

Kevin Gross
+1-303-447-0517
Media Network Consultant
AVA Networks - www.AVAnw.com <http://www.avanw.com/>

On Thu, Oct 10, 2013 at 8:28 AM, Piers O'Hanlon <[email-masked]> wrote:

> Hi Mirja,
>
> You bring up two points:
>
> - I'd agree we do need to have some sort of communication between the
> congestion control and media/codec delivery - probably some API is needed
> to communicate current parameters such as operating rate, latency, and
> possibly some time limited projections (or confidence) of such parameters.
>
> - The issue in your second point as I see is the issue of how deal with
> application limited and/or silence periods - which are a tricky area. I
> guess the corresponding concept for TCP is Congestion Window Validation,
> and the changes in TFRC from RFC3448 to  RFC5348 were largely down to such
> data-limited behaviours. For general info Gorry Fairhurst has a couple of
> recent papers on the subject:
> But you are suggesting a further addition to this - maybe when CWV limits
> are exceed - by adding a probe phase. Although I think it's possible to
> monitor the 'available' bandwidth on the link - the problem maybe that
> other other flows will rapidly take up their 'fair share' of the that
> available capacity. So probably the only way to maintain one's capacity is
> send some sort of dummy traffic - or if we're WebRTC mode - send some
> HTML/JS/CSS etc in between the lulls of media.
>
> I guess this mix could be handled via some coupled congestion service or
> suitable multi-substream low latency transport with optional reliability
> per substream...
>
> Cheers,
>
> Piers
>
>

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Thu, 10 Oct 2013 19:42:34 +0000

Mirja,

Comments below with "MAR:".

Michael

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mirja Kühlewind
Sent: Thursday, October 10, 2013 8:24 AM
To: [email-masked]
Subject: [rmcat] application interface

Hi group,

I have a question as an individual contributor. Looking at the current proposals which are all rate-based (right?), I was wondering if we maybe need a higher layer interface to announce the current rate to the application?

MAR: Although I agree that most of the discussion on the list has assumed "rate" as an application controlling variable, I have posted early on that I believe that we should not rule out other mechanisms at this point. 

MAR: I would like to keep in scope any mechanism (i.e., passing of any necessary congestion control parameter) that the RMCAT participants may find as necessary to attain our goals of bandwidth sharing with a constraint of minimizing delay (when possible).

MAR: So, from my perspective (as an independent contributor, I am not speaking for any group/team) I believe that some interface (API or whatever) will be required to tell the application about the known/observed congestion state and behaviors of the application (e.g., you can speed up at a given rate until informed otherwise ... or please stop sending video frames for the next X milliseconds ... or ?).

Moreover, if the sending rate is application limited, don't we need a way to tell the application that there is more space on the link and e.g. the quality could be increased? Therefore we would need to know how much more capacity would be available.

MAR: One could opportunistically send probes (when traffic is known or believed to be light) to estimate what the maximum rate could be. As you say, knowledge that a given applications traffic is causing congestion is only known AFTER such congestion can be measured. Thus you don't know how high your ceiling is until you bump your head against it.

Normally congestion control does this by probing/slowly increasing the sending rate, but if the traffic is application limited that's not possible. Thus we maybe need some kind of measurement/testing method, that maintains an average rate as provided by the application, but can send for very short intervals with a higher and respective lower rate, like chirping (see PathChirp or TCP Rapid).

MAR: Agree on that point. I would classify it as "opportunistic probing" as I mentioned above. If the application would like to know if more bandwidth is available, then such probing can occur.

Any opinions on this?

Mirja

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 11 Oct 2013 13:52:34 +0200

Hi Piers,

inline...

> possibly some time limited projections (or confidence) of such parameters.
I would be careful to make the interface too large and too complicated. The 
question is which information is really needed for the application to decide 
to switch the coding and also (maybe even more important) which information 
can be understood by every application designer. I currently believe that the 
information, what the estimated largest possible rate is, might be enough.

> recent papers on the subject: "TCP-Friendly Rate Control (TFRC) for bursty
> media flows", ComCom, 2011 And for TCP:
These mechanisms do not address the same problem. They mainly handle the case 
when you've already been sending at a larger rate but now you have a smaller 
rate or stopped sending for a little while. In my case you would like to 
indicate also that the application could switch to a higher rate. That means 
the link is currently not full and you would like to know how much space is 
available. Of course other might also try to use some of the space but that 
means simply you probably want to increase your rate step-wise (on the higher 
layer) and not try to grab all at once.
The other problem is, when the coding is changed, the rate will increase 
step-wise while the proposed congestion control schemes ramp up slowly. When 
you already did some probing before, it might be (more or less) safe to also 
increase by a step (if not too large) at congestion control level.

> HTML/JS/CSS etc in between the lulls of media.
Yes and no. I partly addressed this already above basically saying don't grab 
the whole available capacity at once but increase the application rate 
step-wise. But in fact you bring up one more problem. Let's assume the link 
is already full. How do I know that it still would be possible to increase my 
sending rate because other (potentially less important traffic) might adapt 
and free some capacity for me (without increasing the total congestion 
level)?

> suitable multi-substream low latency transport with optional reliability
You mean, sending dummy data? Not sure if that's the right solution. For sure 
it's a solution which probably works but I would favor any other solution if 
we could find one.

Mirja

> > proposals which are all rate-based (right?), I was wondering if we maybe
> > need a higher layer interface to announce the current rate to the
> > application?
> >
> > Moreover, if the sending rate is application limited, don't we need a way
> > to tell the application that there is more space on the link and e.g. the
> > probing/slowly increasing the sending rate, but if the traffic is
> > application limited that's not possible. Thus we maybe need some kind of
> > measurement/testing method, that maintains an average rate as provided by
> > the application, but can send for very short intervals with a higher and

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 11 Oct 2013 13:55:32 +0200

Hi Kevin,

sure the congestion control could also be implemented in the application. I 
was assuming that the congestion control is part if the transport layer. But 
even if you implement it in the application, you potentially would like to 
define some kind of interface such that the same congestion control could 
more easily be implemented (or even code could be re-used) by different 
applications and the application could then implement an addition control 
loop on top to change the coding. This additional higher layer control loop 
is not part of the rmcat charter thus having a well defined interface would 
be useful form my point of view.

Mirja

On Thursday 10 October 2013 17:05:19 Kevin Gross wrote:
> > and the changes in TFRC from RFC3448 to  RFC5348 were largely down to
> > such data-limited behaviours. For general info Gorry Fairhurst has a
> > couple of recent papers on the subject:
> > available capacity. So probably the only way to maintain one's capacity
> > is send some sort of dummy traffic - or if we're WebRTC mode - send some
> > need
> > > Moreover, if the sending rate is application limited, don't we need a
> >
> > way to
> >
> > > quality could be increased? Therefore we would need to know how much
> > > more capacity would be available. Normally congestion control does this
> > > by probing/slowly increasing the sending rate, but if the traffic is
> >
> > application
> >
> > > measurement/testing method, that maintains an average rate as provided
> >
> > by the
> >

tel: +49(0)711/685-67973

---

From: Manuel Gorius <[email-masked]>
Date: Fri, 11 Oct 2013 14:10:19 +0200

Hi Mirja, Hi Piers,

We are dealing with similar problems in the context of real-time dynamic media streaming. We decided (as we just designed a kind of real-time and media-friendly transport protocol for research purposes) to communicate the transport protocol's current goodput estimate (after deducting error control and header overhead from the throughput) to the application as a part of some socket state information that might be polled by the application. Measurements of the end-to-end communication delay and the packet loss rate are available as well.

In the context of dynamic media streaming we also observe the problem of biased congestion feedback as a result of the multimedia flow's rigid rate requirement. As you mention, it makes rate-adaptive flows weak in the competition for available bandwidth. I'm just wondering whether it wouldn't be more beneficial to fill the gap between the congestion control's rate estimate and the application's actual rate requirement with error control data instead of dummy traffic. This information could even protect the real-time flows against congestion losses during the (probably) upcoming congestion event caused by the probing activities (e. g. chirping).

Best,
Manuel Gorius

Telecommunications Lab
Saarland University

Campus C6 3, 9.06
66123 Saarbruecken
Germany

T   +49 681 302 6544
F   +49 681 302 6542
[email-masked]
www.nt.uni-saarland.de/people/gorius

---

From: Michael Welzl <[email-masked]>
Date: Fri, 11 Oct 2013 15:12:57 +0200

Hi,

I don't understand why you call it a control loop, it's just an API, and I don't understand why you say it's not in the charter? The charter says:

"Identify interactions between applications and RTP flows to enable conveying
helpful cross-layer information such as per-packet priorities, flow elasticity, etc.
This information might be used to populate an API, but the WG will not define a
specific API itself."

The intention behind this really was an API, conceptually - as far as I can tell the (later added) "anti-API" wording is just there to ensure that there won't be an API defined in the sense of defining datatypes and such machine-specific things, but something more abstract. But it's about defining how applications communicate with the transport layer.

FWIW, the result on slide 16:
http://www.ietf.org/proceedings/87/slides/slides-87-rmcat-2.pdf
is only possible because we assume that coupled congestion control gets not only the rate that the congestion controller has calculated, but also the "desired rate" from the application (which may be smaller than the congestion controller's rate).

Now this is the other direction than what you mean - the transport getting the rate from the application - but it does require at least some rudimentary communication between applications and the transport. About the rest of your musings, I agree, and I also agree with the concerns that were raised about defining too much, we'd have to be careful here. In your answer to Piers, you wrote:

"I would be careful to make the interface too large and too complicated. The 
question is which information is really needed for the application to decide 
to switch the coding and also (maybe even more important) which information 
can be understood by every application designer. I currently believe that the 
information, what the estimated largest possible rate is, might be enough."

and I agree, probably you won't need more than this, *upward*, and what we called the application's "desired rate", *downward*.

As another comment to address Kevin's concern: this would be only about things that are of definite concern to the application and can't be dealt with below without involving the application. ECN is in fact a nice example of a case that doesn't need the application, it could be dealt with below - even for TCP, as Mirja et al's related draft shows:
http://tools.ietf.org/html/draft-kuehlewind-tcpm-ecn-fallback

Cheers,
Michael

---

From: Varun Singh <[email-masked]>
Date: Fri, 11 Oct 2013 16:36:36 +0300

Hi Piers, Mirja,

Snipping away and remarking only on the dummy/redundant data inline.

> it's a solution which probably works but I would favor any other solution if

We experimented with sending redundant data for congestion control, it was not
exactly done to fill up the gap between the expected rate by the rate-controller
and the rate generated by the codec but to actually perform congestion control.

Basic idea is to use higher redundancy when aggressively probing and less
redundancy when conservative. This adaptive FEC mechanism could be used
with other congestion control algorithms. We recently published this on
arXiv (http://arxiv.org/pdf/1310.1582v1.pdf). Take a look and comment
on if it is closer to what you had in mind.

-- 
http://www.netlab.tkk.fi/~varun/

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 11 Oct 2013 13:41:07 +0000

> on if it is closer to what you had in mind.

Full disclosure: Nokia may have IPR on the mechanism described in the paper.

Lars

---

From: Wesley Eddy <[email-masked]>
Date: Fri, 11 Oct 2013 10:10:02 -0400

On 10/11/2013 9:12 AM, Michael Welzl wrote:
> I don't understand why you call it a control loop, it's just an API, and
> I don't understand why you say it's not in the charter? The charter says:
> 
> The intention behind this really was an API, conceptually - as far as I
> can tell the (later added) "anti-API" wording is just there to ensure
> that there won't be an API defined in the sense of defining datatypes
> and such machine-specific things, but something more abstract. But it's
> about defining how applications communicate with the transport layer.

Yes, the need is to define what information needs to be available
"above" the CC algorithm.  "API" suggests definition of how that
information is accessed (polled, evented up, etc), which we don't
need to get into.

-- 
Wes Eddy
MTI Systems

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 11 Oct 2013 16:19:02 +0200

Hi Michael,

To clarify this: the API is in charter! I was talking about the control loop 
on the application layer that might decide to change codec based on 
information provided by this API. 

In fact, my original question was explicitly about how this API should look 
like. As you mention correctly, we also have this milestone on "Interactions 
between applications and RTP flows as an Informational RFC". As a chair, I 
believe (and I hope that this discussion will help) we should focus next on 
actually writing a first draft to specify the API. Because my feeling that 
this is also needed for the people designing/proposing an congestion control 
scheme to orientate. 

Moreover, what I also took from the on-going discussion is, that it might be 
important to get an application model (this is also related to the evaluation 
design team though). I was always having in mind, that die audio/video 
applications can switch rates only in discrete steps. But I guess but adding 
more or less FEC, the rate spectrum can be more continuously as well...?
I think we as a working group need to have a clear view, what a RT application 
can do and where the limits are, before designing and evaluation the 
congestion control. So eventhough we will not design and standardize this  
application layer control loop, I was talking about, we should at least be 
aware what applications are doing or what should/would be possible in future.

Mirja

On Friday 11 October 2013 15:12:57 Michael Welzl wrote:
> Hi,
>
> I don't understand why you call it a control loop, it's just an API, and I
> don't understand why you say it's not in the charter? The charter says:
>
> "Identify interactions between applications and RTP flows to enable
> conveying helpful cross-layer information such as per-packet priorities,
> flow elasticity, etc. This information might be used to populate an API,
> but the WG will not define a specific API itself."
>
> The intention behind this really was an API, conceptually - as far as I can
> tell the (later added) "anti-API" wording is just there to ensure that
> there won't be an API defined in the sense of defining datatypes and such
> machine-specific things, but something more abstract. But it's about
> defining how applications communicate with the transport layer.
>
> is only possible because we assume that coupled congestion control gets not
> only the rate that the congestion controller has calculated, but also the
> "desired rate" from the application (which may be smaller than the
> congestion controller's rate).
>
> Now this is the other direction than what you mean - the transport getting
> the rate from the application - but it does require at least some
> rudimentary communication between applications and the transport. About the
> rest of your musings, I agree, and I also agree with the concerns that were
> raised about defining too much, we'd have to be careful here. In your
> answer to Piers, you wrote:
>
> "I would be careful to make the interface too large and too complicated.
> The question is which information is really needed for the application to
> decide to switch the coding and also (maybe even more important) which
> information can be understood by every application designer. I currently
> believe that the information, what the estimated largest possible rate is,
> might be enough."
>
> and I agree, probably you won't need more than this, *upward*, and what we
> called the application's "desired rate", *downward*.
>
> As another comment to address Kevin's concern: this would be only about
> things that are of definite concern to the application and can't be dealt
> with below without involving the application. ECN is in fact a nice example
> of a case that doesn't need the application, it could be dealt with below -
> even for TCP, as Mirja et al's related draft shows:
> > sure the congestion control could also be implemented in the application.
> > I was assuming that the congestion control is part if the transport
> > layer. But even if you implement it in the application, you potentially
> > would like to define some kind of interface such that the same congestion
> > control could more easily be implemented (or even code could be re-used)
> > by different applications and the application could then implement an
> > addition control loop on top to change the coding. This additional higher
> > layer control loop is not part of the rmcat charter thus having a well
> > defined interface would be useful form my point of view.
> >
> > Mirja
> >
> >> internal to the application, there's no need for an API to an
> >> application. We may need to have interfaces to the network for ECN or
> >> whatnot.
> >>
> >> Kevin Gross
> >> +1-303-447-0517
> >> Media Network Consultant
> >>> congestion control and media/codec delivery - probably some API is
> >>> needed to communicate current parameters such as operating rate,
> >>> latency, and possibly some time limited projections (or confidence) of
> >>> such parameters.
> >>>
> >>> guess the corresponding concept for TCP is Congestion Window
> >>> Validation, and the changes in TFRC from RFC3448 to  RFC5348 were
> >>> largely down to such data-limited behaviours. For general info Gorry
> >>> Fairhurst has a couple of recent papers on the subject:
> >>> But you are suggesting a further addition to this - maybe when CWV
> >>> limits are exceed - by adding a probe phase. Although I think it's
> >>> possible to monitor the 'available' bandwidth on the link - the problem
> >>> maybe that other other flows will rapidly take up their 'fair share' of
> >>> the that available capacity. So probably the only way to maintain one's
> >>> capacity is send some sort of dummy traffic - or if we're WebRTC mode -
> >>> send some HTML/JS/CSS etc in between the lulls of media.
> >>>
> >>> I guess this mix could be handled via some coupled congestion service
> >>> or suitable multi-substream low latency transport with optional
> >>> reliability per substream...
> >>>
> >>> Cheers,
> >>>
> >>> Piers
> >>>
> >>>> more capacity would be available. Normally congestion control does
> >>>> this by probing/slowly increasing the sending rate, but if the traffic
> >>>> is
> >>>
> >>> application
> >>>

tel: +49(0)711/685-67973

---

From: Michael Welzl <[email-masked]>
Date: Sat, 12 Oct 2013 08:01:31 +0200

> this is also needed for the people designing/proposing an congestion control 

I agree;

> aware what applications are doing or what should/would be possible in future.

One aspect is that there is the possibility of either doing FEC, as has been brought up, or content-based ARQ in the application. What you choose, or how much of each you do, is a non-trivial design application question - I have advised a Ph.D. thesis on that, see:
http://heim.ifi.uio.no/michawe//research/students/michael_schier-diss_final.pdf
I'm pointing to this thesis because I think that the video coding introduction in this document is a very nice read - easy to understand, yet going beyond what most "networkers" normally understand about video (I think).

Whether an application uses FEC or ARQ, such decisions are based on knowledge about the importance of parts of the media stream. This extends beyond simply I- vs. P- vs. B-Frame, as each frame consists of metablocks and blocks, of different types, with different dependencies... the importance of a network packet can be derived at run-time, and if the transport layer could make use of that information, then that would be a useful part of the API.

This brings me to another, more general network question, for which I'll cc tsv-area, in a separate email.

Cheers,
Michael

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 12 Oct 2013 13:40:58 -0700

> Whether an application uses FEC or ARQ, such decisions are based on knowledge about the importance of parts of the media stream. This extends beyond simply I- vs. P- vs. B-Frame, as each frame cons…

[BA] I couldn't agree more.  Indeed, the linkage between cc and video coding is critical 
-  particularly when dealing with simulcast and scalable video coding, which is becoming increasingly popular.  This has important implications for the congestion control  (and circuit breaker) algorithms.   Here is a reference to one of the early papers on the subject (2006), which, while austensibly about streaming with SVC, has many aspects (such as a critique of TFRC drawbacks) that apply to interactive use as well: 
http://www.zju.edu.cn/jzus/2006/A0605/A060510.htm