Subject: [rmcat] Design Team Meeting Minutes & Recording July 1

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Mon, 1 Jul 2013 21:10:05 +0000

RMCAT Eval Design Team Meeting
Monday, July 1, 2013, 11:00AM New York Time (EDT UTC-4)
66 minutes, 69.29MB
Streaming recording link:
https://cisco.webex.com/ciscosales/ldr.php?AT=pb&SP=MC&rID=69593382&rKey=1ca60ece9b6ec407
Download recording link:
https://cisco.webex.com/ciscosales/lsr.php?AT=dw&SP=MC&rID=69593382&rKey=2b90b6a2d9bca514

Next meeting will be in Berlin before the RMCAT session.

Meeting Minutes:

Fairness criteria:
  - More consensus on defining unfairness as gross inequality rather than optimal fairness.
  - Relative throughput rates exceeding 3:1 (Kevin) or 10:1 (Greg) have been proposed as unfair.
  - Others (Xiaoqing, Mirja) prefer unfair criteria other than relative throughput rates.
  - Weighted fairness was discussed as a possible goal of some RMCAT applications.
  - Should RMCAT solution candidates explicitly support weighted fairness? Mandatory, optional, or prohibited?
  - The requirements draft does not mention weighted fairness nor equal-rate fairness, just that fairness is “a hard-to-define term”, “a tough question”. Need to decide what firmer position the req draft should take, so further drafts (eval, candidates, etc.) have clear reqs.
  - It was noted that in the absence of weighted fairness within RMCAT itself, applications will game multiple RMCATs to achieve weighted fairness.

Stability:
  - It has been assumed that throughput rate stability is a desirable trait of solution candidates, and that evaluation criteria should reflect this in some quantitative way.
  - However, research into highly variable transports such as wireless may beg the question of whether novel forms of congestion control, and perhaps even video rate control, can better utilize the highly variable statistics (ref: Sprout) by exploiting them rather than focusing on rate stability.
  - Noted that rate feedback speed from receiver (or network) to sender can be decoupled from rate feedback loop between the sender and its local encoder.

Drafts:
  - Still waiting on an update to the requirements draft. Randell?
  - Still waiting on an update to the evaluation draft. Varun hopes to get it out next week.
  - What will we do with Michael R.’s doc? Current thought is add as an appendix in the eval draft.

Please add anything else I may have missed.

Mo

---

From: Dan Weber <[email-masked]>
Date: Tue, 2 Jul 2013 10:03:10 -0600

On Mon, Jul 1, 2013 at 3:10 PM, Mo Zanaty (mzanaty) <[email-masked]>wrote:

>  RMCAT Eval Design Team Meeting****
>
> Monday, July 1, 2013, 11:00AM New York Time (EDT UTC-4)****
>
> 66 minutes, 69.29MB****
>
> Streaming recording link:****
>
>
> ****
> Download recording link:****
>
>
> ****
> ** **
> Next meeting will be in Berlin before the RMCAT session.****
>
> ** **
>
> Meeting Minutes:****
>
> ** **
>
> Fairness criteria:****
>
>   - More consensus on defining unfairness as gross inequality rather than
> optimal fairness.****
>
>   - Relative throughput rates exceeding 3:1 (Kevin) or 10:1 (Greg) have
> been proposed as unfair.****
>
>   - Others (Xiaoqing, Mirja) prefer unfair criteria other than relative
> throughput rates.****
>
>   - Weighted fairness was discussed as a possible goal of some RMCAT
> applications.****
>
>   - Should RMCAT solution candidates explicitly support weighted fairness?
> Mandatory, optional, or prohibited?****
>
>   - The requirements draft does not mention weighted fairness nor
> equal-rate fairness, just that fairness is “a hard-to-define term”, “a
> tough question”. Need to decide what firmer position the req draft should
> take, so further drafts (eval, candidates, etc.) have clear reqs.****
>
>   - It was noted that in the absence of weighted fairness within RMCAT
> itself, applications will game multiple RMCATs to achieve weighted fairness.
> ****
>
> **
>
DTM> Mentions were made about updating Randell's draft to see if weighted
fairness should be a general requirement of an RMCAT implementation.

>  **
>
> Stability:****
>
>   - It has been assumed that throughput rate stability is a desirable
> trait of solution candidates, and that evaluation criteria should reflect
> this in some quantitative way.****
>
>   - However, research into highly variable transports such as wireless may
> beg the question of whether novel forms of congestion control, and perhaps
> even video rate control, can better utilize the highly variable statistics
> (ref: Sprout) by exploiting them rather than focusing on rate stability.**
> **
>
>   - Noted that rate feedback speed from receiver (or network) to sender
> can be decoupled from rate feedback loop between the sender and its local
> encoder.
>
DTM> The basis for a rate stability requirement comes from supporting both
fat and skinny buffer networks.  Delay based or interarrival time based
metrics like those used in Sprout and other implementations are unlikely to
be effective in networks where buffer bloat does not exist.  In the skinny
buffer networks, packet loss will replace the excessive delay.  Seeking
rate stability should account for both cases as packet loss will lower
receiver side throughput, and bufferbloated networks will cause receiver
side throughput to bounce up and down quite a bit.

DTM> Supporting LTE based networks where the throughput available changes
rapidly was one concern brought up about the rate stability case.  Varun
also added that a web browser bursting for data connections can produce
similar behavior, where stability may then be optimal.  In either case, it
may be optimal to target the minimum number of rate changes as network
conditions change that allow general reliability of the stream.

> ****
>
> ** **
>
> Drafts:****
>
>   - Still waiting on an update to the requirements draft. Randell?****
>
>   - Still waiting on an update to the evaluation draft. Varun hopes to get
> it out next week.****
>
>   - What will we do with Michael R.’s doc? Current thought is add as an
> appendix in the eval draft.****
>
> ** **
>
> Please add anything else I may have missed.****
>
> **
>
DTM> We're also looking to come up with a standardized structure for
defining test cases to be included in the evaluation criteria appendix.

> **
> Mo****
> ** **

---

From: Michael Welzl <[email-masked]>
Date: Wed, 3 Jul 2013 15:41:12 +0200

Hi,

See below a comment about one bit in the meeting minutes:

> On Mon, Jul 1, 2013 at 3:10 PM, Mo Zanaty (mzanaty) <[email-masked]> wrote:

(snip)

> DTM> Mentions were made about updating Randell's draft to see if weighted fairness should be a general requirement of an RMCAT implementation.

Weighted fairness is requirement F34 in
http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requirements-10

… to address this requirement, the "identifying and controlling groups of flows" milestone ended up in the charter. Therefore I think there should be no debate about "do we want to support this?".

Whether or in which way this has to be directly supported by the "RMCAT solution candidates" (which, I guess, refers to the congestion control mechanisms done by RMCAT) is another question; I think the requirement here is to be able to interact with the entity that deals with the "controlling groups of flows" bit. 

For the draft we're working on ( http://tools.ietf.org/html/draft-welzl-rmcat-coupled-cc ), this translates into a different requirement: to be able to work with this, a mechanism must be able to use an enforced rate that is different from its own derived rate. "Lower" is really the same as "must cope well with non-greedy applications", whereas "higher" is probably less common and somewhat strange from the perspective of a normal congestion control mechanism up to now.

Cheers,
Michael

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 4 Jul 2013 21:06:44 +0200

Hi,

My comment below.

BR
Zahed
> … to address this requirement, the "identifying and controlling groups of flows" milestone ended up in the charter. Therefore I think there should be no debate about "do we want to support this?".
The charter says this WG should "Develop a mechanism for identifying 
shared bottlenecks between groups of
flows, and means to flexibly allocate their rates within the aggregate 
hitting
the shared bottleneck." "Flexibly allocate rate" does not explicitly 
mean "weighted fairness", at least not to me. I see "weighted fairness" 
is one of the ways to solve the problem.

When a set of flows are grouped based on common shared bottleneck that 
group of flows may have same priority or different. Moreover, different 
applications might have different meaning of priorities and different 
policies to handle lower priority flows when congestion occurs. Some of 
them would like to distribute the available bandwidth based on their 
priorities and some other weights, some might like to kill the lower 
priority flow before touching the higher priority flows, some might even 
like to have multiple level of priorities, meaning different preference 
within the same priority group. Hence dealing with all these variations 
will be a great challenge. It will need proper APIs between application 
and RMCAT solution and proper definition of "flexible allocation of rate".

Regarding RTCWEB req F34, I don't think even RTCWEB WG has clear idea 
how that will be realized in the end.

Hence, I would like to discuss " means to flexibly allocate rate and how 
to realize it" before saying "weighted fairness" is the way forward.

Until we do so I would suggest that in our simple (the one Micheal 
Ramalho worked on) fairness test scenario we assume all flows have same 
priority and ensure RMCAT solution shows some sort of fairness among 
those flow. My current feeling is that it is very hard to do so.
**

BR

-- 

Zahed

==================================================
ANM ZAHEDUZZAMAN SARKER

Ericsson AB
Multimedia Technologies
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Fax +46 920 996 21
SMS/MMS +46 76 115 37 43
[email-masked]
www.ericsson.com

==================================================

---

From: Michael Welzl <[email-masked]>
Date: Thu, 4 Jul 2013 22:13:45 +0200

On Jul 4, 2013, at 9:06 PM, Zaheduzzaman Sarker <[email-masked]> wrote:

> The charter says this WG should "Develop a mechanism for identifying shared bottlenecks between groups of
> flows, and means to flexibly allocate their rates within the aggregate hitting
> the shared bottleneck." "Flexibly allocate rate" does not explicitly mean "weighted fairness", at least not to me. I see "weighted fairness" is one of the ways to solve the problem. 
> 
> When a set of flows are grouped based on common shared bottleneck that group of flows may have same priority or different. Moreover, different applications might have different meaning of priorities and different policies to handle lower priority flows when congestion occurs. Some of them would like to distribute the available bandwidth based on their priorities and some other weights, some might like to kill the lower priority flow before touching the higher priority flows, some might even like to have multiple level of priorities, meaning different preference within the same priority group. Hence dealing with all these variations will be a great challenge. It will need proper APIs between application and RMCAT solution and proper definition of "flexible allocation of rate".

I have the impression that you're over-complicating this.

> Regarding RTCWEB req F34, I don't think even RTCWEB WG has clear idea how that will be realized in the end. 

But we should: it was this requirement that motivated the RMCAT charter text that we're discussing here. Hence, at least when RMCAT was conceived, the intention was that this group should take care of this requirement.

> Hence, I would like to discuss " means to flexibly allocate rate and how to realize it" before saying "weighted fairness" is the way forward.

"Develop a means to do something" indicates that stuff isn't happening automatically; instead, we're building a tool that lets someone take control. That someone could be one application, or multiple, or the user. We should make it possible for something (or someone) to "flexibly allocate rate". The word "flexibly" stresses that the control should have as few limitations as possible.

So, this sentence really means: "make it possible for someone or something to pretty arbitrarily control the rates of the flows in an aggregate that shares a bottleneck."

Once someone/-thing is in control of the rates, that someone/-thing can realize the policies you describe above. The policies themselves (re-assigning priorities vs. priority groups vs. killing a flow to support the next vs. …..) are nothing to worry about for RMCAT, then, as long as the possibility to assign rates within the aggregate is given. Not sure if "weighted fairness" is the right word for this functionality or not, but anyway, that's what we need.

> Until we do so I would suggest that in our simple (the one Micheal Ramalho worked on) fairness test scenario we assume all flows have same priority and ensure RMCAT solution shows some sort of fairness among those flow. My current feeling is that it is very hard to do so.

I'm fine with that, actually. A fair allocation gives the mechanism for flexible rate allocation a starting point. We need that. However, to be able to really be in control of these rates, I think that RTT-independence would be a desirable property. I'm not sure if we should prescribe this because it's asking much, but I think we should recommend it.

Cheers,
Michael

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Fri, 5 Jul 2013 16:11:40 +0000

Michael & Zahed (mostly),

The thread below is a little rugged ... so let me just comment here at the top ;-).

The RTCWeb requirement F34 (The browser MUST support prioritization of streams and data) can be taken at least two different ways of importance to RMCAT.

The first way is that RMCAT transport should provide an API to the application that allows it to "appropriately prioritize" the application traffic provided to it when it puts that traffic on the wire. I think that will eventually be a must - although we are still quite far from specifying how that API behaves as we aren't done at the transport level yet ;-). Obviously, the prioritization of flows within one host is doable with information on that one host.

The second way applies when ALL FLOWS through a bottleneck are RMCAT flows - and that a given RMCAT association (borrowing SCTP terminology here) could have multiple RMCAT streams of various priorities within it. Let's say your RMCAT association has streams of different priorities. Then you could have a desire  to compete fairly IN PRIORITY with other RMCAT streams of priority foo. That is a MUCH harder requirement. And possibly impossible if your flow is mixing with other existing transports.

Is it your understanding that RTCWeb hopes we  (RMCAT) handle the second case? And if so, with what desired mechanism (perhaps congestion control rate estimation be a function of the priority value provided to the RMCAT transport - but that only works with an assumed traffic mix at the bottleneck - which is unknowable at a given host)?

Thanks for your input ...

Michael Ramalho

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Welzl
Sent: Thursday, July 04, 2013 4:14 PM
To: Zaheduzzaman Sarker
Cc: Dan Weber; Mo Zanaty (mzanaty); [email-masked]
Subject: Re: [rmcat] Design Team Meeting Minutes & Recording July 1

On Jul 4, 2013, at 9:06 PM, Zaheduzzaman Sarker <[email-masked]> wrote:

>>> On Mon, Jul 1, 2013 at 3:10 PM, Mo Zanaty (mzanaty) 
>>> <[email-masked]>
>>>  wrote:
>>>   - The requirements draft does not mention weighted fairness nor equal-rate fairness, just that fairness is "a hard-to-define term", "a tough question". Need to decide what firmer position the req draft should take, so further drafts (eval, candidates, etc.) have clear reqs.
>>> 
>> ... to address this requirement, the "identifying and controlling groups of flows" milestone ended up in the charter. Therefore I think there should be no debate about "do we want to support this?".
>> 
> shared bottlenecks between groups of flows, and means to flexibly 
> allocate their rates within the aggregate hitting the shared bottleneck." "Flexibly allocate rate" does not explicitly mean "weighted fairness", at least not to me. I see "weighted fairness" is one of the ways to solve the problem.
> 

Once someone/-thing is in control of the rates, that someone/-thing can realize the policies you describe above. The policies themselves (re-assigning priorities vs. priority groups vs. killing a flow to support the next vs. .....) are nothing to worry about for RMCAT, then, as long as the possibility to assign rates within the aggregate is given. Not sure if "weighted fairness" is the right word for this functionality or not, but anyway, that's what we need.

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Fri, 5 Jul 2013 18:29:52 +0200

Hi,

I'll try to summarize in what I think are simpler words (but sticking to SCTP terminology):
case 1: priorities WITHIN a single RMCAT association
case 2: priorities BETWEEN multiple RMCAT associations (which may embed case 1, i.e. one RMCAT asocation may have multiple flows with priorities between them)

I think we agree that supporting case 1 is a must. Supporting case 2, however, is much more difficult, and may not even be a reasonable goal to pursue when we're at the same time trying to be roughly TCP-compatible and minimize delay. It might be possible to do this in some way, but I would prefer to avoid dealing with case 2, I was only concerned with case 1. I do not believe that anyone expects us to solve case 2.

I see that case 1 may not be what you were concerned with when discussing "weighted fairness", but enabling case 1 with a method akin to the one in our coupled-cc draft still requires that some external entity can set the rate of a flow to a rate that wasn't determined by it. That is a bit unusual and should be a part of the tests that people should do with their mechanisms, I think. Yes, I am sorry for not having brought this up earlier!

Cheers,
Michael

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Fri, 5 Jul 2013 18:23:59 +0000

Michael,

Thanks for your reply. I think you and Zahed were talking past each other.

On the RMCAT design team call, the "weighted fairness topic" came up in the context of Case 2 (weighted fairness BETWEEN different RMCAT associations). The "between sessions" topic was initiated by Xiaoqing Zhu (as that is what a NADA does - see her draft for that). I believe that Case 2 was the context behind Zahed's reply.

>From what I understand, your draft deals primarily with Case 1 (within a single RMCAT session/association/whatever-we-will-call-it). And I believe that was the context of your reply.

And now we all agree that our initial work should focus on RMCAT fairness for flows with equal priority ... and that RTT-fairness is a desirable but not-yet-determined-to-be-feasible goal.

Michael Ramalho

PS - Fairness independent of RTT may be attainable, but it would need to be measured on timescales that are many multiples of the RMCAT flow with the largest RTT. This is because the maximum time rate of change of the BW estimate must be a function of the RTT (feedback is not instantaneous). This is easily seen in the dynamics of any window-based congestion control (such as TCP). Indeed, "end-to-end capacity" may need to be defined via a bandwidth measure on a similar timescale (this is a shortcoming of many IETF drafts - where they just say "capacity" without even defining it ...  but I digress).

-----Original Message-----
From: Michael Welzl [mailto:[email-masked]] 
Sent: Friday, July 05, 2013 12:30 PM
To: Michael Ramalho (mramalho)
Cc: Zaheduzzaman Sarker; Dan Weber; Mo Zanaty (mzanaty); [email-masked]
Subject: Re: [rmcat] Design Team Meeting Minutes & Recording July 1

Hi,

I'll try to summarize in what I think are simpler words (but sticking to SCTP terminology):
case 1: priorities WITHIN a single RMCAT association case 2: priorities BETWEEN multiple RMCAT associations (which may embed case 1, i.e. one RMCAT asocation may have multiple flows with priorities between them)

Cheers,
Michael

On Jul 5, 2013, at 6:11 PM, "Michael Ramalho (mramalho)" <[email-masked]> wrote:

>>> http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requireme
>>> n
>>> ts-10
>>> 
>>>

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Mon, 8 Jul 2013 13:57:51 +0200

HI Michael & Michael,

Thanks for clarifying stuffs while I was enjoying my weekend (+ vacation).

yes, if you hear the recoding of the design team meeting the "between 
sessions" fairness was the topic we were concerned about. There I said I 
can see the idea make sense for case 1 from a single application (or 
host) but was not sure about "between sessions".

All these cases has one thing in common " shared bottleneck" and my 
working assumption for that is " flows in the Internet are not sharing 
same bottleneck until something tells they are" " . This WG has an 
agenda of designing mechanism to identify shared bottleneck and lets see 
how far we go with only an end-to-end solution. Luckily for testbeds we 
know that :-).

in Berlin, it will be nice to discuss the topics (issues), like flow 
prioritization and its understanding, shared bottleneck discovery, 
interaction between RTCWEB and RMCAT, along with the issues the media 
design team has identified. May be one of the media team member  need to 
present the topics in the session or the chairs can do so.

BR

Zahed

-- 

Zahed

Ericsson AB
Multimedia Technologies
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Fax +46 920 996 21
SMS/MMS +46 76 115 37 43

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Mon, 8 Jul 2013 14:00:46 +0200

> I have the impression that you're over-complicating this.
I would love to hear your non-complicated or less-complicated version . :-)
> But we should: it was this requirement that motivated the RMCAT charter text that we're discussing here. Hence, at least when RMCAT was conceived, the intention was that this group should take care…
This group should only work with stuffs that this group find 
interesting. Is there any agreement that RTCWEB will blindly accept what 
this group will come up as a solution (algorithms, API for interaction 
etc.)? this question is addressed to the whole group not only to 
Michael. Maybe something to discuss in Berlin on how to proceed that 
both RMCAT and RTCWEB will be served well.
> Once someone/-thing is in control of the rates, that someone/-thing can realize the policies you describe above. The policies themselves (re-assigning priorities vs. priority groups vs. killing a f…
if RMCAT does not need to worry about the policies then why would RMCAT 
differentiate between flows?

BR

-- 

Zahed

Ericsson AB
Multimedia Technologies
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Fax +46 920 996 21
SMS/MMS +46 76 115 37 43

---

From: Michael Welzl <[email-masked]>
Date: Mon, 8 Jul 2013 15:11:53 +0200

hi,

yep, i think the fairness thing was just a misunderstanding. sorry, i had only looked at the minutes, not the audio.

about shared bottleneck detection, e2e is ok i think cause parameters could always be tuned conservatively... i.e. a false *negative* is not a big deal, it's just status quo.. we're working on something which we hope to be ready for presentation in vancouver. berlin is too early i think

Sent from my iPhone

On 8. juli 2013, at 13:57, Zaheduzzaman Sarker <[email-masked]> wrote:

> yes, if you hear the recoding of the design team meeting the "between sessions" fairness was the topic we were concerned about. There I said I can see the idea make sense for case 1 from a single application (or host) but was not sure about "between sessions".
> 
> All these cases has one thing in common " shared bottleneck" and my working assumption for that is " flows in the Internet are not sharing same bottleneck until something tells they are" " . This WG has an agenda of designing mechanism to identify shared bottleneck and lets see how far we go with only an end-to-end solution. Luckily for testbeds we know that :-).
> 
> in Berlin, it will be nice to discuss the topics (issues), like flow prioritization and its understanding, shared bottleneck discovery, interaction between RTCWEB and RMCAT, along with the issues the media design team has identified. May be one of the media team member  need to present the topics in the session or the chairs can do so. 
> 
> 
> BR
> 
> Zahed
>

---

From: Michael Welzl <[email-masked]>
Date: Mon, 8 Jul 2013 15:15:09 +0200

about the response directly to me here, i suspect that this is related to our misunderstanding. i was thinking of flows that leave the same host + are in the same "association", you thought of multiple "associations"?

Sent from my iPhone

On 8. juli 2013, at 14:00, Zaheduzzaman Sarker <[email-masked]> wrote:

> This group should only work with stuffs that this group find interesting. Is there any agreement that RTCWEB will blindly accept what this group will come up as a solution (algorithms, API for interaction etc.)? this question is addressed to the whole group not only to Michael. Maybe something to discuss in Berlin on how to proceed that both RMCAT and RTCWEB will be served well.
>> 
>> 
> if RMCAT does not need to worry about the policies then why would RMCAT differentiate between flows?
> 
> BR
> 
> -- 
> 
> Zahed
>