Subject: [rmcat] Piggybacking

From: John Leslie <[email-masked]>
Date: Wed, 3 Oct 2012 11:00:37 -0400

Randell Jesup <[email-masked]> wrote:
> On 9/24/2012 10:46 AM, Harald Alvestrand wrote:
>>
>> In the TCP case, the feedback mechanism is the ack, which is needed 
>> for reasons beyond congestion control. In the RTP case, there isn't 
>> such a single, simple mechanism needed for other reasons.
> 
> Agreed, though Michael indicated that we can piggyback congestion info 
> on reverse traffic.
> 
> This may be possible and I've thought about it.  Part of the problem is 
> that if you want to signal data back quickly, you have to wait up to 
> 20ms and in some cases longer to piggyback (and on 
> low-upstream-bandwidth links, the uplink transit time if you're 
> piggybacking on a 1500-byte video packet might be 20+ms).

   This is missing the point, IMHO.

   We are dealing in IP packets -- "unreliable" packets: meaning that
we aren't entitled to expect any particular packet to be delivered
at all.

   The signal we wish to send is tiny -- probably on the order of one
byte. If we allocate one byte per packet for feedback, it's almost
lost in the noise.

   But if we expect that byte and it doesn't arrive, that's a clear
congestion signal.

   (Granted its loss signals congestion in the path _to_ us...)

   Trying to optimize 20 msec of reaction time for real-time traffic
isn't the right paradigm. We want real-time traffic to adapt more
slowly than web-surfing traffic, but stay low for longer (to avoid
self-congestion).

   Thus, we can be pretty sure that the absence of an expected feedback
justifies a delay in any ramp-up of outgoing bit-rate currently planned.

   Beyond that, it gets more complicated. Nonetheless, we can easily
signal in our next scheduled outgoing packet that we missed an
expected feedback; and that signal (or the failure to receive our
next outgoing packeti) can trigger a retransmission -- catching up
typically faster than TCP can do.

   I'd like to suggest that we plan RMCAT around bidirectional
(though possibly very small in one direction) media packets. There
aren't many cases where the traffic is _so_ unidirectional that
an "I'm still interested in this stream" isn't appropriate.
(I don't mean to imply that the packet-rate needs to be the same
in both directions, least of all the bit-rate.)

> If you do piggyback, it has to be optional on a instance-by-instance
> basis 

   This is precisely what I _don't_ want to see. Perhaps the size of
the feedback might vary (though I doubt that's needed); but I want the
_absence_ of feedback to be a primary signal.

> (perhaps even estimating when the next chance will be, and the priority 
> of the data - but that speaks against a dumb receiver).

   (I don't think it makes sense to even think about a "dumb receiver.)

> TCP can "get away" with high packet count feedback because flows are 
> typically largely unidirectional or alternating flows, or on 
> well-connected devices, not edge nodes.  The PacketCable/etc issue with 
> shared upstreams and slot allocation is an example of reacting to this - 
> there even with the typically largely downstream flows, the upstream 
> acks become a limiting factor.

   Yes!

   This issue is very real in cable-upstream, and also significant in
many wireless setups.

   But the right approach is to measure "upstream" congestion and adapt
our packet rate to it. Really-tiny upstream packet aren't much "cheaper"
than medium-sized upstream packets.

   Granted, it's easy to confuse upstream-congestion with downstream-
congestion; but it's not _that_ hard to separate them.

>> Stickler: Amount of feedback has little correlation with packet drop. 
>> If feedback is lost due to congestion, more feedback will lead to more 
>> packet drops. What is true is that when feedback occurs more rarely, 
>> losing one feedback packet will lead to a greater gap between feedback 
>> packets.

   This is true, but some possible inferences are not true...

> I'd say less feedback means the odds that a drop will hit a feedback 
> packet are *lower* (as routers typically either tail-drop or 
> random-drop, but in either case in a static evaluation a 
> lower-bandwidth/packet flow (i.e. the feedback packets) will get hit 
> less often), but as Harald says the impact is higher.  And in a dynamic 
> evaluation this may shift some.

   In particular, inferring that we must do slavish AIMD is not true.

>>> - receiver-side-congestion-control-based feedback reduction as in 
>>> RRTCC means to reduce the feedback whenever feedback isn't urgent for 
>>> the sender. I'd argue that, in fact, we should reduce feedback only 
>>> when the channel carrying it (the backwards channel) is congested, 
>>> and that is a generic function, not bound to the sender-side 
>>> congestion control behavior.

   I agree that reducing feedback when the feedback path is not
congested really doesn't help.

   (When we _do_ need to reduce feedback, I suggest we need to lower
the packet-rate, letting the chips fall where they may; and be slow
to increase it again.)

> Personally, I assume both channels are always congested, or will be soon 
> (if we're doing our job right and aren't on a huge link),

   The assumption isn't particularly bad; but I envision many cases
where it isn't true.

> so bandwidth reduction for feedback always allows more bandwidth for
> useful data.

   While technically true, it's a bad tradeoff to discard information
about congestion -- this leads to growing your bit-rate too fast, and
more packet-loss than necessary.

>... 
> And my position is that any *good* two-way communication will run 
> near-but-not-quite-over the bandwidth limit as much of the time as 
> possible, so ACKs always have the potential to cause problems (and 
> always will force you a little further down the quality curve).

   I don't agree with this conclusion.

   With "ideal" feedback, we can be sure we don't initiate any
congestion problems, while "minimal" feedback means some of our
guesses will be wrong.

   And I don't think we have a good understanding of "the quality
curve" yet.

   Some real-time stacks will run very close to the maximum data-rate,
and hopefully will have good redundancy algorithms to reconstruct
lost packets. Others will choose to stay below the maximum. We shouldn't
prejudge that tradeoff, but limit ourselves to timely delivery of the
best congestion information, and algorithms for rate-adjustment.

--
John Leslie <[email-masked]>

---

From: Michael Welzl <[email-masked]>
Date: Thu, 4 Oct 2012 12:44:08 +0200

Hi,

> justifies a delay in any ramp-up of outgoing bit-rate currently planned.

Indeed, I'm not sure a quick signal is always needed. Either way, as John says, a packet may be lost and the signal is tiny. We could agree to always piggyback - if the signal really can't wait, we can send it immediately but include it piggybacked onto the next datapacket too, for redundancy, making the whole thing more stable in case a feedback packet gets lost.

> typically faster than TCP can do.

I don't get that, and it sounds complicated and inefficient - you're adding round-trips here?

>   (I don't think it makes sense to even think about a "dumb receiver.)

Call it a *generic* receiver. That's what was meant in this discussion.

The problem is really that it's a (major, I think) benefit for deployment to have one side generic, and a generic sender is a more complicated setting than a generic receiver... in my opinion, none of the two choices are really ideal. To have things combined in the right place, I'll address this in a separate email in which I extend my previous analysis of the sender/receiver situation.

> than medium-sized upstream packets.

That was my argument too; it does add some complexity though. See my next email on sender vs receiver...

As for John's comments below, I agree with many of them.

> _______________________________________________
> rmcat mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rmcat

---

From: Michael Welzl <[email-masked]>
Date: Thu, 4 Oct 2012 12:58:03 +0200

I herewith extend my own analysis of the sender- vs. receiver-based congestion control question, with one (in my opinion) significant plus in each camp:

> Ok, I think you got me convinced. The way I see it, there's a choice between:
> 1) sending a lot of feedback, and only reducing it when there is backwards congestion, by doing congestion control on the backwards channel like ACK-CC
> 2) simply always sending as little feedback as possible, based on the (forward) congestion control mechanism's requirements
> 
> If we think of e.g. a videoconference, indeed congestion from all the feedback on the uplink is quite realistic. 1) can then become rather complicated... you may get congestion on your uplink, and it might be necessary to reduce the rate of your data stream, or it might suffice to reduce it on some of the feedback streams, but which? And then we have to apply priorities between feedback packets and your payload...
> 

> 2) seems simpler, but it has the problem of making the mechanism more fragile, and perhaps unnecessarily so (in a situation where it might be fine to send much more feedback). I say "unnecessarily" because I'm not convinced about "both channels are always congested, or will be soon" because codecs may not be greedy, and streams such as VoIP may simply not generate enough traffic for that.

2), i.e. receiver-side congestion control, also has the ADVANTAGE that it doesn't misinterpret the loss of feedback as an indication of forward congestion. A receiver-side controller has a clear notion of all packets that have arrived in an interval and knows exactly what was dropped on the way from the sender to the receiver and what wasn't.

2), i.e. receiver-side congestion control, also has the DISADVANTAGE that tight interaction with the sending application gets harder - e.g., to consider per-packet priorities in the congestion control mechanism. With a receiver-side scheme, this requires richer feedback (in the style: "for high-priority packets: this rate; for low-priority and high-priority packets combined: that rate"). Then, the receiver would have to know how many priorities are there on the sender side to give the right amount of feedback...  this will make things unnecessarily complicated or, as I rather suspect, not happen, and then greatly limit us.

Cheers,
Michael

---

From: Randell Jesup <[email-masked]>
Date: Thu, 04 Oct 2012 10:06:57 -0400

> Indeed, I'm not sure a quick signal is always needed. Either way, as John says, a packet may be lost and the signal is tiny. We could agree to always piggyback - if the signal really can't wait, we…

That's closer to what I was thinking - use independent packets 
*sometimes*, but mostly use piggybacked (which reduced the overall 
impact of feedback).  I'd use independent packets when there's an 
expected (or measured) long delay waiting to piggyback (for example if 
muted or using VAD), and when we have new (unexpected) information to send.

There is an issue with 'how' piggybacking is done on RTP.  In RTCP, it's 
fairly easy (though more than one byte!)  In RTP, you either are 
changing the payload formats or you're using a header extension.  (Or 
we're designing RTP v3....  That would be 'interesting'.)

WebRTC 'bundle' helps us here (more packets to piggyback on), and in 
general if we have multiple flows sharing 
congestion-measurement/reporting domains.  (Also more data (faster 
smoothing of noise), etc).  In my previous implementation one common 
case was that on packet loss (video), the system would send an immediate 
RTCP NACK/rtcpfb pli/etc to ask for a refresh, and congestion info would 
hitchhike along with any RTCP packet we sent.

> I don't get that, and it sounds complicated and inefficient - you're adding round-trips here?

Basically PLI for congestion control.  However, you need a jitter/delay 
buffer to handle transit jitter and reordering, so it's not so simple.  
Or you decide fairly frequent false positives are ok, which might be 
possible.

>> (though possibly very small in one direction) media packets. There

I think it has to handle *well* both one-directional and bidirectional 
flows.  But opportunistic piggybacking would do that (if you solve the 
'how' question above for RTP).  You definitely can piggyback with other 
RTCP traffic.

>> in both directions, least of all the bit-rate.)

Right, though that also means we're not talking about ack-per-packet in 
unidirectional.
While I'm not suggesting this, when there was no other RTCP traffic to 
piggyback on (and no loss) I ran ~1 second congestion reports.

>> _absence_ of feedback to be a primary signal.

That's not at odds with what I was saying - I meant 
packet-by-packet/signal-by-signal you decide to piggyback or not - and 
if you're waiting to piggyback you can react to a timer expiry and send 
it standalone.  So you *could* design it to have "feedback every Nms" 
(with adjustable N), and have the receiver send feedback every <=N, 
using piggyback or not as available.

> That was my argument too; it does add some complexity though. See my next email on sender vs receiver...

Agreed.  Many/most routers buffer by packet not by packet-size. Each 
WiFi packet is much heavierweight than the payload size would indicate, 
etc.  Which is why I like reducing ACK *packet* flows.

-- 
Randell Jesup
[email-masked]

---

From: John Leslie <[email-masked]>
Date: Thu, 4 Oct 2012 10:46:39 -0400

Michael Welzl <[email-masked]> wrote:
> 
> I herewith extend my own analysis of the sender- vs. receiver-based
> congestion control question, with one (in my opinion) significant plus
> in each camp:
> 
>>... The way I see it, there's a choice between:
>> 1) sending a lot of feedback, and only reducing it when there is
>>    backwards congestion, by doing congestion control on the backwards
>>    channel like ACK-CC
>> 2) simply always sending as little feedback as possible, based on the
>>    (forward) congestion control mechanism's requirements
> 
> 2), i.e. receiver-side congestion control, also has the ADVANTAGE that
>     it doesn't misinterpret the loss of feedback as an indication of
>     forward congestion.

   We should try pretty hard to avoid the need to interpret the lack
of feedback as indicating what happened on the forward path.

>     A receiver-side controller has a clear notion of all packets that
>     have arrived in an interval and knows exactly what was dropped on
>     the way from the sender to the receiver and what wasn't.

   (Assuming, of course, it knows the schedule of what the sender sent.)

> 2), i.e. receiver-side congestion control, also has the DISADVANTAGE
>     that tight interaction with the sending application gets harder -
>     e.g., to consider per-packet priorities in the congestion control
>     mechanism.

   I don't see any necessary coupling between priorities of what the
sender would like to send and what the receiver would like to receive.

>     With a receiver-side scheme, this requires richer feedback (in
>     the style: "for high-priority packets: this rate; for low-priority
>     and high-priority packets combined: that rate"). Then, the receiver
>     would have to know how many priorities are there on the sender
>     side to give the right amount of feedback...  this will make
>     things unnecessarily complicated or, as I rather suspect, not
>     happen, and then greatly limit us.

   Congestion is somewhat orthogonal to priorities: we're interested
in avoiding congestion far more than we're worried about which packets
have the highest priority.

   The receiver may or may not wish to express an opinion of which
streams should have highest priority. If it does, this preference
should be stable over any particular period of congestion.

   The sender may well believe that certain events should have high
priority regardless of the receiver's priorities. It will no doubt
find a way to send these with higher priority regardless of what we
may specify.

   We should concentrate our efforts on getting both endpoints a
measure of congestion along the path they're using to send, and
one or more algorithms to back off the _overall_ sending rate. The
network doesn't care one whit for our application-layer priorities.

---

From: Michael Welzl <[email-masked]>
Date: Fri, 5 Oct 2012 00:09:18 +0200

> sender would like to send and what the receiver would like to receive.

Not "what the receiver would like to receive", but between the  
sender's priorities and congestion control.

> network doesn't care one whit for our application-layer priorities.

First, to clarify, I'm talking about per-packet priorities within a  
stream. Second, my point is that congestion control could be done  
better if packet priorities would be known, and could then be taken  
into account.
Right now, we are essentially facing greedy or non-greedy traffic, and  
we have the LEDBAT notion of a generally lower priority. In fact a  
mechanism could opt for a more LEDBAT'ish behavior when its buffer is  
full of low-priority packets and be more aggressive when it has high- 
priority packets.

To the best of my knowledge, these things don't really exist yet, but  
I believe that this is largely because this knowledge is not typically  
available to a congestion control mechanism. Note that conforming to  
such priorities is not just some obscure idea, as there are hard facts  
for a significant qualitative impact, e.g. in the case of video.

But if a mechanism does such things, and it's on the receiver side,  
whereas the send buffer is on the sender side, things get problematic.  
Either we fix this, or we move the mechanism to the sender, or we  
ignore the chance we have, here, to do things better. I guess these  
are our choices.

Cheers,
Michael

---

From: John Leslie <[email-masked]>
Date: Thu, 4 Oct 2012 18:31:15 -0400

> sender's priorities and congestion control.

   Two different things...

> better if packet priorities would be known, and could then be taken  

   I guess I'm missing why packet priorities would ever _not_ be the
domain of the sender -- even if advised by receiver preferences.

> Right now, we are essentially facing greedy or non-greedy traffic,

   Do you mean we're "contending against" those? If so, I agree.

> and we have the LEDBAT notion of a generally lower priority. In fact a  

   If you're saying that sending rate may deserve to increase faster
when there are higher priority packets to send, I could agree but it
seems unlikely such a situation will happen. If OTOH, the sender has
_only_ higher priority packets, a more aggressive sending rate _may_
be justified. Either way, it seems unlikely to be a good idea to
increase sending rate too aggressively for low-priority packets...

> for a significant qualitative impact, e.g. in the case of video.

   This sounds worth discussing. (I'm seriously hopeful that the
"video codec" BoF will lead to a family of video codecs with smoother
bit-rate than is typical today.)

> ignore the chance we have, here, to do things better. I guess these  

   To tell truth, I think it's premature to specify what will be
receiver-side vs. sender-side.

   Especially, I think we need intelligence at both ends in order to
communicate useful congestion information in the two directions.
Once we have designed that we will be better able to specify the
control loops.

   (Assuming ConEx ever finishes, we are likely to see some kind of
congestion-based pricing both for senders and receivers -- and the
control loops ideally would serve the price-based needs of both
senders and receivers.)

---

From: Michael Welzl <[email-masked]>
Date: Fri, 5 Oct 2012 11:46:21 +0200

> domain of the sender -- even if advised by receiver preferences.

It's just not what I meant. I meant priorities influencing the congestion control behavior (I should say the "sender's" behavior, but this discussion is about having the congestion control logic on the sender or receiver side... so...).

>   Do you mean we're "contending against" those? If so, I agree.

I would agree with that too, but I meant: congestion control mechanism design is considering only these two classes of traffic. Sorry for my imprecise wording here!

> seems unlikely such a situation will happen. If OTOH, the sender has

Yes that's what I'm saying. I don't understand why that would be unlikely?

> increase sending rate too aggressively for low-priority packets...

Yes, exactly. We can be less aggressive when we have low-priority packets to send.

> Once we have designed that we will be better able to specify the

I hear you, but have a hard time imagining this without envisioning whether the control decision is made on the sender or receiver side, as this influences what needs to be signalled.

So I think it's a key architectural decision that we'd better get right from the outset. This is because having one side generic ("dumb") is probably very important for deployment. Look at TCP, with its many congestion control flavors in Linux' or FreeBSD's pluggable congestion control. If I run a TCP based (e.g. web) server, I can just pick a mechanism with a simple command. No need to hope for clients to support it; no need to signal to the client what mechanism I use. No need to even standardize it, as we had to learn the hard way...

In case of rmcat in rtcweb, this would mean that browser vendor X could simply start playing with a new congestion control mechanism (offer it to its users or enable it by default), and that would work with all other rtcweb/rmcat browsers, without ever having support for this mechanism from browser vendor Y or Z. If we want that to be possible, we need to decide which side should be the generic one at some point, and then agree on the exact functionality of this generic side.

> control loops ideally would serve the price-based needs of both

I would prefer to leave this consideration aside, because 1) rmcat works on a much tighter schedule than ConEx, as it will probably be deployed in the rtcweb context fast, while I don't know how long we'd have to wait for reasonably wide-spread ConEx deployment...   2) I don't think that this matters much for our decision - in the worst case, we would find that ConEx related information needs to be signalled from one end to the other, and then we could consider extending to whatever signalling scheme we define.

Cheers,
Michael

---

From: Colin Perkins <[email-masked]>
Date: Fri, 5 Oct 2012 10:53:32 +0100

...
> There is an issue with 'how' piggybacking is done on RTP.  In RTCP, it's fairly easy (though more than one byte!)  In RTP, you either are changing the payload formats or you're using a header extension.  (Or we're designing RTP v3....  That would be 'interesting'.)

Or defining a new RTP Profile (see section 5.3 of RFC 3550).

-- 
Colin Perkins
http://csperkins.org/

---

From: John Leslie <[email-masked]>
Date: Fri, 5 Oct 2012 10:58:38 -0400

Michael Welzl <[email-masked]> wrote:
> It's just not what I meant. I meant priorities influencing the
> congestion control behavior (I should say the "sender's" behavior,
> but this discussion is about having the congestion control logic
> on the sender or receiver side... so...).

   "Priorities influencing" is actually quite separable from packet
scheduling.

   I think we're hopelessly talking past each other here. It might
help if you outlined some "likely" mixes of priorities for packets.

> I would agree with that too, but I meant: congestion control
> mechanism design is considering only these two classes of traffic.

   Alas, I don't understand that statement either (and it sounds as if
I would disagree if I did understand it).

>>> In fact a mechanism could opt for a more LEDBAT'ish behavior when
>>> its buffer is full of low-priority packets and be more aggressive
>>> when it has high-priority packets.
>> 
>> when there are higher priority packets to send, I could agree but
>> it seems unlikely such a situation will happen.
> 

   I expressed myself poorly, but I think we're suffering from the
lack of a shared vision of "priority" again...

>> If OTOH, the sender has _only_ higher priority packets, a more
>> aggressive sending rate _may_ be justified. Either way, it seems
>> unlikely to be a good idea to increase sending rate too aggressively
>> for low-priority packets...
> 

   In essence, I was trying to say that an increased aggressiveness
for sending high-priority packets shouldn't spill over into the rate
for low-priority packets. (I still think we're talking past each
other, though.)

> I hear you, but have a hard time imagining this without envisioning
> whether the control decision is made on the sender or receiver side,
> as this influences what needs to be signalled.

   I think it's way to early to talk bits on the wire.

> So I think it's a key architectural decision that we'd better get
> right from the outset. This is because having one side generic ("dumb")
> is probably very important for deployment.

   To the extent that _is_ important for deployment (IMHO) we'd need
to design for both sending _and_ receiving to be "dumb" on one side
of a two-way session.

   I don't think of optimizing for "one side dumb" as important for
deployment, though I would certainly agree that fallback when one side
turns out to be "dumb" is critical.

> Look at TCP, with its many congestion control flavors in Linux' or
> FreeBSD's pluggable congestion control. If I run a TCP based (e.g. web)
> server, I can just pick a mechanism with a simple command. No need to
> hope for clients to support it; no need to signal to the client what
> mechanism I use. No need to even standardize it, as we had to learn
> the hard way...

   Those are all sender-based-control. Are you arguing we should give
up considering receiver-based-control?

> In case of rmcat in rtcweb, this would mean that browser vendor X
> could simply start playing with a new congestion control mechanism
> (offer it to its users or enable it by default), and that would work
> with all other rtcweb/rmcat browsers, without ever having support for
> this mechanism from browser vendor Y or Z.

   That indeed is a worthy goal. But do understand that X, Y, and Z
all will be both sender and receiver.

> If we want that to be possible, we need to decide which side should
> be the generic one at some point, and then agree on the exact
> functionality of this generic side.

   I don't agree that's necessary or sufficient. OTOH, I _do_ believe
we need to establish a feedback mechanism with a clear definition
of what the feedback means (in both directions, of course).

>> control loops ideally would serve the price-based needs of both

   I expressed myself poorly again. We will (IMHO) see congestion-
based pricing for both senders and receivers anyway -- it's happening
already, though the measure of "congestion" is very poor.

   I meant to indicate that, even if absent from the first version,
our control loops should allow receivers to request "lower congestion"
and senders to moderate their sending rate to avoid "congestion"
charges.

> I would prefer to leave this consideration aside, because
> 1) rmcat works on a much tighter schedule than ConEx, as it will
>    probably be deployed in the rtcweb context fast, while I don't
>    know how long we'd have to wait for reasonably wide-spread ConEx
>    deployment...

   Agreed: we may not see widespread ConEx deployment in our lifetime.

> 2) I don't think that this matters much for our decision - in the
>    worst case, we would find that ConEx related information needs
>    to be signalled from one end to the other, and then we could
>    consider extending to whatever signalling scheme we define.

   Although there may (someday) be value to selective ConEx marking
by the sender, it's way premature to talk about such bits on the wire.

---

From: Michael Welzl <[email-masked]>
Date: Sun, 7 Oct 2012 15:17:40 +0200

Hi,

Very sorry for any disadvantages that might have come from having  
expressed myself unclear. I'll try to clean things up in this email now:

> help if you outlined some "likely" mixes of priorities for packets.

Maybe you're considering one type of priority and me another.

Indeed, in rmcat we're facing two kinds of priorities. One, priorities  
between streams. I think we *must* be able to support these priorities  
because they are required by rtcweb (A23 in draft-ietf-rtcweb-use- 
cases-and-requirements). In case of rtcweb, these priorities are under  
control of the user or at least the web site programmer via the  
javascript API, and should therefore probably be expected to be  
reasonably long-lived.

Second (and this is what I meant in this current discussion),  
individual packets in a multimedia stream differ in importance. One  
easy way to say this is that a video consists of e.g. I-frames, P- 
frames and B-frames, where the priority is I > P > B. In reality,  
things are more complex: a frame typically spans multiple packets, but  
there are blocks in frames of different type (e.g. a P-frame may  
contain I- P- and B-blocks), and then there is also a dependency  
chain: some blocks may have many other blocks depend upon them, making  
them more important, and others may have less, making them less  
important. Honoring these priorities is important for the receiver's  
subjective quality - here's one out of many, many existing  
publications documenting this in one way or another:
Michael Schier, Michael Welzl: "Optimizing Selective ARQ for H.264  
Live Streaming: A Novel Method for Predicting Loss-Impact in  
Realtime", IEEE Transactions on Multimedia 14(2), April 2012.
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6095636

Suffice to say, we can have different priorities per packet as a  
function of the multimedia encoding, and these priorities could  
quickly change (this is a way to represent the dependence - if the  
sending application is in control of its data until the last moment,  
and if it is informed about the loss of packet X, it could quickly  
assign a very low priority to packets that depend on X.

Supporting these priorities in rmcat doesn't seem to be a MUST in  
accordance with the charter and rtcweb requirements. I am arguing that  
it would be great if we could support them.

What I mean with supporting them via congestion control is: going back  
to the overly simplistic example of I- P- and B-"packets", one may  
decide to be less aggressive when the buffer is full of "B-packets"  
and more aggressive when it is full of "I-packets". One out of many  
possibilities... my point is, such functionality should be enabled by  
handing over the right information to the congestion controller, and  
this requires the controller to be close to the send buffer, or the  
information to be signalled to the other side. Both can be okay, but  
we should be clear about this.

> I would disagree if I did understand it).

Let's leave it at that, this bit doesn't really matter for the  
discussion anyway.

> lack of a shared vision of "priority" again...

Perhaps; I hope my explanation above helps.

> up considering receiver-based-control?

I'm arguing that we should have either this, or strictly only-receiver- 
based control. Then we could get the same functionality: you could  
just change the receiver to use a new mechanism X, without waiting for  
the sender to support the new mechanism X or even signalling to the  
sender what mechanism to use. Sender- or receiver- based, but not both.

Note that this doesn't exclude two-way behavior at all. To get back to  
the example of TCP and sender-based control, it's absolutely possible  
to have two way communication on a connection there, where one side  
uses say CUBIC congestion control and the other Compound TCP. Each  
side doesn't require any specific behavior from the other, because it  
simply expects the other side to show the simple generic receiver  
behavior. This is what I'd like to see in rmcat.

(again, the situation is the same for the case of receiver-based  
control with a generic, generally agreed upon sender behavior - but  
spreading each congestion controller over both sides ruins this ability)

> all will be both sender and receiver.

See just above, I hope this was clear enough.

> of what the feedback means (in both directions, of course).

I do believe it's necessary (sufficient, no - indeed we'd need a  
clearly defined feedback mechanism). For example, with strict receiver- 
based control, only having a clearly defined feedback mechanism will  
not make flexible use of controls as with TCP possible. Or, in other  
words: having e.g. RRTCC's sender behavior but multiply defined  
receiver behaviors would make this flexibility possible, but having  
multiple sender- AND receiver-behaviors will not.

> and senders to moderate their sending rate to avoid "congestion"

Agreed, but I don't see a problem in doing this with either sender- or  
receiver-based control.

Cheers,
Michael

---

From: Ingemar Johansson S <[email-masked]>
Date: Mon, 8 Oct 2012 07:26:37 +0000

Hi

This brings up old memories (OK, not that old)
I proposed something very 3GPP AMR specific ~6 years ago. 
http://tools.ietf.org/html/draft-johansson-avt-rtp-shim-01 
The message in the AVT WG was then to define a new RTP profile or use RTCP, we ended up with reduced size RTCP (RFC5506)

/Ingemar

-----Original Message-----
From: Michael Welzl [mailto:[email-masked]] 
Sent: den 4 oktober 2012 12:44
To: rmcat WG
Subject: Re: [rmcat] Piggybacking

Hi,

On 3. okt. 2012, at 17:00, John Leslie wrote:

>> Agreed, though Michael indicated that we can piggyback congestion 
>> info on reverse traffic.
>> 
>> This may be possible and I've thought about it.  Part of the problem 
>> is that if you want to signal data back quickly, you have to wait up 
>> to 20ms and in some cases longer to piggyback (and on 
> we aren't entitled to expect any particular packet to be delivered at 
> all.
> 

> expected feedback; and that signal (or the failure to receive our next 
> outgoing packeti) can trigger a retransmission -- catching up 

>   I'd like to suggest that we plan RMCAT around bidirectional (though 
> possibly very small in one direction) media packets. There

> aren't many cases where the traffic is _so_ unidirectional that an 
> "I'm still interested in this stream" isn't appropriate.
> (I don't mean to imply that the packet-rate needs to be the same in 
> both directions, least of all the bit-rate.)
> 
>> (perhaps even estimating when the next chance will be, and the 
>> priority of the data - but that speaks against a dumb receiver).
> 

>> well-connected devices, not edge nodes.  The PacketCable/etc issue 
>> with shared upstreams and slot allocation is an example of reacting 
>> to this - there even with the typically largely downstream flows, the 
>> upstream acks become a limiting factor.
> 
>   Yes!
> 

>>> If feedback is lost due to congestion, more feedback will lead to 
>>> more packet drops. What is true is that when feedback occurs more 
>>> rarely, losing one feedback packet will lead to a greater gap 
>>> between feedback packets.
> 
>> less often), but as Harald says the impact is higher.  And in a 
>> dynamic evaluation this may shift some.
> 
>>>> RRTCC means to reduce the feedback whenever feedback isn't urgent 
>>>> for the sender. I'd argue that, in fact, we should reduce feedback 
>>>> only when the channel carrying it (the backwards channel) is 
>>>> congested, and that is a generic function, not bound to the 
>>>> sender-side congestion control behavior.
> 
> the packet-rate, letting the chips fall where they may; and be slow to 
> increase it again.)
> 
>> Personally, I assume both channels are always congested, or will be 
>> soon (if we're doing our job right and aren't on a huge link),
> 
> and hopefully will have good redundancy algorithms to reconstruct lost 
> packets. Others will choose to stay below the maximum. We shouldn't

---

From: Michael Welzl <[email-masked]>
Date: Mon, 8 Oct 2012 09:47:41 +0200

This gave me an extra thought:

> Second (and this is what I meant in this current discussion), individual packets in a multimedia stream differ in importance. One easy way to say this is that a video consists of e.g. I-frames, P-frames and B-frames, where the priority is I > P > B. In reality, things are more complex: a frame typically spans multiple packets, but there are blocks in frames of different type (e.g. a P-frame may contain I- P- and B-blocks), and then there is also a dependency chain: some blocks may have many other blocks depend upon them, making them more important, and others may have less, making them less important. Honoring these priorities is important for the receiver's subjective quality - here's one out of many, many existing publications documenting this in one way or another:
> Michael Schier, Michael Welzl: "Optimizing Selective ARQ for H.264 Live Streaming: A Novel Method for Predicting Loss-Impact in Realtime", IEEE Transactions on Multimedia 14(2), April 2012.

I don't know how many systems do these sort of thing in practice, but I know that there are lots and lots of papers on such content-aware mechanisms. What they tend to do is either ARQ or FEC, and yes, these things can operate in real-time (also the mechanism in our paper above was designed for that), i.e. they work for interactive media.

For this stuff to work, what is needed at the sender is precise knowledge about which packets have been lost, as quickly as possible. This means that, if we go for a receiver-side-congestion-control-sending-as-little-feedback-as-possible scheme, we should have "special" feedback whenever loss happens: provide fine-grain information about exactly which packets were lost, and probably increase the feedback frequency.

Given that this is about loss on the sender=>receiver path which may not be affected by increased traffic on the receiver=>sender path, I don't see a big problem in doing this. It just complicates the minimum-feedback-due-to-receiver-side-cc. case somewhat, I suppose.

Cheers,
Michael

---

From: Stefan Holmer <[email-masked]>
Date: Mon, 8 Oct 2012 13:43:09 +0200

On Sun, Oct 7, 2012 at 3:17 PM, Michael Welzl <[email-masked]> wrote:

> because they are required by rtcweb (A23 in draft-ietf-rtcweb-use-cases-**and-requirements).
> In case of rtcweb, these priorities are under control of the user or at
> least the web site programmer via the javascript API, and should therefore
> probably be expected to be reasonably long-lived.
>
> Second (and this is what I meant in this current discussion), individual
> packets in a multimedia stream differ in importance. One easy way to say
> this is that a video consists of e.g. I-frames, P-frames and B-frames,
> where the priority is I > P > B. In reality, things are more complex: a
> frame typically spans multiple packets, but there are blocks in frames of
> different type (e.g. a P-frame may contain I- P- and B-blocks), and then
> there is also a dependency chain: some blocks may have many other blocks
> depend upon them, making them more important, and others may have less,
> making them less important. Honoring these priorities is important for the
> receiver's subjective quality - here's one out of many, many existing
> Michael Schier, Michael Welzl: "Optimizing Selective ARQ for H.264 Live
> Streaming: A Novel Method for Predicting Loss-Impact in Realtime", IEEE
> Transactions on Multimedia 14(2), April 2012.
> http://ieeexplore.ieee.org/**xpls/abs_all.jsp?arnumber=**6095636<http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6095636>
>
> Suffice to say, we can have different priorities per packet as a function
> of the multimedia encoding, and these priorities could quickly change (this
> is a way to represent the dependence - if the sending application is in
> control of its data until the last moment, and if it is informed about the
> loss of packet X, it could quickly assign a very low priority to packets
> that depend on X.
>
> accordance with the charter and rtcweb requirements. I am arguing that it
> would be great if we could support them.
>
> What I mean with supporting them via congestion control is: going back to
> the overly simplistic example of I- P- and B-"packets", one may decide to
> be less aggressive when the buffer is full of "B-packets" and more
> aggressive when it is full of "I-packets". One out of many possibilities...
> my point is, such functionality should be enabled by handing over the right
> information to the congestion controller, and this requires the controller
> to be close to the send buffer, or the information to be signalled to the
> other side. Both can be okay, but we should be clear about this.

You want to allow the sender to, for a short time, override the bandwidth
estimate by sending at a higher rate because it has important packets to
send? Isn't this possible even though the estimator is at the receiver? I'm
also not sure that is something you want to do. Sending important packets
at a higher rate increases the risk of losing those important packets,
which is the opposite of what you want, right?

> Let's leave it at that, this bit doesn't really matter for the discussion
> anyway.
>
>
>
> I'm arguing that we should have either this, or strictly
> only-receiver-based control. Then we could get the same functionality: you
> could just change the receiver to use a new mechanism X, without waiting
> for the sender to support the new mechanism X or even signalling to the
> Note that this doesn't exclude two-way behavior at all. To get back to the
> example of TCP and sender-based control, it's absolutely possible to have
> two way communication on a connection there, where one side uses say CUBIC
> congestion control and the other Compound TCP. Each side doesn't require
> any specific behavior from the other, because it simply expects the other
> side to show the simple generic receiver behavior. This is what I'd like to
> see in rmcat.
>
> (again, the situation is the same for the case of receiver-based control
> with a generic, generally agreed upon sender behavior - but spreading each
> congestion controller over both sides ruins this ability)
>
>
>
> I do believe it's necessary (sufficient, no - indeed we'd need a clearly
> defined feedback mechanism). For example, with strict receiver-based
> control, only having a clearly defined feedback mechanism will not make
> flexible use of controls as with TCP possible. Or, in other words: having
> e.g. RRTCC's sender behavior but multiply defined receiver behaviors would
> make this flexibility possible, but having multiple sender- AND
> receiver-behaviors will not.
>
>
>

---

From: Michael Welzl <[email-masked]>
Date: Mon, 8 Oct 2012 13:54:01 +0200

On 8. okt. 2012, at 13:43, Stefan Holmer wrote:

> Very sorry for any disadvantages that might have come from having expressed myself unclear. I'll try to clean things up in this email now:
> 
> 
> 
> Indeed, in rmcat we're facing two kinds of priorities. One, priorities between streams. I think we *must* be able to support these priorities because they are required by rtcweb (A23 in draft-ietf-rtcweb-use-cases-and-requirements). In case of rtcweb, these priorities are under control of the user or at least the web site programmer via the javascript API, and should therefore probably be expected to be reasonably long-lived.
> 
> Suffice to say, we can have different priorities per packet as a function of the multimedia encoding, and these priorities could quickly change (this is a way to represent the dependence - if the sending application is in control of its data until the last moment, and if it is informed about the loss of packet X, it could quickly assign a very low priority to packets that depend on X.
> 
> Supporting these priorities in rmcat doesn't seem to be a MUST in accordance with the charter and rtcweb requirements. I am arguing that it would be great if we could support them.
> 
> What I mean with supporting them via congestion control is: going back to the overly simplistic example of I- P- and B-"packets", one may decide to be less aggressive when the buffer is full of "B-packets" and more aggressive when it is full of "I-packets". One out of many possibilities... my point is, such functionality should be enabled by handing over the right information to the congestion controller, and this requires the controller to be close to the send buffer, or the information to be signalled to the other side. Both can be okay, but we should be clear about this.
> 
> You want to allow the sender to, for a short time, override the bandwidth estimate by sending at a higher rate because it has important packets to send? Isn't this
> possible even though the estimator is at the receiver? I'm also not sure that is something you want to do. Sending important packets at a higher rate increases the risk of losing those important packets, which is the opposite of what you want, right?

That's a possibility; I was actually thinking of the exact opposite - to be overly cautious when you only have low-priority packets to send. Tune your filter to say "yes there's congestion, don't increase" when it might actually be noise in the measured delay signal in these conditions. 

Yes this can also be done when the controller is at the receiver, but to do such a thing you need to inform the controller about the current state of the sender buffer (how many low-, medium- and high-priority packets are waiting there) => more signalling. I'm not saying this is a big problem, but it's probably another weight in the "minus" part of the scale. Either way, it's just something to keep in mind, I think.

Cheers,
Michael

---

From: John Leslie <[email-masked]>
Date: Mon, 8 Oct 2012 09:19:40 -0400

> Maybe you're considering one type of priority and me another.

   Indeed!

> because they are required by rtcweb (A23 in
> draft-ietf-rtcweb-use-cases-and-requirements).
] 
] A23             The Web API MUST provide means for the
]                 application to specify the priority to
]                 apply for outgoing streams and data.

   Agreed.

> In case of rtcweb, these priorities are under control of the user or
> at least the web site programmer via the javascript API, and should
> therefore probably be expected to be reasonably long-lived.

   Yes.

> frames and B-frames, where the priority is I > P > B.

   Hmmm...

   One should tread carefully here. Most video codecs are designed to
expect in-order delivery; and priorities can mess that up.

   (Or are you actually talking about trying to jiggle probability-of-
loss, rather than what goes first?)

> In reality, things are more complex: a frame typically spans multiple
> packets, but there are blocks in frames of different type (e.g. a
> P-frame may contain I- P- and B-blocks),

   You're starting to lose me... :^(

   There are a heck of a lot of video codec standards. Are you referring
to one in particular?

   I think of an I-frame as a picture complete in itself, and both
P-frames and B-frames as trying to describe motion between frames of
the original video signal -- the difference being that P-frames refer
to previous I-frames while B-frames refer to following I-frames as well.
(This in itself is enough to make my head ache!)

   I'm reduced to guesswork to try to attach a meaning to "P-block" and
"B-block"... I'd rather not go too far into guesswork...

> and then there is also a dependency chain: some blocks may have many
> other blocks depend upon them, making them more important, and others
> may have less, making them less important. Honoring these priorities
> is important for the receiver's subjective quality

   That sounds like probability-of-loss...

> - here's one out of many, many existing publications documenting this
> in one way or another:

   Many of us won't have time to access and read this...

> assign a very low priority to packets that depend on X.

   I have been assuming the sender _will_ be in control of what goes
into each packet sent (in other words, the sender will compose
packets, not streams).

   But you're losing me at "assign a very low priority"... There are
many things that could mean.

   The most obvious is QoS bits. I dislike that because it _asks_ to
mess with order of delivery.

   I won't list any other meanings -- this email is complicated enough
already...

> it would be great if we could support them.

   To the extent you mean in what order the sender packs the pieces
into outgoing packets, I very much want to support that. If you mean
some "priority" mechanism other than QoS bits attached to packets, I'm
all ears to learn of it...

> we should be clear about this.

   Philosophically, I'm more friendly to congestion-control-at-sender;
but I still don't think we need to settle that yet.

>> Are you arguing we should give up considering receiver-based-control?
> 

   This is a bit convoluted... I agree we could accomplish what we need
with either receiver-based or sender-based congestion control; thus I
don't see the need to settle it yet.

   I disagree that we can't do it with a mix (though at first blush
that would seem a very poor choice).

>> I _do_ believe we need to establish a feedback mechanism with a clear
>> definition of what the feedback means (in both directions, of course).
 
> clearly defined feedback mechanism).

   :^)

> For example, with strict receiver-based control, only having a
> clearly defined feedback mechanism will not make flexible use of
> controls as with TCP possible. Or, in other words: having e.g.
> RRTCC's sender behavior but multiply defined receiver behaviors
> would make this flexibility possible, but having multiple sender-
> AND receiver-behaviors will not.

   I think you've lost most of our readers here. :^(

   Presumably you mean:

http://tools.ietf.org/id/draft-alvestrand-rtcweb-congestion-02.txt

I _could_ follow the math there, had I more time to spare... I'm
guessing the critical part comes in section 3.4:
] 
] 3.4.  Rate control
] 
]  The rate control at the receiving side is designed to increase the
]  receive-side estimate of the available bandwidth A_hat as long as the
]  detected state is normal.  Doing that assures that we, sooner or
]  later, will reach the available bandwidth of the channel and detect
]  an over-use.
]...

any you're saying A_hat isn't sufficient to the purpose. (I don't even
see why that's necessarily true, unless you mean that any intelligence
whatsoever at sender-side is forbidden.)

---

From: Michael Welzl <[email-masked]>
Date: Mon, 8 Oct 2012 21:24:55 +0200

> loss, rather than what goes first?)

Well, there is only so much you can squeeze through a pipe. When what  
you'd like to send is more than the network allows, a decision has to  
be taken, and that should be done on priorities.
In a way that's similar to a loss - it's a decision on the sender  
side, to not send certain packets in favor of others.

>   There are a heck of a lot of video codec standards. Are you  

AFAIK many codecs are based on MPEG in one way or another. This basis  
is what I'm referring to.

> "B-block"... I'd rather not go too far into guesswork...

A block is just a part of a frame. Anyway, as I said in my previous  
email, for simplicity it's enough to think about I- vs. P- vs. B- 
frames, exactly as you describe them here.

>   That sounds like probability-of-loss...

... in the sender. See above.

>   Many of us won't have time to access and read this...

Not needed to get my point. It's just one out of many examples, to  
clarify that honoring such priorities can make a pronounced quality  
difference.

> into each packet sent (in other words, the sender will compose

That's good. I hope the sender will be in control until the very last  
moment in actual implementations coming out of rmcat. This is not as  
simple and obvious as it may sound, I think.

>   I won't list any other meanings -- this email is complicated enough

Let's stay with the I-, P-, B-frame model. That makes priority 1, 2,  
3. I am then saying that priorities could be changed to be 3. My  
context is the send buffer, influencing the congestion control  
behavior but nothing else. Nothing on the wire. I'm not trying to make  
things THAT complicated!  :-)

> all ears to learn of it...

I mean: if the sender's buffer is full of 1-priority packets, be more  
aggressive. If the sender's buffer is full of 3-priority packets, be  
less aggressive.
This can be implemented in more or less reasonable ways. One, to me,  
reasonable method is what I described in the email answering Stefan:  
only implement low priorities, by being overly cautious in increasing  
the rate when your buffer is full of low-priority packets. ("Overly  
cautious" has a bad sound to it, but in fact any delay-based mechanism  
must be based on some parameters which can never be ideal - always  
trade-offs there, and I'm merely suggesting to tune these trade-offs  
based on what is in the sender buffer). Here's another example:
RRTCC: http://tools.ietf.org/id/draft-alvestrand-rtcweb-congestion-02.txt
has the sender send at least as much as one standard TCP would, using  
the TCP equation. With the MulTCP equation:
http://heim.ifi.uio.no/~michawe/research/projects/multfrc/index.html
one could easily tune this to send at least as much as N standard  
TCP's, where N could for example be 1.5 or 0.8. N could depend on the  
number of priority-1, priority-2 or priority-3 packets waiting to be  
transmitted.

> that would seem a very poor choice).

I don't understand how a mix would be as easily deployable as a one- 
sided approach.

> any you're saying A_hat isn't sufficient to the purpose. (I don't even

Err... now you're losing me. I think your interpretation of what I'm  
saying is way more complicated than what I mean  :-)
Sorry if I'm not clear enough, I promise I'm doing my best here...

> whatsoever at sender-side is forbidden.)

I mean that any mechanism-specific intelligence at sender-side should  
be forbidden, if we settle for receiver-based congestion control.  
That's what I mean with "generic": not mechanism-specific.
RMCAT will perhaps standardize 3 mechanisms. A, B and C. Say each is  
split into a sender and receiver behavior: A_send and A_recv, B_send  
and B_recv, C_send and C_recv. Say they are not compatible, i.e.  
A_send cannot talk to B_recv, and C_send cannot talk to A_recv and so  
on. Then, for any mechanism X to work, you need both X_send and X_recv  
to be in place.

Say, we decide to do receiver-based congestion control. Then I argue  
that we should agree on G_send, which is a generic sender behavior.  
Then, A, B and C only consist of A_recv, B_recv and C_recv, and you  
always only need to deploy this receiving end to use the new mechanism.

Symmetrically, the same is true for pure sender-based congestion  
control. It's a mix of the two that's hard to deploy.

Cheers,
Michael

---

From: John Leslie <[email-masked]>
Date: Tue, 9 Oct 2012 09:06:40 -0400

> be taken, and that should be done on priorities.

   I'm not sure "priorities" is a good name for this -- it's allowed me
to get confused with packet-forwarding priorities.

>> There are a heck of a lot of video codec standards. Are you  
>> referring to one in particular?
> 

   That's probably a good family of codecs to think in terms of; but
we might find ourselves working with something different. Not
surprisingly, "Moving Pictures Experts Group" concentrated on motion,
while we will have uses where motion is limited...

> simple and obvious as it may sound, I think.

   It may indeed not be "simple" for web-browsers in general. Our
charter calls for:
" 
" - Find or develop candidate congestion control algorithms, verify that
" these can be tested on the Internet without significant risk, and
" publish one or more of these as Experimental RFCs.

which I think gives us sufficient wiggle-room for demonstrating a
congestion control algorithm that not all web-browsers can initially
support.

   Nonetheless, I see sender control of what gets packed into packets
as essential to our eventual success; and I think that browsers can
evolve to escape the TCP-stream paradigm.

>> many things that could mean...
> 

   :^)

   In some sense, you seem to be talking of pre-determining x-frame
loss, rather than waiting for the network to drop something. "Priority"
IMHO isn't a good word for that; though it may turn out to be a very
good paradigm.

   Fundamentally, under "congestion control" we want to back off
sending bit-rate when congestion is detected, and ramp it back up
when congestion appears to have cleared. Packing less into the packets
we send is certainly an appropriate mechanism.

   "Priority" may be a good term for academic types to describe
policies for which frames don't get sent, but I'm afraid that to
operations types it brings too much baggage. Also, "priority" suggests
we're talking about a scalar, when IMHO the problem space is multi-
dimensional.

> aggressive. If the sender's buffer is full of 3-priority packets, be  

   This deserves its own thread. (I agree it's a promising path.)

> the rate when your buffer is full of low-priority packets.

   (I have gone beyond this, to suggest maintaining two rates: a
slower one for less important packets and a faster one for the most
important packets; both to be reduced when congestion is detected.
This may deserve discussion in some other thread...)

> ("Overly cautious" has a bad sound to it, but in fact any delay-based
> mechanism must be based on some parameters which can never be ideal -
> always trade-offs there, and I'm merely suggesting to tune these
> trade-offs based on what is in the sender buffer).

   This may not be the easiest thing to model... :^(

   The way I think about it is that we want to avoid a situation where
_our_ packets are growing the bit-rate at the bottleneck too quickly.
There are plenty of uses out there which deserve to be called greedy:
where every second of delay in delivering a fixed-size web-page
reduces the user experience. For these, multiple TCP streams are "the"
way to go; but this leads to overshoot and congestion more severe
than "should" be needed.

   Thus, by the "first, do no harm" principle, we should avoid driving
a bottleneck into severe congestion. In trade, we should be able to
back off a bit more slowly when congestion _is_ encountered. Our true
aim is to deliver a _consistent_ user experience -- not an "optimized"
delivery of a fixed-size message.

> Here's another example:

   At first blush, that seems a reasonable way to model the situation;
but I suspect we can do better. N>1 TCP is a big part of the problem...

> I don't understand how a mix would be as easily deployable as a one- 

   I'm not talking about "ease of deployment". We need first to
establish we have something worth deploying.

>>... Presumably you mean:
>> and you're saying A_hat isn't sufficient to the purpose. (I don't
>> even see why that's necessarily true, unless you mean that any
>> intelligence whatsoever at sender-side is forbidden.)
> 

   I don't agree that would be helpful.

> on. Then, for any mechanism X to work, you need both X_send and X_recv  

   Logically true; but not particularly interesting...

   We're chartered to explore Experimental protocols. There's every
reason to believe when the time comes for Proposed Standard, we will
have learned from the experiments. IMHO, at this stage it's more
important to design good experiments than worry about deployment of
something not yet designed.

---

From: Michael Welzl <[email-masked]>
Date: Tue, 9 Oct 2012 15:38:44 +0200

> to get confused with packet-forwarding priorities.

I understand; sorry. I herewith coin the term "Packet  
Importance" (PI)   :-)

> while we will have uses where motion is limited...

Different importance of different parts of media is a natural  
occurrence if you want to compress, really. And compression is a  
natural occurrence if you want to squeeze, say, video through the  
network.
We might not only consider video, yes, but for cases where you have  
similar-importance packets, well, you have no problem. It's just that  
WHEN different PIs (defined as per above) occur, it can be useful to  
let the congestion controller know about them.

> evolve to escape the TCP-stream paradigm.

I'm completely with you here.

> IMHO isn't a good word for that; though it may turn out to be a very

PI hereinafter  :-)   and "pre-determining x-frame loss" is a rather  
complicated phrase for what really is: "reducing the send rate".  
Adaptive multimedia.

> we're talking about a scalar, when IMHO the problem space is multi-

Nononononono please! A scalar please!!!
I herewith very very quickly define that "Packet Importance" is a  
scalar with values 1, 2 and 3. Not multiple dimensions!!!

> This may deserve discussion in some other thread...)

Ok about another thread. About your suggestion: another possibility...  
there are many. All I'm saying is that such stuff could be done, and  
would probably be beneficial, and for that to be possible the  
congestion controller must be informed about the PIs of packets in the  
buffer. Which, in case of sender-side control, is straightforward, and  
in case of receiver-side control, needs signalling.

> but I suspect we can do better. N>1 TCP is a big part of the  

Our equation also allows for N<1 TCP  :-)    Like, 0.5  :-)

> something not yet designed.

I disagree on both accounts.
1) About the importance: the IETF has standardized a wealth of  
undeployed / undeployable stuff, e.g. DCCP.
2) About the design aspect: I don't see any notable benefit from  
having a per-mechanism sender *and* receiver behavior, but, as stated,  
a significant deployment problem with two-sided approaches. And I  
believe it's important to get these major architectural decisions  
right from the outset.

But I don't believe we will come to an agreement here, and have yet  
converged to a point where we can probably just leave it at that, for  
now  :)

Cheers,
Michael

---

From: Randell Jesup <[email-masked]>
Date: Tue, 09 Oct 2012 16:04:08 -0400

> side, to not send certain packets in favor of others.

Except in most cases we're not sitting with a pile of frames/packets and 
deciding which ones get dropped (liek a router); we're encoding data to 
meet the current-moment bandwidth requirements.  There is a separate 
component of feeding data out onto the wire, which would be applied to 
non-encode-controlled streams like a SCTP/DTLS DataChannel sharing the 
pipe.  For encoder-based streams, we generally want to control via the 
encoder not by inducing possible internal delay in how we select packets 
for transmission.

If the data doesn't fit in the current pipe estimate, we shouldn't be 
generating it in the first place.  And intermediate nodes don't know our 
priorities to adjust probability of drop except maybe indirectly via 
markings, which may instead mess with order of delivery (not a good thing).

> email, for simplicity it's enough to think about I- vs. P- vs. 
> B-frames, exactly as you describe them here.

Most packets are at a much higher level than blocks, so barring 
considerable swizzling of internal order it's likely that all packets 
for a frame should have similar priority and likelyhood of drop.

> ... in the sender. See above.

We should never drop in the sender. :-)

> control. It's a mix of the two that's hard to deploy.

Not that I'm arguing for it at this moment: but you could design things 
so that 'generic'/dumb mechanisms are available to support either sender 
or receiver-based control, with the decision on which done by 
negotiation or even in-band decision.  It simply requires defining both, 
and defining a way to select the one that's non-default.

This would allow innovation at either end in the future.

As to whether this is useful... I haven't given it the thought it 
deserves yet.  But it's quite doable.

(And on a separate note, a 'generic'/dumb mechanism doesn't mean there 
can't be feedback from the other side affecting how it operates, or that 
it can't be a more complex state machine than "send one packet per 
packet received").  Just that it be relatively simple and well-defined.  
(A bit like the whole video-codec "we'll define the quite-complex 
decoder and let the encoder innovate")

-- 
Randell Jesup
[email-masked]

---

From: Michael Welzl <[email-masked]>
Date: Wed, 10 Oct 2012 18:10:18 +0200

On Oct 9, 2012, at 10:04 PM, Randell Jesup wrote:

>> Well, there is only so much you can squeeze through a pipe. When  
>> what you'd like to send is more than the network allows, a decision  
>> has to be taken, and that should be done on priorities.
> Except in most cases we're not sitting with a pile of frames/packets  
> and deciding which ones get dropped (liek a router); we're encoding  
> data to meet the current-moment bandwidth requirements.  There is a  
> separate component of feeding data out onto the wire, which would be  
> applied to non-encode-controlled streams like a SCTP/DTLS  
> DataChannel sharing the pipe.  For encoder-based streams, we  
> generally want to control via the encoder not by inducing possible  
> internal delay in how we select packets for transmission.
>
> If the data doesn't fit in the current pipe estimate, we shouldn't  
> be generating it in the first place.  And intermediate nodes don't  
> know our priorities to adjust probability of drop except maybe  
> indirectly via markings, which may instead mess with order of  
> delivery (not a good thing).

I agree about marking, or at least I'd like to keep this a separate  
issue, so I ignore your last sentence above.

About the rest of what you say: I guess that may be case-dependent? I  
would think that always generating exactly the right amount, as you  
describe, is an ideal case - but sometimes you may not be able to get  
exactly that amount, and then you will have to have a (of course, very  
small) send buffer, and then honoring the importance of packets in  
this buffer is useful.

BUT I am waving my hands here. Note that the case I previusly dealt  
with, this Ph.D. thesis:
http://heim.ifi.uio.no/~michawe/research/students/michael_schier-diss_final.pdf
... is about real-time communication all right, but for real-time one- 
way TV transmission with interactive controls, having timing  
constraints which are probably not as strict as e.g. in WebRTC. Maybe  
we're only interested in codecs that would give you just the right  
amount of data at the right moment. I don't know (after all, rmcat is  
not exclusively about WebRTC too). I thought that we could probably  
have such situations with a short send buffer, and then honoring the  
importance of packets in this buffer would be useful, that's all.

> considerable swizzling of internal order it's likely that all  
> packets for a frame should have similar priority and likelyhood of  
> drop.

That depends on various factors and is not always true. Going back to  
the Ph.D. thesis I linked to above, and papers that have come out of  
it, in this case we had a clearly distinguishable importance for  
packets within the same frame, depending on block and macroblock  
types, their dependencies and such. I guess this will greatly depend  
on video configuration and whatnot.

> We should never drop in the sender. :-)

That might have to depend on the granularity of send rates the codec  
produces. But again, I'm waving my hands.

>> I mean that any mechanism-specific intelligence at sender-side  
>> should be forbidden, if we settle for receiver-based congestion  
>> control. That's what I mean with "generic": not mechanism-specific.
>> RMCAT will perhaps standardize 3 mechanisms. A, B and C. Say each  
>> is split into a sender and receiver behavior: A_send and A_recv,  
>> B_send and B_recv, C_send and C_recv. Say they are not compatible,  
>> i.e. A_send cannot talk to B_recv, and C_send cannot talk to A_recv  
>> and so on. Then, for any mechanism X to work, you need both X_send  
>> and X_recv to be in place.
>>
>> Say, we decide to do receiver-based congestion control. Then I  
>> argue that we should agree on G_send, which is a generic sender  
>> behavior. Then, A, B and C only consist of A_recv, B_recv and  
>> C_recv, and you always only need to deploy this receiving end to  
>> use the new mechanism.
>>
> Not that I'm arguing for it at this moment: but you could design  
> things so that 'generic'/dumb mechanisms are available to support  
> either sender or receiver-based control, with the decision on which  
> done by negotiation or even in-band decision.  It simply requires  
> defining both, and defining a way to select the one that's non- 
> default.
>

This is an interesting idea! Not sure if it's useful either. Perhaps?

> (And on a separate note, a 'generic'/dumb mechanism doesn't mean  
> there can't be feedback from the other side affecting how it  
> operates, or that it can't be a more complex state machine than  
> "send one packet per packet received").  Just that it be relatively  
> simple and well-defined.  (A bit like the whole video-codec "we'll  
> define the quite-complex decoder and let the encoder innovate")

Yes, agree 100%

Cheers,
Michael

---

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 11 Oct 2012 13:56:10 +0200

Hi,

skipped the part on (intra-flow packet) priorities because that a topic on its 
own. 

> decoder and let the encoder innovate")

I want to note that the algorithm design is mostly independent of this 
question who sends feedback to whom. A certain algorithm should be 
implementable at sender or receiver side. But as soon as you have one 
specific algorithm you might be able to analyze which implementation causes 
how much feedback to send. 

I personally agree that it would be nice to have on one side a very simple and 
well defined state maschine (dumb side) such that only one side needs to be 
changed to deploy new algorithms. But actually having this simple state 
maschine for both sides and then negotiate at the beginning who will do the 
actual control would be an option.

For my understand in TCP the reason why sometmes a receiver control would be 
more useful, is because the receiver might be close to be bottleneck and thus 
might be able to use additional information on the bottleneck (without 
additional signaling and signaling delay to the sender). In case of 
interactive audio/video up- and downstream traffic between to home costumers 
it might be not that clear where the bottleneck is...

Mirja

---

From: Michael Welzl <[email-masked]>
Date: Thu, 11 Oct 2012 14:05:16 +0200

Hi,

Just saying, I agree 100% with every word here.

Cheers,
Michael

On Oct 11, 2012, at 1:56 PM, Mirja Kühlewind wrote:

>>> should
>> things
>> there

---

From: Michael Welzl <[email-masked]>
Date: Thu, 11 Oct 2012 14:19:09 +0200

Hi all,

Just a quick thought, below, quoting my own old email again (sorry):

On Oct 4, 2012, at 12:58 PM, Michael Welzl wrote:

> congestion control question, with one (in my opinion) significant  
> plus in each camp:
>
>> 2) simply always sending as little feedback as possible, based on  
>> the (forward) congestion control mechanism's requirements
>>
>> If we think of e.g. a videoconference, indeed congestion from all  
>> the feedback on the uplink is quite realistic. 1) can then become  
>> rather complicated... you may get congestion on your uplink, and it  
>> might be necessary to reduce the rate of your data stream, or it  
>> might suffice to reduce it on some of the feedback streams, but  
>> which? And then we have to apply priorities between feedback  
>> packets and your payload...
>>
>
>> 2) seems simpler, but it has the problem of making the mechanism  
>> more fragile, and perhaps unnecessarily so (in a situation where it  
>> might be fine to send much more feedback). I say "unnecessarily"  
>> because I'm not convinced about "both channels are always  
>> congested, or will be soon" because codecs may not be greedy, and  
>> streams such as VoIP may simply not generate enough traffic for that.

This increased fragility is, to me, the biggest problem of 2) above  
(feedback minimization via receiver-based control).

But: I think this could largely be compensated for if we make that  
feedback reliable (a la ACKs-of-ACKs).
What I mean is: a sender could piggyback a short message in its data  
packets to the receiver, to acknowledge the reception of feedback. If  
a receiver only sends feedback very rarely, it could use a quite  
aggressive timeout to send the feedback again if it doesn't get an  
acknowledgement fast.

Yes, this message would be piggybacked onto an unreliable data stream,  
but such problems have been fixed before (e.g. TCP's ECN-ECE-CWR  
sequence).

What does everyone think? Good idea, bad idea?
(yes, I know, such stuff would ideally be discussed in the context of  
the RTCP extensions draft, but I clearly don't know enough about RTP/ 
RTCP to lead this effort... happy to contribute though!)

> 2), i.e. receiver-side congestion control, also has the ADVANTAGE  
> that it doesn't misinterpret the loss of feedback as an indication  
> of forward congestion. A receiver-side controller has a clear notion  
> of all packets that have arrived in an interval and knows exactly  
> what was dropped on the way from the sender to the receiver and what  
> wasn't.
>
> mechanism. With a receiver-side scheme, this requires richer  
> feedback (in the style: "for high-priority packets: this rate; for  
> low-priority and high-priority packets combined: that rate"). Then,  
> the receiver would have to know how many priorities are there on the  
> sender side to give the right amount of feedback...  this will make

---

From: John Leslie <[email-masked]>
Date: Fri, 12 Oct 2012 06:41:02 -0400

Michael Welzl <[email-masked]> wrote:
>... 
>>> The way I see it, there's a choice between:
>>>    backwards congestion, by doing congestion control on the
>>>    backwards channel like ACK-CC

   I'm not commenting on this choice!

> (feedback minimization via receiver-based control).

   (I'm not commenting on that either.)

> feedback reliable (a la ACKs-of-ACKs).

   We should be careful about using the term "reliable".

> packets to the receiver, to acknowledge the reception of feedback.
> If a receiver only sends feedback very rarely, it could use a quite  

   This is, indeed, possible.

   But since we're inherently talking "media" here, there will inherently
be a fairly-regular succession of packets from "sender" to "receiver".
Keeping an event-counter and always sending a few low-order bits of that
counter gives us a better approximation of what we want than any attempt
at "reliable" by re-transmission.

   (I also claim that many of our use-cases will involve media in both
directions, and that the remainder deserve periodic "I'm still listening"
messages; so there is always a packet stream to attach these low-order
bits to, although the packet rates may be quite different in the two
directions.)

> but such problems have been fixed before (e.g. TCP's ECN-ECE-CWR  

   ECN-ECE-CWR is doing essentially the event-counter sending one
low-order bit. I prefer to think of the general case, and recommend
two or three low-order bits at least. (Bits just aren't as scarce today
as they were ten years ago -- heck, RFC3168 was trying to squeeze into
a 1981 straitjacket!)

> What does everyone think? Good idea, bad idea?

   I prefer the always-send-counter approach to reliable-by-retransmit.

   I have no objection to _also_ sending an immediate event-report,
but I'm not sure it's worth the trouble. We care about avoiding
self-congestion; but that's mostly a matter of avoiding the TCP habit
of ramping-up too quickly. Depending on an unreliable event report to
tune the ramp-up is a suboptimal design.

> RTCP to lead this effort... happy to contribute though!)

   Hint to chairs: I think Michael would be a good Document Editor for
this...