Subject: [rmcat] About the importance of packets in a single data stream

From: Michael Welzl <[email-masked]>
Date: Sat, 12 Oct 2013 08:11:59 +0200

Hi,

I'll begin by apologizing for asking what I'm quite sure is a terribly stupid question to many of you.

The question is:

It has long been known that media data can have different levels of importance. Simply put, if packet #1 from a single source contains parts of a video's I-Frame and packet #2 contains only parts of B-Frames, and both end up in the same queue, controlled by an AQM (for instance), it would be better for the video stream if packet #2 would be dropped rather than packet #1.

There is nothing new with this story; it's not hard to find research papers that document various variations of this theme, showing benefits in video quality.

My question is, why is none of this happening?

Is it because DSCP values are typically associated with sources, and hence, marking packet #2 as "less" important would put the source at the risk of having its packets less important than not only its own other packets, but anybody else's?  But there is equipment that does per-connection stuff, and such things could probably better be done near the edges, where the bottleneck typically is... so if that's the whole issue, we could define DSCP values that mean relative importance *within the same five-tuple only*. Surely that has been thought about and probably proposed by folks before, so what happened? Why isn't it done?

Or is it because per-five-tuple-functionality in the network is regarded as being too costly, and not encouraged, and hence not standardized?

I'm just trying to understand the reasons for this particular long-standing difference between research an reality.

Cheers,
Michael

---

From: Varun Singh <[email-masked]>
Date: Sat, 12 Oct 2013 09:53:31 +0300

Hi Michael,

In my experience, Video telephony applications do not generate any
b-frames. So #2 packets do not exist. Secondly some generate I-frames
only when they really have to (scene change or decoding packet chain
is corrupted) otherwise their GOP is set to infinity.

 For sending I-frames, there are many tricks that a sending endpoint
can play, it doesn't need to burst the whole frame immediately, it can
pace out (there are 10s of milliseconds) before the next frames is
generated. Alternatively, if the frame is too large, they can send
parts of the I-frame with the subsequent frames, this potentially
delays rendering. I don't know if any of these have been standardized.

If a part of a frame is lost, the receiver can send a NACK or packet
loss indication Or reference picture selection.

Another way is that the sending point uses unequal level of protection
(ULP) ie., not all packets or frames are FEC'ed.
Lastly it can vary the packet size, ie., it may do this by varying the
size of slices (combinations of macro blocks).

Applicability of ARQ typically depends on observed end-to-end delay
and observed losses. The endpoint won't send a nack or a
retransmission if there are too many losses or if it thinks it won't
reach in time for decoding.

Regards,
Varun

----
http://www.netlab.tkk.fi/~varun

> On 12.10.2013, at 9.12, Michael Welzl <[email-masked]> wrote:
>
> Hi,
>

---

From: Michael Welzl <[email-masked]>
Date: Sat, 12 Oct 2013 10:28:48 +0200

On 12. okt. 2013, at 08:28, Mikael Abrahamsson <[email-masked]> wrote:

> On Sat, 12 Oct 2013, Michael Welzl wrote:
> 
> My guess is that the rule to always deliver packets within a stream in order. If packets within the same stream have different DCSP values, there is a risk that they will go into different queues and they might be delivered out of order. A lot of applications that do real-time will have a PDV buffer, but won't have much of a out-of-order buffer. If a packet with sequence number 10 arrives after sequence number 8, packet number 9 is often assumed to be lost.
> 
> AQM is not widely deployed at all on the Internet as it is today, and a lot of devices will just have 4 different queues. Also as you say below, putting packets into different queues might mean they compete with other traffic and you might get packet loss due to that.
> 
> It would of course be of interest going forward to look into your proposal. AQM/Bufferbloat movement could perhaps benefit from this, but the current thinking is to make sure that all streams get equal access to the media with minimal influence between them, not that much to decide what packets within a stream to select for drop.

Thanks, that's about what I guessed... strange, a bit, given that it's not hard to show quite large quality gains if packet priorities within a stream (five-tuple) are honored, and doing per-5-tuple things near the edge doesn't seem all too unusual to me...

So indeed it wasn't a proposal, I was just wondering why it's not done. Maybe not enough interest from the parties involved (because application developers are not usually the same people that build middle-boxes)

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Sat, 12 Oct 2013 10:32:50 +0200

> is corrupted) otherwise their GOP is set to infinity.

But packet importance is more than just I- vs. P- vs. B-Frames - there are blocks that depend on each other, and the more data depends on data in a packet, the more important that packet gets.

> delays rendering. I don't know if any of these have been standardized.

Probably no need to standardize this?

> reach in time for decoding.

Yes - the choices are really:
1. add FEC for the more important packets
2. do ARQ when applicable for the more important packets
3. somehow tell the network about the more important packets  (but you can't, normally)

and there are trade-offs between these choices. I just wonder if we need 3. as a part of the API to transport?

Cheers,
Michael

---

From: Varun Singh <[email-masked]>
Date: Sat, 12 Oct 2013 11:49:22 +0300

Hi Michael,

> But packet importance is more than just I- vs. P- vs. B-Frames - there are blocks that depend on each other, and the more data depends on data in a packet, the more important that packet gets.

At the moment, some systems use unequal level of protection (UEP) to protect
the packets they think are more important, I suppose what you are proposing is
to add a kill bit in the packets that are less important.

I think there is a tradeoff, redundancy causes overhead (which the
endpoint should
control) and adding priorities pushes the responsibility to the
middlebox and in both
the mechanisms, the sender should probably adapt its rate based on
input from the receiver.

> Probably no need to standardize this?

Depends where this is done, at the RTP level or elsewhere. If at the RTP level
there probably needs to be some guidance on how the bits are fed to the decoding
pipeline from the RTP layer.

> and there are trade-offs between these choices. I just wonder if we need 3. as a part of the API to transport?

when you say transport, you mean DSCP codepoint to indicate to the
middleboxes along the path or
an API from the codec to the transport (RTP/UDP) at the sending endpoint?

-- 
http://www.netlab.tkk.fi/~varun/

---

From: Michael Welzl <[email-masked]>
Date: Sat, 12 Oct 2013 11:43:12 +0200

> to add a kill bit in the packets that are less important.

Exactly...

> the mechanisms, the sender should probably adapt its rate based on

Exactly

> an API from the codec to the transport (RTP/UDP) at the sending endpoint?

I was really only thinking aloud - but what I was thinking about was indeed a DSCP codepoint that is set because the codec has told the transport about it, in the API, defined by RMCAT.

Cheers,
Michael

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 12 Oct 2013 14:00:05 -0700

[BA] A few points: 

a. No B-frames in interactive uses, as others  have said. 
b. With SVC, you can apply FEC or RTX only to the base layer.   So if there is loss in the base, you can recover.  Loss in enhancement layers may be "lived with" by dropping down to base frame rate/resolution/quality.  So not essential to apply FEC, RTX (or QoS) to enhancement layers, especially since if there is high loss in enhancement layers a logical response is to stop sending them (duh).

---

From: Michael Welzl <[email-masked]>
Date: Sun, 13 Oct 2013 10:01:33 +0200

> b. With SVC, you can apply FEC or RTX only to the base layer.   So if there is loss in the base, you can recover.  Loss in enhancement layers may be "lived with" by dropping down to base frame rate…

I see. Well, should we perhaps really suggest a "lower-priority-than-other-packets-in-the-same-5-tuple" DSCP then?

Cheers,
Michael

---

From: Kevin Gross <[email-masked]>
Date: Sun, 13 Oct 2013 09:29:13 -0600

There have been mechanisms proposed or implemented to some extent in
Ethernet, ATM and IP for indicating drop precedence (as opposed to class).
These mechanisms are designed to solve exactly this problem. I'm not sure
why this capability has, by and large, not been widely deployed. I think,
for one, we network propeller-heads tend to overestimate the importance of
QoS or at least the ability of application designers and IT personnel to
get their heads around it well enough to be able to configure it so that it
actually improves things.

Kevin Gross
+1-303-447-0517
Media Network Consultant
AVA Networks - www.AVAnw.com <http://www.avanw.com/>

On Sun, Oct 13, 2013 at 2:01 AM, Michael Welzl <[email-masked]> wrote:

> b. With SVC, you can apply FEC or RTX only to the base layer.   So if
> there is loss in the base, you can recover.  Loss in enhancement layers may
> be "lived with" by dropping down to base frame rate/resolution/quality.  So
> not essential to apply FEC, RTX (or QoS) to enhancement layers, especially
> since if there is high loss in enhancement layers a logical response is to
> stop sending them (duh).
>
>
> I see. Well, should we perhaps really suggest a
> "lower-priority-than-other-packets-in-the-same-5-tuple" DSCP then?
>
> Cheers,
> Michael
>
>
>
>
>
>
> Michael said:
>
> > It has long been known that media data can have different levels of
> importance. Simply put, if packet #1 from a single source contains parts of
> a video's I-Frame and packet #2 contains only parts of B-Frames, and both
> end up in the same queue, controlled by an AQM (for instance), it would be
> better for the video stream if packet #2 would be dropped rather than
> packet #1.
> >
> > There is nothing new with this story; it's not hard to find research
> papers that document various variations of this theme, showing benefits in
> video quality.
> >
> > Is it because DSCP values are typically associated with sources, and
> hence, marking packet #2 as "less" important would put the source at the
> risk of having its packets less important than not only its own other
> packets, but anybody else's? But there is equipment that does
> per-connection stuff, and such things could probably better be done near
> the edges, where the bottleneck typically is... so if that's the whole
> issue, we could define DSCP values that mean relative importance *within
> the same five-tuple only*. Surely that has been thought about and probably
> proposed by folks before, so what happened? Why isn't it done?
> >
> > Or is it because per-five-tuple-functionality in the network is regarded
> as being too costly, and not encouraged, and hence not standardized?
> >
> > I'm just trying to understand the reasons for this particular
> long-standing difference between research an reality.
> >
> > Cheers,
> > Michael
> >
>
>
>

---

From: Michael Welzl <[email-masked]>
Date: Sun, 13 Oct 2013 18:28:34 +0200

IP? with a clear "per-5-tuple-only" rule?

Sent from my iPhone

On 13. okt. 2013, at 17:29, Kevin Gross <[email-masked]> wrote:

> There have been mechanisms proposed or implemented to some extent in Ethernet, ATM and IP for indicating drop precedence (as opposed to class). These mechanisms are designed to solve exactly this problem. I'm not sure why this capability has, by and large, not been widely deployed. I think, for one, we network propeller-heads tend to overestimate the importance of QoS or at least the ability of application designers and IT personnel to get their heads around it well enough to be able to configure it so that it actually improves things.
> 
> Kevin Gross
> +1-303-447-0517
> Media Network Consultant
> AVA Networks - www.AVAnw.com
> 
>

---

From: Kevin Gross <[email-masked]>
Date: Mon, 14 Oct 2013 15:35:15 -0600

No, not per 5 tuple (per flow). I don't think that is something we should
ask of the network. It doesn't sit well with the end-to-end principle and
it won't scale.

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Sun, Oct 13, 2013 at 10:28 AM, Michael Welzl <[email-masked]> wrote:

>> > Or is it because per-five-tuple-functionality in the network is
>> regarded as being too costly, and not encouraged, and hence not
>> standardized?
>> >

---

From: Steven Blake <[email-masked]>
Date: Mon, 14 Oct 2013 20:58:25 -0400

On Sat, 2013-10-12 at 08:11 +0200, Michael Welzl wrote:

> importance. Simply put, if packet #1 from a single source contains
> parts of a video's I-Frame and packet #2 contains only parts of
> B-Frames, and both end up in the same queue, controlled by an AQM (for
> instance), it would be better for the video stream if packet #2 would
> be dropped rather than packet #1.
> 
> papers that document various variations of this theme, showing
> benefits in video quality.
> 
> hence, marking packet #2 as "less" important would put the source at
> the risk of having its packets less important than not only its own
> other packets, but anybody else's?  But there is equipment that does
> per-connection stuff, and such things could probably better be done
> near the edges, where the bottleneck typically is... so if that's the
> whole issue, we could define DSCP values that mean relative importance
> *within the same five-tuple only*. Surely that has been thought about
> and probably proposed by folks before, so what happened? Why isn't it
> done?
> 

The Assured Forwarding (AF) PHB group (RFC2597) was defined precisely to
enable this drop-precedence-without-reordering-packets-within-a-flow
behavior.  In Diffserv jargon (RFC3260), an instance of an AF behavior
aggregate (BA), e.g., AF11/AF12/AF13, would be an ordered aggregate
(OA).  Packets in a flow DSCP-encoded with either AF11/AF12/AF13 (for
example) are guaranteed to be transmitted in the order received, but
will have different drop precedences (AF11 highest; AF13 lowest).  This
is usually achieved by serving all packets on a port with the AF1* DSCP
via the same FIFO queue.

Regards,

// Steve

---

From: Michael Welzl <[email-masked]>
Date: Tue, 15 Oct 2013 09:59:06 +0200

Hi,

I'll dissect your second and third sentences and tell you what I think about each part  :-)   below:

On 14. okt. 2013, at 23:35, Kevin Gross wrote:

> No, not per 5 tuple (per flow). I don't think that is something we should ask of the network.

I agree that asking this of the network may seem like encouraging per-flow functionality; in particular, if we'd put this into IP, that would be ugly. However, it's just a matter of fact that there are devices that do identify flows, and then, why not give such devices some guidance?  I like Toerless' suggestion of making this an RTP header extension field: if a device is identifying flows, we can expect such a device to identify such a field too. Plus, if we think of end systems playing an intermediate system role in RTP (mixers, ..), these devices might also benefit from that sort of information.

> It doesn't sit well with the end-to-end principle

I disagree, this is an all-too-common misunderstanding of the end-to-end principle: it only says that whatever can be done in the application should be left up to the application, but it doesn't say that you mustn't identify application streams inside the network or "keep the network simple and stupid". If you do "keep the network stupid", you're definitely in line with the e2e argument, but the argument isn't as strict as that (a typical example is routing, which can well become as complex as you wish, without harming the end-to-end principle). Now here we're discussing a function where the application really can't help you: when congestion happens inside the network and a packet must be dropped there, the best the application can do is tell you which packet to drop.

> and it won't scale.

I agree. But it doesn't have to, just because we introduce the signal we don't harm scalability - devices at the edges of the network could deal with it, all other devices could ignore it.

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Tue, 15 Oct 2013 10:14:45 +0200

> is usually achieved by serving all packets on a port with the AF1* DSCP

No, this is not what I mean. From RFC2597:

In a DS node, the level of forwarding assurance of an IP packet thus
depends on (1) how much forwarding resources has been allocated to
the AF class that the packet belongs to, (2) what is the current load
of the AF class, and, in case of congestion within the class, (3)
what is the drop precedence of the packet.

So while AF PHB groups are indeed meant to address such per-flow drop precedence, we still have a per-aggregate function, meaning that it's risky business for an application to decide that some of its packets have a higher drop precedence (it might put the application at a disadvantage in comparison with other *users*).

If you just look at page 4 of http://tools.ietf.org/html/draft-lai-tsvwg-normalizer-02 (Flow A vs. Flow B) you'll see an example that explains exactly the problem I'm talking about. Indeed I think that this draft suggests a nice solution, but as long as an application doesn't know that the mechanism in this draft is in place everywhere, the incentive for an application to assign a higher drop precedence to some packets still isn't there.

For how I think this could be solved, see my previous email to Toerless.

Cheers,
Michael

---

From: Kevin Gross <[email-masked]>
Date: Tue, 15 Oct 2013 11:04:17 -0600

I think you're stretching here. I don't think the balance is right. The
proposed information is difficult to retrieve and of little use to much of
the network. Even if standardized, it is unlikely to be successful.

The ideas in the normalizer draft don't seem to have so many issues and I
think it answers your original question and sets out to achieve what you're
looking for.

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Tue, Oct 15, 2013 at 1:59 AM, Michael Welzl <[email-masked]> wrote:

> I agree that asking this of the network may seem like encouraging per-flow
> functionality; in particular, if we'd put this into IP, that would be ugly.
> However, it's just a matter of fact that there are devices that do identify
> flows, and then, why not give such devices some guidance?  I like Toerless'
> suggestion of making this an RTP header extension field: if a device is
> identifying flows, we can expect such a device to identify such a field
> too. Plus, if we think of end systems playing an intermediate system role
> in RTP (mixers, ..), these devices might also benefit from that sort of
> information.
>
>
> I disagree, this is an all-too-common misunderstanding of the end-to-end
> principle: it only says that whatever can be done in the application should
> be left up to the application, but it doesn't say that you mustn't identify
> application streams inside the network or "keep the network simple and
> stupid". If you do "keep the network stupid", you're definitely in line
> with the e2e argument, but the argument isn't as strict as that (a typical
> example is routing, which can well become as complex as you wish, without
> harming the end-to-end principle). Now here we're discussing a function
> where the application really can't help you: when congestion happens inside
> the network and a packet must be dropped there, the best the application
> can do is tell you which packet to drop.
>
>
> > and it won't scale.
>
> I agree. But it doesn't have to, just because we introduce the signal we
> don't harm scalability - devices at the edges of the network could deal
> with it, all other devices could ignore it.
>
> Cheers,
> Michael
>
>

---

From: Michael Welzl <[email-masked]>
Date: Tue, 15 Oct 2013 22:43:51 +0200

Hmm. Whether the balance is right or not might depend on how the signaling is done. Anyway, unless I misunderstood something, the way the normalizer now stands it won't really make a change: it gives an incentive to applications to mark some packets with higher drop precedence, yet requires applications to know that the normalizer is in place or else the service would be different.

Maybe different DSCP values would be needed - but then this can't be used in combination with "regular" DiffServ usage. It really depends on the signaling - ideas are now being tossed around by Colin and Toerless, who clearly both know more about what the right signaling could be than me.

Cheers,
Michael

On 15. okt. 2013, at 19:04, Kevin Gross <[email-masked]> wrote:

> I think you're stretching here. I don't think the balance is right. The proposed information is difficult to retrieve and of little use to much of the network. Even if standardized, it is unlikely to be successful.
> 
> The ideas in the normalizer draft don't seem to have so many issues and I think it answers your original question and sets out to achieve what you're looking for.
> 
> Kevin Gross
> +1-303-447-0517
> Media Network Consultant

---

From: Kevin Gross <[email-masked]>
Date: Tue, 15 Oct 2013 16:45:11 -0600

Maybe I skimmed it too quickly but my understanding is that the proposal is
to remark DSCP values at the ingress point of the network and possibly at
other key boundaries too. Such DSCP remapping is already common practice in
enterprise networks.

The twist here is that the remapper will do the remapping dynamically based
on the proportion of AFn1, AFn2 and AFn3 packets received from a sender. A
greedy sender that it producing only AF41 markings will get some of those
randomly remarked as AF42 and AF43.

The proposal uses existing DSCP values and it doesn't look like signaling
is necessary nor is it necessary for the application to know that this is
happening.

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Tue, Oct 15, 2013 at 2:43 PM, Michael Welzl <[email-masked]> wrote:

> Hmm. Whether the balance is right or not might depend on how the signaling
> is done. Anyway, unless I misunderstood something, the way the normalizer
> now stands it won't really make a change: it gives an incentive to
> applications to mark some packets with higher drop precedence, yet requires
> applications to know that the normalizer is in place or else the service
> would be different.
>
> Maybe different DSCP values would be needed - but then this can't be used
> in combination with "regular" DiffServ usage. It really depends on the
> signaling - ideas are now being tossed around by Colin and Toerless, who
> clearly both know more about what the right signaling could be than me.
>
> Cheers,
> Michael
>
>
>> I agree that asking this of the network may seem like encouraging
>> per-flow functionality; in particular, if we'd put this into IP, that would
>> be ugly. However, it's just a matter of fact that there are devices that do
>> identify flows, and then, why not give such devices some guidance?  I like
>> Toerless' suggestion of making this an RTP header extension field: if a
>> device is identifying flows, we can expect such a device to identify such a
>> field too. Plus, if we think of end systems playing an intermediate system
>> role in RTP (mixers, ..), these devices might also benefit from that sort
>> of information.
>>
>>

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Oct 2013 17:23:02 -0700

The method described in the draft is applicable to whatever marking one wants
to use. I wish DSCP was the solution because as you said it would avoid the need
for figuring out how to carry a new marking (like RTP header extensions).

But: There are no existing DSCP with the PHB of "intra-flow-priority" with a
guarantee that flows with different distributions of those priorities would see
the same percentage of drops under queue congestion.

If we wanted to go the DSCP route, we would need at least 3 new DSCP to indicate
high,normal,low intra-flow priority (and just for intelligent dropping, 3 actually
might be good enough). But 3 new DSCP is an act of god in the IETF.

Then those three new DSCP would not only have to indicate intra-flow priority but
also the overall class PHB that the traffic should have even without priority 
indications. Aka: lets say low loss, low latency, appropriate for RMcat.
So when the next class of traffic with different intra-flow priorities comes
along, we'd have to redo the whole DSCP exercise again. And again.

And then the DSCP still get reset along the path, and apps still have problems
setting DSCP across various OSs.

Cheers
    Toerless

-- 
---
Toerless Eckert, [email-masked]
Cisco NSSTG Systems & Technology Architecture
SDN: Let me play with the network, mommy!

---

From: Kevin Gross <[email-masked]>
Date: Tue, 15 Oct 2013 19:34:18 -0600

If we create a situation where all flows are "normalized" (i.e. a certain
percentage of their packets are marked with higher drop precedence) by the
network then we shouldn't have to deal with things on a per-flow basis. If
we're not dealing with things on a per-flow basis, we can use the existing
AF codepoints. Dropping according to drop precedence markings on a class
basis should be statistically fair to all flows so long as all flows have
been normalized.

Kevin Gross
+1-303-447-0517
Media Network Consultant

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Oct 2013 19:13:24 -0700

Partially true IMHO.

Yes, if we could have all flows agree to have a marking let's say 20% high,
60% normal and 20% low prio bits (long term average), then we would not
need normalization. If there is any chance to establish such a standard, that
would be gret.

It just doesn't change the problem of DSCP. It would change the PHB though
by making it much more strict. Which makes it more questionable whether it
would be accepted.

Cheers
    Toerless

---

From: Kevin Gross <[email-masked]>
Date: Tue, 15 Oct 2013 21:31:55 -0600

As I read it, the proposal in the draft is to have network equipment tweak
DSCP values at ingress points so that drop precedence proportions are
normalized in some way. This does not require that all senders produce
normalized streams. The only thing that needs to be accepted is common
normalization profiles for all the ingress points in the network. Under the
proposal, useful drop precedence behavior becomes a network configuration
issue, not a standards compliance issue.

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Tue, Oct 15, 2013 at 8:13 PM, Toerless Eckert <[email-masked]> wrote:

> > > But: There are no existing DSCP with the PHB of "intra-flow-priority"
> with
> > > a
> > > high,normal,low intra-flow priority (and just for intelligent
> dropping, 3
> > > actually
> > > > Maybe I skimmed it too quickly but my understanding is that the
> proposal
> > > is
> > > > other key boundaries too. Such DSCP remapping is already common
> practice
> > > in
> > > > enterprise networks.
> > > >
> > > > on the proportion of AFn1, AFn2 and AFn3 packets received from a
> sender.
> > > A
> > > > > Maybe different DSCP values would be needed - but then this can't
> be
> > > used
> > > > > signaling - ideas are now being tossed around by Colin and
> Toerless,
> > > who
> > > > > think it answers your original question and sets out to achieve
> what
> > > you're
> > > > > looking for.
> > > > >
> > > > > Kevin Gross
> > > > > +1-303-447-0517
> > > > > Media Network Consultant
> > > > > On Tue, Oct 15, 2013 at 1:59 AM, Michael Welzl <[email-masked]
> >
> > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I'll dissect your second and third sentences and tell you what I
> think
> > > > >> about each part  :-)   below:
> > > > >>
> > > > >>
> > > > >> > No, not per 5 tuple (per flow). I don't think that is something
> we
> > > > >> should ask of the network.
> > > > >>
> > > > >> per-flow functionality; in particular, if we'd put this into IP,
> that
> > > would
> > > > >> be ugly. However, it's just a matter of fact that there are
> devices
> > > that do
> > > > >> identify flows, and then, why not give such devices some
> guidance?  I
> > > like
> > > > >> Toerless' suggestion of making this an RTP header extension
> field: if
> > > a
> > > > >> device is identifying flows, we can expect such a device to
> identify
> > > such a
> > > > >> field too. Plus, if we think of end systems playing an
> intermediate
> > > system
> > > > >> role in RTP (mixers, ..), these devices might also benefit from
> that
> > > sort
> > > > >> of information.
> > > > >>
> > > > >>
> > > > >> principle: it only says that whatever can be done in the
> application
> > > should
> > > > >> where the application really can't help you: when congestion
> happens
> > > inside
> > > > >> I agree. But it doesn't have to, just because we introduce the
> signal
> > > we

---

From: Michael Welzl <[email-masked]>
Date: Wed, 16 Oct 2013 09:22:00 +0200

> high,normal,low intra-flow priority (and just for intelligent dropping, 3 actually

Exactly.

> along, we'd have to redo the whole DSCP exercise again. And again.

....so we'd really need "intra-flow drop precedence" bits, to inter-work with existing DSCP values and get the expected PHB, right?
(or do as you say, define the PHB + precedence, for all possible combinations)

> setting DSCP across various OSs.

There is also an architectural side to it: Kevin rightfully started this conversation by saying: "No, not per 5 tuple (per flow). I don't think that is something we should ask of the network."
Indeed, doing this right *requires* routers (or whatever box does that) to identify flows and keep track of them. Such per-flow functionality isn't what we want "normal" routers to do, I think, and so binding  DSCP values to such a functionality just seems wrong to me. In a way, it makes the IP header less self-contained: in order to implement all the functions encoded in it, you need to look beyond the IP header. This doesn't strike me as good design.

I think it's not too much to ask of devices that identify flows to look beyond the IP header for a shim, RTP-something, or whatever we define. Then, this device can aggregate flows into the right DSCPs if it knows that traffic's going to traverse a path where the normalizer is present - and the normalizing devices again only need to look at the IP header, where it's all per-aggregate.
(sorry for repeating myself, I thought I'd say this again here just for context, but I did say this before)

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Wed, 16 Oct 2013 09:36:06 +0200

[I'm adding tsv-area: I noticed it was dropped from some part of this thread, but I think unintentionally? BTW I hope tsv-area is appropriate, and not tsvwg? I started off to rmcat and tsv-area because I had a general DiffServ'ish question, yet related to rmcat, and so I thought this would fit.]

Hi,

Imagine that you write an application and based on the intra-flow priorities of your packets you decide that this would be your profile:
(copy+paste from the draft)

***
      Flow B

         40% or 2Mbps in AF41

         40% or 2Mbps in AF42

         20% or 1Mbps in AF43
***

...and the normalizer is not present in the network, and you get "normal" DiffServ behavior (because we're using the standard AF codepoints). Now, if someone else uses:

***
      Flow A

         80% or 4Mbps in AF41

         20% or 1Mbps in AF42

          0% or 0Mbps in AF43
***

...then this someone (flow A) would be better off in the presence of congestion, as described in the draft. This means that you'll never decide for a profile as in Flow B in your application because you could hurt yourself, you might be treated worse than some other users.

This is what the normalizer fixes, but as long as you don't know it's there, going for a profile as in Flow B is still risky business.

"You" in my story here is the application programmer, and it really has to be the application programmer doing this - the priorities would be derived from knowledge about the codec, FEC usage etc. What the application programmer needs is a signal that is definitely going to be ignored unless it hits a normalizing ingress device.

Cheers,
Michael

On 16. okt. 2013, at 05:31, Kevin Gross wrote:

> As I read it, the proposal in the draft is to have network equipment tweak DSCP values at ingress points so that drop precedence proportions are normalized in some way. This does not require that all senders produce normalized streams. The only thing that needs to be accepted is common normalization profiles for all the ingress points in the network. Under the proposal, useful drop precedence behavior becomes a network configuration issue, not a standards compliance issue.
> 
> Kevin Gross
> +1-303-447-0517
> Media Network Consultant

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Wed, 16 Oct 2013 10:21:42 +0200

Hi Michael,

On 16.10.2013 09:36, Michael Welzl wrote:
> [I'm adding tsv-area: I noticed it was dropped from some part of this
> thread, but I think unintentionally? BTW I hope tsv-area is appropriate,
> and not tsvwg? I started off to rmcat and tsv-area because I had a
> general DiffServ'ish question, yet related to rmcat, and so I thought
> this would fit.]

Yes, I'm following this on TSVAREA since I'm not subscribed on rmcat yet.

> ...then this someone (flow A) would be better off in the presence of
> congestion, as described in the draft. This means that you'll never
> decide for a profile as in Flow B in your application because you could
> hurt yourself, you might be treated worse than some other users.
> 
> This is what the normalizer fixes, but as long as you don't know it's
> there, going for a profile as in Flow B is still risky business.
> 
> "You" in my story here is the application programmer, and it really has
> to be the application programmer doing this - the priorities would be
> derived from knowledge about the codec, FEC usage etc. What the
> application programmer needs is a signal that is definitely going to be
> ignored unless it hits a normalizing ingress device.

Just some few observations from my point of view:
- DiffServ doesn't distinguish between individual flows within
  a behavior aggregate - it was intentionally designed that way.
  Per micro-flow unfairness within the same class is not prevented
  by DiffServ - conceptionally you should only use the first-hop router
  for this kind of traffic conditioning measures. However, this doesn't
  help to prevent other aggregation effects inside a DiffServ domain
  (cf. Per-Hop Behavior vs. Per Domain Behavior).

- Following the end-to-end argument, it doesn't make sense to build
  somewhat complex mechanisms into the network, which may even depend
  on the particular application (like WebRTC etc.) - so building a
  "normalizer" (w/o having fully understood the concept) into the
  network doesn't seem to be the right approach in the long run.

- Therefore, I agree very much with your last statement: it's only
  the application that has the knowledge and context to set any
  kind of forwarding priority (or drop precendence) for individual
  packets correctly.
  Especially, the logic to react on dropped packets should reflect
  the marking and corresponding dropping behavior.

Regards,
 Roland

---

From: Michael Welzl <[email-masked]>
Date: Wed, 16 Oct 2013 10:49:02 +0200

> Yes, I'm following this on TSVAREA since I'm not subscribed on rmcat yet.

... and dropping tsv-area was a mistake, as your message shows  :(   'cause we've been around this block I think...

>  (cf. Per-Hop Behavior vs. Per Domain Behavior).

Sure-

>  network doesn't seem to be the right approach in the long run.

I disagree with what you say about e2e, see:
http://www.ietf.org/mail-archive/web/rmcat/current/msg00519.html
but anyway that's a separate debate, as the normalizer draft works on aggregates and is really compliant with DiffServ as far as I can tell.
That's the draft we're debating here:
http://tools.ietf.org/html/draft-lai-tsvwg-normalizer-02

>  the marking and corresponding dropping behavior.

Yeah, the issue we're debating is that an application won't have an incentive to go for the profile of Flow B (page 4 of the draft) unless it knows that the normalizer is there, because otherwise it could get treated in the normal DiffServ way and that would make it worse off than Flow A (page 4 of the draft). Thus you need a way for the application to signal intra-flow packet priorities to an edge without "standard DiffServ" operating on that signal. The signal must be ignored unless the normalizer is there.

I hope that helped to clarify the discussion a bit...

Cheers,
Michael

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 16 Oct 2013 10:53:16 -0700

> (or do as you say, define the PHB + precedence, for all possible combinations)

Right. And independent of those intra-flow priority bits there is
IMHO still the open question whether the existing DSCP/PHB best suit
what we would want for RMcat.

> Indeed, doing this right *requires* routers (or whatever box does that) to identify flows and keep track of them. Such per-flow functionality isn't what we want "normal" routers to do, I think, and…

Flow-awareness is not evil, and there are a lot more proof points of
deployment where it works and scales than IMHO places where its needed and
does not scale. All enterprise routers i know do heavily leverage flow
awareness. 

Flow awareness for intra-flow priority can easily be designed to require
the same amount of flow awareness as existing QoS problems. Specifically
for QoS-authorization on the edge (and for intra-flow priority, that
authorization could mean normalization).  I think that's waht you're also saying
below.

> (sorry for repeating myself, I thought I'd say this again here just for context, but I did say this before)

Right.

There is also some fundamental insight into what a "correct" layer is that
i think the IETF is missing. With IPv6, they just carried over the IPv4
design without reviewing why router-alert and IP packet options didn't
catch on in IPv4 for onpath operations. After 30 years regurgitating what's
not working, it's IMHO rally time to give IETF blessings to some other
approach, and clean well reviewed designs with such network aware
extensions in the transport layer IMHO is exactly that (whether it's shim
layer, SCTP extensions, RTP header extension or what hae you - that's
 just a mattter of use-case priorities).

Cheers
    toerless

Cheers,

---

From: Roland Bless <[email-masked]>
Date: Wed, 16 Oct 2013 21:43:16 +0200

Hi Michael and all,

Am 16.10.2013 10:49, schrieb Michael Welzl:
> but anyway that's a separate debate, as the normalizer draft works on
> aggregates and is really compliant with DiffServ as far as I can tell.

Thanks for the pointer, I quickly read over it.
My understanding is: if we have an example flow
of AF41:AF42:AF43 80:20:0 the normalizer would
reassign those packets to 40:20:20 (or whatever
the configured proportion is). Is this perception
correct?

Regarding the e2e argument:
- you said "it only says that whatever can be done in the application
  should be left up to the application" - I don't think this catches
  the gist of it: "The function in question can completely and
  correctly be implemented only with the knowledge and help of the
  application standing at the end points of the communication system.
  Therefore, providing that questioned function as a feature of the
  communication system itself is not possible". => especially,
  application knowledge is often required when reacting on failures,
  i.e., the right decision cannot be made when implemented within the
  network. Thus, dropping packets according to the markings set by
  the application is fully in accordance with the e2e argument, whereas
  building marking mechanisms into the network that require application
  knowledge is usually not in accordance with it.

- if I understood the normalizer proposal in draft-lai-tsvwg-normalizer-02
  correctly, it is not in line with the e2e argument since it reassigns
  drop precedences without application knowledge. Thus, it may make
  the wrong decisions about the importance of packets, not knowing how
  apps react to loss.

> Yeah, the issue we're debating is that an application won't have an
> incentive to go for the profile of Flow B (page 4 of the draft) unless
> it knows that the normalizer is there, because otherwise it could get
> treated in the normal DiffServ way and that would make it worse off than
> Flow A (page 4 of the draft). Thus you need a way for the application to
> signal intra-flow packet priorities to an edge without "standard
> DiffServ" operating on that signal. The signal must be ignored unless
> the normalizer is there.
> 

Thanks, I think I understand the rationale now. However, I see several
potential problems with this approach and have some thoughts to share
as well:
0) I think an application with a 40:20:20 assignment has no real problem
   if it experiences loss in PHBs AFx2, AFx3 since it was designed that
   way. Whether it's "unfair" w.r.t. to other streams doesn't really
   matter. Remember that QoS support is usually managed unfairness
   anyway. Fairness within AF usually matters for the "yellow" packets,
   e.g., AFx2. This excess bandwidth should be fairly shared among all
   streams in this AF class if possible.

1) reassignment of drop precedences w/o application knowledge

2) how can one find the right default proportions suitable for different
   streams or applications?

3) Is this AF PHB class restricted to forward this special kind of
   encoded video streams? If so, we already have the problem of
   knowing that this is actually the case in the particular DS
   domain/region...

4) Normalizing other (non video) streams within the same AF class may
   cause adverse effects to the application behavior.

5) Aggregation effects within a domain in interior nodes may cause
   a different proportion than what the normalizer is configured for.
   Thus the normalizer may not cause the desired effects for a whole
   domain.

6) The AF PHB group per se is difficult to handle, since drop
   precedences may be even inconsistenly configured within a DS domain
   or DS region (see RFC 2597, sec. 4 at the end: "Inconsistent discard
   behaviors lead to inconsistent end-to-end service semantics and
   limit the range  of possible uses of the AF PHB in a multi-vendor
   environment.").
   Furthermore, packets of lowest drop precedence may suffer from
   packets with higher drop precedences waiting before them in the
   queue.

7) For an AF class you basically need a sort of admission control for
   the assured (let's say "green") AFx1 packets to guarantee the low
   loss probability. Every usage of excess bandwidth is purely optional
   for which the application must be prepared.

Regards,
 Roland

---

From: "Cheng-Jia Lai (chelai)" <[email-masked]>
Date: Wed, 16 Oct 2013 21:53:23 +0000

Hi Michael, Roland, Kevin, and all,

As an author of this draft, I am grateful to see so many great comments and insights from all of you. I do agree with most of those. Michael has made a good point that the proposal's incentive won't be completely in effect unless the applications *know* the normalizer is deployed in the network. This is probably the hardest problem (not technical, maybe). Usually we see that in an enterprise network, routers are managed as DS domains while service provider edges have SLA, so it may be safe to say that the IT admins can manually configure AF PHB in their network planning for video endpoints deployment. Suppose the vendor of a video endpoint implements an option that allows generating more packets in lower priorities such as AF43 if so configured. Since the IT admin *knows* the enterprise network has deployed a normalizer at DS edges, the IT admin can turn on that option on the video endpoint. However, if the DS domains don't have a single administration such as on extranet or in service-provider networks, then the applications will need some way to *know* exactly if a normalizer is used.

One way to do that is make the normalizer deployment to become a standardized mechanism, so that the interaction (i.e. apps know or discover normalizers' existence) is possible by some sort of signaling or protocol. Exactly how to do this (or whether it is wise to do this) is open to discussion, so we tentatively changed the draft's intended status from standard (v0) to informational (v2) before it was presented to the WG.

Also, some inline comments below.

Regards,
CJ

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Roland Bless
> Sent: Wednesday, October 16, 2013 12:43 PM
> To: Michael Welzl
> Cc: Bernard Aboba; rmcat WG; [email-masked]; Kevin Gross
> Subject: Re: [rmcat] About the importance of packets in a single data stream
> 
> Hi Michael and all,
> 

Yes.

>   the wrong decisions about the importance of packets, not knowing how

If the incoming packets of the flow are already marked by the application, the normalizer (in a color-aware mode, i.e. aware of the existing markings) should learn from the existing distribution of the existing markings while it reassigns the markings. This might not be perfect, but it is making use of application knowledge which is retrieved from the existing markings, for right decisions. If the incoming packets have no good markings (i.e. all the same), there is no application knowledge for the normalizer to retrieve, so yes, I agree, it can make wrong decisions at remarking. But our thinking is: if the applications do not care enough to mark its own packets, why should the normalizer care? :)

> > DiffServ" operating on that signal. The signal must be ignored unless

Agree. This signaling could be designed in various forms, so it might need a few standardized mechanisms for this interaction. Either a new DiffServ service (like a contract) with DSCP assignments to be used in configuration & deployment, or a more dynamic way, i.e. discovery of normalizer existence via some protocol or application-layer message.

>    streams in this AF class if possible.

I agree that "unfairness" is OK to some degree, as long as it is not super unfair. The normalizer is designed to avoid the super unfairness, instead of offering perfect fairness which we can hardly achieve in DiffServ, as I agree with you.

>    knowing that this is actually the case in the particular DS

We do not want it to specialize on particular video encoding. The H.264 notion in the draft is for example.

>    cause adverse effects to the application behavior.

Yes, agree. It is still necessary to classify/categorize the traffic so only relevant streams should go into the AF class where a normalizer is deployed.

>    Thus the normalizer may not cause the desired effects for a whole

Assume all the edges are deployed with a normalizer, while the interior nodes are not. Then, all flows traversing the interior nodes have already been normalized, so they should see similar-fair drop effects everywhere inside the DS domain.

>    packets with higher drop precedences waiting before them in the

If the suffering here refers to queuing delays, then it is the same for all packets (no matter high or low drop precedence) in the AF PHB class, on average in the FIFO queue. Instead, the normalizer addresses the drop probability for AQM.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 17 Oct 2013 15:47:36 +0200

Hi, 

I have a (quick) question as you are talking about using existing DSCP vs. 
some kind of "intra-flow drop precedence" signaling. You only have been 
talking about the signaling so far. But what would be the expected network 
behavior/reaction?

With AF4x classed if believe you will have to calculate different drop 
probabilities for each class based on a different parameter set for RED (or 
something similar...?), right?

I would always have envisioned an behavior where you calculate the same drop 
probability for every packet or even use DropTail. Then if a packet is about 
to be dropped, you look into the queue and try to find another packet of the 
same flow but with the "drop precedence" bit set. If you find one, you drop 
this packet instead, if not you drop the original (incoming) packet.
Does such a mechanism exist anywhere?

Mirja

On Wednesday 16 October 2013 19:53:16 Toerless Eckert wrote:
> > ....so we'd really need "intra-flow drop precedence" bits, to inter-work
> > with existing DSCP values and get the expected PHB, right? (or do as you
> > say, define the PHB + precedence, for all possible combinations)
>
> > > And then the DSCP still get reset along the path, and apps still have
> > > problems setting DSCP across various OSs.
> >
> > There is also an architectural side to it: Kevin rightfully started this
> > conversation by saying: "No, not per 5 tuple (per flow). I don't think
> > that is something we should ask of the network." Indeed, doing this right
> > *requires* routers (or whatever box does that) to identify flows and keep
> > track of them. Such per-flow functionality isn't what we want "normal"
> > routers to do, I think, and so binding  DSCP values to such a
> > functionality just seems wrong to me. In a way, it makes the IP header
> > less self-contained: in order to implement all the functions encoded in
> > it, you need to look beyond the IP header. This doesn't strike me as good
> > design.
>
> authorization could mean normalization).  I think that's waht you're also
> saying below.
>
> > I think it's not too much to ask of devices that identify flows to look
> > beyond the IP header for a shim, RTP-something, or whatever we define.
> > Then, this device can aggregate flows into the right DSCPs if it knows
> > that traffic's going to traverse a path where the normalizer is present -
> > and the normalizing devices again only need to look at the IP header,
> > where it's all per-aggregate. (sorry for repeating myself, I thought I'd
> > say this again here just for context, but I did say this before)
>
> Right.
>
> > >> proposal is to remark DSCP values at the ingress point of the network
> > >> and possibly at other key boundaries too. Such DSCP remapping is
> > >> already common practice in enterprise networks.
> > >>
> > >> The twist here is that the remapper will do the remapping dynamically
> > >> based on the proportion of AFn1, AFn2 and AFn3 packets received from a
> > >> sender. A greedy sender that it producing only AF41 markings will get
> > >> some of those randomly remarked as AF42 and AF43.
> > >>
> > >> The proposal uses existing DSCP values and it doesn't look like
> > >> signaling is necessary nor is it necessary for the application to know
> > >> that this is happening.
> > >>
> > >> Kevin Gross
> > >> +1-303-447-0517
> > >> Media Network Consultant
> > >>> Hmm. Whether the balance is right or not might depend on how the
> > >>> signaling is done. Anyway, unless I misunderstood something, the way
> > >>> the normalizer now stands it won't really make a change: it gives an
> > >>> incentive to applications to mark some packets with higher drop
> > >>> precedence, yet requires applications to know that the normalizer is
> > >>> in place or else the service would be different.
> > >>>
> > >>> Maybe different DSCP values would be needed - but then this can't be
> > >>> used in combination with "regular" DiffServ usage. It really depends
> > >>> on the signaling - ideas are now being tossed around by Colin and
> > >>> Toerless, who clearly both know more about what the right signaling
> > >>> could be than me.
> > >>>
> > >>> Cheers,
> > >>> Michael
> > >>>
> > >>>
> > >>> I think you're stretching here. I don't think the balance is right.
> > >>> The proposed information is difficult to retrieve and of little use
> > >>> to much of the network. Even if standardized, it is unlikely to be
> > >>> successful.
> > >>>
> > >>> The ideas in the normalizer draft don't seem to have so many issues
> > >>> and I think it answers your original question and sets out to achieve
> > >>> what you're looking for.
> > >>>
> > >>> Kevin Gross
> > >>> +1-303-447-0517
> > >>> Media Network Consultant
> > >>>> think about each part  :-)   below:
> > >>>>
> > >>>>> No, not per 5 tuple (per flow). I don't think that is something we
> > >>>>
> > >>>> that would be ugly. However, it's just a matter of fact that there
> > >>>> are devices that do identify flows, and then, why not give such
> > >>>> devices some guidance?  I like Toerless' suggestion of making this
> > >>>> an RTP header extension field: if a device is identifying flows, we
> > >>>> can expect such a device to identify such a field too. Plus, if we
> > >>>> think of end systems playing an intermediate system role in RTP
> > >>>> (mixers, ..), these devices might also benefit from that sort of
> > >>>> information.
> > >>>>
> > >>>> I disagree, this is an all-too-common misunderstanding of the
> > >>>> end-to-end principle: it only says that whatever can be done in the
> > >>>> application should be left up to the application, but it doesn't say
> > >>>> that you mustn't identify application streams inside the network or
> > >>>> "keep the network simple and stupid". If you do "keep the network
> > >>>> stupid", you're definitely in line with the e2e argument, but the
> > >>>> argument isn't as strict as that (a typical example is routing,
> > >>>> which can well become as complex as you wish, without harming the
> > >>>> end-to-end principle). Now here we're discussing a function where
> > >>>> the application really can't help you: when congestion happens
> > >>>> inside the network and a packet must be dropped there, the best the
> > >>>> application can do is tell you which packet to drop.
> > >>>>
> > >>>>> and it won't scale.
> > >>>>
> > >>>> signal we don't harm scalability - devices at the edges of the
> > >>>> network could deal with it, all other devices could ignore it.
> > >>>>
> > >>>> Cheers,
> > >>>> Michael

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 17 Oct 2013 07:15:55 -0700

Mirja,

Simple drop-tail configs for the different classes yielded IMHO
quite good results.

WRED just adds more parameters (with the slope). Routers may support only
WRED and no explicit drop-tail, so one can easily use WRED to create
a drop-tail config via a WRED config, but leveraging the slope in WREP
is something IMHO related to the CC mechanism used so we didn't play around
too much with that yet (priority dropping is meant to react faster than CC).

What you describe last is what we called the "oracle", aka the method
that we thought could be used in simulation at least as a theoretical 
"best-possible-drop-priority" mechanism. And no, you're right, it does not
exist in router products. Luckily it turned out to actually be worse than
droptail. I have some analsysis slide that tries to explain why. Hint:
bufferbloat ;-)

Cheers
    Toerless

---

From: "LOCHIN Emmanuel" <[email-masked]>
Date: Fri, 18 Oct 2013 07:59:30 +0200

Hi Mirja

Your proposal is not random because in terms of probability, all packets are not considered as an independent event anymore.
IMHO the simply thing is to classify packets with and without "drop precedence" bit set into two separate queues (can be FIFO or RED) for a given class 
and to schedule both queues with a round robin or priority queuing scheme.

EL

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Fri, 18 Oct 2013 09:25:04 +0200

Hi,

> and to schedule both queues with a round robin or priority queuing scheme.

But by using separate queues you'll get reordered packets within a
microflow (e2e data stream) and AF class, which is not allowed by the
AF PHB group specification.

Regards,
 Roland

---

From: Emmanuel Lochin <[email-masked]>
Date: Fri, 18 Oct 2013 09:29:23 +0200

But when you declassify AF trafic in another AF class or in the best-effort one you might also get reordering. 

If this is a problem, the only solution is, I think, to implement a pushout mechanism that drops marked packets when the queue is full.

Emmanuel

"Bless, Roland (TM)" <[email-masked]> wrote:

>> Your proposal is not random because in terms of probability, all
>packets are not considered as an independent event anymore.
>> IMHO the simply thing is to classify packets with and without "drop
>precedence" bit set into two separate queues (can be FIFO or RED) for a
>given class

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Fri, 18 Oct 2013 14:05:15 +0200

Hi Emmanuel,

> But when you declassify AF trafic in another AF class or in the best-effort one you might also get reordering. 

That's actually a different story and falls more into the PDB concept
than into a PHB concept. Furthermore, we are talking about reordering
caused by PHB _implementations_, so getting reordered packets by routing
is quite a different story, too.

With DiffServ one usually assumes that the DS domains have at least
some agreement on how they treat the aggregates. For PHBs it's quite
well defined (though there is still some room left for variation), but
if you remark traffic between different PHBs, it's something the
service user should be aware of (could be part of a PDB specification).

> If this is a problem, the only solution is, I think, to implement a pushout mechanism that drops marked packets when the queue is full.

Yes, maybe. But how do you determine eligible packets?
Starting from the front of the queue, considering makred packets of
all flows, of only your own microflow or by using some randomized
selection scheme?

Regards,
 Roland

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 18 Oct 2013 06:51:33 -0700

IMHO, whatever PHB is most feasible to achieve the desired priority-dropping
is the one to use or to define. If packet reordering between the priorities
would be a result, then IMHO thats would be a perfectly good price to pay given how
receivers have playout buffers. And if that means that AF would be unusable
becaue of it's "no-reorder" policy, then so be it.

Its just that i don't think that it is either necessary nor beneficial to
use a mechanism that leads to persistent reordering to solve the priority-dropping
requirement. If you just created different weighted queues for the different
priorities, i think you've achieved nothing towards priority dropping:
If the ratio of traffic in the priority bucket stays the same, but you've got bursts
that in total exceed the aggregate bandwidth available, you just start dropping
across all priorities proportional to their BW-share. Thats not the desired
behavior.

So, IMHO, single queue with different drop thresholds for different priorities
is perfectly fine and does not cause reordering.

Cheers
    Toerless

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 18 Oct 2013 16:08:14 +0200

Hi Toerless,

can you provide these results? I can't imagine that this will be worse than 
droptail without this mechanism. Sure it doesn't help in terms of 
bufferbloat. This is a problem you have to address differently. But it 
shouldn't worsen the problem, no? And at least it will help the application 
to better deal with packets drops/losses.

Mirja

On Thursday 17 October 2013 16:15:55 Toerless Eckert wrote:
> > I have a (quick) question as you are talking about using existing DSCP
> > vs. some kind of "intra-flow drop precedence" signaling. You only have
> > been talking about the signaling so far. But what would be the expected
> > network behavior/reaction?
> >
> > probabilities for each class based on a different parameter set for RED
> > (or something similar...?), right?
> >
> > I would always have envisioned an behavior where you calculate the same
> > drop probability for every packet or even use DropTail. Then if a packet
> > is about to be dropped, you look into the queue and try to find another
> > packet of the same flow but with the "drop precedence" bit set. If you
> > find one, you drop this packet instead, if not you drop the original
> > (incoming) packet. Does such a mechanism exist anywhere?
> >
> > Mirja
> >
> > > > ....so we'd really need "intra-flow drop precedence" bits, to
> > > > inter-work with existing DSCP values and get the expected PHB, right?
> > > > > And then the DSCP still get reset along the path, and apps still
> > > > > have problems setting DSCP across various OSs.
> > > >
> > > > There is also an architectural side to it: Kevin rightfully started
> > > > this conversation by saying: "No, not per 5 tuple (per flow). I don't
> > > > think that is something we should ask of the network." Indeed, doing
> > > > this right *requires* routers (or whatever box does that) to identify
> > > > flows and keep track of them. Such per-flow functionality isn't what
> > > > we want "normal" routers to do, I think, and so binding  DSCP values
> > > > to such a functionality just seems wrong to me. In a way, it makes
> > > > the IP header less self-contained: in order to implement all the
> > > > functions encoded in it, you need to look beyond the IP header. This
> > > > doesn't strike me as good design.
> > >
> > > deployment where it works and scales than IMHO places where its needed
> > > and does not scale. All enterprise routers i know do heavily leverage
> > > flow awareness.
> > >
> > > Flow awareness for intra-flow priority can easily be designed to
> > > require the same amount of flow awareness as existing QoS problems.
> > > Specifically for QoS-authorization on the edge (and for intra-flow
> > > priority, that authorization could mean normalization).  I think that's
> > > waht you're also saying below.
> > >
> > > > I think it's not too much to ask of devices that identify flows to
> > > > look beyond the IP header for a shim, RTP-something, or whatever we
> > > > define. Then, this device can aggregate flows into the right DSCPs if
> > > > it knows that traffic's going to traverse a path where the normalizer
> > > > is present - and the normalizing devices again only need to look at
> > > > the IP header, where it's all per-aggregate. (sorry for repeating
> > > > myself, I thought I'd say this again here just for context, but I did
> > > > say this before)
> > >
> > > Right.
> > >
> > > There is also some fundamental insight into what a "correct" layer is
> > > that i think the IETF is missing. With IPv6, they just carried over the
> > > IPv4 design without reviewing why router-alert and IP packet options
> > > didn't catch on in IPv4 for onpath operations. After 30 years
> > > regurgitating what's not working, it's IMHO rally time to give IETF
> > > blessings to some other approach, and clean well reviewed designs with
> > > such network aware extensions in the transport layer IMHO is exactly
> > > that (whether it's shim layer, SCTP extensions, RTP header extension or
> > > what hae you - that's just a mattter of use-case priorities).
> > >
> > >
> > > Cheers
> > >     toerless
> > >
> > > Cheers,
> > >
> > > > Michael
> > > >
> > > > > Cheers
> > > > >    Toerless
> > > > >
> > > > >> proposal is to remark DSCP values at the ingress point of the
> > > > >> network and possibly at other key boundaries too. Such DSCP
> > > > >> remapping is already common practice in enterprise networks.
> > > > >>
> > > > >> The twist here is that the remapper will do the remapping
> > > > >> dynamically based on the proportion of AFn1, AFn2 and AFn3 packets
> > > > >> received from a sender. A greedy sender that it producing only
> > > > >> AF41 markings will get some of those randomly remarked as AF42 and
> > > > >> AF43.
> > > > >>
> > > > >> signaling is necessary nor is it necessary for the application to
> > > > >> know that this is happening.
> > > > >>
> > > > >> Kevin Gross
> > > > >> +1-303-447-0517
> > > > >> Media Network Consultant
> > > > >> On Tue, Oct 15, 2013 at 2:43 PM, Michael Welzl
> > > > >> <[email-masked]>
> >
> > wrote:
> > > > >>> signaling is done. Anyway, unless I misunderstood something, the
> > > > >>> way the normalizer now stands it won't really make a change: it
> > > > >>> gives an incentive to applications to mark some packets with
> > > > >>> higher drop precedence, yet requires applications to know that
> > > > >>> the normalizer is in place or else the service would be
> > > > >>> different.
> > > > >>>
> > > > >>> be used in combination with "regular" DiffServ usage. It really
> > > > >>> depends on the signaling - ideas are now being tossed around by
> > > > >>> Colin and Toerless, who clearly both know more about what the
> > > > >>> right signaling could be than me.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Michael
> > > > >>>
> > > > >>>
> > > > >>> I think you're stretching here. I don't think the balance is
> > > > >>> right. The proposed information is difficult to retrieve and of
> > > > >>> little use to much of the network. Even if standardized, it is
> > > > >>> unlikely to be successful.
> > > > >>>
> > > > >>> The ideas in the normalizer draft don't seem to have so many
> > > > >>> issues and I think it answers your original question and sets out
> > > > >>> to achieve what you're looking for.
> > > > >>>
> > > > >>> Kevin Gross
> > > > >>> +1-303-447-0517
> > > > >>> Media Network Consultant
> > > > >>> On Tue, Oct 15, 2013 at 1:59 AM, Michael Welzl
> > > > >>> <[email-masked]>
> >
> > wrote:
> > > > >>>> Hi,
> > > > >>>>
> > > > >>>> I agree that asking this of the network may seem like
> > > > >>>> encouraging per-flow functionality; in particular, if we'd put
> > > > >>>> this into IP, that would be ugly. However, it's just a matter of
> > > > >>>> fact that there are devices that do identify flows, and then,
> > > > >>>> why not give such devices some guidance?  I like Toerless'
> > > > >>>> suggestion of making this an RTP header extension field: if a
> > > > >>>> identify such a field too. Plus, if we think of end systems
> > > > >>>> playing an intermediate system role in RTP (mixers, ..), these
> > > > >>>> devices might also benefit from that sort of information.
> > > > >>>>
> > > > >>>> end-to-end principle: it only says that whatever can be done in
> > > > >>>> the application should be left up to the application, but it
> > > > >>>> doesn't say that you mustn't identify application streams inside
> > > > >>>> the network or "keep the network simple and stupid". If you do
> > > > >>>> "keep the network stupid", you're definitely in line with the
> > > > >>>> e2e argument, but the argument isn't as strict as that (a
> > > > >>>> typical example is routing, which can well become as complex as
> > > > >>>> you wish, without harming the end-to-end principle). Now here
> > > > >>>> we're discussing a function where the application really can't
> > > > >>>> help you: when congestion happens inside the network and a
> > > > >>>> packet must be dropped there, the best the application can do is
> > > > >>>> tell you which packet to drop.
> > > > >>>>
> > > > >>>>> and it won't scale.
> > > > >>>>

tel: +49(0)711/685-67973

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 18 Oct 2013 16:17:26 +0200

Hi Emmanuel,

yes, it's not random because the goal to potentially avoid losses for some 
packets. But it's still random on a per flow basis; which means flows will 
see drops proportional to their bandwidth share. That's the important part!

I believe my proposal is actually more simple that handling different queues, 
and you can avoid such problems like reordering (as already discussed) or 
disincentive to set the bit because your traffic might get treated worse.

If you have different queues and use round robin, you might actually end up 
that you 'drop precedence' packet get treated better than the other traffic, 
if e.g. the bandwidth share of these packets is smaller... I believe this 
makes things really complicated...

Mirja

On Friday 18 October 2013 07:59:30 LOCHIN Emmanuel wrote:
> Hi Mirja
>
> Your proposal is not random because in terms of probability, all packets
> are not considered as an independent event anymore. IMHO the simply thing
> is to classify packets with and without "drop precedence" bit set into two
> separate queues (can be FIFO or RED) for a given class and to schedule both
> queues with a round robin or priority queuing scheme.
>
> EL
>
> Le Jeudi 17 Octobre 2013 15:47 CEST, Mirja Kuehlewind 
<[email-masked]> a écrit:
> > (incoming) packet.

tel: +49(0)711/685-67973

---

From: Emmanuel Lochin <[email-masked]>
Date: Fri, 18 Oct 2013 17:51:49 +0200

> Hi Emmanuel,

Hi Roland,

> service user should be aware of (could be part of a PDB specification).

Ok, thanks for these clarifications :)

> all flows, of only your own microflow or by using some randomized

Considering a simple Push-Out implementation, as soon as the queue is 
full and you get an incoming non-marked packet, you have to drop one 
marked packet previously enqueued.
Might be the last one or the first one enqueued. This has to be evaluated.

Emmanuel

-- 
Emmanuel Lochin
Professeur ISAE - OSSI
Institut Supérieur de l'Aéronautique et de l'Espace (ISAE)
Issu du rapprochement SUPAERO et ENSICA
10 avenue Edouard Belin - BP 54032 - 31055 Toulouse cedex 4
Tel : 05 61 33 91 85 - Fax : 05 61 33 91 88
Web : http://personnel.isae.fr/emmanuel-lochin/
---
"This email and any attachments are confidential. They may contain legally privileged information or copyright material. You should not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages. We do not accept liability in connection with computer virus, data corruption, delay, interruption, unauthorised access or unauthorised amendment. This notice should not be removed"

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 18 Oct 2013 19:45:49 +0200

Hi Emmanuel,

> >> If this is a problem, the only solution is, I think, to implement a
> >> pushout mechanism that drops marked packets when the queue is full.
> >

If you do this, there is a disincentive to mark packets (as 'drop precedence') 
as this will treat you worse than flows that don't mark at all or at least 
less.

That's why I proposed to look for a marked packet of the same flow only. If 
there is no marked packet of the same flow in the queue, you drop the 
original packets. Because than this flow presumably does not have any packets 
that are important to not get dropped or the flow did just ignore the 
possibility of marking but than it's its own fault.

Mirja

---

From: "LOCHIN Emmanuel" <[email-masked]>
Date: Sat, 19 Oct 2013 08:46:15 +0200

Le Vendredi 18 Octobre 2013 19:45 CEST, Mirja Kühlewind <[email-masked]> a écrit: 

Hi Mirja,

That's interesting :) I did a presentation at the last ICCRG meeting in Berlin about an AQM that does this kind of operation. 
I mean, looking whether an incoming packet belongs to a flow of packets previously enqueued. 
Except that in my case, the objective is to dequeue this incoming packet first if it does not belong to a flow previously enqueued.
The goal is to decrease the overall latency following a well-known queuing theory property which states that short jobs should be served first. See for details: http://arxiv.org/abs/1103.2303
One big comment during my presentation in Berlin came from a router vendor (I think) who said that such kind of classification (based for instance on a hash table) is not scalable and
not deployable inside a core router. On my side, I've implemented such thing inside ns-2 and I am currently porting the code inside TC GNU/Linux API for edge router (I do not target core router anyway).
Other thing, you'll see inside my paper, that such classification is strongly linked to the traffic pattern and that the dropping policy based on such classification can be greatly unfair as a function of
the traffic pattern (in particular the burstiness and the pacing) and how the sizing of the queue is done (sized with the BDP product, larger, smaller ...).
So, following my experience and looking at my statistical analysis (http://arxiv.org/abs/1103.2303), I believe that you might have unfairness in terms of video quality between flows crossing 
the same path by using such dropping policy. If I have a bit of time, I'll send you some simulation results based on your proposal to check this intuition.

Emmanuel

---

From: Michael Welzl <[email-masked]>
Date: Sun, 20 Oct 2013 22:06:34 +0200

Hi Emmanuel,

The last sentence here: (below)

On 19. okt. 2013, at 08:46, LOCHIN Emmanuel <[email-masked]> wrote:

> the same path by using such dropping policy.

... sounds like a pretty general (not scoped to your mechanism only) statement related to Mirja's proposal, and is, as such, perhaps too pessimistic?

I think it would be extremely important to avoid such unfairness (giving the right incentives is the whole point!), and it should also be quite easy, by simply requiring: *only* drop a packet from a flow if you would have dropped a packet from that flow anyway, based on the "regular" AQM rules => *only* change *which* of the packets from that flow are dropped, i.e. if there's a higher drop precedence packet, drop this one instead of a lower drop precedence one.

...or am I completely off track??

Cheers,
Michael

> If I have a bit of time, I'll send you some simulation results based on your proposal to check this intuition.
> 
> Emmanuel
> 
>> Hi Emmanuel,
>>

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 20 Oct 2013 16:56:58 -0700

Michael,

I really think it's hard to see how to build this mechanism efficiently
into HW. Even in general purpose CPU/memory it seems to be quite expensive.

Given how we want low-latency, i think it is unlikley and undesirable
that a single flow would have to have another low-priority frame in the
queue if it has a high-priority frame that randomnly got elected to be dropped.

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 20 Oct 2013 18:34:58 -0700

Fundamentally, any form of low-priority dropping in itself is
a form of AQM even without other AQM enabled. Simple thought experiment:
20% packets/frames in flows are low priority. Assume we never have more 
than 20% drops in our queue, so we only drop low-priority packets. 

RED for example randomnly subsets flows to avoid global synchronization.
priority dropping would select the flows who can best afford a loss
(eg: the 20% flows with a low priority packet in the queue right now).

Priority dropping is IMHO very beneficial, but it's not totally "free",
aka: the application tht is marking priorities should better know
what to expect and be preapred to react accordingly:

If one in five frames in a flow is low priority (unrealistic but simple
example for the sake of explanation), then a flow with 50 fps would
see drops only once every 100 msec. So the rate-control should better not
upspeed faster than that.

A sender that marks priorities and only sees drop in low priority packets
can likewise tune its rate , for example to linger around a low drop
rate of low priority packets. 

Cheers
   Toerless

On Fri, Oct 18, 2013 at 07:45:49PM +0200, Mirja K?hlewind wrote:
> Hit Emmanuel,

---

From: Michael Welzl <[email-masked]>
Date: Mon, 21 Oct 2013 09:12:56 +0200

> into HW. Even in general purpose CPU/memory it seems to be quite expensive.

1) you said yourself that you target edge equipment. I think we can be a bit more relaxed about such constraints there...
2) So what about CHOKe? Was that not implementable? CHOKe, upon arrival of a packet, picked a random packet from the queue and dropped both if they were from the same sender. This way it punished aggressive senders. A variation of Mirja's suggestion could be a simple extension to CHOKe: if the rule I just described says "yes, drop!", then you only drop the packet with the higher drop precedence, not both.

Someone could argue that dropping both is a key requirement of CHOKe, but then my rule would make it less aggressive at dropping? So consider gCHOKe, a scheme investigated by Addisu Eshete in his Ph.D. thesis:  http://ntnu.diva-portal.org/smash/record.jsf?pid=diva2:583155
a related paper: http://www.sciencedirect.com/science/article/pii/S138912861200326X
gCHOKe applies the CHOKe rule not once, but up to N times ("up to" because it also stops if the check on a similar packet fails) in a row, thereby dropping multiple packets from the queue in one go. As a variation of gCHOKe that incorporates drop precedence, one could apply "only drop the higher drop precedence packet" from iteration M<N onward or something.

Just an example.

> queue if it has a high-priority frame that randomnly got elected to be dropped.

Well yes, the scheme would operate only quite rarely, given that the queues are supposed to be small anyway. This would potentially limit its benefits - by how much would have to be investigated (I think this is one of the issues Toerless mentioned early on in this conversation). Then again, I can't see how you can guarantee to a flow that setting higher drop precedence for some of its packets wouldn't give the flow a worse overall treatment than other flows in the network - and so there is no incentive to use higher drop precedence marking in applications, as today.

Cheers,
Michael

---

From: Emmanuel Lochin <[email-masked]>
Date: Mon, 21 Oct 2013 12:41:08 +0200

On 21/10/2013 09:12, Michael Welzl wrote:
> 1) you said yourself that you target edge equipment. I think we can be 
> a bit more relaxed about such constraints there...
> 2) So what about CHOKe? Was that not implementable? CHOKe, upon 
> arrival of a packet, picked a random packet from the queue and dropped 
> both if they were from the same sender. This way it punished 
> aggressive senders. A variation of Mirja's suggestion could be a 
> simple extension to CHOKe: if the rule I just described says "yes, 
> drop!", then you only drop the packet with the higher drop precedence, 
> not both.

Hi Michael,

As far I understand and to follow Mirja proposal, this could be based on 
my algorithm which in this case, would drop a low priority packet if it 
belongs to another flow inside the queue.
I've proposed to Mirja to look at the way to implement this starting 
from my own code.

Emmanuel

>
> Someone could argue that dropping both is a key requirement of CHOKe, 
> but then my rule would make it less aggressive at dropping? So 
> consider gCHOKe, a scheme investigated by Addisu Eshete in his Ph.D. 
> thesis: http://ntnu.diva-portal.org/smash/record.jsf?pid=diva2:583155
> a related paper: 
> http://www.sciencedirect.com/science/article/pii/S138912861200326X
> gCHOKe applies the CHOKe rule not once, but up to N times ("up to" 
> because it also stops if the check on a similar packet fails) in a 
> row, thereby dropping multiple packets from the queue in one go. As a 
> variation of gCHOKe that incorporates drop precedence, one could apply 
> "only drop the higher drop precedence packet" from iteration M<N 
> onward or something.
>
> Just an example.
>
>
> Well yes, the scheme would operate only quite rarely, given that the 
> queues are supposed to be small anyway. This would potentially limit 
> its benefits - by how much would have to be investigated (I think this 
> is one of the issues Toerless mentioned early on in this 
> conversation). Then again, I can't see how you can guarantee to a flow 
> that setting higher drop precedence for some of its packets wouldn't 
> give the flow a worse overall treatment than other flows in the 
> network - and so there is no incentive to use higher drop precedence 
> marking in applications, as today.
>
> Cheers,
> Michael
>
>
>
>>
>>> I think it would be extremely important to avoid such unfairness 
>>> (giving the right incentives is the whole point!), and it should 
>>> also be quite easy, by simply requiring: *only* drop a packet from a 
>>> flow if you would have dropped a packet from that flow anyway, based 
>>> on the "regular" AQM rules => *only* change *which* of the packets 
>>> from that flow are dropped, i.e. if there's a higher drop precedence 
>>> packet, drop this one instead of a lower drop precedence one.
>>>
>>>> If I have a bit of time, I'll send you some simulation results 
>>>> based on your proposal to check this intuition.
>>>>
>>>> Emmanuel
>>>>
>>>>> Hi Emmanuel,
>>>>>
>> Toerless Eckert, [email-masked] <mailto:[email-masked]>

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 21 Oct 2013 09:17:38 -0700

> 1) you said yourself that you target edge equipment. I think we can be a bit more relaxed about such constraints there...

Sure. But i'd still like to find the most easily implementable solution.

Also: There is important edge equipment thats not generl purpose CPU.
For example MetroE edge equipment (both PE/CE) is often all ASIC based.

> 2) So what about CHOKe? ...

No final opinions on any of the new innovative AQM methods. I am
certainly a fan of getting as much flow awareness into solutions as
feasible, but thats just a personal preference. Pragmatically speaking,
i think AQM-WG will have to vet and decide on AQM methods, and i
would like to track that and see how priority-dropping can be done
via eiter parameterizing or enhancing those AQM methods - and not
adding unreasonalbe implementation complexity in the process.

And (W)RED seems to be the only really currently widely availabl AQM
method and it also serves as a reference for better AQM methods,
so from that perspective it seems most appropriate to have a WRED
based priority-dropping reference.

> Well yes, the scheme would operate only quite rarely, given that the queues are supposed to be small anyway. This would potentially limit its benefits - by how much would have to be investigated (I…

As i said in other emails. Lets think of the most stripped down
priority scheme: There is a standard definition to mark flows
with 20% high-prio, 60% normal, 20% low-prio packets (long term
average) - or not to mark at all. Whatever the marking is going to be.
Edge-equipment  that today authenticates and permit existing markings
(like DSCP) would equally have to authenticate this new priority-marking. 
Hoever that authentication is done. Could just be trust for specific
senders or could be a policer.

So, that leaves authenticated 20/60/20 or unmarked flows, and those
IMHO could essily be made to fairly co-exist in a single queue with
an appropriate WRED profile or hopefully in the future better AQM
that is parameterized/enhanced to do priority-dropping..

And IMHO, the real cool goal would be that all flows with priority
markings would experience only low-priority packet loss 
under all rasonable situations. And that not only helps video
quality under congestion, it also permits a simple improvement
to the rate adaptation to hopefully become better than what's
possible without low-prio dropping.

Cheers
    Toerless