Subject: Re: [AVT] I-D ACTION:draft-ietf-avt-rtcpxr-audio-01.txt

From: Colin Perkins <[email-masked]>
Date: Sun, 2 Dec 2007 08:18:15 -0800

On 19 Nov 2007, at 15:15, [email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
> This draft is a work item of the Audio/Video Transport Working  
> Group of the IETF.
>
> 	Title		: RTCP XR - Audio Metrics Report Block
> 	Author(s)	: A. Clark, A. Pendleton
> 	Filename	: draft-ietf-avt-rtcpxr-audio-01.txt
> 	Pages		: 0
> 	Date		: 2007-11-19
> 	
> This document defines extensions to the RTCP XR extended report
>    packet type blocks to support the performance monitoring of audio
>    streams transmitted using RTP.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-avt-rtcpxr-audio-01.txt

While this is a relatively simple extension, it seems to suffer from  
the problem of being too focussed on a specific application, and  
lacks architectural vision. I'd generally consider it better to  
define metrics that are of general use, which can be combined when  
new applications arise, rather than defining a succession of similar,  
but not quite identical, reports for each class of application. This  
concern was raised in the development of RTCP XR, and is doubly an  
issue given the range of new metrics being defined now.

Details:

An MPEG Program ID is specified both here and in the RTCP XR video  
metrics draft. This implies that it should be a reusable extension,  
rather than being defined in both drafts. Perhaps an (optional) new  
RTCP SDES item would be appropriate?

Why is Round Trip Time reported - can't this be derived from RTCP SR/ 
RR packets?/

The playout buffer metrics seem very similar to the jitter buffer  
metrics defined in RFC 3611. Can't the existing metrics be reused?

Why is there a need to signal the measurement interval? The baseline  
RTCP reports operate on the interval since the last report - is that  
not simpler?

Why are intervals measured in milliseconds, rather than RTCP  
timestamp units? The use of milliseconds seems to require an  
additional clock at the receiver.

-- 
Colin Perkins
http://csperkins.org/

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Schwarz Albrecht" <[email-masked]>
Date: Mon, 3 Dec 2007 13:53:10 +0100

> Why is there a need to signal the measurement interval? The 
> baseline RTCP reports operate on the interval since the last 
> report - is that not simpler?

The baseline RTCP report transmission interval is following a stochastic
model. The measurement interval relates to the RTCP interdeparture time,
i.e. the measurement interval size is *variable*.

Such a statistically distributed interval size is OK for the original
applications considered, when RFC 1889 was developed, but not for
measurement applications (like the ones which could be based on XR or
HR).
Constant measurement interval sizes are more advantegeous than variable
ones.

Further: absolutes times are in the end required with regards to
measurement and statistics data. That's why I do support the proposed
time unit of ms.

-Albrecht

> -----Original Message-----
> From: Colin Perkins [mailto:[email-masked]] 
> Sent: Sonntag, 2. Dezember 2007 17:18
> To: AVT WG
> Subject: Re: [AVT] I-D ACTION:draft-ietf-avt-rtcpxr-audio-01.txt 
> 
> > This draft is a work item of the Audio/Video Transport 
> Working Group 
> > of the IETF.
> >
> While this is a relatively simple extension, it seems to 
> suffer from the problem of being too focussed on a specific 
> application, and lacks architectural vision. I'd generally 
> consider it better to define metrics that are of general use, 
> which can be combined when new applications arise, rather 
> than defining a succession of similar, but not quite 
> identical, reports for each class of application. This 
> concern was raised in the development of RTCP XR, and is 
> doubly an issue given the range of new metrics being defined now.
> 
> Details:
> 
> An MPEG Program ID is specified both here and in the RTCP XR 
> video metrics draft. This implies that it should be a 
> reusable extension, rather than being defined in both drafts. 
> Perhaps an (optional) new RTCP SDES item would be appropriate?
> 
> Why is Round Trip Time reported - can't this be derived from 
> RTCP SR/ RR packets?/
> 
> The playout buffer metrics seem very similar to the jitter 
> buffer metrics defined in RFC 3611. Can't the existing 
> metrics be reused?
> 
> Why is there a need to signal the measurement interval? The 
> baseline RTCP reports operate on the interval since the last 
> report - is that not simpler?
>

---

From: Colin Perkins <[email-masked]>
Date: Mon, 3 Dec 2007 05:29:32 -0800

> i.e. the measurement interval size is *variable*.

Sure, but you know the exact interval covered.

> Constant measurement interval sizes are more advantegeous than  

Why?

> measurement and statistics data. That's why I do support the proposed

You can convert to milliseconds for display should you wish, so  
that's a non-issue. Architecturally, one shouldn't introduce another  
clock into the system with no reason. The existing clocks are the  
media clock, and the NTP-format clock used for SR/RR reports  
(reported in 1/65535 of a second intervals). One of those should be  
used.

-- 
Colin Perkins
http://csperkins.org/

---

From: "Alan Clark" <[email-masked]>
Date: Mon, 3 Dec 2007 11:27:35 -0500

Colin

>An MPEG Program ID is specified both here and in the RTCP XR video metrics
draft. This implies that it 
should be a reusable extension, rather than being defined in both drafts.
Perhaps an (optional) new RTCP SDES item would be appropriate?

Certainly the approach of a header to identify the stream could be useful.
However - in the case of MPEG Transport, the audio and video streams have
separate PIDs hence the audio and video metrics blocks would need to be
identified separately anyway.  

>Why is Round Trip Time reported - can't this be derived from RTCP SR/ RR
packets?/

I agree that endpoints can do this.  It is non-trivial for any midpoint
system (e.g. a probe or router) to calculate RTD from SR/RR and requires the
capture of a series of SR/RR exchanges, maintaining state etc.  

>The playout buffer metrics seem very similar to the jitter buffer metrics
defined in RFC 3611. Can't the existing metrics be reused?

The main issues are:

(i) Size - playout buffers can be much larger, potentially 10's of seconds

(ii) The playout metrics allow problems such as playout interruptions due to
buffer starvation in TCP based systems to be monitored.

>Why is there a need to signal the measurement interval? The baseline RTCP
reports operate on the interval since the last report - is that not simpler?

This requires monitoring systems to recognize sequential RTCP reports, would
not deal properly with lost RTCP reports, and for midpoint systems would
cause problems if routes changed.  

>Why are intervals measured in milliseconds, rather than RTCP timestamp
units? The use of milliseconds seems to require an additional clock at the
receiver.

Really just convenience - this does not require an additional clock, just
some simple math.  It also means that if the RTP clock rate associated with
the stream changed, or the video clock rate was different to the audio clock
rate, it may be complex for any entity other than the endpoint to figure out
what the interval was. 

Regards

Alan

---

From: Randell Jesup <[email-masked]>
Date: Mon, 03 Dec 2007 13:58:50 -0500

>capture of a series of SR/RR exchanges, maintaining state etc.  

This is a valid point - it's hard to calculate if you're not an endpoint;
various types of midboxes can use this info, and it can be useful for
overall network-health monitoring.  I've seen it proposed (in the area of
RFC 3611) recently (and also via end-of-call SIP PUBLISH).

I should note that RFC 3611 also reports delays in ms (section 4.7.3),
using RTCP NTP times per the normal spec for DLRR/etc.

>(i) Size - playout buffers can be much larger, potentially 10's of seconds

Greater than 65 seconds?  That's how long the jitter buffer params in 3611
can cover (16 bits at 1ms/per).

>buffer starvation in TCP based systems to be monitored.

That may be useful, depending on definition (note: I haven't read the spec
to see how this is defined/transmitted).

>cause problems if routes changed.  

I don't think this adds siginificant complexity, at least not the
sequential aspect.  The RTCP has info that lets you know the sequence
numbers/timestamps received/transmitted, and the time and period.  It's
easy to figure out if an RTCP was "lost", just by retaining the last one
(see note).  However, whether the period is fixed or not, what does a
midbox/etc do if an RTCP is lost anyways?  How is this really different
(let alone tough) if the packets are of variable length?

(Note: in AVPF scenarios, you can get a number of RTCPs sent in a very
short period, and *in theory*, they could be received out-of-order due to
route-flapping, etc.  Something to realize can happen, even if it's rare.
The simplest solution (depending on what you use RTCP for) is to ignore the
"stale" out-of-order RTCP as if it were lost.)

>rate, it may be complex for any entity other than the endpoint to figure out

There still is confusion over RTP streams switching timestamps - whether
it's really allowed (it appears to be), and the side-effects (very
confusing for RTCP, and other things):

Random example (I probably have the G722 media type wrong):
m=audio 2345 RTP/AVP 0 99
a=rtpmap:0 PCMU/8000
a=rtpmap:99 G722/16000

This means the timestamp rate can change at any point, on a
packet-by-packet basis.  It's even theoretically allowable to alternate
G711 and G722 packets.  Totally odd and non-useful, but it illustrates the
point.  More realistic is a change from one to the other half-way through
an RTCP monitoring period.

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: "Alan Clark" <[email-masked]>
Date: Mon, 3 Dec 2007 14:57:01 -0500

Well.... most would be up to 20 seconds however you could potentially have a
TCP based streaming system with a much larger buffer 

>>RTCP reports operate on the interval since the last report - is that not
simpler?
>
>This requires monitoring systems to recognize sequential RTCP reports, 
>would not deal properly with lost RTCP reports, and for midpoint 
>systems would cause problems if routes changed.

sequential aspect.  The RTCP has info that lets you know the sequence
numbers/timestamps received/transmitted, and the time and period.  It's easy
to figure out if an RTCP was "lost", just by retaining the last one (see
note).  However, whether the period is fixed or not, what does a midbox/etc
do if an RTCP is lost anyways?  How is this really different (let alone
tough) if the packets are of variable length?

If the midpoint is a router or SBC handling 50,000 concurrent streams
through a distributed network processor architecture (i.e. each packet may
be assigned to any microengine) then it is more practical to simply capture
an RTCP packet and relay it up to a metrics processing / traffic analysis
function than to have to track relationships between adjacent RTCP payloads.

Regards

Alan

---

From: Randell Jesup <[email-masked]>
Date: Mon, 03 Dec 2007 18:46:58 -0500

>TCP based streaming system with a much larger buffer 

Ok, but you'll need to make a justification why the 65 seconds available
for reporting in 3611 isn't sufficient.  (65 seconds doesn't mean you can't
have a deeper buffer, just that the 3611 stats won't show more than 65.)

>function than to have to track relationships between adjacent RTCP payloads.

Except that the "metrics-processing/traffic-analysis" engine *can* track
relationships between adjacent RTCP payloads, I assume.  And it only
matters if you need to do something active based on the loss of an RTCP
packet (which is normally possible, though over TCP it wouldn't be lost).

So I understand the router/SBC scenario, but you haven't shown why that
leads to a requirement.

---

From: "Schwarz Albrecht" <[email-masked]>
Date: Wed, 5 Dec 2007 08:46:35 +0100

Hi Collin,

there's an RTCP protocol oriented view of the problem and an application
perspective.
There might be many different served user instances ('application') for
XR/HR information. The users may be locally (in the RTP endpoint) but
also remotely.
[An example for remote processing is e.g. the ETSI TISPAN R2 network
overlay infrastructure for QoS reporting.]

I may follow your arguments somehow when having the pure local view in
mind.
Please take a more application-centric view of the problem, and consider
also remote served users ("remote means that XR/HR are reported again
via non-RTCP based reporting interfaces, e.g. like SIP or H.248").

Responses to your comments:
a) relative time: ok locally, but remote processing implies a second
piece of information in order to derivate the absolute time; I fail to
see the benefit of relative time

b) equidistant measurement interval size: think about statistical
functions above the time scale of the measurement interval itself; there
are then functions based on time series (i.e., temporal composition of
metrics; note: the spatial composition, as considered by IPPM, is a
different story).
... such time series based functions are much simpler when time
intervals are equidistant 

-Albrecht

> Sent: Montag, 3. Dezember 2007 14:30
> To: Schwarz Albrecht
> Cc: AVT WG
> > The baseline RTCP report transmission interval is following a 
> > stochastic model. The measurement interval relates to the RTCP 
> > interdeparture time, i.e. the measurement interval size is 
> *variable*.
> 
> > Constant measurement interval sizes are more advantegeous than 
> > variable ones.
> 
> Why?
> 
> You can convert to milliseconds for display should you wish, 
> so that's a non-issue. Architecturally, one shouldn't 
> introduce another clock into the system with no reason. The 
> existing clocks are the media clock, and the NTP-format clock 
> used for SR/RR reports (reported in 1/65535 of a second 
> intervals). One of those should be used.
> 
> --
> Colin Perkins
> http://csperkins.org/
> 
> 
>

---

From: Randell Jesup <[email-masked]>
Date: Wed, 05 Dec 2007 14:31:41 -0500

>there's an RTCP protocol oriented view of the problem and an application

Or perhaps more directly: an endpoint-oriented view, and a network/midbox
oriented view.

>via non-RTCP based reporting interfaces, e.g. like SIP or H.248").

Well, the transport for the information shouldn't matter.  The "user"
(network/midbox) in this scenario gets the data blocks, whether they come
by RTCP or by SIP, etc.

>see the benefit of relative time

Colin was saying to use an existing clock/format, such as an NTP time,
instead of ms.  And if a remote processor needs an additional bit of data
from the RTCP, then whatever is sending the report block to the remote 
processor should send the additional data as well (such as SR/RR from the
RTCP, or the relevant data from it).  That said, I'll note that
RFC 3611 has lots of stuff measured in ms in it.  And the media clock is
problematic unless we add restrictions that lock the media clock for all
payloads of a stream to the same value (which isn't going to happen, even
though it causes untold possibilities for annoyance).

>intervals are equidistant 

Sure, they're easier to calculate.  However, even under the best of
circumstances you can't count on the intervals being truly equidistant, and
in some quite possible cases (RTCP bandwidth restricted by the algorithm
due to competing traffic from the same or other endpoints,
endpoint/application issues, etc) you can get wildly non-equidistant
measurements (factors of 2x or more).  So your stats functions *have* to
handle both minor non-uniform periods (due to sampling error and real-time
preemption), and major non-uniformity (due to RTCP
bandwidth/algorithm/etc).

---

From: Colin Perkins <[email-masked]>
Date: Mon, 17 Dec 2007 18:06:32 +0000

On 5 Dec 2007, at 19:31, Randell Jesup wrote:
> Or perhaps more directly: an endpoint-oriented view, and a network/ 
> midbox oriented view.
>
> (network/midbox) in this scenario gets the data blocks, whether  
> they come by RTCP or by SIP, etc.
>
> Colin was saying to use an existing clock/format, such as an NTP  
> time, instead of ms.  And if a remote processor needs an additional  
> bit of data from the RTCP, then whatever is sending the report  
> block to the remote processor should send the additional data as  
> well (such as SR/RR from the RTCP, or the relevant data from it).   
> That said, I'll note that RFC 3611 has lots of stuff measured in ms  
> in it.  And the media clock is problematic unless we add  
> restrictions that lock the media clock for all payloads of a stream  
> to the same value (which isn't going to happen, even though it  
> causes untold possibilities for annoyance).

You've convinced me that media time won't work here, but the  
millisecond timers in RFC 3611 were a mistake we shouldn't repeat.  
The NTP time used in RTCP SR/RR packets seems the best clock to use.

>> functions above the time scale of the measurement interval itself;  
>> there are then functions based on time series (i.e., temporal  
>> composition of metrics; note: the spatial composition, as  
>> considered by IPPM, is a different story). ... such time series  
>> based functions are much simpler when time intervals are equidistant
>
> circumstances you can't count on the intervals being truly  
> equidistant, and in some quite possible cases (RTCP bandwidth  
> restricted by the algorithm due to competing traffic from the same  
> or other endpoints, endpoint/application issues, etc) you can get  
> wildly non-equidistant measurements (factors of 2x or more).  So  
> your stats functions *have* to handle both minor non-uniform  
> periods (due to sampling error and real-time preemption), and major  
> non-uniformity (due to RTCP bandwidth/algorithm/etc).

Right - much as it would be easier to have fixed intervals, it's just  
not an option given an IP network and the RTCP reporting architecture.

-- 
Colin Perkins
http://csperkins.org/