Subject: [AVT] RE: Suggestion for how to handle derived/computedmetricindraft-avt-rtcpxr-*

From: "Ingemar Johansson S (LU/EAB)" <[email-masked]>
Date: Fri, 27 Jul 2007 04:07:47 +0200

Hi

Coming from the area to speech and audio compression I can only second the opinion that you really must specify what algrithm you are using. Lots of work has been done to try to pair subjective quality measurements against objective measurements and the results are not always straightforward. One typical problem is impact of longer packet loss where the models and what people actually perceive differs.

Another big problem is that I cannot see how you account for jitterbuffer adaptation in this case, especially if you use adaptive jitterbuffering. My fear is that you only get to a point where you compare apples and oranges. But I wont argue more about the applications and the reliability. 

Besides all this I find it a bit too static to specify binary RTCP report blocks for every kind of computed metric that one can come up with. A better solution would be some kind of an XML structure that allows for more dynamics. I know it is probably/maybe not doable for bandwidth reasons but extensibility would be alot easier then. 

Why use RTCP at all for this ?, I sort of get the feeling that these are reports that are sent by the clients when he hangs up and the clients gather up all the statistics.

Regards
Ingemar
*******************************************
Ingemar Johansson
Senior Research Engineer
EAB/TVP - Multimedia Technologies 
  Ericsson Research Ericsson AB 
  Box 920 S-971 28 Luleå, Sweden
Tel: +46 (0)8 4043042
ECN: 850-43042
Mobile: +46 (0)730 783289
MSN: [email-masked]
*******************************************

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Colin Perkins <[email-masked]>
Date: Fri, 27 Jul 2007 07:00:31 -0500

Hi,

On 26 Jul 2007, at 21:07, Ingemar Johansson S (LU/EAB) wrote:
> Coming from the area to speech and audio compression I can only  
> second the opinion that you really must specify what algrithm you  
> are using. Lots of work has been done to try to pair subjective  
> quality measurements against objective measurements and the results  
> are not always straightforward. One typical problem is impact of  
> longer packet loss where the models and what people actually  
> perceive differs.
>
> Another big problem is that I cannot see how you account for  
> jitterbuffer adaptation in this case, especially if you use  
> adaptive jitterbuffering. My fear is that you only get to a point  
> where you compare apples and oranges. But I wont argue more about  
> the applications and the reliability.
>
> Besides all this I find it a bit too static to specify binary RTCP  
> report blocks for every kind of computed metric that one can come  
> up with. A better solution would be some kind of an XML structure  
> that allows for more dynamics. I know it is probably/maybe not  
> doable for bandwidth reasons but extensibility would be alot easier  
> then.

I'm not sure we want this to be too easily extensible. A limited set  
of well defined metrics probably makes analysis easier than a very  
extensible and flexible system, filled with "metric of the day"  
parameters.

-- 
Colin Perkins
http://csperkins.org/

---

From: "Ingemar Johansson S (LU/EAB)" <[email-masked]>
Date: Fri, 27 Jul 2007 16:29:45 +0200

Hi

Understand the "metric of the day" concern but I don't believe that this
will happen as the addition of metrics will need to be standardized
anyway. It is however possible that this standardization is done in
other fora such as TISPAN or 3GPP as I believe that detailed speech
quality metrics is not IETF's cup of tea. What IETF can do is to specify
a framework for this that is expected to work well in the packet
switched environment.

Just to second my "why use RTCP". 
RTCP is, as I understand it, to a great extent a means to send signaling
and metrics end to end, possibly modified by a middle box. These more
elaborate metrics are in my opinion mostly of value for an operator but
of lesser value for the user. 
Therefore it is something that the client can transmit to the ISP or
operator in a number of different (more relaible) ways after the call is
completed or during a call if needed. The need to make the whole thing a
binary bit field is then also less important.

Just my 0.05 EUR

Regards
Ingemar

> -----Original Message-----
> From: Colin Perkins [mailto:[email-masked]] 
> Sent: den 27 juli 2007 07:01
> To: Ingemar Johansson S (LU/EAB)
> Cc: [email-masked]
> Subject: Re: [AVT] RE: Suggestion for how to handle 
> derived/computedmetricindraft-avt-rtcpxr-*
> 
> Hi,
> 
> > Coming from the area to speech and audio compression I can 
> only second 
> > the opinion that you really must specify what algrithm you 
> are using. 
> > Lots of work has been done to try to pair subjective quality 
> > measurements against objective measurements and the results are not 
> > always straightforward. One typical problem is impact of 
> longer packet 
> > loss where the models and what people actually perceive differs.
> >
> adaptive 
> > jitterbuffering. My fear is that you only get to a point where you 
> > compare apples and oranges. But I wont argue more about the 
> > applications and the reliability.
> >
> > report blocks for every kind of computed metric that one 
> can come up 
> > with. A better solution would be some kind of an XML structure that 
> > allows for more dynamics. I know it is probably/maybe not 
> doable for 
> > bandwidth reasons but extensibility would be alot easier then.
> 
> I'm not sure we want this to be too easily extensible. A 
> limited set of well defined metrics probably makes analysis 
> easier than a very extensible and flexible system, filled 
> with "metric of the day"  
> parameters.
> 
> -- 
> Colin Perkins
> http://csperkins.org/
> 
> 
>