Subject: Re: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

From: [email-masked]
Date: Wed, 2 Nov 2011 15:26:46 +0800 (CST)

I would rather like to regard PPDV described in RFC3550 as cumulative metric since it rely on all the past observation as input.

Also if the digital signal processors (DSPs) can be used to calculate metrics, both Interval metric and cumulative can be calculated and reported immedieately.

Cheers!

-Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark

时　间：2011/11/02 05:28:30 星期三

收件人：[email-masked], Qin Wu

抄送人：xrblock

主　题：Re: 答复: [xrblock] 答复: General comments on xrblock metrics drafts

Jing

I’m not describing how the metric is calculated but what it refers to.. So “Sample” in this case means the sampled value of a continuously varying metric, such as PPDV or audio signal level.

Best Regards

Alan

On 11/1/11 4:33 AM, "[email-masked]" <[email-masked]> wrote:

I agree more samples, more accurate of the metric.

but my interpretation to what Alan said here is if we have 6 samples in one designated interval(x0,x1,x2,x3,x4,x5), the sampling interval bewteen two adjacent samples is set different, in this case, how to calculate average?

Cheers!

-Jing

------------------ 原始邮件 ------------------

发件人：Qin Wu
时　间：2011/11/01 16:17:07 星期二
收件人：Alan Clark , [email-masked]
抄送人：xrblock
主　题：答复: [xrblock] 答复: General comments on xrblock metrics drafts

Interval averages can be quite misleading if the measurements are strongly time varying, which PDV typically is.

[Qin]: I agree people often confuse some statistics measurement that use sample with average. But Sample is not a new concept and widely used in the RFC 3550(e.g., page 15 for the definition of timestample), for example,

you can sample SSRC, you can sample packet, the sample instant is choosen as reference point. jitter calculation is based on the RTP timestamp which
represents the instant when the first data in the packet was sampled. The sample implementation is allowed in the RFC3550.

Also, in my understanding the sample will be used only when it is impractical to keep track of all the packets.

So it looks to me if you gather enough samples within interval, you will get a more accurate average or other statistics measurment that does not rely on per-packet statistics.

Best Regards

Alan

On 10/31/11 5:23 AM, "[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:

Hi, Alan:

Question for clarification.

Isn't the sampled metric same as the Interval metric? Is the sampled metric only applied to per-packet reporting ? what's the difference between sampled metric and non-sampled metric?

what's the difference between sampled metric and Interval metric? Would you like to clarify more?

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark
时　间：2011/10/31 06:51:03 星期一
收件人：[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
抄送人：
主　题：[xrblock] General comments on xrblock metrics drafts

1. Cumulative, Interval and Sample metrics
The drafts current have one bit assigned for Cumulative/Interval indication.... Some metrics are however sampled and don’t have validity for an interval or a call/session; an example of this is PPDV, which is a sampled value of a running average and only has validity for the last 200-300ms before the value is sampled.
Proposal - increase this to two bits to allow for the indication of a Sampled value.

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Wed, 2 Nov 2011 12:10:13 +0100

Folks,
this is getting more and more confusing.

Jing,
don't mix up implementation with specification. This is firstly a specification issue, looks like that the terminology is not sufficiently precisely defined.

Further: separate measurements from reporting, both are different functions.

All considered (by RTCP XR) metrics could be modelled as time-dependent stochastical variables X(t).
Using such an approach you may describe the observation, measurement, estimation or/and the generation of statistics based on that variable.

If the term 'metric' is already defined, please provide a reference to a correspondent RFC.

You may also use ITU-T E.800 as baseline, in particular the terms parameter, characteristic, measure, metric, etc.

And there would be finally a difference between a sample or an average-based (like over a single interval, or over all intervals (= cumulative) value of a particular performance parameter.

Regards,
Albrecht

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Mittwoch, 2. November 2011 08:27
To: Alan Clark; Qin Wu
Cc: xrblock
Subject: Re: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

------------------ 原始邮件 ------------------

发件人：Alan Clark

Jing

Best Regards

Alan

On 11/1/11 4:33 AM, "[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>" <[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>> wrote:

Cheers!

-Jing

------------------ 原始邮件 ------------------

发件人：Qin Wu
时　间：2011/11/01 16:17:07 星期二
收件人：Alan Clark , [email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>
抄送人：xrblock
主　题：答复: [xrblock] 答复: General comments on xrblock metrics drafts

Best Regards

Alan

On 10/31/11 5:23 AM, "[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:

Hi, Alan:

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark
时　间：2011/10/31 06:51:03 星期一
收件人：[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件人：[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
抄送人：
主　题：[xrblock] General comments on xrblock metrics drafts

---

From: Alan Clark <[email-masked]>
Date: Wed, 2 Nov 2011 22:10:07 -0400

Jing

One would generally expect that a cumulative metrics indicates performance
from the start of a session to the point in time at which the report is sent
(or at least created).  This would be meaningful for a parameter such as
packet loss, but not PPDV.

PPDV is a running average of the packet-to-packet delay variation, as in
PPDV = (PPDV*15 + X) / 16

PDV (jitter) is often strongly time varying. If there was a period of high
jitter early in the session, the PPDV value would go up during this period
however would then drop by 1/16th per packet. If the “high” value of PPDV
was “P”  then after 100 packets this value would have dropped to 0.002P.  So
PPDV only relates to the jitter level during the most recent 20-30 packets.

You “could” measure the average PDV value for an interval or the session
using 

Interval Average PDV = Sum(per packet PDV) / Number of packets

This would be regarded as an interval or cumulative metric however the
RFC3550 PPDV metric is better described as the sampled value of the running
average 

Best Regards

Alan

On 11/2/11 3:26 AM, "[email-masked]" <[email-masked]> wrote:

>    
>    
>   
>  I would rather like to regard PPDV described in RFC3550 as cumulative metric
> since it rely on all the past observation as input.
>   
>    
>   
>  Also if the digital signal processors (DSPs) can be used to calculate
> metrics, both Interval metric and cumulative can be calculated and reported
> immedieately.  
>   
>    
>   
>  Cheers! 
>   
>  -Jing 
>   
> I’m not describing how the metric is calculated but what it refers to.. So
> “Sample” in this case means the sampled value of a continuously varying
> metric, such as PPDV or audio signal level.
> 
> Best Regards
> 
> Alan
> 
> 
> On 11/1/11 4:33 AM, "[email-masked]
> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@stt
> ri.com.cn> " <[email-masked]
> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@stt
> ri.com.cn> > wrote:
> 
>     
>   
>   
>>  
>> but my interpretation to what Alan said here is if we have 6 samples in one
>> designated interval(x0,x1,x2,x3,x4,x5), the sampling interval bewteen two
>> adjacent samples is set different, in this case, how to calculate average?
>> 
>> 
>> 
>> Cheers! 
>> 
>> -Jing 
>> 
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  
>> 抄送人：xrblock 
>>  [Qin]: I agree people often confuse some statistics measurement that use
>> sample with average. But Sample is not a new concept and widely used in the
>> RFC 3550(e.g., page 15 for the definition of timestample), for example,
>>   
>> 
>>  you can sample SSRC, you can sample packet, the sample instant is choosen as
>> reference point. jitter calculation is based on the RTP timestamp which
>> represents the instant when the first data in the packet was sampled. The
>> sample implementation is allowed in the RFC3550.
>>   
>> 
>>  Also, in my understanding the sample will be used only when it is
>> impractical to keep track of all the packets.
>>   
>> 
>>  So it looks to me if you gather enough samples within interval, you will get
>> a more accurate average or other statistics measurment that does not rely on
>> per-packet statistics.
>> 
>> Best Regards
>> 
>> Alan
>> 
>> 
>> On 10/31/11 5:23 AM, "[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> "
>> <[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:
>> 
>>   
>>  
>> 
>>   
>>  
>> 
>>   
>>>  
>>>  Hi, Alan: 
>>> 
>>> Isn't the sampled metric same as the Interval metric? Is the sampled metric
>>> only applied to per-packet reporting ? what's the difference between sampled
>>> metric and non-sampled metric?
>>> 
>>> 收件人：[email-masked]
>>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件
>>> 人：[email-masked]>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
>>> 抄送人： 
>>> The drafts current have one bit assigned for Cumulative/Interval
>>> indication.... Some metrics are however sampled and don’t have validity for
>>> an interval or a call/session; an example of this is PPDV, which is a
>>> sampled value of a running average and only has validity for the last
>>> 200-300ms before the value is sampled.

---

From: Qin Wu <[email-masked]>
Date: Thu, 03 Nov 2011 14:54:52 +0800

Jing:
Albrecht is right. "DSP" is not specification term but implmentation term.
The metric we are talking about here is QoS/QoE Metric and is referred to each measurement result carried in the same XR Block payload.
we can refer to RFC6390 for QoS/QoE metric explaination. This is also what charter said we should get alignment with PMOL work. The PMOL
work is referred to as RFC6390.

Regards!
-Qin
  ----- Original Message ----- 
  From: Schwarz, Albrecht (Albrecht) 
  To: [email-masked] ; Alan Clark ; Qin Wu 
  Cc: xrblock 
  Sent: Wednesday, November 02, 2011 7:10 PM
  Subject: RE: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

  Folks, 

  Jing,

  Regards,
  Albrecht

----------------------------------------------------------------------------
    From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
    Sent: Mittwoch, 2. November 2011 08:27
    To: Alan Clark; Qin Wu
    Cc: xrblock
    Subject: Re: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

    ------------------ 原始邮件 ------------------ 

    发件人：Alan Clark 

    Jing

    Best Regards

    Alan

      Cheers! 

      -Jing 

      ------------------ 原始邮件 ------------------ 

      发件人：Qin Wu 

      Best Regards

      Alan

        Hi, Alan: 

        Cheers! 

        Jing 

        ------------------ 原始邮件 ------------------ 

        发件人：Alan Clark