Subject: Re: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

From: [email-masked]
Date: Tue, 1 Nov 2011 16:33:40 +0800 (CST)

I agree more samples, more accurate of the metric.

but my interpretation to what Alan said here is if we have 6 samples in one designated interval(x0,x1,x2,x3,x4,x5), the sampling interval bewteen two adjacent samples is set different, in this case, how to calculate average?

Cheers!

-Jing

------------------ 原始邮件 ------------------

发件人：Qin Wu

时　间：2011/11/01 16:17:07 星期二

收件人：Alan Clark

, [email-masked]

抄送人：xrblock

主　题：答复: [xrblock] 答复: General comments on xrblock metrics drafts

Interval averages can be quite misleading if the measurements are strongly time varying, which PDV typically is.

[Qin]: I agree people often confuse some statistics measurement that use sample with average. But Sample is not a new concept and widely used in the RFC 3550(e.g., page 15 for the definition of timestample), for example,

you can sample SSRC, you can sample packet, the sample instant is choosen as reference point. jitter calculation is based on the RTP timestamp which
represents the instant when the first data in the packet was sampled. The sample implementation is allowed in the RFC3550.

Also, in my understanding the sample will be used only when it is impractical to keep track of all the packets.

So it looks to me if you gather enough samples within interval, you will get a more accurate average or other statistics measurment that does not rely on per-packet statistics.

Best Regards

Alan

On 10/31/11 5:23 AM, "[email-masked]" <[email-masked]> wrote:

Hi, Alan:

Question for clarification.

Isn't the sampled metric same as the Interval metric? Is the sampled metric only applied to per-packet reporting ? what's the difference between sampled metric and non-sampled metric?

what's the difference between sampled metric and Interval metric? Would you like to clarify more?

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark
时　间：2011/10/31 06:51:03 星期一
收件人：[email-masked]
抄送人：
主　题：[xrblock] General comments on xrblock metrics drafts

1. Cumulative, Interval and Sample metrics
The drafts current have one bit assigned for Cumulative/Interval indication... Some metrics are however sampled and don’t have validity for an interval or a call/session; an example of this is PPDV, which is a sampled value of a running average and only has validity for the last 200-300ms before the value is sampled.
Proposal - increase this to two bits to allow for the indication of a Sampled value.

---

From: Qin Wu <[email-masked]>
Date: Tue, 01 Nov 2011 16:52:53 +0800

Are yo saying interval(xi-1, xi) does not equal to interval(xi,xi+1), it looks weird to me.
My counter question is:
Do we have the use case for such measurment?
Do we have metrics in the currect xrblock metrics drafts that use such measurement?

Regards!
-Qin
  ----- Original Message ----- 
  From: [email-masked] 
  To: Qin Wu ; Alan Clark 
  Cc: xrblock 
  Sent: Tuesday, November 01, 2011 4:33 PM
  Subject: Re: [xrblock] 答复: 答复: General comments on xrblock metrics drafts

   
  Cheers! 
  -Jing 
  ------------------ 原始邮件 ------------------ 

  发件人：Qin Wu 
  时　间：2011/11/01 16:17:07 星期二 
  收件人：Alan Clark , [email-masked] 
  抄送人：xrblock 
  主　题：答复: [xrblock] 答复: General comments on xrblock metrics drafts 

  Best Regards

  Alan

    Hi, Alan: 

    Cheers! 

    Jing 

    ------------------ 原始邮件 ------------------ 

    发件人：Alan Clark 

------------------------------------------------------------------------------

  _______________________________________________
  xrblock mailing list
  [email-masked]
  https://www.ietf.org/mailman/listinfo/xrblock

---

From: Alan Clark <[email-masked]>
Date: Tue, 1 Nov 2011 17:28:30 -0400

Jing

I’m not describing how the metric is calculated but what it refers to.  So
“Sample” in this case means the sampled value of a continuously varying
metric, such as PPDV or audio signal level.

Best Regards

Alan

On 11/1/11 4:33 AM, "[email-masked]" <[email-masked]> wrote:

>  but my interpretation to what Alan said here is if we have 6 samples in one
> designated interval(x0,x1,x2,x3,x4,x5), the sampling interval bewteen two
> adjacent samples is set different, in this case, how to calculate average?
>   
>    
>   
>  Cheers! 
>   
>  -Jing 
>   
> [Qin]: I agree people often confuse some statistics measurement that use
> sample with average. But Sample is not a new concept and widely used in the
> RFC 3550(e.g., page 15 for the definition of timestample), for example,
>   
> 
> you can sample SSRC, you can sample packet, the sample instant is choosen as
> reference point. jitter calculation is based on the RTP timestamp which
> represents the instant when the first data in the packet was sampled. The
> sample implementation is allowed in the RFC3550.
>   
> 
> So it looks to me if you gather enough samples within interval, you will get a
> more accurate average or other statistics measurment that does not rely on
> per-packet statistics.
> 
> Best Regards
> 
> Alan
> 
> 
> On 10/31/11 5:23 AM, "[email-masked]
> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked]
> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:
> 
>   
>   
>   
>   
>   
>   
>>  
>> Hi, Alan: 
>> 
>> Isn't the sampled metric same as the Interval metric? Is the sampled metric
>> only applied to per-packet reporting ? what's the difference between sampled
>> metric and non-sampled metric?
>> 
>> 收件人：[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
>> 抄送人： 
>> The drafts current have one bit assigned for Cumulative/Interval
>> indication... Some metrics are however sampled and don’t have validity for an
>> interval or a call/session; an example of this is PPDV, which is a sampled
>> value of a running average and only has validity for the last 200-300ms
>> before the value is sampled.

---

From: Alan Clark <[email-masked]>
Date: Tue, 1 Nov 2011 17:36:56 -0400

Qin

Some measurable parameters of an IP stream are non-stationary, for example
PDV and loss rate.  So these would certainly vary from interval to interval.
It is easy to find cases in which loss and jitter are highly variable - I
can provide some references if this would be useful

Since the MONARCH/XR work should be about establishing a framework that can
be used for both the current and future metrics drafts it would be sensible
to make provision for this, whether or not the current drafts include
metrics that should be reported as sampled.  The PDV draft should be
reviewed to see whether it would be preferable to report PDV for an interval
or as a sample, as this is a case in which the network property being
measured is strongly time varying.

In terms of a use case for the measurement - PPDV is an existing RTP metric
of this type and hence there is a case in which a sampled metric has been
used

Best Regards

Alan

---

From: Qin Wu <[email-masked]>
Date: Wed, 02 Nov 2011 12:31:25 +0800

Re: [xrblock] ??:  ??:  General comments on xrblock metrics draftsHi, Alan:
I am not doubt some measurement parameters vary from interval to interval, but it does not matter since  what we look for is 
the statistics value that presents key features of any transport impairments during a period rather than raw data value varied from time to time or 
average value at a sample instance. You can not prevent statistics value changing.

Except Loss RLE, Duplicate RLE, and Packet Receipt Times Report Blocks defined in RFC3611, most of other metrics blocks defined
in XRBLock are sampled metric you called or derived metric I called. For details, Please see my response in the previous email.

Also MONARCH/XR work has already distinguish basic metric from derived metric in the section 3.

Regards!
-Qin
  ----- Original Message ----- 
  From: Alan Clark 
  To: Qin Wu ; [email-masked] 
  Cc: xrblock 
  Sent: Wednesday, November 02, 2011 5:36 AM
  Subject: Re: [xrblock] µªÎ`: µªÎ`: General comments on xrblock metrics drafts

  Qin

  Some measurable parameters of an IP stream are non-stationary, for example PDV and loss rate.  So these would certainly vary from interval to interval.  It is easy to find cases in which loss and jitter are highly variable - I can provide some references if this would be useful

  Since the MONARCH/XR work should be about establishing a framework that can be used for both the current and future metrics drafts it would be sensible to make provision for this, whether or not the current drafts include metrics that should be reported as sampled.  The PDV draft should be reviewed to see whether it would be preferable to report PDV for an interval or as a sample, as this is a case in which the network property being measured is strongly time varying.

  In terms of a use case for the measurement - PPDV is an existing RTP metric of this type and hence there is a case in which a sampled metric has been used

  Best Regards

  Alan

     
    Regards!
    -Qin