Subject: Re: [xrblock] 答复: General comments on xrblock metrics drafts

From: [email-masked]
Date: Wed, 2 Nov 2011 14:59:13 +0800 (CST)

I know you intend to seperate Sampled metric from Interval Metric. However according to the defintion of Interval metric definition:

"

The Interval metric is refered to as the reported values apply to the most recent measurement

interval duration between successive metrics reports

"

In your understanding, the samepled metric is referred to as the measured value is sampled at the end of interval or apply to the end of interval.

Is that what you say?

But I am not convinced. It looks to me the sampled metric can be classified into cumulative metric.

Take PPDV as example, the J(i)= J(i-1)++ (|D(i-1,i)| - J(i-1))/16

Isn't J(i) a cumulative metric since the J(i) is the accumulation period characteristic that combine J(i-1),J(i-2),..J(0)

during cumulative duration?

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark

时　间：2011/11/02 05:25:01 星期三

收件人：[email-masked], Qin Wu

抄送人：xrblock

主　题：Re: 答复: [xrblock] General comments on xrblock metrics drafts

Jing

An Interval metric is calculated over the interval - so packet loss for an interval would be the packets lost during the interval divided by the number of packets expected.

If the packet loss rate was continuously calculated as a running average (updated per packet) and the value of this packet loss rate metric was sampled at the end of the interval, it would be a sampled value of the metric and not the interval value.

Hence there is a difference between an interval metric and a sampled metric, which IMHO means that we should have a way to state that a metric is Cumulative, Interval or Sampled.

Best Regards

Alan

On 11/1/11 4:21 AM, "[email-masked]" <[email-masked]> wrote:

Are you saying the sample semantics has already been incorporate into Interval/Cumulative indication?

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Qin Wu
时　间：2011/11/01 15:19:54 星期二
收件人：Alan Clark , [email-masked]
抄送人：xrblock
主　题：答复: [xrblock] General comments on xrblock metrics drafts

Hi, Alan:

So the Interval metric you understand is for per RTP packet monitoring or observation while the sampled metric is not.
In other words,
a. The Interval metric is obtained by keeping trace of each RTP packet during a specified Interval and treat all the sequence number valid.
b.The sampled metric is obtained by sampling a sub set of RTP packets during a specified Interval.
Is that what you say?

However according to PDV drafts and other similar xrblock metrics drafts, per-packet statistics doesn't apply to these metrics(See security consideration section).

That seems to implicate that both Interval metrics and cumulative metric are not used for per-packet accounting but for sampling use.

Regards!

-Qin

发件人: [email-masked] [[email-masked]] 代表 Alan Clark [[email-masked]]
发送时间: 2011年10月31日 20:08
到: [email-masked]
Cc: xrblock
主题: Re: [xrblock] General comments on xrblock metrics drafts

Hi Jing

An interval metric should be measured throughout an interval however a sampled metric is an instantaneous value.

Using Jitter/PDV as an example:

- Interval PDV should be calculated by measuring the delay variation of each packet and averaging these over the interval. This requires the implementation to sum the individual measurements and divide by the number of measurements.

- PPDV (RFC3550) is calculated by keeping a running average of the individual delay variation measurements using a scaling factor of 16 and then reporting the value of this running average at the end of the interval. With a scaling factor of 16 the PPDV metric is most strongly affected by the delay variation of the most recent 10-20 packets and hence is not representative of the whole interval.

Interval averages can be quite misleading if the measurements are strongly time varying, which PDV typically is.

Best Regards

Alan

On 10/31/11 5:23 AM, "[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:

Hi, Alan:

Question for clarification.

Isn't the sampled metric same as the Interval metric? Is the sampled metric only applied to per-packet reporting ? what's the difference between sampled metric and non-sampled metric?

what's the difference between sampled metric and Interval metric? Would you like to clarify more?

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark
时　间：2011/10/31 06:51:03 星期一
收件人：[email-masked] <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
抄送人：
主　题：[xrblock] General comments on xrblock metrics drafts

1. Cumulative, Interval and Sample metrics
The drafts current have one bit assigned for Cumulative/Interval indication.... Some metrics are however sampled and don’t have validity for an interval or a call/session; an example of this is PPDV, which is a sampled value of a running average and only has validity for the last 200-300ms before the value is sampled.
Proposal - increase this to two bits to allow for the indication of a Sampled value.

---

From: Qin Wu <[email-masked]>
Date: Wed, 02 Nov 2011 07:11:32 +0000

I tend to agree with you. But It looks to me Interval metric also can be used to calculate cumulative charastertistics. The only difference to the Cumulative metric is one has short interval, one has longer interval.

Am I right?

Or  should we view Interval metric as a metric that is used to report instant value at the end of the interval set for the interval metric? in that case, there will be no difference between Iinteval metric and Sampled metric,

Or should we view Interval metric as a basic metric that can be directly measured and calculated and does not rely on other metric.

which one is correct?

Before answering this question,  I think introducing new term "sampled metric" is very confused.

Regards!

-Qin

________________________________
发件人: [email-masked] [[email-masked]]
发送时间: 2011年11月2日 14:59
到: Alan Clark; Qin Wu
Cc: xrblock
主题: Re: Re: 答复: [xrblock] General comments on xrblock metrics drafts

------------------ 原始邮件 ------------------

发件人：Alan Clark

Jing

Best Regards

Alan

On 11/1/11 4:21 AM, "[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>" <[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>> wrote:

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Qin Wu
时　间：2011/11/01 15:19:54 星期二
收件人：Alan Clark , [email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>
抄送人：xrblock
主　题：答复: [xrblock] General comments on xrblock metrics drafts

Hi, Alan:

Regards!

-Qin

________________________________

发件人: [email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> [[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>] 代表 Alan Clark [[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>]
发送时间: 2011年10月31日 20:08
到: [email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>
Cc: xrblock
主题: Re: [xrblock] General comments on xrblock metrics drafts

Hi Jing

Best Regards

Alan

On 10/31/11 5:23 AM, "[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:

Hi, Alan:

Cheers!

Jing

------------------ 原始邮件 ------------------

发件人：Alan Clark
时　间：2011/10/31 06:51:03 星期一
收件人：[email-masked]<http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件人：[email-masked]> <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
抄送人：
主　题：[xrblock] General comments on xrblock metrics drafts

---

From: Alan Clark <[email-masked]>
Date: Wed, 2 Nov 2011 22:41:05 -0400

Qin

An interval metric is measured over a time interval.  A cumulative metric is
the same as an interval metric if there is only one interval.

A metric for an interval should not be affected by previous intervals.

A “sampled” metric is the value of a metric at the instant of sampling.

(a) Simple case

If we consider 10 intervals, each with 1000 packets expected.  Intervals 1
and 3 have 100 packets missing.

Cumulative packet loss rate, reported at the end of each interval would be

10%, 5%, 6.7%, 5%, 4%, 3.3%, 2.9%, 2.5%, 2.2%, 2.0%

Interval packet loss rate would be
10%, 0%, 10%, 0%, 0%, 0%, 0%, 0%, 0%, 0%

Averaging the interval loss rates gives (10+10)/10 = 2.0%

calculating the running average loss rate and reporting sampled values would
give something roughly like

10%, 3%, 10%, 3%, 1%, 0.1%, 0.01%, 0%, 0%, 0%

(b) Slightly less simple case

Assume the same intervals and loss rates however say that some of the
packets from interval 1 were not actually lost, but arrive in interval 2.
So at the time the packet loss rate was calculated for interval 1 it
appeared to be 8% instead of 10%.

We don’t typically report negative loss rates, so the interval loss rate
would be reported as

8%, 0%.....

If we compute loss rate cumulatively then this problem would only occur at
the end of the cumulative period, so the loss rates reported would be

8%, 5%, ...

In other words the cumulative metric corrects itself.

This would mean that if we added the interval packet loss rates together the
sum may not match the cumulative value.

Best Regards

Alan

On 11/2/11 3:11 AM, "Qin Wu" <[email-masked]> wrote:

> I tend to agree with you. But It looks to me Interval metric also can be used
> to calculate cumulative charastertistics. The only difference to the
> Cumulative metric is one has short interval, one has longer interval.
> 
> Am I right? 
> 
>  
> 
> Or  should we view Interval metric as a metric that is used to report instant
> value at the end of the interval set for the interval metric? in that case,
> there will be no difference between Iinteval metric and Sampled metric,
> 
>  
> 
> Or should we view Interval metric as a basic metric that can be directly
> measured and calculated and does not rely on other metric.
> 
> which one is correct?
> 
>  
> 
> I know you intend to seperate Sampled metric from Interval Metric. However
> according to the defintion of Interval metric definition:
> " 
> In your understanding, the samepled metric is referred to as the measured
> value is sampled at the end of interval or apply to the end of interval.
> Is that what you say?
> Isn't J(i) a cumulative metric since the J(i) is the accumulation period
> characteristic that combine J(i-1),J(i-2),..J(0)
> An Interval metric is calculated over the interval - so packet loss for an
> interval would be the packets lost during the interval divided by the number
> of packets expected.
> 
> If the packet loss rate was continuously calculated as a running average
> (updated per packet) and the value of this packet loss rate metric was sampled
> at the end of the interval, it would be a sampled value of the metric and not
> the interval value.
> 
> Hence there is a difference between an interval metric and a sampled metric,
> which IMHO means that we should have a way to state that a metric is
> Cumulative, Interval or Sampled.
> 
> Best Regards
> 
> Alan
> 
> 
> On 11/1/11 4:21 AM, "[email-masked]
> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@stt
> ri.com.cn> " <[email-masked]
> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@stt
> ri.com.cn> > wrote:
> 
>> 
>> 
>> 
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  
>> 抄送人：xrblock 
>> So the Interval metric you understand is for per RTP packet monitoring or
>> observation while the sampled metric is not.
>> In other words,
>> a. The Interval metric is obtained by keeping trace of each RTP packet during
>> a specified Interval and treat all the sequence number valid.
>> However according to PDV drafts and other similar xrblock metrics drafts,
>> per-packet statistics doesn't apply to these metrics(See security
>> consideration section).
>> 
>> 
>> That seems to implicate that both Interval metrics and cumulative metric are
>> not used for per-packet accounting but for sampling use.
>> 
>> 
>> 
>> 
>> 
>> Regards!
>> 
>> 
>> -Qin
>> 
>> 
>> 
>> 
>> 发件人: [email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/xrblock-bou
>> [email-masked]>  [[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/xrblock-bou
>> [email-masked]> ] 代表 Alan Clark [[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/alan.d.clar
>> [email-masked]> ]
>> 发送时间: 2011年10月31日 20:08
>> 到: [email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn> 
>> Cc: xrblock
>> An interval metric should be measured throughout an interval however a
>> sampled metric is an instantaneous value.
>> 
>> - Interval PDV should be calculated by measuring the delay variation of each
>> packet and averaging these over the interval. This requires the
>> implementation to sum the individual measurements and divide by the number of
>> measurements.
>> 
>> - PPDV (RFC3550) is calculated by keeping a running average of the individual
>> delay variation measurements using a scaling factor of 16 and then reporting
>> the value of this running average at the end of the interval. With a scaling
>> factor of 16 the PPDV metric is most strongly affected by the delay variation
>> of the most recent 10-20 packets and hence is not representative of the whole
>> interval.
>> 
>> On 10/31/11 5:23 AM, "[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> "
>> <[email-masked]
>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/zhaojing@st
>> tri.com.cn>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:
>> 
>> 
>>> 
>>> Hi, Alan: 
>>> 
>>> Isn't the sampled metric same as the Interval metric? Is the sampled metric
>>> only applied to per-packet reporting ? what's the difference between sampled
>>> metric and non-sampled metric?
>>> 
>>> 收件人：[email-masked]
>>> <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件
>>> 人：[email-masked]>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx>
>>> 抄送人： 
>>> The drafts current have one bit assigned for Cumulative/Interval
>>> indication.... Some metrics are however sampled and don’t have validity for
>>> an interval or a call/session; an example of this is PPDV, which is a
>>> sampled value of a running average and only has validity for the last
>>> 200-300ms before the value is sampled.

---

From: Qin Wu <[email-masked]>
Date: Thu, 03 Nov 2011 15:15:14 +0800

Re: 答复: Re: 答复: [xrblock] General comments on xrblock metrics draftsHi, Alan:
Thank for your clarification on the difference between Interval metric, Cumulative metric the existing draft described and sampled metric you proposed.
It looks Sample metric is established upon the Interval metric and present different features of any transport impairments from Culmulative metric.
e.g., we have a list of Interval metric
x1, x2, x3,x4,x5,x6,x7,x8,x9,x10
The sample metric is as follows:
S1,S2,S3,S4,S5,S6,S7,S8,S9,S10
So the sampled metic can be calculated using the following example:
S1=x1; S2= func(x1,x2); S3=func(x2,x3); S4=func(x3,x4);...
In this example, Si only rely on the xi and xi-1.
 
So the question is:
Does sample metric fix the problem you raised below for Cumulative metric?

Also what's the sampled algorithm used for calculating such metric and presented at the end of each Interval? the algorithm similar to PPDV? or any algorithm we can find?
Do you think measurement Interval for Interval metric is same as one for Sampled metric?

It looks to me that it is not that simple to just add one more bit in the XRBlock header to indicate Sampled metric is used? You should make sure the measurement interval and specific sampled algorithm?
But Interval metric and Cumulative metric don't have such issue.

Regards!
-Qin
  ----- Original Message ----- 
  From: Alan Clark 
  To: Qin Wu ; [email-masked] 
  Cc: xrblock 
  Sent: Thursday, November 03, 2011 10:41 AM
  Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts

  Qin

  An interval metric is measured over a time interval.  A cumulative metric is the same as an interval metric if there is only one interval.

  (a) Simple case

  If we consider 10 intervals, each with 1000 packets expected.  Intervals 1 and 3 have 100 packets missing.

  calculating the running average loss rate and reporting sampled values would give something roughly like

  Assume the same intervals and loss rates however say that some of the packets from interval 1 were not actually lost, but arrive in interval 2.  So at the time the packet loss rate was calculated for interval 1 it appeared to be 8% instead of 10%.

  We don’t typically report negative loss rates, so the interval loss rate would be reported as 

  8%, 0%.....

  If we compute loss rate cumulatively then this problem would only occur at the end of the cumulative period, so the loss rates reported would be

  8%, 5%, ...

  This would mean that if we added the interval packet loss rates together the sum may not match the cumulative value.

  Best Regards

  Alan

    Am I right? 

    which one is correct? 

    Regards!

    -Qin

----------------------------------------------------------------------------
    发件人: [email-masked] [[email-masked]]
    发送时间: 2011年11月2日 14:59
    到: Alan Clark; Qin Wu
    Cc: xrblock
    主题: Re: Re: 答复: [xrblock] General comments on xrblock metrics drafts

    ------------------ 原始邮件 ------------------ 

    发件人：Alan Clark 

    Jing

    Best Regards

    Alan

    On 11/1/11 4:21 AM, "[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> " <[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> > wrote:

      Cheers! 

      Jing 

      ------------------ 原始邮件 ------------------ 

      发件人：Qin Wu 
      时　间：2011/11/01 15:19:54 星期二 
      收件人：Alan Clark , [email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>  
      抄送人：xrblock 
      主　题：答复: [xrblock] General comments on xrblock metrics drafts 

      Hi, Alan:

      Regards!

      -Qin

--------------------------------------------------------------------------

      发件人: [email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>  [[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> ] 代表 Alan Clark [[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> ]
      发送时间: 2011年10月31日 20:08
      到: [email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]> 
      Cc: xrblock
      主题: Re: [xrblock] General comments on xrblock metrics drafts

      Hi Jing

      Best Regards

      Alan

      On 10/31/11 5:23 AM, "[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> " <[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> > wrote:

        Hi, Alan: 

        Cheers! 

        Jing 

        ------------------ 原始邮件 ------------------ 

        发件人：Alan Clark 
        时　间：2011/10/31 06:51:03 星期一 
        收件人：[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件人：[email-masked]>  <https://imail.huawei.com/OWA/UrlBlockedError.aspx> 
        抄送人： 
        主　题：[xrblock] General comments on xrblock metrics drafts 

------------------------------------------------------------------------------

  _______________________________________________
  xrblock mailing list
  [email-masked]
  https://www.ietf.org/mailman/listinfo/xrblock