Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts

From: Qin Wu <[email-masked]>
Date: Fri, 04 Nov 2011 10:27:13 +0800

Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics draftsI am not comfortable to impose the sample indication  into all the future metrics, but that only stands for my personal opinion.:-)
I like to hear other opinion and will follow consensus.

Regards!
-Qin
  ----- Original Message ----- 
  From: Alan Clark 
  To: Qin Wu ; [email-masked] 
  Cc: xrblock 
  Sent: Friday, November 04, 2011 10:07 AM
  Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts

  Hi Qin

  The goal in establishing a framework should be to make provision for things that you would reasonably expect.  There are quite a few continuously varying parameters that could be reported using a sampled approach - audio signal level, noise level, video bandwidth to name a few. 
   
  The first part of the header comprises 
  - Block type (8 bits)
  - Interval/Cumulative ( 1 bit)
  - Tag (3 bits)
  - Reserved (4 bits)

  Expanding the Interval/Cumulative to 2 bits still leaves 3 reserved bits, so this would not break the budget.

  Best Regards

  Alan

  On 11/3/11 9:26 PM, "Qin Wu" <[email-masked]> wrote:

    Hi, Alan:
    I tend to agree with you. But currently I don't see the sampled metric is applied to all the metrics except PDV metric, 
    So my proposal is to add one bit in PDV metric block header to indicate if sample will be used. 
    If there is any future metric that need to support sample, we can add indication bit later.
     
    Regards!
    -Qin

      ----- Original Message ----- 
       
      From:  Alan Clark <mailto:[email-masked]>  
       
      To: Qin Wu <mailto:[email-masked]>  ; [email-masked] 
       
      Cc: xrblock <mailto:[email-masked]>  
       
      Sent: Thursday, November 03, 2011 8:26  PM
       
      Subject: Re: [xrblock] 答复: Re: 答复: General  comments on xrblock metrics drafts

      Qin

      You cannot derive the value of a sampled  metric from two interval metrics.

      A sampled metric is useful for  measuring and reporting on parameters that vary in time. Sometimes it is  acceptable to report the min/max/average of a time varying parameter and  sometimes you may want to report the value at points in time.

      At risk  of causing complete confusion - I’m going to use a temperature example.   

      (i) Between 2pm and 3pm the average temperature was 10 degrees  C, is an interval metric
      (ii) Between 2pm and 3pm the maximum temperature  was 11 degrees C, is an interval metric
      (ii) The temperature at 3pm is 2  degrees C is a sampled measurement

      It is remarkably easy to add one bit  to the XRBlock head at this time to allow for current or future metrics that  are sampled. Given that there are existing RTP examples of this type of metric  then we know that it is already useful to report sampled values. 

      Best  Regards

      Alan

      On 11/3/11 3:15 AM, "Qin Wu" <[email-masked]>  wrote:

        Hi, Alan:
        Thank for your clarification on the  difference between Interval metric, Cumulative metric the existing draft  described and sampled metric you proposed.
        It looks Sample metric is  established upon the Interval metric and present different features of any  transport impairments from Culmulative metric.
        e.g., we have a list of  Interval metric
        x1, x2, x3,x4,x5,x6,x7,x8,x9,x10
        The sample metric is  as follows:
        S1,S2,S3,S4,S5,S6,S7,S8,S9,S10
        So the sampled metic can be  calculated using the following example:
        S1=x1; S2= func(x1,x2);  S3=func(x2,x3); S4=func(x3,x4);...
        In this example, Si only rely on the  xi and xi-1.
         
        So the question is:
        Does sample metric fix the  problem you raised below for Cumulative metric?
         
        Also what's the  sampled algorithm used for calculating such metric and presented at the end  of each Interval? the algorithm similar to PPDV? or any algorithm we can  find?
        Do you think measurement Interval for Interval metric is same as  one for Sampled metric?
         
        It looks to me that it is not that  simple to just add one more bit in the XRBlock header to indicate Sampled  metric is used? You should make sure the measurement interval and specific  sampled algorithm?
        But Interval metric and Cumulative metric don't have  such issue.

        Regards!
        -Qin

          ----- Original Message -----  
           
          From:  Alan Clark <mailto:[email-masked]>   
           
          To: Qin Wu <mailto:[email-masked]>  ;  [email-masked]  
           
          Cc: xrblock <mailto:[email-masked]>   
           
          Sent: Thursday, November 03, 2011 10:41   AM
           
          Subject: Re: [xrblock] 答复: Re: 答复: General   comments on xrblock metrics drafts

          Qin

          An  interval metric is measured over a time  interval.  A cumulative  metric is the same as an interval metric if there  is only one  interval.

          A metric for an interval should not be affected  by  previous intervals.

          A “sampled” metric is the value of a metric at   the instant of sampling.

          (a) Simple case

          If we  consider 10  intervals, each with 1000 packets expected.   Intervals 1 and 3 have 100  packets missing.

          Cumulative  packet loss rate, reported at the end of  each interval would  be

          10%, 5%, 6.7%, 5%, 4%, 3.3%, 2.9%, 2.5%, 2.2%,   2.0%

          Interval packet loss rate would be
          10%, 0%, 10%, 0%,  0%, 0%,  0%, 0%, 0%, 0%  

          Averaging the interval loss  rates gives  (10+10)/10 = 2.0%

          calculating the running average  loss rate and  reporting sampled values would give something roughly  like

          10%, 3%,  10%, 3%, 1%, 0.1%, 0.01%, 0%, 0%, 0%

          (b)  Slightly less simple  case

          Assume the same intervals and loss  rates however say that some of  the packets from interval 1 were not  actually lost, but arrive in interval 2.   So at the time the  packet loss rate was calculated for interval 1 it  appeared to be 8%  instead of 10%.

          We don’t typically report negative  loss  rates, so the interval loss rate would be reported as 

          8%,   0%.....

          If we compute loss rate cumulatively then this  problem would  only occur at the end of the cumulative period, so the  loss rates reported  would be

          8%, 5%, ...

          In other  words the cumulative metric  corrects itself.

          This would mean  that if we added the interval packet  loss rates together the sum may  not match the cumulative value.

          Best   Regards

          Alan

          On 11/2/11 3:11 AM, "Qin Wu" <[email-masked]>   wrote:

            I   tend to agree with you. But It looks to me Interval metric also  can be used  to calculate cumulative charastertistics. The only  difference to the  Cumulative metric is one has short interval, one  has longer  interval.

            Am I right? 

            Or   should we view  Interval metric as a metric that is used to  report instant value at the end  of the interval set for the  interval metric? in that case, there will be no  difference between  Iinteval metric and Sampled  metric,

            Or should  we view Interval metric as a basic  metric that can be directly  measured and calculated and does not rely on  other  metric.

            which one is correct? 

            Before   answering this question,  I think introducing new term  "sampled metric"  is very   confused.

            Regards!

            -Qin

--------------------------------------------------------------------
            发件人:  [email-masked] [[email-masked]]
            发送时间:   2011年11月2日 14:59
            到: Alan Clark; Qin Wu
            Cc:   xrblock
            主题: Re: Re: 答复: [xrblock] General comments on  xrblock  metrics drafts

            I know you  intend to seperate Sampled metric from  Interval Metric. However  according to the defintion of Interval metric  definition: 
            "  
            The Interval metric is refered to as the reported  values apply  to the most recent measurement 
             interval duration  between  successive metrics reports 
            " 
            In your understanding, the   samepled metric is referred to as the measured value is sampled at  the end  of interval or apply to the end of interval. 
            Is that  what you say?  
            But I am not convinced. It looks to me the  sampled metric can be  classified into cumulative metric. 
            Take  PPDV as example, the J(i)=  J(i-1)++ (|D(i-1,i)| - J(i-1))/16  
            Isn't J(i) a cumulative metric since  the J(i) is the  accumulation period characteristic that combine   J(i-1),J(i-2),..J(0) 
            during cumulative duration? 
            Cheers!  
            Jing  
            ------------------ 原始邮件 ------------------  

            发件人：Alan Clark  
            时　间：2011/11/02 05:25:01 星期三 
            收件人：[email-masked], Qin Wu   
            抄送人：xrblock 
            主　题：Re: 答复: [xrblock] General comments on  xrblock  metrics drafts 

            Jing

            An Interval metric is calculated  over  the interval - so packet loss for an interval would be the  packets lost  during the interval divided by the number of packets  expected. 

            If  the packet loss rate was continuously  calculated as a running average  (updated per packet) and the value  of this packet loss rate metric was  sampled at the end of the  interval, it would be a sampled value of the  metric and not the  interval value.

            Hence there is a difference  between an  interval metric and a sampled metric, which IMHO means that we   should have a way to state that a metric is Cumulative, Interval  or  Sampled.

            Best Regards

            Alan

            On 11/1/11  4:21 AM, "[email-masked]  <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>   " <[email-masked]  <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>   > wrote:

              Are you saying the sample  semantics  has already been incorporate into Interval/Cumulative  indication?  

              Cheers! 

              Jing  

              ------------------ 原始邮件  ------------------  

              发件人：Qin Wu 
              时　间：2011/11/01 15:19:54 星期二  
              收件人：Alan  Clark , [email-masked]  <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>    
              抄送人：xrblock 
              主　题：答复: [xrblock] General comments on  xrblock  metrics drafts 

              Hi, Alan:

              So the Interval   metric you understand is for per RTP packet monitoring or  observation  while the sampled metric is not. 
              In other  words,
              a. The Interval  metric is obtained by keeping trace of  each RTP packet during a specified  Interval and treat all the  sequence number valid.
              b.The sampled metric  is obtained by  sampling a sub set of RTP packets during a specified  Interval.  
              Is that what you say?

              However   according to PDV drafts and other similar xrblock metrics  drafts,  per-packet statistics doesn't apply to these metrics(See  security  consideration section).

              That seems to  implicate that both  Interval metrics and cumulative metric are  not used for per-packet  accounting but for sampling   use.

              Regards!

              -Qin

------------------------------------------------------------------

              发件人:  [email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>    [[email-masked]   <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>   ] 代表 Alan Clark [[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>   ]
              发送时间: 2011年10月31日 20:08
              到: [email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>   
              Cc: xrblock
              主题: Re: [xrblock] General  comments on  xrblock metrics drafts

              Hi  Jing

              An interval metric should  be measured throughout an interval  however a sampled metric is  an instantaneous value.

              Using  Jitter/PDV as an  example:

              - Interval PDV should be calculated by  measuring  the delay variation of each packet and averaging these over the   interval. This requires the implementation to sum the individual   measurements and divide by the number of measurements.

              -  PPDV  (RFC3550) is calculated by keeping a running average of the  individual  delay variation measurements using a scaling factor  of 16 and then  reporting the value of this running average at  the end of the interval.  With a scaling factor of 16 the PPDV  metric is most strongly affected by  the delay variation of the  most recent 10-20 packets and hence is not  representative of the  whole interval.

              Interval averages can be  quite misleading  if the measurements are strongly time varying, which PDV   typically is.

              Best Regards

              Alan

              On  10/31/11 5:23  AM, "[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>    <https://imail.huawei.com/OWA/UrlBlockedError.aspx>   " <[email-masked]  <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/[email-masked]>    <https://imail.huawei.com/OWA/UrlBlockedError.aspx>   > wrote:

                Hi, Alan: 

                Question for  clarification.  

                Isn't the sampled metric same as the  Interval metric? Is the  sampled metric only applied to  per-packet reporting ? what's the  difference between sampled  metric and non-sampled metric? 

                what's  the difference  between sampled metric and Interval metric? Would you  like to  clarify more? 

                Cheers! 

                Jing   

                ------------------ 原始邮件 ------------------  

                发件人：Alan  Clark 
                时　间：2011/10/31 06:51:03 星期一 
                收件人：[email-masked] <http://corpw6.webmail.21cn.com/webmail/jsp/pentsimple/htmlEditor/收件人：[email-masked]>    <https://imail.huawei.com/OWA/UrlBlockedError.aspx>   
                抄送人： 
                主　题：[xrblock] General comments on xrblock metrics  drafts  

                1. Cumulative, Interval and Sample  metrics
                The drafts  current have one bit assigned for  Cumulative/Interval indication....  Some metrics are however  sampled and don’t have validity for an interval  or a  call/session; an example of this is PPDV, which is a sampled value   of a running average and only has validity for the last  200-300ms before  the value is sampled.
                Proposal - increase  this to two bits to allow  for the indication of a Sampled  value.  

----------------------------------------------------------------------

          _______________________________________________
          xrblock mailing   list
          [email-masked]
          https://www.ietf.org/mailman/listinfo/xrblock

---

From: Qin Wu <[email-masked]>
Date: Tue, 08 Nov 2011 09:43:48 +0800

Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics draftsHi, Alan:
Based on our discussion, would you like to propose the change to the drafts regarding sample metric?
e.g., what's your proposed change to the Interval metric flag? or just allocate additional falg to sample?
Also it will be great you provide the clear definition of sample metric.
My understanding to sample metrics is:
"
Sample metrics

 It is referred to the metrics of which the reported values only 

 apply to the sample instant which is any time of the measurement duration. 

"
If this is not precise, please correct me. Thanks!

Regards!
-Qin
  ----- Original Message ----- 
  From: Alan Clark 
  To: Qin Wu ; [email-masked] 
  Cc: xrblock 
  Sent: Friday, November 04, 2011 10:07 AM
  Subject: Re: [xrblock] 答复: Re: 答复: General comments on xrblock metrics drafts

  Hi Qin

  Best Regards

  Alan

    Hi, Alan:
     
    Regards!
    -Qin

      ----- Original Message ----- 

       

      Qin

      Best  Regards

      Alan

        Hi, Alan:
         
        So the question is:

        Regards!
        -Qin

          ----- Original Message -----  

           

          Qin

          (a) Simple case

          8%,   0%.....

          8%, 5%, ...

          Best   Regards

          Alan

           

            Am I right? 

            which one is correct? 

            Regards!

            -Qin

            ------------------ 原始邮件 ------------------  

            发件人：Alan Clark  

            Jing

            Best Regards

            Alan

             

              Cheers! 

              Jing  

              ------------------ 原始邮件  ------------------  

              发件人：Qin Wu 

              Hi, Alan:

              Regards!

              -Qin

              Hi  Jing

              Best Regards

              Alan

               

                Hi, Alan: 

                Cheers! 

                Jing   

                ------------------ 原始邮件 ------------------  

                发件人：Alan  Clark