Subject: Re: [AVTCORE] Comments on draft-ietf-avtcore-monarch-09

From: Qin Wu <[email-masked]>
Date: Thu, 01 Mar 2012 18:57:42 +0800

Hi, Magnus:
See inline belows.

Regards!
-Qin
----- Original Message ----- 
From: "Magnus Westerlund" <[email-masked]>
To: "Qin Wu" <[email-masked]>
Cc: "IETF AVTCore WG" <[email-masked]>
Sent: Wednesday, February 29, 2012 10:06 PM
Subject: Re: [AVTCORE] Comments on draft-ietf-avtcore-monarch-09

> Hi,
> 
> Thanks for addressing my comment. See inline the ones where I have
> comments or feedback on the suggestions for fixing the issues.
> 
> 
> 
> On 2012-02-24 10:13, Qin Wu wrote:
>> --------------------------------------------
>> 3. In general I think 3.1 should discuss what exist already for
>> monitoring which there is no discussion. Like the RTP MIB. I can also
>> see other solutions like using IPfix to capture a packet flow and then
>> process using an RTP and RTCP receiver.
>>  
>> 
>> [Qin] I agree. In order to fix this, We propose
>> 
>> the following text at the beginning of section
>> 
>> 3.1 to say:
>> 
>> “
>> 
>> There are many ways in which the performance
>> 
>> of an RTP session can be monitored. These
>> 
>> include RTP-based mechanisms such as the RTP
>> 
>> SNMP MIB [RFC2959], or the SIP event package
>> 
>> for RTCP summary reports [RFC6035], or non-RTP
>> 
>> mechanisms such as generic MIBs, NetFlow, IPFix,
>> 
>> and so on. Together, these provide useful
>> 
>> mechanisms for exporting data on the performance
>> 
>> of an RTP session to non-RTP network management
>> 
>> systems. It is desirable to also perform in-session
>> 
>> monitoring of RTP performance. RTCP provides the
>> 
>> means to do this. In the following, we specify an
>> 
>> architecture for using and extending RTCP for
>> 
>> monitoring RTP sessions. One major benefit of
>> 
>> such architecture is with ease of integration
>> 
>> with other RTP/RTCP mechanism.
>> 
>> ”
> 
> Yes, that makes the scope and context much more clearer.

[Qin]: Great and has fixed in version (-10).

> 
>> --------------------------------------
>> 4. The document argues for RTCP XR based monitoring, but can you please
>> provide a clear indication of what the benefits are.
>>  
>> 
>> [Qin]: One major benefit I think is Ease of
>> 
>> integration with other RTP/RTCP mechanisms.
>> 
>> Another benefit I can see is RTCP-based monitoring
>> 
>> Takes both transport quality and application quality
>> 
>> into account and can provide more accurate measurement
>> 
>> for the Real time application quality.
> 
> My intention was to have that made clear in the draft.

[Qin]: I  see and like to put additional text in the above proposed text at the beginning of section 3.1.
> 
>> 6. Section 1.
>>   "As more users and subscribers rely on real time application services,
>>    uncertainties in the performance and availability of these services
>>    are driving the need to support new standard methods for gathering
>>    performance metrics from RTP applications."
>> 
>> What users and subscribers are you discussing above? The ones using RTP
>> to build services or the actual user of the service using RTP?
>>  
>> 
>> [Qin]: I think it is the latter. How about change it as:
>> 
>> “
>> 
>> As the delivery of multimedia services using the Real-Time
>> 
>>  Transport Protocol (RTP) over IP network is gaining an
>> 
>> increasing popularity, uncertainties in the performance and
>> 
>>  availability of these services are driving the need to
>> 
>> support new standard methods for gathering performance
>> 
>> metrics from RTP applications.
>> 
> 
> 
> Yes, that is clearer.

[Qin]: Great and has already fixed in the version (-10).

> 
>> ---------------------------------------
>> 8. Section 1. " Quality of Experience (QoE)." Can you please include
>> some reference that explains the concept of QoE?
>>  
>> 
>> [Qin]: Good suggestion, I think we should reference
>> 
>> RFC6390 or P.10/G.100 Amendment 2. I prefer to use
>> 
>> RFC6390.
> 
> RFC 6390 is fine.

[Qin]: Great and has already fixed in the version (-10).

>> 
>> 
>> -------------------------------------
>> 9. Section 1: "Since different applications layered on RTP may have
>>    some monitoring requirements in common, therefore these metrics
>>    should be satisfied by a common design."
>>  
>> The above sentence needs to be reformulated.
>>  
>> 
>> [Qin]: How about rephrase like:
>> 
>> “Given that different applications layered on RTP may have
>> 
>>    some monitoring requirements in common, these metrics
>> 
>>    should be satisfied by a common design.
>> 
> 
> Hmm, I guess it works in the context it is used in.

[Qin]: Okay, so you agree it fits into the document?!

> 
>> 11. Section 2: One example of such metrics is the QoE Metric
>>       specified in QoE metric reporting Block [MQ].
>> 
>> I don't think that reference is a good example. The reference defines
>> how to transport and identify a large number of application level
>> metrics in RTCP XR, rather than being an example of a metric.
>>  
>> 
>> [Qin]: Do we have better reference than this one.
>> 
>> I am aware that some ITU-T documents (e.g., ITU-T P.NAMS,
>> 
>> P.NBAMS) define measurement method and model for
>> 
>> QoE evaluation while other documents like RFC6390,
>> 
>> E.800, P.10/G.100 Amendment 2 only give definition
>> 
>> of QoE. Also I think QoE metric reporting Block [MQ]
>> 
>> not only define data format for QoE metric and but
>> 
>> also provide Metric name and description for QoE metric.
>> 
>> If you really concerns this, I can remove this sentence.
> 
> No, I can live with what is there.

[Qin]: Great. 
>> 
>> ---------------------------------
>> 12. Section 2:
>> Composed metrics
>> 
>>       Metrics that are calculated based on direct metric or combination
>>       of direct metric and derived metrics.
>> 
>> What is an derived metric? Appears to be an reference without an
>> explanation.
>> 
>> [Qin]: Derived metric is actually one kind of composed metric.
>> 
>> How about fix this like:
>> 
>> “
>> 
>> Composed metrics
>> 
>>      Metrics that are calculated based on Direct metric that have been
>>      measured or combination of Direct metrics that are identical to the
>>      metric being composed.
>> ”
> 
> Sorry, that looks strange in the second part: "combination of Direct
> metrics that are identical to the metric being composed" How can they be
> identical when they are a combination?

[Qin]: I agree it is confusing. I am just thinking we may have case where one metric 
A is calculated by one direct metric B and another metric C and the metric C is 
calculated from the same direct metric B. 
Also we may have the case where one metric D is calculated from two other 
composed metrics E and F.  In these two cases, A,C,D,E,F are all composed metrics. 
How about change it as:
“
 Composed metrics
 
      Metrics that are not directly measured and derived from other metrics. 
      One example of such metrics is the ones calculated based on Direct 
     metric that have been measured.
”

> 
> 
>> ----------------------------------------------
>> 13. Section 2:
>>    Interval metrics
>> 
>>       It is referred to as the metrics of which the reported values
>>       apply to the most recent measurement interval duration between
>>       successive metrics reports
>> 
>> Is "interval duration" actually good English? To me the duration of an
>> interval is the actual interval.
>>  
>> [Qin]:My understanding is Yes, Interval metrics is defined
>>  in relation to “cumulative metric” as a metric that apply
>>  to the duration of an interval. The interval should be the
>> most recent interval to be measured rather than accumulative
>>  period within the whole session.
> 
> Hmm, let see if someone else gets tripped up by this.

>> -------------------------------------------------
>> 14. Section 2:
>>    Sampled metrics
>> 
>>       It is referred to as the metrics of which the reported values only
>>       apply to the value of a continuously measured or calculated that
>>       has been sampled at end of the interval.
>> 
>> Doesn't the above exclude metrics measured at give instance. Or is that
>> what "sampled at end of the interval" means? But then it isn't actually
>> an interval, is it?
>>  
>> [Qin]: Ideally, I think sample metric can be applied to any
>> instance of one measurement interval. But I assume the more
>> sampled data you gather during one measurement interval, the
>> more accurate the sampled metric you can calculate. Does this
>> make sense?
> 
> No, there is to much context missing of what type of metering one
> attempts and the behavior of the subject of the metering.

 [Qin] Sample metric is one metric that is different from interval metric and cumulative metric.
Take interarrival jitter defined in Section 6.4.1 of [RFC3550] as a example, the value J(i) is
 the example of sampled metric and should be calculated according to [RFC3550] at the time 
the report is generated, i.e.,
J(i) = J(i-1) + (|D(i-1,i)| - J(i-1))/16
Therefore I think your suggestion is reasonable. That is to say, the sampled metric can be 
calculated at any instance of measurement interval rather than only at the end of measurement 
interval. If you agree with this, I like to propose the following change to sampled metrics definition:
“
    Sampled metrics
 
       It is referred to as the metrics of which the reported values only
       apply to the value of a continuously measured or calculated that
       has been sampled at any given instance of the interval.
”
Does this address your comments?
> 
> 
> 
>> ----------------------------------------------------
>> 17. Section 3.2:
>>    o  RTCP NACK is used to provide feedback on the RTP sequence number
>>       of the lost packets [RFC4585].
>> 
>> In this case I would be a bit careful. Because the sequence numbers
>> reported on in NACK may in fact be a subset of all lost packets. Already
>> the media stream receiver creating NACK requests may in fact filter on
>> which sequence number is reported. There was a proposal for a
>> informative block (draft-ietf-avt-rtp-selret-05.txt) that allowed for
>> this type of information. There is other cases where a receiver will be
>> able to determine that it doesn't need to ask for some lost packets.
>>  
>> 
>> [Qin]: Okay, How about rephrase this sentence as:
>> 
>> “
>> 
>>    o  RTCP NACK is used to provide feedback on the RTP sequence number
>> 
>>       on a subset of the lost packets or the total lost packets[RFC4585].
>> 
>> ”
> 
> Except that "numbers" and "for a subset". I also think "total" is likely
> wrong and "all the currently"

[Qin]: Do you suggest to the following change:
“
RTCP NACK is used to provide feedback on the RTP sequence numbers 
for a subset of the lost packets or all the currently lost packets [RFC4585].

”
If yes, I like to fix this as you suggested. If not, please let me know.

> 
> Cheers
> 
> Magnus Westerlund
> 
> ----------------------------------------------------------------------
> Multimedia Technologies, Ericsson Research EAB/TVM
> ----------------------------------------------------------------------
> Ericsson AB                | Phone  +46 10 7148287
> Färögatan 6                | Mobile +46 73 0949079
> SE-164 80 Stockholm, Sweden| mailto: [email-masked]
> ----------------------------------------------------------------------
>

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 5 Mar 2012 08:39:13 +0100

Hi,

I think the fixes are good enough. Now I am hoping for some broader
review during the WG last call which I will start in a separate message.

Cheers

Magnus

On 2012-03-01 11:57, Qin Wu wrote:
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
> 

-- 

Magnus Westerlund