Subject: Re: [AVT] Audio to discuss monitoring architecture for RTP

From: Qin Wu <[email-masked]>
Date: Sat, 16 Oct 2010 12:18:49 +0800

Hi, 
Sorry for late reply. Thank geoff and philip for arranging this discussion. 
I am willing to work on the aspect of draft that describe transport metrics 
and application metrics. I think they are worth more works. I would like
to contribute on these aspect.

Regards!
-Qin
----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, September 21, 2010 5:36 PM
Subject: Re: [AVT] Audio to discuss monitoring architecture for RTP

> Here are the notes of the informal AVT call to discuss a monitoring architecture for RTP,
> held on 14-Sep-2010.
> 
> In summary, a number of potential work areas were identified within the broad scope of a monitoring 
> architecture but there was no commitment of resource on the call to work on any of these topics. We 
> agreed that, if a consensus emerges on the list regarding area(s) of immediate interest where people 
> will commit to work, these area(s) could be taken forward.
> 
> Thanks to Philip for taking these notes. If you were on the call and find any errors or omissions, 
> please let us know.
> 
> Participants:
> 
> Geoff Hunt (chair)
> Philip Arden (notes)
> Gunnar Heikkila
> Keith Drage
> Qin Wu
> Glen Zorn
> Marie Josee Drouin
> Dan Romascanu
> Michael Romalho
> Christian Hoene
> 
> Keith said that the audio was an informal IETF call and asked whether 
> there was agreement to adopt the conventions of the IETF Note well for 
> the call.  All were in agreement.
> 
> 1.  Agenda
> 
> Geoff introduced the agenda as follows and asked for any additions - 
> there were none.
> 
> 1. Agenda bashing
> 2. Scene setting
> - Current chartered work
> - Existing drafts and previous work
> 3. What must be defined in an architecture?
>    - candidate topics
>      - transport metrics?
> - application metrics?
>      - how to choose re-usable metrics
>      - source stream identification
>      - measurement point identification (where/what)
>      - measurement correlation keys
>      - destination entities
>      - delivery methods
>      - packaging of metrics (compact? re-usable?)
>      - security
> 4. How much of this is within AVT remit?
> 5. Proposal for scope of AVT deliverable 
> 6. Next steps
> 
> Geoff saw two aims for the call:
> 
> i)  Agree on a short-term AVT deliverable
> ii)  Discuss any longer term deliverables, which may not necessarily be 
> part of AVT.
> 
> Keith - work needs to be defined first and then a decision can be made 
> as to where it fits. Don't distinguish between short- and long-term 
> work on this call
> 
> 2.  Scene Setting
> Geoff gave a brief summary of Monarch 01 and Monarch 00:
> 
> Monarch 01 is narrower in scope and describes how to transport XR 
> metrics (independently of application) in RTCP and how to structure the 
> format of the packets whilst also making sure that the metrics are 
> transportable.
> 
> Monarch 00 describes how to choose metrics for applications and is 
> wider in scope.  It includes application metrics for voice and requires 
> expansion to include video metrics.
> 
> Although AVT has a milestone for a monitoring architecture, neither 
> draft is adopted as an AVT WG work item
> 
> Geoff - do other documents need to be considered?
> 
> Qin - video quality monitoring needed to be addressed, and the 
> following documents need to be considered:
> RFC 4710
> RFC 4712
> RFC 3611
> He also felt that the current scope was very narrow and needed to be 
> extended.  The architecture title implied a broader scope.
> 
> Keith - scope needs to reflect the available resource.  There is no 
> point in having a work item that could cannot be resourced.
> 
> Qin - a more suitable title for the document might be something like 
> "Guidelines for extending metrics blocks using RTCP-XR".
> 
> 3. What must be defined in the architecture?
> 
> i) Transport and application metrics
> 
> Geoff - AVT has a primary obligation to define RTCP transport metric.
> 
> Qin - RFC3611 contains both transport and application metrics and 
> transport metrics should be included as part of the architecture.  The 
> architecture should also include metrics guidelines and how to classify 
> the information in metrics.
> 
> Geoff - should northbound metrics be included as well as RTP peer 
> metrics. Do peer metrics need to be distinguished from northbound 
> metrics?
> 
> Qin - agreed and said that translators play a role in extracting 
> information which may or may not be sent in RTCP.
> 
> ii) Metric re-use
> 
> Geoff - the architecture should describe transport metrics.  When Colin
> (Perkins) asked for a draft he was keen not to invent new metrics and 
> existing metrics should be re-used as far as possible for different 
> applications.
> 
> Qin and Marie both agreed with this view.
> 
> Keith - list of metrics needs to be reviewed and also assuming one 
> needs new metrics information is needed about how they can be made re-usable.
> 
> Dan - agreed.  Should the doc cover scalability in gathering and 
> reporting metrics based on experience gained in XR deployment, eg why 
> MIB development didn't succeed, and examples of probe deployments.
> 
> iii) Security
> 
> Geoff - there is also a related security issue about gathering and 
> transporting metrics.
> 
> Dan - agreed.  The impact of encrypted signals also needs to be 
> considered.
> 
> Geoff - do we need all the topics on the list and are there any 
> omissions?
> 
> iv) Source stream identification and measurement point ID
> 
> Marie - Monarch draft 01 mentioned raised a number of questions.  For 
> example in Section 4 about the identity block there are unresolved 
> questions about the overlap of identity information between XR and RR 
> information.
> 
> Geoff - need to identify the overlaps and then take them to the AVT 
> list. Also, why does RTCP not have some of the identity information 
> mentioned in Monarch 01?
> 
> Marie - missing from Monarch 01 is what could be transported in RTP as 
> opposed to RTCP.
> 
> v) Measurement correlation keys
> 
> Geoff - could we use the SSRC and IP address as a correlation key to ID 
> a stream from end to end?
> 
> Michael - the SIPPING group is working on a session ID for this purpose.
> 
> Geoff - you need a session ID to compare metrics from point A with 
> those from point B.
> 
> Michael - could a hop count identifier be used?
> 
> Geoff - measurements could be made at several nodes and then sent on, 
> but would this technique scale?
> 
> Michael - if you only use RTCP metrics, then one possibility would be 
> to enable them as required.  Ought not to enable all the metrics all 
> the time except for a basic set.  The architecture needs to define a 
> rudimentary set of metrics.
> 
> Geoff - so another aspect of the architecture would be methods for 
> control of metrics collection
> 
> vi) Destination entities and delivery methods
> 
> Geoff - is it appropriate to include a range of destination entities, 
> delivery methods and protocols?
> 
> Michael - this is a hard problem.  If you have separate RTP sessions, 
> all sorts of ID are different.  The DSPs don't have access to all the 
> information required.  Also provider A won't want to give metrics 
> relating to problems in its own network to provider B.
> 
> Geoff - are you saying that it's not worth defining interfaces because 
> of inter-operator politics?
> 
> Michael - one could define metrics for a single network.
> 
> 6. Summary and next steps
> 
> Geoff - next steps are to put out the notes.  Should AVT be doing work 
> in this area?
> 
> Keith - asked whether people are prepared to commit resource.
> 
> Gunnar - the requirements for troubleshooting and for monitoring 
> end-user quality are different. Could collect application metrics from endpoints.
> Should consider both sets of requirements.
> 
> Geoff - need to decide how they fit into the architecture.
> 
> Marie - would like to contribute, but scope needs to be narrowed.
> 
> Keith - guidelines for defining reporting blocks would be useful - or 
> do we need a review?  How much resource would be required to do this?
> 
> Geoff - should we therefore suggest a work item on guidelines for 
> defining new XR blocks?
> 
> Qin - the draft needs to be restructured as the title doesn't reflect 
> the content.
> 
> Keith - keen not to propose work items that have no resource. From AVT 
> charter perspective, key criteria are that work must be useful and that 
> it must be resourced. Just "an idea and an editor" are not enough.
> 
> Geoff - suggest putting the resource question to the AVT list and 
> propose a work item on draft guidelines for XR blocks.  Also note what 
> was discussed on the call and see what topics elicit volunteers.
> 
> Keith - mentioned next meeting in Beijing and asked whether it would be 
> possible to have a face to face meeting? This seemed unlikely for the 
> responses.  He also suggested a follow-on call once responses had been 
> obtained form the list.
> 
> Geoff - will arrange follow on call.
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Qin Wu <[email-masked]>
Date: Sat, 16 Oct 2010 12:29:30 +0800

Hi, Keith:
It seems we have two things to do for the next step:
a) contribute to some parts of monitoring architecture draft
or
b) contribute to monitoring topic by writing some new draft
It is not clear to me what one you suggest us to do? Would you 
like to clarify this?

Regards!
-Qin
----- Original Message ----- 
From: "DRAGE, Keith (Keith)" <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, September 22, 2010 9:48 PM
Subject: Re: [AVT] Audio to discuss monitoring architecture for RTP

> All,
> 
> Please comment on the notes, and propose where we go next. It would be nice to see words like "I am prepared to work on the parts of a draft that describe this".

> My understanding is that the relevant people will not be in Beijing, so we will not be scheduling it as part of the AVT meeting there. A further conference call (which could potentially be official) has been suggested, but that requires some input before we get to that point. If you want this work, please contribute.
> 
> regards
> 
> Keith 
> 
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On 
>> Behalf Of [email-masked]
>> Sent: Tuesday, September 21, 2010 10:36 AM
>> To: [email-masked]
>> Cc: [email-masked]
>> Here are the notes of the informal AVT call to discuss a 
>> monitoring architecture for RTP, held on 14-Sep-2010.
>> 
>> In summary, a number of potential work areas were identified 
>> within the broad scope of a monitoring architecture but there 
>> was no commitment of resource on the call to work on any of 
>> these topics. We agreed that, if a consensus emerges on the 
>> list regarding area(s) of immediate interest where people 
>> Thanks to Philip for taking these notes. If you were on the 
>> call and find any errors or omissions, please let us know.
>> 
>> Participants:
>> 
>> Geoff Hunt (chair)
>> Philip Arden (notes)
>> Gunnar Heikkila
>> Keith Drage
>> Qin Wu
>> Glen Zorn
>> Marie Josee Drouin
>> Dan Romascanu
>> Michael Romalho
>> Christian Hoene
>> 
>> Keith said that the audio was an informal IETF call and asked 
>> whether there was agreement to adopt the conventions of the 
>> IETF Note well for the call.  All were in agreement.
>> 
>> 1.  Agenda
>> 
>> Geoff introduced the agenda as follows and asked for any 
>> additions - there were none.
>> 
>> 1. Agenda bashing
>> 2. Scene setting
>> - Current chartered work
>> ii)  Discuss any longer term deliverables, which may not 
>> necessarily be part of AVT.
>> 
>> Keith - work needs to be defined first and then a decision 
>> can be made as to where it fits. Don't distinguish between 
>> short- and long-term work on this call
>> 
>> 2.  Scene Setting
>> Monarch 01 is narrower in scope and describes how to 
>> transport XR metrics (independently of application) in RTCP 
>> and how to structure the format of the packets whilst also 
>> making sure that the metrics are transportable.
>> 
>> Monarch 00 describes how to choose metrics for applications 
>> and is wider in scope.  It includes application metrics for 
>> voice and requires expansion to include video metrics.
>> 
>> Although AVT has a milestone for a monitoring architecture, 
>> neither draft is adopted as an AVT WG work item
>> 
>> Qin - video quality monitoring needed to be addressed, and 
>> the following documents need to be considered:
>> RFC 4710
>> RFC 4712
>> RFC 3611
>> He also felt that the current scope was very narrow and 
>> needed to be extended.  The architecture title implied a 
>> broader scope.
>> 
>> Keith - scope needs to reflect the available resource.  There 
>> is no point in having a work item that could cannot be resourced.
>> 
>> Qin - a more suitable title for the document might be 
>> something like "Guidelines for extending metrics blocks using 
>> RTCP-XR".
>> 
>> Qin - RFC3611 contains both transport and application metrics 
>> and transport metrics should be included as part of the 
>> architecture.  The architecture should also include metrics 
>> guidelines and how to classify the information in metrics.
>> 
>> Geoff - should northbound metrics be included as well as RTP 
>> peer metrics. Do peer metrics need to be distinguished from 
>> northbound metrics?
>> 
>> Qin - agreed and said that translators play a role in 
>> extracting information which may or may not be sent in RTCP.
>> 
>> ii) Metric re-use
>> 
>> (Perkins) asked for a draft he was keen not to invent new 
>> metrics and existing metrics should be re-used as far as 
>> possible for different applications.
>> 
>> Keith - list of metrics needs to be reviewed and also 
>> assuming one needs new metrics information is needed about 
>> how they can be made re-usable.
>> 
>> Dan - agreed.  Should the doc cover scalability in gathering 
>> and reporting metrics based on experience gained in XR 
>> deployment, eg why MIB development didn't succeed, and 
>> examples of probe deployments.
>> 
>> iii) Security
>> 
>> Geoff - there is also a related security issue about 
>> gathering and transporting metrics.
>> 
>> Dan - agreed.  The impact of encrypted signals also needs to 
>> be considered.
>> 
>> Geoff - do we need all the topics on the list and are there 
>> any omissions?
>> 
>> Marie - Monarch draft 01 mentioned raised a number of 
>> questions.  For example in Section 4 about the identity block 
>> there are unresolved questions about the overlap of identity 
>> information between XR and RR information.
>> 
>> Geoff - need to identify the overlaps and then take them to 
>> the AVT list. Also, why does RTCP not have some of the 
>> identity information mentioned in Monarch 01?
>> 
>> Marie - missing from Monarch 01 is what could be transported 
>> in RTP as opposed to RTCP.
>> 
>> Geoff - could we use the SSRC and IP address as a correlation 
>> key to ID a stream from end to end?
>> 
>> Geoff - you need a session ID to compare metrics from point A 
>> with those from point B.
>> 
>> Geoff - measurements could be made at several nodes and then 
>> sent on, but would this technique scale?
>> 
>> Michael - if you only use RTCP metrics, then one possibility 
>> would be to enable them as required.  Ought not to enable all 
>> the metrics all the time except for a basic set.  The 
>> architecture needs to define a rudimentary set of metrics.
>> 
>> Geoff - so another aspect of the architecture would be 
>> methods for control of metrics collection
>> 
>> Geoff - is it appropriate to include a range of destination 
>> entities, delivery methods and protocols?
>> 
>> Michael - this is a hard problem.  If you have separate RTP 
>> sessions, all sorts of ID are different.  The DSPs don't have 
>> access to all the information required.  Also provider A 
>> won't want to give metrics relating to problems in its own 
>> network to provider B.
>> 
>> Geoff - are you saying that it's not worth defining 
>> interfaces because of inter-operator politics?
>> 
>> Geoff - next steps are to put out the notes.  Should AVT be 
>> doing work in this area?
>> 
>> Gunnar - the requirements for troubleshooting and for 
>> monitoring end-user quality are different. Could collect 
>> application metrics from endpoints.
>> Keith - guidelines for defining reporting blocks would be 
>> useful - or do we need a review?  How much resource would be 
>> required to do this?
>> 
>> Geoff - should we therefore suggest a work item on guidelines 
>> for defining new XR blocks?
>> 
>> Qin - the draft needs to be restructured as the title doesn't 
>> reflect the content.
>> 
>> Keith - keen not to propose work items that have no resource. 
>> From AVT charter perspective, key criteria are that work must 
>> be useful and that it must be resourced. Just "an idea and an 
>> editor" are not enough.
>> 
>> Geoff - suggest putting the resource question to the AVT list 
>> and propose a work item on draft guidelines for XR blocks.  
>> Also note what was discussed on the call and see what topics 
>> elicit volunteers.
>> 
>> Keith - mentioned next meeting in Beijing and asked whether 
>> it would be possible to have a face to face meeting? This 
>> seemed unlikely for the responses.  He also suggested a 
>> follow-on call once responses had been obtained form the list.
>>