Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-monarch-08.txt

From: [email-masked]
Date: Mon, 28 Nov 2011 23:56:13 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Audio/Video Transport Core Maintenance Working Group of the IETF.

	Title           : Monitoring Architectures for RTP
	Author(s)       : Qin Wu
                          Geoff Hunt
                          Philip Arden
	Filename        : draft-ietf-avtcore-monarch-08.txt
	Pages           : 25
	Date            : 2011-11-28

   This memo proposes an architecture for extending RTCP with a new RTCP
   XR (RFC3611) block type to report new metrics regarding media
   transmission or reception quality, following RTCP guideline
   established in RFC5968.  This memo suggests that a new block should
   contain a single metric or a small number of metrics relevant to a
   single parameter of interest or concern, rather than containing a
   number of metrics which attempt to provide full coverage of all those
   parameters of concern to a specific application.  Applications may
   then "mix and match" to create a set of blocks which covers their set
   of concerns.  Where possible, a specific block should be designed to
   be re-usable across more than one application, for example, for all
   of voice, streaming audio and video.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-avtcore-monarch-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-avtcore-monarch-08.txt

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Tue, 29 Nov 2011 12:32:46 +0100

Dear All,

there are different types of monitoring architectures, particularily from perspective of QoE related performance parameters. 
Let's recall QoE (e.g., ITU-T P.10 Amendment 2):

Quality of Experience (QoE): 
	The overall acceptability of an application or service, as perceived subjectively by the end-user.  
	NOTE 1 - Quality of experience includes the complete end-to-end system effects (client, terminal, network, services infrastructure, etc.). 
	NOTE 2 - Overall acceptability may be influenced by user expectations and context. 
 
Important for MONARCH:
1st ... perceived by end-user
2nd QoE includes the complete end-to-end sytem effects ...

That said, present RTP monitoring architecture in MONARCH according clause 3 is not specific on the type of monitoring architecture. For instance, Fig. 1, it is possibly supposed that the "RTP Sender" and "RTP Receiver" entities are equal to a terminal/user equipment (UE). This would be an end-to-end IP network scenario.

However, the "RTP Sender" and "RTP Receiver" entities may be also located at the edge of an IP network, e.g. in a gateway for IP-to-non-IP interworking, i.e. an UE located in the non-IP network part. 
Note: such scenarios are considered by MEGACO gateways (I've Cc'd megaco list).

An RTP Sender/Receiver located NOT in the UE would question the value of RTP/RTCP QoE related performance measurements (because the UE is located outside the RTP domain). 

Conclusions:
- the set of meaningful performance metrics may be dependent on the location of the RTP Sender/Receiver, e.g. whether in a UE, gateway, ...
- don't see any impact on transport level metrics
- ... but the set of meaningful application level metrics

Proposal for MONARCH:
#1: insert sub-clause title "3.1 End-to-end IP network architecture" below clause 3

#2: add a sub-clause 3.2 at the end of clause 3, sth like

"3.2 RTP Sender/Receiver entities located in network nodes
The location of the RTP Sender/Receiver entity may impact the set of meaningful metrics.
For instance: application level metrics for QoE related performance metrics are questionable (or doesn't really make sense at all) when measured in a network node (terminating the RTP) instead of the user equipment."

Something like this would help to avoid confusion.
I even believe that a QoE measurement in a network node would violate the QoE definition. Would be rather something like QoE* ("remote QoE"):-(

Such information would improve the MONARCH doc in my opinion.
Sorry for that late comment.
Any views?

Regards,
Albrecht

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of [email-masked]
> Sent: Dienstag, 29. November 2011 08:56
> To: [email-masked]
> Cc: [email-masked]
> Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-monarch-08.txt
> 
> 
> A New Internet-Draft is available from the on-line 
> Internet-Drafts directories. This draft is a work item of the 
> Audio/Video Transport Core Maintenance Working Group of the IETF.
> 
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>

---

From: Qin Wu <[email-masked]>
Date: Wed, 30 Nov 2011 11:50:41 +0800

Hi,
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, November 29, 2011 7:32 PM
Subject: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

> Sorry for that late comment.

[Qin]: 
Albrrecht, if you follow ealry list discussion on draft-hunt-avt-monarch-01, you will see the rough consensus to position this document was 
this daft should focus on guidelines for defining reporting blocks. That is to say the main purpose of MONARCH draft is how to 
provide guidelines for the development of new RTCP XR block types in XRBlock working group.

Therefore how and where to collect the metric from RTP endpoint is not scope of this document, in my opinion. since you may have many ways to do this and different vendors have their different appoaches. It will be better to leave this beyond the scope. But you are right, it will be more reasonable to measure QoE related performance metric at the RTP receiver.
 
Another point is it is not clear to me how RTP monitoring has to do with architectureend to end IP network architecture .
this draft mostly focus on discussing RTP level stuff. Therefore I am quite doubt that we should describe anything in this document like how
RTP entties is mapped into network entities/nodes at the IP layer.

But as the author of this document, I am interested and open to other opinions.:-)

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Wed, 30 Nov 2011 18:24:16 +0100

My replies below inline,
Albrecht 

> From: Qin Wu [mailto:[email-masked]] 
> Sent: Mittwoch, 30. November 2011 04:51
> To: Schwarz, Albrecht (Albrecht); [email-masked]
> Cc: [email-masked]
> Subject: Re: [AVTCORE] Types of monitoring architectures; RE: 
> I-D Action: draft-ietf-avtcore-monarch-08.txt
> 
> Hi,
> Subject: [AVTCORE] Types of monitoring architectures; RE: I-D 
> Action: draft-ietf-avtcore-monarch-08.txt
> 
> 
> > Dear All,
> > 
> > there are different types of monitoring architectures, 
> particularily from perspective of QoE related performance parameters. 
> > NOTE 1 - Quality of experience includes the complete 
> end-to-end system effects (client, terminal, network, 
> services infrastructure, etc.). 
> > That said, present RTP monitoring architecture in MONARCH 
> according clause 3 is not specific on the type of monitoring 
> architecture. For instance, Fig. 1, it is possibly supposed 
> that the "RTP Sender" and "RTP Receiver" entities are equal 
> to a terminal/user equipment (UE). This would be an 
> end-to-end IP network scenario.
> > 
> > However, the "RTP Sender" and "RTP Receiver" entities may 
> be also located at the edge of an IP network, e.g. in a 
> gateway for IP-to-non-IP interworking, i.e. an UE located in 
> the non-IP network part. 
> > An RTP Sender/Receiver located NOT in the UE would question 
> the value of RTP/RTCP QoE related performance measurements 
> (because the UE is located outside the RTP domain). 
> > 
> > Conclusions:
> > - the set of meaningful performance metrics may be 
> dependent on the location of the RTP Sender/Receiver, e.g. 
> whether in a UE, gateway, ...
> > For instance: application level metrics for QoE related 
> performance metrics are questionable (or doesn't really make 
> sense at all) when measured in a network node (terminating 
> the RTP) instead of the user equipment."
> > 
> > I even believe that a QoE measurement in a network node 
> would violate the QoE definition. Would be rather something 
> like QoE* ("remote QoE"):-(
> > 
> Albrrecht, if you follow ealry list discussion on 
> draft-hunt-avt-monarch-01, you will see the rough consensus 
> to position this document was 
> this daft should focus on guidelines for defining reporting 
> blocks. That is to say the main purpose of MONARCH draft is how to 

[ABS] ... then the title "monitoring architectures" is misleading.
Proposal: keep title, revise scope section on focus

> 
> Therefore how and where to collect the metric from RTP 
> endpoint is not scope of this document, in my opinion. since 
> you may have many ways to do this and different vendors have 
> their different appoaches. It will be better to leave this 
> beyond the scope. But you are right, it will be more 
> reasonable to measure QoE related performance metric at the 
> RTP receiver.

[ABS] I would disagree. At least your statement should be conditional, sth like 
"QoE related performance metrics may be measured in a RTP receiver, IF the RTP receiver function is located in user equipment (e.g. IP terminal), because QoE performance metrics are tightly coupled to the point of view of a 'user'.
... IF NOT, then ..."

>  
> Another point is it is not clear to me how RTP monitoring has 
> to do with architectureend to end IP network architecture .
> this draft mostly focus on discussing RTP level stuff. 
> Therefore I am quite doubt that we should describe anything 
> in this document like how

[ABS] I don't got any problems to talk about architectural framework for monitoring in this document. Such information is required and this is the only IETF AVT doc about it.
The solution to this issue is rather simple to me:
- split the MONARCH "monitoring architecture" in
	1) a functional monitoring architecture (let's call it 'FMA') and
	2) a physical monitoring architecture (... 'PMA').

- the present MONARCH doc would describe the FMA at the level of RTP entities (i.e., Fig. 1 entities, incl. all the RTP topology components according RFC 5117)

- there are many options of functional-to-physical mappings, i.e. how a FMA would be mapped on a PMA
- the mapping is typically dependent on considered network architecture (like SIP, H.323, MGCP, IMS, wireless, etc networks)

- the present MONARCH doc would describe the FMA-to-PMA mapping IF all RTP receiver/sender entities would be located in physical IP terminals (UEs)
- etc

> But as the author of this document, I am interested and open 

[ABS] Like to add a further aspect: a "functional monitoring architecture" may be considered as a network overlay, something which adds additional capabilities (here: measurement services, reporting services on top of basic RTP services).
Such an overlay concept is nothing new: e.g., the MultiService Forum did define a "Performance Measurement & Monitoring (PMM)" overlay network, or IETF RMON may be also considered as such.

---

From: Qin Wu <[email-masked]>
Date: Thu, 01 Dec 2011 09:42:00 +0800

Hi,
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, December 01, 2011 1:24 AM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

My replies below inline,
Albrecht 

[Qin]: No, the title is *RTP* monitoring architecture, we can not expand scope into IP monitoring architecture.

[Qin]: I am not sure user equipment or IP terminal is the right term from RTP perspective.

Also if you read P.NAMS, P.NAMS model for calculating QoE related metric can be placed either at the network or at the client.
So I disagree with what you said.

Also as I said, how and where to collect the metric from RTP 
 endpoint is not scope of this document. It depends on the model you are choosing, e.g., four operation modes specified in P.NAMS.
Apparently this should not be specified again in RTP monitoring architecture document.

[Qin]: I can understand your point, but you can not do that. Since this document is only focus on *RTP* monitoring architecture, that is what it said in AVTCore milestone:
"

Goals and Milestones:

  Feb 2011 - Submit Monitoring Architecture for RTP for Informational

"
Also if you really don't see how to map FMA to PMA, you may look at the P.NAMS models specifed in ITU-T P.NAMS. 
As for how to use RTP monitoring architecture or RTP XR Report in the IMS network architecture, you may look at:
http://tools.ietf.org/html/rfc6035
ITU-T H.248.48.
But I haven't check the current status of H.248.48.