Subject: Re: [Megaco] [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Thu, 1 Dec 2011 17:20:28 +0100

My replies below ...
-Albrecht 

> -----Original Message-----
> From: Qin Wu [mailto:[email-masked]] 
> Sent: Donnerstag, 1. Dezember 2011 02:42
> To: Schwarz, Albrecht (Albrecht); [email-masked]
> Cc: [email-masked]
> Subject: Re: [AVTCORE] Types of monitoring architectures; RE: 
> I-D Action: draft-ietf-avtcore-monarch-08.txt
> 
> Hi,
> ----- Original Message -----
> From: "Schwarz, Albrecht (Albrecht)" 
> <[email-masked]>
> To: "Qin Wu" <[email-masked]>; <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, December 01, 2011 1:24 AM
> Subject: RE: [AVTCORE] Types of monitoring architectures; RE: 
> I-D Action: draft-ietf-avtcore-monarch-08.txt
> 
> 
> My replies below inline,
> Albrecht 
> 
> > -----Original Message-----
> > From: Qin Wu [mailto:[email-masked]] 
> > Sent: Mittwoch, 30. November 2011 04:51
> > To: Schwarz, Albrecht (Albrecht); [email-masked]
> > Cc: [email-masked]
> > Subject: Re: [AVTCORE] Types of monitoring architectures; RE: 
> > I-D Action: draft-ietf-avtcore-monarch-08.txt
> > 
> > Hi,
> > ----- Original Message -----
> > From: "Schwarz, Albrecht (Albrecht)" 
> > <[email-masked]>
> > To: <[email-masked]>
> > Cc: <[email-masked]>
> > Sent: Tuesday, November 29, 2011 7:32 PM
> > Subject: [AVTCORE] Types of monitoring architectures; RE: I-D 
> > Action: draft-ietf-avtcore-monarch-08.txt
> > 
> > 
> > > Dear All,
> > > 
> > > there are different types of monitoring architectures, 
> > particularily from perspective of QoE related performance 
> parameters. 
> > > Let's recall QoE (e.g., ITU-T P.10 Amendment 2):
> > > 
> > > Quality of Experience (QoE): 
> > > The overall acceptability of an application or service, as 
> > perceived subjectively by the end-user.  
> > > NOTE 1 - Quality of experience includes the complete 
> > end-to-end system effects (client, terminal, network, 
> > services infrastructure, etc.). 
> > > NOTE 2 - Overall acceptability may be influenced by user 
> > expectations and context. 
> > > 
> > > Important for MONARCH:
> > > 1st ... perceived by end-user
> > > 2nd QoE includes the complete end-to-end sytem effects ...
> > > 
> > > That said, present RTP monitoring architecture in MONARCH 
> > according clause 3 is not specific on the type of monitoring 
> > architecture. For instance, Fig. 1, it is possibly supposed 
> > that the "RTP Sender" and "RTP Receiver" entities are equal 
> > to a terminal/user equipment (UE). This would be an 
> > end-to-end IP network scenario.
> > > 
> > > However, the "RTP Sender" and "RTP Receiver" entities may 
> > be also located at the edge of an IP network, e.g. in a 
> > gateway for IP-to-non-IP interworking, i.e. an UE located in 
> > the non-IP network part. 
> > > Note: such scenarios are considered by MEGACO gateways 
> > (I've Cc'd megaco list).
> > > 
> > > An RTP Sender/Receiver located NOT in the UE would question 
> > the value of RTP/RTCP QoE related performance measurements 
> > (because the UE is located outside the RTP domain). 
> > > 
> > > Conclusions:
> > > - the set of meaningful performance metrics may be 
> > dependent on the location of the RTP Sender/Receiver, e.g. 
> > whether in a UE, gateway, ...
> > > - don't see any impact on transport level metrics
> > > - ... but the set of meaningful application level metrics
> > > 
> > > Proposal for MONARCH:
> > > #1: insert sub-clause title "3.1 End-to-end IP network 
> > architecture" below clause 3
> > > 
> > > #2: add a sub-clause 3.2 at the end of clause 3, sth like
> > > 
> > > "3.2 RTP Sender/Receiver entities located in network nodes
> > > The location of the RTP Sender/Receiver entity may impact 
> > the set of meaningful metrics.
> > > For instance: application level metrics for QoE related 
> > performance metrics are questionable (or doesn't really make 
> > sense at all) when measured in a network node (terminating 
> > the RTP) instead of the user equipment."
> > > 
> > > Something like this would help to avoid confusion.
> > > I even believe that a QoE measurement in a network node 
> > would violate the QoE definition. Would be rather something 
> > like QoE* ("remote QoE"):-(
> > > 
> > > Such information would improve the MONARCH doc in my opinion.
> > > Sorry for that late comment.
> > > Any views?
> > 
> > [Qin]: 
> > Albrrecht, if you follow ealry list discussion on 
> > draft-hunt-avt-monarch-01, you will see the rough consensus 
> > to position this document was 
> > this daft should focus on guidelines for defining reporting 
> > blocks. That is to say the main purpose of MONARCH draft is how to 
> > provide guidelines for the development of new RTCP XR block 
> > types in XRBlock working group.
> 
> [ABS] ... then the title "monitoring architectures" is misleading.
> Proposal: keep title, revise scope section on focus
> 
> [Qin]: No, the title is *RTP* monitoring architecture, we can 
> not expand scope into IP monitoring architecture.

[ABS] That's wasn't the intent! Did just focus on the key word.

> 
> > 
> > Therefore how and where to collect the metric from RTP 
> > endpoint is not scope of this document, in my opinion. since 
> > you may have many ways to do this and different vendors have 
> > their different appoaches. It will be better to leave this 
> > beyond the scope. But you are right, it will be more 
> > reasonable to measure QoE related performance metric at the 
> > RTP receiver.
> 
> [ABS] I would disagree. At least your statement should be 
> conditional, sth like 
> "QoE related performance metrics may be measured in a RTP 
> receiver, IF the RTP receiver function is located in user 
> equipment (e.g. IP terminal), because QoE performance metrics 
> are tightly coupled to the point of view of a 'user'.
> ... IF NOT, then ..."
> 
> [Qin]: I am not sure user equipment or IP terminal is the 
> right term from RTP perspective.
> 
> Also if you read P.NAMS, P.NAMS model for calculating QoE 
> related metric can be placed either at the network or at the client.
> So I disagree with what you said.

[ABS] Not convinced at all.
Guess you are referring to the "mid-point" models, right?
The usage of such that models depends on "prior knowledge ..." information.
Thus, the applicability is conditional. You may not make such a general statement.

> 
> Also as I said, how and where to collect the metric from RTP 
>  endpoint is not scope of this document. It depends on the 
> model you are choosing, e.g., four operation modes specified 
> in P.NAMS.
> Apparently this should not be specified again in RTP 
> monitoring architecture document.
> 
> >  
> > Another point is it is not clear to me how RTP monitoring has 
> > to do with architectureend to end IP network architecture .
> > this draft mostly focus on discussing RTP level stuff. 
> > Therefore I am quite doubt that we should describe anything 
> > in this document like how
> > RTP entties is mapped into network entities/nodes at the IP layer.
> 
> [ABS] I don't got any problems to talk about architectural 
> framework for monitoring in this document. Such information 
> is required and this is the only IETF AVT doc about it.
> The solution to this issue is rather simple to me:
> - split the MONARCH "monitoring architecture" in
> 1) a functional monitoring architecture (let's call it 'FMA') and
> 2) a physical monitoring architecture (... 'PMA').
> 
> - the present MONARCH doc would describe the FMA at the level 
> of RTP entities (i.e., Fig. 1 entities, incl. all the RTP 
> topology components according RFC 5117)
> 
> - there are many options of functional-to-physical mappings, 
> i.e. how a FMA would be mapped on a PMA
> - the mapping is typically dependent on considered network 
> architecture (like SIP, H.323, MGCP, IMS, wireless, etc networks)
> 
> - the present MONARCH doc would describe the FMA-to-PMA 
> mapping IF all RTP receiver/sender entities would be located 
> in physical IP terminals (UEs)
> - etc
> 
> [Qin]: I can understand your point, but you can not do that. 
> Since this document is only focus on *RTP* monitoring 
> architecture, that is what it said in AVTCore milestone:
> "
> 
> Goals and Milestones:
> 
>   Feb 2011 - Submit Monitoring Architecture for RTP for Informational
> 
> "

[ABS] If you go back to my first email on this subject, and if you look at my proposal: that's not a lot, nothing which violates the charter, goals and milestones.

> Also if you really don't see how to map FMA to PMA, you may 
> look at the P.NAMS models specifed in ITU-T P.NAMS. 
> As for how to use RTP monitoring architecture or RTP XR 
> Report in the IMS network architecture, you may look at:
> http://tools.ietf.org/html/rfc6035

[ABS] That's the SIP reporting interface, i.e., applicable primarily for physical IP user equipment with embedded RTP sender/receiver function and SIP UA function.
However, the applicability of RFC 6035 is limited for a SIP-to-PSTN gateway (because the RTP sender/receiver would be located NOT in user equipment. Also the P.NAMS mid-point model fails for such a scenario.

> ITU-T H.248.48.
> But I haven't check the current status of H.248.48.

[ABS] Work is finalized, see http://www.itu.int/md/meetingdoc.asp?lang=en&parent=T09-SG16-111121-TD-PLEN-0419

> 
> > 
> > But as the author of this document, I am interested and open 
> > to other opinions.:-)
> 
> [ABS] Like to add a further aspect: a "functional monitoring 
> architecture" may be considered as a network overlay, 
> something which adds additional capabilities (here: 
> measurement services, reporting services on top of basic RTP 
> services).
> Such an overlay concept is nothing new: e.g., the 
> MultiService Forum did define a "Performance Measurement & 
> Monitoring (PMM)" overlay network, or IETF RMON may be also 
> considered as such.
> 
> > 
> > > Regards,
> > > Albrecht
> > > 
> > > 
> > > 
> > > 
> > >> -----Original Message-----
> > >> From: [email-masked] [mailto:[email-masked]] On 
> > >> Behalf Of [email-masked]
> > >> Sent: Dienstag, 29. November 2011 08:56
> > >> To: [email-masked]
> > >> Cc: [email-masked]
> > >> Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-monarch-08.txt
> > >> 
> > >> 
> > >> A New Internet-Draft is available from the on-line 
> > >> Internet-Drafts directories. This draft is a work item of the 
> > >> Audio/Video Transport Core Maintenance Working Group of the IETF.
> > >> 
> > >> Title           : Monitoring Architectures for RTP
> > >> Author(s)       : Qin Wu
> > >>                           Geoff Hunt
> > >>                           Philip Arden
> > >> Filename        : draft-ietf-avtcore-monarch-08.txt
> > >> Pages           : 25
> > >> Date            : 2011-11-28
> > >> 
> > >>    This memo proposes an architecture for extending RTCP with 
> > >> a new RTCP
> > >>    XR (RFC3611) block type to report new metrics regarding media
> > >>    transmission or reception quality, following RTCP guideline
> > >>    established in RFC5968.  This memo suggests that a new 
> > block should
> > >>    contain a single metric or a small number of metrics 
> > relevant to a
> > >>    single parameter of interest or concern, rather than 
> > containing a
> > >>    number of metrics which attempt to provide full coverage 
> > >> of all those
> > >>    parameters of concern to a specific application.  
> > Applications may
> > >>    then "mix and match" to create a set of blocks which 
> > >> covers their set
> > >>    of concerns.  Where possible, a specific block should be 
> > >> designed to
> > >>    be re-usable across more than one application, for 
> > example, for all
> > >>    of voice, streaming audio and video.
> > >> 
> > >> 
> > >> A URL for this Internet-Draft is:
> > >> 
> > 
> http://www.ietf.org/internet-drafts/draft-ietf-avtcore-monarch-08.txt
> > >> 
> > >> Internet-Drafts are also available by anonymous FTP at:
> > >> ftp://ftp.ietf.org/internet-drafts/
> > >> 
> > >> This Internet-Draft can be retrieved at:
> > >> 
> > ftp://ftp.ietf.org/internet-drafts/draft-ietf-avtcore-monarch-08.txt
> > >> 
> > >> _______________________________________________
> > >> Audio/Video Transport Core Maintenance
> > >> [email-masked]
> > >> https://www.ietf.org/mailman/listinfo/avt
> > >> 
> > > _______________________________________________
> > > Audio/Video Transport Core Maintenance
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/avt
> >
>

---

From: Qin Wu <[email-masked]>
Date: Thu, 01 Dec 2011 09:42:00 +0800

Hi,
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, December 01, 2011 1:24 AM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

My replies below inline,
Albrecht 

> particularily from perspective of QoE related performance parameters. 

[Qin]: No, the title is *RTP* monitoring architecture, we can not expand scope into IP monitoring architecture.

> reasonable to measure QoE related performance metric at the 

[ABS] I would disagree. At least your statement should be conditional, sth like 
"QoE related performance metrics may be measured in a RTP receiver, IF the RTP receiver function is located in user equipment (e.g. IP terminal), because QoE performance metrics are tightly coupled to the point of view of a 'user'.
... IF NOT, then ..."

[Qin]: I am not sure user equipment or IP terminal is the right term from RTP perspective.

Also if you read P.NAMS, P.NAMS model for calculating QoE related metric can be placed either at the network or at the client.
So I disagree with what you said.

Also as I said, how and where to collect the metric from RTP 
 endpoint is not scope of this document. It depends on the model you are choosing, e.g., four operation modes specified in P.NAMS.
Apparently this should not be specified again in RTP monitoring architecture document.

> RTP entties is mapped into network entities/nodes at the IP layer.

[ABS] I don't got any problems to talk about architectural framework for monitoring in this document. Such information is required and this is the only IETF AVT doc about it.

- the present MONARCH doc would describe the FMA at the level of RTP entities (i.e., Fig. 1 entities, incl. all the RTP topology components according RFC 5117)

- there are many options of functional-to-physical mappings, i.e. how a FMA would be mapped on a PMA
- the mapping is typically dependent on considered network architecture (like SIP, H.323, MGCP, IMS, wireless, etc networks)

- the present MONARCH doc would describe the FMA-to-PMA mapping IF all RTP receiver/sender entities would be located in physical IP terminals (UEs)
- etc

[Qin]: I can understand your point, but you can not do that. Since this document is only focus on *RTP* monitoring architecture, that is what it said in AVTCore milestone:
"

Goals and Milestones:

"
Also if you really don't see how to map FMA to PMA, you may look at the P.NAMS models specifed in ITU-T P.NAMS. 
As for how to use RTP monitoring architecture or RTP XR Report in the IMS network architecture, you may look at:
http://tools.ietf.org/html/rfc6035
ITU-T H.248.48.
But I haven't check the current status of H.248.48.

> But as the author of this document, I am interested and open 

[ABS] Like to add a further aspect: a "functional monitoring architecture" may be considered as a network overlay, something which adds additional capabilities (here: measurement services, reporting services on top of basic RTP services).
Such an overlay concept is nothing new: e.g., the MultiService Forum did define a "Performance Measurement & Monitoring (PMM)" overlay network, or IETF RMON may be also considered as such.

---

From: Qin Wu <[email-masked]>
Date: Fri, 02 Dec 2011 09:54:33 +0800

Hi, Albrecht:
No offence, I am not really convinced with your proposal on physical monitoring architecture.
 my counterquestion is what does physical monitoring architecture looks like in your mind?
How many components do we have in such physical monitroing architecture? what are they?
Take IMS network architecture as a example, they may have P-CSCF,S-CSCF, SIP PBX, SBC,
why do you believe monitor described in draft-ietf-avtcore-monarch is mapped into P-CSCF,
S-CSCF, SBC, Media Gateway, Media Gateway Controller should be in the scope of IETF RTP 
monitoring archtiecture draft?

Monitor described in  draft-ietf-avtcore-monarch is well defined in the section 3 of RFC3550 as follows:
"

   Monitor: An application that receives RTCP packets sent by
      participants in an RTP session, in particular the reception
      reports, and estimates the current quality of service for
      distribution monitoring, fault diagnosis and long-term statistics.
      The monitor function is likely to be built into the application(s)
      participating in the session, but may also be a separate
      application that does not otherwise participate and does not send
      or receive the RTP data packets (since they are on a separate
      port).  These are called third-party monitors.  It is also
      acceptable for a third-party monitor to receive the RTP data
      packets but not send RTCP packets or otherwise be counted in the
      session.

"
Isn't how to implent monitor into IMS netowork archtiecture a implentation specific issue?
If this issue are really interesting, isn't it a good idea to produce another new document to talk about this?
One good example for this motivation is this draft I like to share with you:
http://tools.ietf.org/html/draft-kaplan-dispatch-b2bua-taxonomy-00#section-4.6

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, December 02, 2011 12:20 AM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

My replies below ...
-Albrecht 

> =

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Mon, 5 Dec 2011 15:48:35 +0100

Hello Qin,

to 1) Physical monitoring architecture
Of course consideration of generic physical entities (PE), not specific to any network reference architectures defined by SDOs (like your refered entities).
There would be just a few PE types ("given by the limited set of RTP topology components (RFC 5117)").

to 2) (RFC 3550) Monitor function
Right, that's a functional entity. The applicability of a monitor may be limited, dependent on ... (like in P.NAMS).

to 3) your IMS network example
Don't see the need to refer from MONARCH to a specific network architecture like IMS.
Please also note that IMS is a functional architecture, not a physical architecture.

Regards,
Albrecht

PS
There are multiple options for adding the "RTP monitoring architecture" to a functional/physical network architecture:
a) integrated approach: extension of existing functional entities or/and addition of RTP monitoring architectural components, extension or/and addition of signalling/management interfaces for control and configuration of RTP monitoring services, and reporting interfaces for measurement data ...
b) overlay approach: separate specification to ...

PSS
BTW: there's a discussion paper with regards to the location of a monitor function in an IMS H.248 gateway entity:
Measurements and statistics in IMS - Use Cases
http://www.3gpp.org/ftp/tsg_ct/WG3_interworking_ex-CN3/TSGC3_58_Kyoto_Japan/Docs/C3-100469.zip
This is just fyi.

> Behalf Of Qin Wu
> Sent: Freitag, 2. Dezember 2011 02:55
> Take IMS network architecture as a example, they may have
> P-CSCF,S-CSCF, SIP PBX, SBC, why do you believe monitor
> described in draft-ietf-avtcore-monarch is mapped into
> P-CSCF, S-CSCF, SBC, Media Gateway, Media Gateway Controller
> should be in the scope of IETF RTP monitoring archtiecture draft?
>
> Monitor described in  draft-ietf-avtcore-monarch is well
> defined in the section 3 of RFC3550 as follows:
> "
>
> If this issue are really interesting, isn't it a good idea to
> produce another new document to talk about this?
> [ABS] If you go back to my first email on this subject, and
> if you look at my proposal: that's not a lot, nothing which
> violates the charter, goals and milestones.
>
>
> [ABS] That's the SIP reporting interface, i.e., applicable
> primarily for physical IP user equipment with embedded RTP
> sender/receiver function and SIP UA function.
> However, the applicability of RFC 6035 is limited for a
> SIP-to-PSTN gateway (because the RTP sender/receiver would be
> located NOT in user equipment. Also the P.NAMS mid-point
> model fails for such a scenario.
>
> > ITU-T H.248.48.
> [ABS] Work is finalized, see
> http://www.itu.int/md/meetingdoc.asp?lang=en&parent=T09-SG16-1
> 11121-TD-PLEN-0419
>
> >
> > >

---

From: Qin Wu <[email-masked]>
Date: Tue, 06 Dec 2011 14:13:17 +0800

If you talk about existing RFC5117 RTP topology components, this has already been covered by the current version of draft-ietf-avtcore-monarch.
In most cases, an RTP system using RTCP XR is architecturally no different from an RTP system which does not use RTCP XR.

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, December 05, 2011 10:48 PM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

Hello Qin,

Regards,
Albrecht

PS

PSS

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Wed, 7 Dec 2011 17:24:24 +0100

Qin,
that's NOT the point.

All RFC 3550 and RFC 5117 RTP components are FUNCTIONAL ENTITIES (FE).
e.g.,
- the RTP Monitor or RTP receiver FE may be located in different PHYSICAL ENTITIES (PE), e.g.
        a1) a user equipment
        a2) a RTP-to-PSTN gateway
        a3) an IP-to-IP session border gateway (in a back-to-back RTP endsystem topology)
        a3) ...

- the RTP Monitor may be associated to an RTP transport translator ...
        b1) an IP-to-IP session border gateway

The "RTP monitoring architecture" is IMPACTED by each functional-to-physical mapping because e.g.
* QoE related metrics are questionable in case of a2 ("despite the fact of a RTP receiver function")
* application level metrics at the location of a RTP transport translator are a contradiction in itself
* ...

Again, it's not really changing the MONARCH draft because in the very majority of scenarios it is correct. However, there are few cases where the MONARCH description would be "too high level", "not fully correct", "lead to possible wrong expectations", ...

Regards,
Albrecht

> Sent: Dienstag, 6. Dezember 2011 07:13
> If you talk about existing RFC5117 RTP topology components,
> this has already been covered by the current version of
> draft-ietf-avtcore-monarch.
> In most cases, an RTP system using RTCP XR is architecturally
> no different from an RTP system which does not use RTCP XR.
>
> Regards!
> -Qin
> Of course consideration of generic physical entities (PE),
> not specific to any network reference architectures defined
> by SDOs (like your refered entities).
> Right, that's a functional entity. The applicability of a
> monitor may be limited, dependent on ... (like in P.NAMS).
>
> There are multiple options for adding the "RTP monitoring
> architecture" to a functional/physical network architecture:
> a) integrated approach: extension of existing functional
> entities or/and addition of RTP monitoring architectural
> components, extension or/and addition of
> signalling/management interfaces for control and
> configuration of RTP monitoring services, and reporting
> interfaces for measurement data ...
> BTW: there's a discussion paper with regards to the location
> of a monitor function in an IMS H.248 gateway entity:

---

From: Qin Wu <[email-masked]>
Date: Thu, 08 Dec 2011 10:25:54 +0800

You reiterate what you propose for physical entities in the early email.
I am still not convicing with that.

Since we have RTP components like RTP end system, RTP mixer, RTP media translator, RTP transport translator,
RTP monitor specified in RFC3550 can be built into these component as a application. also monitor can be disjointed 
from these RTP components as a third party monitor. why we need to specify how to map RTP end system into user equipment,
RTP translator or RTP mixer can be mapped into RTP-PSTN gateway, or IP to IP border gateway since we have already mapped monitor
into different RTP components. 
How many PE types you can enumerate? why they should be scope of RTP monitoring architecture?

Also as specified in MONARCH draft, we have end sytem metrics, transport level metric, transport level metric, they can be measured and collected from the different RTP
comoponents specified in RFC3550, RFC5117. Apparently end system metric can only be collected from RTP end system.
transport metric is collected from any RTP system in the RTP monitoring architecture, 
regarding application level metric, apparently they should be measured at the application layer and can not be collected from RTP transport translator, Also RTP transport translator is still 
a functional entity rather than a physical entity. I am not convincing with what you said about "contradiction", "lead to possible wrong expectations".

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, December 08, 2011 12:24 AM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

Qin,
that's NOT the point.

Regards,
Albrecht

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Fri, 9 Dec 2011 17:01:00 +0100

How do you explain to a network operator the value of QoE measurements by an RTP receiver / RTP monitor component located in an RTP-to-PSTN gateway?

This would be a monitoring scenario inline with MONARCH, but with valueless QoE results, right?

> Sent: Donnerstag, 8. Dezember 2011 03:26
> Since we have RTP components like RTP end system, RTP mixer,
> RTP media translator, RTP transport translator, RTP monitor
> specified in RFC3550 can be built into these component as a
> application. also monitor can be disjointed from these RTP
> components as a third party monitor. why we need to specify
> how to map RTP end system into user equipment, RTP translator
> or RTP mixer can be mapped into RTP-PSTN gateway, or IP to IP
> border gateway since we have already mapped monitor into
> different RTP components.
> Also as specified in MONARCH draft, we have end sytem
> metrics, transport level metric, transport level metric, they
> can be measured and collected from the different RTP
> comoponents specified in RFC3550, RFC5117. Apparently end
> system metric can only be collected from RTP end system.
> transport metric is collected from any RTP system in the RTP
> monitoring architecture, regarding application level metric,
> apparently they should be measured at the application layer
> and can not be collected from RTP transport translator, Also
> RTP transport translator is still a functional entity rather
> than a physical entity. I am not convincing with what you
> said about "contradiction", "lead to possible wrong expectations".
>
> Regards!
> -Qin
> The "RTP monitoring architecture" is IMPACTED by each
> functional-to-physical mapping because e.g.
> * QoE related metrics are questionable in case of a2
> ("despite the fact of a RTP receiver function")
> * application level metrics at the location of a RTP
> transport translator are a contradiction in itself
> * ...
>
> Again, it's not really changing the MONARCH draft because in
> the very majority of scenarios it is correct. However, there
> are few cases where the MONARCH description would be "too
> high level", "not fully correct", "lead to possible wrong
> expectations", ...
>
> Regards,
> Albrecht
>
>
>

---

From: Qin Wu <[email-masked]>
Date: Mon, 12 Dec 2011 12:04:31 +0800

Hi, Albrecht:
My understanding to QoE measurment is:
The factors that influence the QoE metric include encoding system, transport network, access network,home network and end device.
If the customer(e.g., network operator) doesn't know the original content, the QoE metric should be usually measured at the user device.
If the customer knows the original content, the QoE metric can be measured either in the network or at the user device.
Given the growing popularity of connected devices, these devices may have different capability corresponding to CPU load, processing overhead, storage, batteries.
If the cusomer want to measure QoE without considering connected devices with various capability , the QoE metric can be measured at the point between
access network and home network.

Regarding the monitoring scenario you mentioned, I am not familiar with such scenario with RTP-to-PSTN gateway deployed.
But I am aware that there are scenarios with SIP to PSTN gateway or SIP to H.323 gateway widely deployed,  RFC6035 defines a new SIP event package, vq-rtcpxr
that enable the collection and  reporting of metrics that measure quality for RTP sessions. In such cases, RTCP XR VOIP metric with QoE metric included is measured  
at the user device and mapped into the equivalent SIP vq-rtcpxr parameter since SIP session call is established end to end and RTP media stream is exchanged directly between
one endpoint and the other endpoint.

Regards!
-Qin

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Mon, 12 Dec 2011 11:33:08 +0100

Hi Qin,

I'm very aware about all that.
It looks like that you are now agreeing that QoE measurements are CONDITIONAL ("you got a number of IF statements, there might be even more ...").

MONARCH is not talking about such conditions, thus some readers may got the impression that the metrics related to RTP traffic are UNCONDITIONAL (which is not true as you confirmed).

If you go back to my very first email: I did just propose to indicate the conditional aspect. Don't understand why such kind of valuable info would be not in scope of MONARCH. Hm?

Regards,
Albrecht

PS
I did use the notion of a control plane agnostic "RTP-to-PSTN gateway". You did indicate control plane specific gateway types like SIP-to-PSTN or SIP-to-H.323. Now issue to use such signalling protocol specific examples, but it doesn't change the underlying issue.
The SIP side reporting capability based on RFC 6035 allows to report the outlined measurements, but the RFC itself does not specify whether the reported information is meaningful (which is OK because this is not subject of a reporting interface spec, it is rather subject of a network monitoring doc like MONARCH).
QoE measurements would be conditional in both gateway types (because such a gateway is NOT a user device despite the fact of an embedded SIP user agent entity).
Actually I would even question here the condition of "customer knows the original content" in many real network scenarios.

> Sent: Montag, 12. Dezember 2011 05:05
> The factors that influence the QoE metric include encoding
> system, transport network, access network,home network and end device.
> If the customer(e.g., network operator) doesn't know the
> original content, the QoE metric should be usually measured
> at the user device.
> If the customer knows the original content, the QoE metric
> can be measured either in the network or at the user device.
> Given the growing popularity of connected devices, these
> devices may have different capability corresponding to CPU
> load, processing overhead, storage, batteries.
> If the cusomer want to measure QoE without considering
> connected devices with various capability , the QoE metric
> can be measured at the point between access network and home network.
>
> Regarding the monitoring scenario you mentioned, I am not
> familiar with such scenario with RTP-to-PSTN gateway deployed.
> But I am aware that there are scenarios with SIP to PSTN
> gateway or SIP to H.323 gateway widely deployed,  RFC6035
> defines a new SIP event package, vq-rtcpxr that enable the
> collection and  reporting of metrics that measure quality for
> RTP sessions. In such cases, RTCP XR VOIP metric with QoE
> metric included is measured at the user device and mapped
> into the equivalent SIP vq-rtcpxr parameter since SIP session
> call is established end to end and RTP media stream is
> exchanged directly between one endpoint and the other endpoint.
>
> Regards!
> -Qin
> How do you explain to a network operator the value of QoE
> measurements by an RTP receiver / RTP monitor component
> located in an RTP-to-PSTN gateway?
>

---

From: Qin Wu <[email-masked]>
Date: Tue, 13 Dec 2011 09:55:17 +0800

I am just a little doubt whether mapping RTP component into specific network node is the scope of MONARCH draft
since it was agreed early that the focus of monarch draft is to provide guideline on how to define XRBlock.
But I am fine if nobody has objection to your proposal, i.e., wide scope a little bit to talk about how to map RTP 
monitoring architecture to the network nodes.

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, December 12, 2011 6:33 PM
Subject: RE: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

Hi Qin,

Regards,
Albrecht

PS

---

From: Qin Wu <[email-masked]>
Date: Tue, 13 Dec 2011 10:00:19 +0800

Hi, Albrecht:
Thank for your proposal. I will how your proposed text can fit into RTP monitoring architecture
and reflect them in the new update. Thanks again.

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, December 12, 2011 6:55 PM
Subject: Update proposal; RE: [Megaco] [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

Might be more effective by continuing the discussion with a concrete text proposal, see attachments.
Any comments?
Albrecht

> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Schwarz,
> Albrecht (Albrecht)
> Sent: Montag, 12. Dezember 2011 11:33
> To: Qin Wu; [email-masked]
> Cc: [email-masked]
> Subject: Re: [Megaco] [AVTCORE] Types of monitoring
> architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt
>
> Hi Qin,
>
> It looks like that you are now agreeing that QoE measurements
> are CONDITIONAL ("you got a number of IF statements, there
> might be even more ...").
>
> MONARCH is not talking about such conditions, thus some
> readers may got the impression that the metrics related to
> RTP traffic are UNCONDITIONAL (which is not true as you confirmed).
>
> If you go back to my very first email: I did just propose to
> indicate the conditional aspect. Don't understand why such
> kind of valuable info would be not in scope of MONARCH. Hm?
>
> Regards,
> Albrecht
>
> PS
> I did use the notion of a control plane agnostic "RTP-to-PSTN
> gateway". You did indicate control plane specific gateway
> types like SIP-to-PSTN or SIP-to-H.323. Now issue to use such
> signalling protocol specific examples, but it doesn't change
> the underlying issue.
> The SIP side reporting capability based on RFC 6035 allows to
> report the outlined measurements, but the RFC itself does not
> specify whether the reported information is meaningful (which
> is OK because this is not subject of a reporting interface
> spec, it is rather subject of a network monitoring doc like MONARCH).
> QoE measurements would be conditional in both gateway types
> (because such a gateway is NOT a user device despite the fact
> of an embedded SIP user agent entity).
> Actually I would even question here the condition of
> "customer knows the original content" in many real network scenarios.
>
>
>
> > The factors that influence the QoE metric include encoding system,
> > transport network, access network,home network and end device.
> > If the customer(e.g., network operator) doesn't know the original
> > content, the QoE metric should be usually measured at the
> user device.
> > If the customer knows the original content, the QoE metric can be
> > measured either in the network or at the user device.
> devices may
> > have different capability corresponding to CPU load, processing
> > overhead, storage, batteries.
> > If the cusomer want to measure QoE without considering connected
> > devices with various capability , the QoE metric can be measured at
> > the point between access network and home network.
> >
> > Regarding the monitoring scenario you mentioned, I am not familiar
> > with such scenario with RTP-to-PSTN gateway deployed.
> > But I am aware that there are scenarios with SIP to PSTN gateway or
> > SIP to H.323 gateway widely deployed,  RFC6035 defines a
> new SIP event
> > package, vq-rtcpxr that enable the collection and  reporting of
> > metrics that measure quality for RTP sessions. In such
> cases, RTCP XR
> > VOIP metric with QoE metric included is measured at the user device
> > and mapped into the equivalent SIP vq-rtcpxr parameter since SIP
> > session call is established end to end and RTP media stream is
> measurements
> > by an RTP receiver / RTP monitor component located in an
> RTP-to-PSTN
> > gateway?
> >
> > > Since we have RTP components like RTP end system, RTP mixer, RTP
> > > media translator, RTP transport translator, RTP monitor
> specified in
> > > RFC3550 can be built into these component as a application. also
> > > monitor can be disjointed from these RTP components as a
> third party
> > > monitor. why we need to specify how to map RTP end system
> into user
> > > equipment, RTP translator or RTP mixer can be mapped into
> RTP-PSTN
> > > gateway, or IP to IP border gateway since we have already mapped
> > > monitor into different RTP components.
> > > How many PE types you can enumerate? why they should be
> scope of RTP
> > > monitoring architecture?
> > >
> > > Also as specified in MONARCH draft, we have end sytem metrics,
> > > transport level metric, transport level metric, they can
> be measured
> > > and collected from the different RTP comoponents specified in
> > > RFC3550, RFC5117. Apparently end system metric can only
> be collected
> > > from RTP end system.
> > > apparently they should be measured at the application
> layer and can
> > > not be collected from RTP transport translator, Also RTP
> transport
> > > translator is still a functional entity rather than a physical
> > > entity. I am not convincing with what you said about
> > > "contradiction", "lead to possible wrong expectations".
> > >
> > > Regards!
> > > -Qin
> > > Again, it's not really changing the MONARCH draft because in the
> > > very majority of scenarios it is correct. However, there are few
> > > cases where the MONARCH description would be "too high
> level", "not
> > > fully correct", "lead to possible wrong expectations", ...
> > >
> > > Regards,
> > > Albrecht
> > >
> > >
> > >
> > > > If you talk about existing RFC5117 RTP topology
> components, this
> > > > has already been covered by the current version of
> > > > In most cases, an RTP system using RTCP XR is
> architecturally no
> > > > different from an RTP system which does not use RTCP XR.
> > > >
> > > > Regards!
> > > > -Qin
> > > > to 1) Physical monitoring architecture Of course
> consideration of
> > > > generic physical entities (PE), not specific to any network
> > > > reference architectures defined by SDOs (like your refered
> > > > entities).
> > > > There would be just a few PE types ("given by the
> limited set of
> > > > RTP topology components (RFC 5117)").
> > > >
> > > > Right, that's a functional entity. The applicability of
> a monitor
> > > > may be limited, dependent on ... (like in P.NAMS).
> > > >
> > > > a) integrated approach: extension of existing
> functional entities
> > > > or/and addition of RTP monitoring architectural components,
> > > > extension or/and addition of signalling/management
> interfaces for
> > > > control and configuration of RTP monitoring services, and
> > > > reporting interfaces for measurement data ...
> > > > BTW: there's a discussion paper with regards to the
> location of a
> > > > monitor function in an IMS H.248 gateway entity:
> > > > > described in draft-ietf-avtcore-monarch is mapped
> into P-CSCF,
> > > > > S-CSCF, SBC, Media Gateway, Media Gateway Controller
> should be
> > > > > in the scope of IETF RTP monitoring
> > archtiecture draft?
> > > > >
> > > > > Monitor described in  draft-ietf-avtcore-monarch is
> well defined
> > > > > in the section 3 of RFC3550 as follows:
> > > > > "
> > > > >
> > > > >       application that does not otherwise participate
> and does
> > > > > not send
> > > > > > > > 1st ... perceived by end-user 2nd QoE includes the
> > > > > > > > complete end-to-end sytem effects ...
> > > > > > > >
> > > > > > > to position this document was this daft should focus on
> > > > > > > guidelines for defining reporting blocks. That is
> to say the
> > > > > > > main purpose of MONARCH draft
> > > > > is how to
> > > > > > conditional, sth like "QoE related performance
> metrics may be
> > > > > > measured in a RTP receiver, IF the RTP receiver function is
> > > > > > located in user equipment (e.g. IP terminal), because QoE
> > > > > > performance metrics are tightly coupled to the
> point of view
> > > > > > of a 'user'.
> > > > > > ... IF NOT, then ..."
> > > > > >
> > > > > The usage of such that models depends on "prior
> knowledge ..."
> > > > > information.
> > > > > Thus, the applicability is conditional. You may not
> make such a
> > > > > general statement.
> > > > >
> > > > >
> > > > > >
> > > > > > > in this document like how RTP entties is mapped
> into network
> > > > > > > entities/nodes at
> > > > the IP layer.
> > > > > >
> > > > > > framework for monitoring in this document. Such
> information is
> > > > > > required and this is the only IETF AVT doc about it.
> > > > > [ABS] If you go back to my first email on this
> subject, and if
> > > > > you look at my proposal: that's not a lot, nothing which
> > > > > > As for how to use RTP monitoring architecture or
> RTP XR Report
> > > > > > in the IMS network architecture, you may look at:
> > > > > located NOT in user equipment. Also the P.NAMS
> mid-point model
> > > > > fails for such a scenario.
> > > > >
> > > > > > ITU-T H.248.48.
> MultiService
> > > > > > Forum did define a "Performance Measurement & Monitoring
> > > > > > (PMM)" overlay network, or IETF RMON may be also
> considered as
> > > > > > such.
> > > > > >
> > > > > > >
> > > > > > > > Regards,
> > > > > > > > Albrecht
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >> Audio/Video Transport Core Maintenance [email-masked]
> > > > > > > > Audio/Video Transport Core Maintenance [email-masked]
> > > > > Audio/Video Transport Core Maintenance [email-masked]
> Megaco mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/megaco
>

---

From: Qin Wu <[email-masked]>
Date: Tue, 13 Dec 2011 10:15:17 +0800

Hi, Colin:
Thank for your suggestion. Besides Albrecht's proposed text, my additional proposal is to fix application level metric defintion in section 2
OLD text:
"
   Application level metrics

      Metrics relating to QoE related parameters.  These metrics are
      measured at the application level and focus on quality of content
      rather than network parameters. 
"
New text
"
   Application level metrics
 
      Metrics relating to application specific parameters or QoE related parameters. 
     Application specific parameters  are measured at the application level and focus
     on quality of content rather than network performance.  QoE related parameters
     reflect the end-to-end performance at the services level and is ususally measured
     at the user endpoint.
"

Regards!
-Qin
----- Original Message ----- 
From: "Colin Perkins" <[email-masked]>
To: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Cc: "Qin Wu" <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, December 13, 2011 12:56 AM
Subject: Re: [AVTCORE] Types of monitoring architectures; RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

Albrecht, Qin,

It doesn't seem controversial that some metrics are only meaningful in some scenarios, or when measured at certain points in a system. I don't see how adding a statement to the monarch draft to recognise that could be problematic. Is there anything else being suggested here?

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Tue, 13 Dec 2011 12:16:31 +0100

> Albrecht, Qin,
>
> It doesn't seem controversial that some metrics are only
> meaningful in some scenarios, or when measured at certain
> points in a system. I don't see how adding a statement to the
> monarch draft to recognise that could be problematic. Is
> there anything else being suggested here?
>
> Colin

Right.
No, nothing else is suggested.
(Send yesterday also an update proposal with initial text.)

Albrecht

> Sent: Dienstag, 13. Dezember 2011 03:15
> To: Colin Perkins; Schwarz, Albrecht (Albrecht)
> Cc: [email-masked]; [email-masked]
> Thank for your suggestion. Besides Albrecht's proposed text,
> my additional proposal is to fix application level metric
> defintion in section 2 OLD text:
> "
> It doesn't seem controversial that some metrics are only
> meaningful in some scenarios, or when measured at certain
> points in a system. I don't see how adding a statement to the
> monarch draft to recognise that could be problematic. Is
> there anything else being suggested here?
>
> Colin
>
>
> > It looks like that you are now agreeing that QoE
> measurements are CONDITIONAL ("you got a number of IF
> statements, there might be even more ...").
> >
> > If you go back to my very first email: I did just propose
> to indicate the conditional aspect. Don't understand why such
> > I did use the notion of a control plane agnostic
> "RTP-to-PSTN gateway". You did indicate control plane
> specific gateway types like SIP-to-PSTN or SIP-to-H.323. Now
> issue to use such signalling protocol specific examples, but
> it doesn't change the underlying issue.
> > The SIP side reporting capability based on RFC 6035 allows
> to report the outlined measurements, but the RFC itself does
> not specify whether the reported information is meaningful
> (which is OK because this is not subject of a reporting
> interface spec, it is rather subject of a network monitoring
> doc like MONARCH).

---

From: Colin Perkins <[email-masked]>
Date: Mon, 12 Dec 2011 16:56:57 +0000

Albrecht, Qin,

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Wed, 14 Dec 2011 15:57:20 +0800

Okay, thanks.
----- Original Message ----- 
From: "Colin Perkins" <[email-masked]>
To: "Qin Wu" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Sent: Wednesday, December 14, 2011 6:54 AM
Subject: Re: [AVTCORE] Types of monitoring architectures;RE: I-D Action: draft-ietf-avtcore-monarch-08.txt

> Qin,
> 
> Both this, and Albrecht's suggested additions, look reasonable to me.
> 
> Colin
> 
> 
> 
> On 13 Dec 2011, at 02:15, Qin Wu wrote:
>> Hi, Colin:

---

From: Colin Perkins <[email-masked]>
Date: Tue, 13 Dec 2011 22:54:26 +0000

Qin,

Colin

-- 
Colin Perkins
http://csperkins.org/