Subject: [xrblock] Call for adopting xr-quality-monitoring-06 as WG item

From: Shida Schubert <[email-masked]>
Date: Mon, 19 Dec 2011 20:55:08 +0900

This is a call to see whether people are happy 
with the latest xr-quality-monitoring-06 draft for addressing 
the milestone "QoE Metrics Reporting".

 http://tools.ietf.org/html/draft-wu-xrblock-rtcp-xr-quality-monitoring-06

 xr-que-00 which used to be an AVT draft also addresses 
the same milestone. 

 http://tools.ietf.org/id/draft-clark-xrblock-rtcp-xr-qoe-00.txt

 The xr-quality-monitoring-06 claims to merge the two 
into one. 

 Please response to express your option for the followings. 

 1. Adopt the draft-wu-xrblock-rtcp-xr-quality-monitoring-06 as a WG item. 

 2. Not ready to adopt.
   * Would be very helpful if you provide your reasonings.

 Regards
  Shida (as chair)

---

From: Colin Perkins <[email-masked]>
Date: Mon, 19 Dec 2011 17:13:44 +0000

Shida,

I don't think this draft is ready to adopt:

- The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into designing the XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. The format seems to follow the RFC 3611 model, throwing together a set of unrelated parameters, rather than the approach documents in the monarch draft.

- The multi-layer segment is ill-specified: does it support audio only, video only, or both depending on the Media Type bit? Which calculation algorithm makes sense here, since there is no guidance given, and it's possible to specify any algorithm? What is the SSID field? How it is used? It's not sufficient to simply say that a "NAL unit type is one example", the draft needs to give normative rules for the use of this field.

- The media type, MOS type, and calculation algorithm for the multi-channel segment are ill-specified in similar ways to the multi-layer segment. Also, it's not clear that the channel mapping in RFC 3550 Section 4.1 is the only one in use.

- In general, it's not clear to me that including multiple different types of MOS report (single-stream, multi-layer, and multi-channel) in a single draft fits with the monitoring architecture, which encourages small, single-use, XR block definitions. I would suggest that the three different MOS report types would be better defined as three drafts: there is little in common between them.

- The draft provides no meaningful guidelines for registration of new MOS algorithms, and has an inadequate IANA registry.

Colin

On 19 Dec 2011, at 11:55, Shida Schubert wrote:
> _______________________________________________
> xrblock mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xrblock

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Tue, 20 Dec 2011 09:54:23 +0800

Hi, Colin:
I understand your concerns. It is painful, but that's the tradeoff we should made.
please see my clarification belows.
----- Original Message ----- 
From: "Colin Perkins" <[email-masked]>
To: "Shida Schubert" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, December 20, 2011 1:13 AM
Subject: Re: [xrblock] Call for adopting xr-quality-monitoring-06 as WG item

> - The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into designing the >XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. 

[Qin]:  We do have guidance to prevent such combination  in three places
a. See MoS Type (MT) field definition, it said:
"
      If MoS type is MoS-LQ and MoS-CQ, the MoS value can be calculated
      based on ITU-T G.107[G.107], ITU-T P.564 [P.564]or ETSI TS 101
      329-5 [ETSI], if the Mos type is MoS-V or MoS-AV, the Mos value
      can be calculated based on ITU-T P.NAMS [P.NAMS]or ITU-T P.NBAMS
      [P.NBAMS]. 
"
b. Also see Mos Type (MT) field defintion, it said:
"
     If MoS Type does not match the MoS algorithm in the report (e.g.,
      specify a voice MOS algorithm for a video quality MOS), the
      receiver should also discard this report.
"
c.  See Calculation Algorithm (CALg) field definition
It distinguish Audio MoS algorithm from video MoS algorithm by adding "(voice)" into Audio MoS algorithm.

>The format seems to follow the RFC 3611 model, throwing together a set of unrelated parameters, rather than the approach documents in the monarch draft.

[Qin]: In this draft, we are not allowed to put "single stream segment","multi-layer segment" and "multi-channel segment" into one XR Block Report. please see the note in the segement i field definition
it said:
"
      Note that in these
      three segment type,any two segment types can not be present in the
      same metric block.
"
That is to say if we choose to use "single stream segment" to report MoS, we will not use "multi-channel segment" or "multi-layer segment" . If we choose to use multi-channel segment, we will not use 
"single stream segment" or "multi-layer segment".

Also we may need to measure MoS-CQ using three different audio MoS algorithms, e.g., 
Using ITU-T P.564 to calculate MoS-CQ1
Using G.107 to calculate MoS-CQ2
Using ETSI TS 101 329-5 to calculate MoS-CQ3

In this case,  MoS-CQ1 and MoS-CQ2 and MoS-CQ3 are all MoS value, but using different audio MoS algorithm, so the question is how to report them, we have two options:
1. Report each MoS-CQ value carrried in the "single stream segment" using three XR Blocks 
2. Report three MoS-CQ vlaues carried in three "single stream segment" using only one XR Block.
I think the option 2 is a better approach, this approach only report MoS value related to one single MoS Type and doesn't put a set of unrelated parameters together and therefore doesn't conflict the rule we set in MONARCH.
However if we also measure MoS-LQ using three different audio MoS algorithms and report MoS-CQ value and MoS-LQ value carried in two "single stream segment" using one single XR Block, that does mean "put a set of unrelated parameters together" and therefore conflict with Monarch.

So my suggestion is to add a sentence to say only one MoS type is allowed to report in one single XR Block.

If you agree with this, I can revise the draft to make this clear. 

> - The multi-layer segment is ill-specified: does it support audio only, video only, or both depending on the Media Type bit? 

[Qin]: multi-layer segment does support both video and audio, 

>Which calculation algorithm makes sense here, since there is no guidance given, and it's possible to specify any algorithm? 

[Qin]: currently we only allow 5 MoS algorithms.since three Audio MoS algorithms have already been standardized and the other two are in standardization progress.

>What is the SSID field? How it is used? 

[Qin]:SSID is sub stream identifier and can be used to identifier each layer if all the layers are carried in the same RTP stream.

>It's not sufficient to simply say that a "NAL unit type is one example", the draft needs to give normative rules for the use of this field.

[Qin]:Agree.

> - The media type, MOS type, and calculation algorithm for the multi-channel segment are ill-specified in similar ways to the multi-layer segment.

[Qin]: Yes, you are right.

> Also, it's not clear that the channel mapping in RFC 3550 Section 4.1 is the only one in use.

[Qin]: Good point, but that's what we have in mind currently.

> - In general, it's not clear to me that including multiple different types of MOS report (single-stream, multi-layer, and multi-channel) in a single draft fits with the monitoring architecture, whi…

[Qin]: As I clarified above, these three MOS report types can not be put into one single XR Block.They are excluded between one another. 
But If WG think it is a good idea to merge them together, I am okay with that.

> - The draft provides no meaningful guidelines for registration of new MOS algorithms, and has an inadequate IANA registry.

[Qin]: We do have in the (-06), section 5.4 of  xr-quality-monitoring-06

"
5.4.  New registry of calculation algorithms

   This document creates a new registry to be called "RTCP XR QoE metric
   block - Calculation Algorithm" as a sub-registry of the "RTP Control
   Protocol Extended Reports (RTCP XR) Block Type Registry".  Policies
   for this new registry are as follows:

   o  Entries in the registry are integers.  The valid range is 0 to 7
      corresponding to the 3-bit field "CAlg" in the block.  Values are
      to be recorded in decimal.

   o  Initial assignments are as follows:

      1.  ITU-T P.564 Compliant Algorithm [P.564] (Voice)
      2.  G.107 [G.107] (Voice)
      3.  ETSI TS 101 329-5 Annex E ETSI] (Voice)
      4.  ITU-T P.NAMS [P.NAMS]
      5.  ITU-T P.NBAMS [P.NBAMS]
"
Hope this makes sense.

---

From: Colin Perkins <[email-masked]>
Date: Tue, 20 Dec 2011 17:06:39 +0000

Qin,

See inline.
Colin

On 20 Dec 2011, at 01:54, Qin Wu wrote:
> Please see my clarification belows.
>> - The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into designing the XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. 
> 

I agree that there is guidance to avoid invalid combinations. However, a better design would prevent invalid combinations by appropriate choice of report block structure. 

I still see no guidance on which algorithm is appropriate for which MOS type.

> If you agree with this, I can revise the draft to make this clear. 

This change doesn't address my concern, which is that a single draft is describing three different types of report block. The monarch draft was intended to recommend different report blocks be developed in different drafts, so they can be implemented separately, in a mix-and-match style, rather than being combined into a single draft.

> [Qin]: multi-layer segment does support both video and audio, 

None of the field descriptions in Section 3.2.2 mention audio; they're all video only.

> [Qin]: currently we only allow 5 MoS algorithms.since three Audio MoS algorithms have already been standardized and the other two are in standardization progress.

This doest address my question. What algorithms make sense in the context of layered media?

> [Qin]:SSID is sub stream identifier and can be used to identifier each layer if all the layers are carried in the same RTP stream.

More detail is needed in the draft.

> But If WG think it is a good idea to merge them together, I am okay with that.

I agree that they should not be included in a single XR block. My question, however, was whether they should be in a single draft. 

> [Qin]: We do have in the (-06), section 5.4 of  xr-quality-monitoring-06

Much more detail on the registration procedures, and guidelines for what should be registered, is needed.

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Wed, 21 Dec 2011 11:24:23 +0800

Hi, Colin and Chairs:
>To: "Qin Wu" <[email-masked]>
>Cc: "Shida Schubert" <[email-masked]>; <[email-masked]>
>Sent: Wednesday, December 21, 2011 1:06 AM

>>> - The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into designing >>>the XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. 
>>> 

>I still see no guidance on which algorithm is appropriate for which MOS type.

[Qin]: I get your point. In the current draft, we list 5 MoS algorithms, 3 MOS algorithms specifed in the G.107 and P.564 and ETSI TS101 329-5 are only used to estimate speech quality or conversation quality. 2 MoS algorithms specified in the P.NAMS and P.NBAMS can be used to estimate either audio quality or video quality or multimedia quality. Also I like to propose to add one MoS Type MoS-A into the draft to distinguish conversation MoS and speech MoS from audio MoS.  

If you think this clarificatin does address your comments, I can put them into the update of the draft.

>> I think the option 2 is a better approach, this approach only report MoS value related to one single MoS Type and doesn't put a set of unrelated parameters together and therefore doesn't conflict the rule >>we set in MONARCH.
>> However if we also measure MoS-LQ using three different audio MoS algorithms and report MoS-CQ value and MoS-LQ value carried in two "single stream segment" using one single XR Block, that >>does mean "put a set of unrelated parameters together" and therefore conflict with Monarch.
>> 

>This change doesn't address my concern, which is that a single draft is describing three different types of report block. The monarch draft was intended to recommend different report blocks be developed >in different drafts, so they can be implemented separately, in a mix-and-match style, rather than being combined into a single draft.

[Qin]: I agree, I have no strong opinon on this. I am okay to either put them toether in one single draft or document them separately.
but my difficulty is these three type of report blocks all report MoS value and are all for QoE metric reporting. I am not worried about block type space but I do worry 
about or need to figure out how to distinguish one another. But if we put them together, we don't have such issue. Also it is not difficult for the user to place his order by choosing different segment type
rather than place his order by choosing different block type for the same purpose(QoE metric reporting).

>None of the field descriptions in Section 3.2.2 mention audio; they're all video only.

[Qin]: You are right, I check RFC6190, draft-ietf-avt-rtp-mvc-01, it looks to me these specification only support video. Audio seems to be carried in the different RTP session or different
RTP stream with different SSRC. And then using CNAME to group audio with layered video. Am I right?
If yes, can we say multi-layer segment only support video?

>This doest address my question. What algorithms make sense in the context of layered media?

[Qin]: Do we really have layered audio using RTP payload format?
If no, I think P.NAMS or PNBAMS is sufficient enough for layed media.

>More detail is needed in the draft.

[Qin]: Okay.

>>> - In general, it's not clear to me that including multiple different types of MOS report (single-stream, multi-layer, and multi-channel) in a single draft fits with the monitoring architecture, which encourages >small, single-use, XR block definitions. I would suggest that the three different MOS report types would be better defined as three drafts: there is little in common between them.
>> 

[Qin]: See above, also I like to hear chairs' opinion.

>Much more detail on the registration procedures, and guidelines for what should be registered, is needed.

[Qin]: Okay.

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Coverdale <[email-masked]>
Date: Wed, 21 Dec 2011 16:44:52 -0500

There is still work to be done, but I support adopting
draft-wu-xrblock-rtcp-xr-quality-monitoring-06 as a WG item.

Paul Coverdale

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On
>Behalf Of Shida Schubert
>Sent: Monday, December 19, 2011 6:55 AM
>To: [email-masked]
>Subject: [xrblock] Call for adopting xr-quality-monitoring-06 as WG item
>
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Wed, 21 Dec 2011 15:38:48 -0800

(as an individual)

It seems the merge efforts resulted in a combination of arguably
separate metrics within a single draft. I think this highlights some of
the reasons why we had two drafts to start with. I agree with Colin's
objection per the principles established of the monarch draft. I think
it is worth looking at the various MOS types and seeing how their usage
is simplified if this draft is in fact separated into three component
drafts. At initial glance, there is some simplification. Furthermore,
seeing as the existing draft already contains the restriction that
two segment types cannot be present in the same metric block, splitting
into multiple drafts does not result in any additional duplication of
information within the resulting metric blocks.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Qin Wu
> Sent: Tuesday, December 20, 2011 7:24 PM
> To: Colin Perkins
> Cc: [email-masked]
> >>> - The single-stream metric mixes audio and video MOS into a single
metric block, with a grab-bag
> of MOS type and calculation algorithm parameters. It seems that little
thought has gone into designing
> >>>the XR block format to prohibit invalid combinations (e.g., MT =
MOS-V with CALg = G.107), and
> there is no guidance on which algorithm is appropriate for which MOS
type.
> >> It distinguish Audio MoS algorithm from video MoS algorithm by
adding "(voice)" into Audio MoS
> algorithm.
> 
> >I agree that there is guidance to avoid invalid combinations.
However, a better design would prevent
> invalid combinations by appropriate choice of report block structure.
> 
> [Qin]: I get your point. In the current draft, we list 5 MoS
algorithms, 3 MOS algorithms specifed in
> the G.107 and P.564 and ETSI TS101 329-5 are only used to estimate
speech quality or conversation
> quality. 2 MoS algorithms specified in the P.NAMS and P.NBAMS can be
used to estimate either audio
> quality or video quality or multimedia quality. Also I like to propose
to add one MoS Type MoS-A into
> the draft to distinguish conversation MoS and speech MoS from audio
MoS.
> 
> If you think this clarificatin does address your comments, I can put
them into the update of the
> draft.
> 
> >>> The format seems to follow the RFC 3611 model, throwing together a
set of unrelated parameters,
> rather than the approach documents in the monarch draft.
> >>
> >> [Qin]: In this draft, we are not allowed to put "single stream
segment","multi-layer segment" and
> "multi-channel segment" into one XR Block Report. please see the note
in the segement i field
> definition
> >> That is to say if we choose to use "single stream segment" to
report MoS, we will not use "multi-
> channel segment" or "multi-layer segment" . If we choose to use
multi-channel segment, we will not use
> >> In this case,  MoS-CQ1 and MoS-CQ2 and MoS-CQ3 are all MoS value,
but using different audio MoS
> algorithm, so the question is how to report them, we have two options:
> >> I think the option 2 is a better approach, this approach only
report MoS value related to one
> single MoS Type and doesn't put a set of unrelated parameters together
and therefore doesn't conflict
> the rule >>we set in MONARCH.
> >> However if we also measure MoS-LQ using three different audio MoS
algorithms and report MoS-CQ
> value and MoS-LQ value carried in two "single stream segment" using
one single XR Block, that >>does
> mean "put a set of unrelated parameters together" and therefore
conflict with Monarch.
> >>
> >> So my suggestion is to add a sentence to say only one MoS type is
allowed to report in one single
> XR Block.
> >This change doesn't address my concern, which is that a single draft
is describing three different
> types of report block. The monarch draft was intended to recommend
different report blocks be
> developed >in different drafts, so they can be implemented separately,
in a mix-and-match style,
> rather than being combined into a single draft.
> 
> [Qin]: I agree, I have no strong opinon on this. I am okay to either
put them toether in one single
> draft or document them separately.
> but my difficulty is these three type of report blocks all report MoS
value and are all for QoE metric
> reporting. I am not worried about block type space but I do worry
> about or need to figure out how to distinguish one another. But if we
put them together, we don't have
> such issue. Also it is not difficult for the user to place his order
by choosing different segment
> type
> rather than place his order by choosing different block type for the
same purpose(QoE metric
> reporting).
> 
> >>> - The multi-layer segment is ill-specified: does it support audio
only, video only, or both
> depending on the Media Type bit?
> >>
> [Qin]: You are right, I check RFC6190, draft-ietf-avt-rtp-mvc-01, it
looks to me these specification
> only support video. Audio seems to be carried in the different RTP
session or different
> >>> Which calculation algorithm makes sense here, since there is no
guidance given, and it's possible
> to specify any algorithm?
> >>>
> >> [Qin]: currently we only allow 5 MoS algorithms.since three Audio
MoS algorithms have already been
> standardized and the other two are in standardization progress.
> 
> >> [Qin]:SSID is sub stream identifier and can be used to identifier
each layer if all the layers are
> carried in the same RTP stream.
> 
> >>> It's not sufficient to simply say that a "NAL unit type is one
example", the draft needs to give
> normative rules for the use of this field.
> >>
> >> [Qin]:Agree.
> >>
> >>> - The media type, MOS type, and calculation algorithm for the
multi-channel segment are ill-
> specified in similar ways to the multi-layer segment.
> >>
> >>> - In general, it's not clear to me that including multiple
different types of MOS report (single-
> stream, multi-layer, and multi-channel) in a single draft fits with
the monitoring architecture, which
> encourages >small, single-use, XR block definitions. I would suggest
that the three different MOS
> report types would be better defined as three drafts: there is little
in common between them.
> >>
> >> [Qin]: As I clarified above, these three MOS report types can not
be put into one single XR
> Block.They are excluded between one another.
> I agree that they should not be included in a single XR block. My
question, however, was whether they
> should be in a single draft.
> 
> >>> - The draft provides no meaningful guidelines for registration of
new MOS algorithms, and has an
> inadequate IANA registry.
> >>
> >Much more detail on the registration procedures, and guidelines for
what should be registered, is
> needed.
http://tools.ietf.org/html/draft-wu-xrblock-rtcp-xr-quality-monitoring-0
6

---

From: Hitoshi Asaeda <[email-masked]>
Date: Thu, 22 Dec 2011 14:14:25 +0900 (JST)

> draft-wu-xrblock-rtcp-xr-quality-monitoring-06 as a WG item.

Same opinion.

Regards,
--
Hitoshi Asaeda

---

From: Glen Zorn <[email-masked]>
Date: Thu, 22 Dec 2011 16:38:39 +0700

>> draft-wu-xrblock-rtcp-xr-quality-monitoring-06 as a WG item.

I agree; as I have stated multiple times, there is no need for any
process of "merging" (except, possibly of author lists) these drafts to
make progress; the only thing needed is adoption of one or the other (or
none, for that matter) as a WG document and the use of the consensus
process to produce a final document from that point.  Maybe I haven't
been clear enough on that point, however, so let me make it crystalline:
can we stop wasting time and effort on non-WG documents and get to work
on something real, PLEASE?

...

---

From: Colin Perkins <[email-masked]>
Date: Thu, 22 Dec 2011 11:31:23 +0000

Qin,

See inline.
Colin

> If you think this clarificatin does address your comments, I can put them into the update of the draft.

Again, this doesn't address the issue, which is that the draft allows one to specify an inappropriate MOS algorithm, since it uses a single registry for all algorithm types, irrespective of the top-level media type.

> [Qin]: I agree, I have no strong opinon on this. I am okay to either put them toether in one single draft or document them separately. but my difficulty is these three type of report blocks all report MoS value and are all for QoE metric reporting. I am not worried about block type space but I do worry about or need to figure out how to distinguish one another. But if we put them together, we don't have such issue. Also it is not difficult for the user to place his order by choosing different segment type rather than place his order by choosing different block type for the same purpose(QoE metric reporting).

They are all MOS values for QoE reporting, I agree, but they are MOS values for three very different classes of application, and a single application is unlikely to need more than one of them. For this reason, they should be separate drafts.

> If yes, can we say multi-layer segment only support video?

That's up to you, but you need to be clear what the draft supports.

> If no, I think P.NAMS or PNBAMS is sufficient enough for layed media.

The draft allows any algorithm, and gives no guidance. It should limit the possible choices to those algorithms that make sense.

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Fri, 23 Dec 2011 10:25:16 +0800

Hi, Colin:
>Sent: Thursday, December 22, 2011 7:31 PM

>>>>> - The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into >>>>>designing >>>the XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. 
>>>>> 
>>> [Qin]: I get your point. In the current draft, we list 5 MoS algorithms, 3 MOS algorithms specifed in the G.107 and P.564 and ETSI TS101 329-5 are only used to estimate speech quality or >>conversation quality. 2 MoS algorithms specified in the P.NAMS and P.NBAMS can be used to estimate either audio quality or video quality or multimedia quality. Also I like to propose to add one >>MoS >Type MoS-A into the draft to distinguish conversation MoS and speech MoS from audio MoS.  
>> 

>Again, this doesn't address the issue, which is that the draft allows one to specify an inappropriate MOS algorithm, since it uses a single registry for all algorithm types, irrespective of the top-level media >type.

[Qin]: That's a good point and what I think as well. I think we should assign three different registries for three segment type.
Since multi-layer per SSRC segment only support video and multi-channel audio segment only support conversation or speech.

>>>> I think the option 2 is a better approach, this approach only report MoS value related to one single MoS Type and doesn't put a set of unrelated parameters together and therefore doesn't conflict the >>rule >>we set in MONARCH.
>>>> However if we also measure MoS-LQ using three different audio MoS algorithms and report MoS-CQ value and MoS-LQ value carried in two "single stream segment" using one single XR Block, >>that >>does mean "put a set of unrelated parameters together" and therefore conflict with Monarch.
>>>> 
>>> This change doesn't address my concern, which is that a single draft is describing three different types of report block. The monarch draft was intended to recommend different report blocks be >>developed in different drafts, so they can be implemented separately, in a mix-and-match style, rather than being combined into a single draft.
>> 
>> [Qin]: I agree, I have no strong opinon on this. I am okay to either put them toether in one single draft or document them separately. but my difficulty is these three type of report blocks all report MoS >>value and are all for QoE metric reporting. I am not worried about block type space but I do worry about or need to figure out how to distinguish one another. But if we put them together, we don't have >>such issue. Also it is not difficult for the user to place his order by choosing different segment type rather than place his order by choosing different block type for the same purpose(QoE metric reporting).

>They are all MOS values for QoE reporting, I agree, but they are MOS values for three very different classes of application, and a single application is unlikely to need more than one of them. 

[Qin]: You are right, the single stream per SSRC segment applies all the applications. However multi-layer per SSRC segment only applies to video application and multi-channel per SSRC segment only applies to voice application.

>For this reason, they should be separate drafts.

[Qin]: Can we have three registries in the same draft since we only have one milestone for QoE metric reporting?

[Qin]: Okay.

>The draft allows any algorithm, and gives no guidance. It should limit the possible choices to those algorithms that make sense.

[Qin]: Yes, as for layered meidia, I think only video MoS algorithm will be applied to it. That is to say, only P.NAMS and P.NBAMS 
will be applied to layered media or multi-layer per SSRC segment. I will add one setence in the draft to make this clear.

>>>>> - In general, it's not clear to me that including multiple different types of MOS report (single-stream, multi-layer, and multi-channel) in a single draft fits with the monitoring architecture, which >>encourages >small, single-use, XR block definitions. I would suggest that the three different MOS report types would be better defined as three drafts: there is little in common between them.
>>>> 

-- 
Colin Perkins
http://csperkins.org/