Subject: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 21 Aug 2013 13:35:33 +0000

Hi,

This is the second WGLC for the Internet-Draft 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks for MoS Metric Reporting' previously known as 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks for QoE Metric Reporting'. Please send your comments, questions, and concerns to the WG list before Wednesday 9/4. If you have no comments or questions and you believe that this document is ready for submission to the IESG for consideration as a Proposed Standard, please send a message stating this. 

The latest version of the document can be retrieved from http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-qoe-10.txt.

Thanks and Regards,

Dan

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Thu, 22 Aug 2013 06:06:31 +0000

Hi, 

I have read this draft and believe it's in good shape. I have only one minor comment:

In the section 3.2
"
Reserved: 6 bits

      This field is reserved for future definition.  In the absence of
      such a definition, the bits in this field MUST be set to zero and
      ignored by the receiver (See RFC6709 section 4.2).

"

In What kind of case such a definition will be absent?

Best regards,
Rachel

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: Wednesday, August 21, 2013 9:36 PM
To: [email-masked]
Subject: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

Hi,

Thanks and Regards,

Dan

_______________________________________________
xrblock mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/xrblock

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 22 Aug 2013 09:23:25 +0000

I think that what is meant here is that until a future usage of this field is defined the transmitter and receiver must have the behavior indicated in section 4.2 of RFC 6709 for reserved fields. 

Maybe it would be more clear: 

OLD: 

Reserved: 6 bits

NEW: 

Reserved: 6 bits
 
      This field is reserved for future use.  Until future use is 
      defined, the bits in this field MUST be set to zero at 
      transmission and ignored by the receiver (See RFC6709 section 4.2).

Regards,

Dan

> From: Huangyihong (Rachel) [mailto:[email-masked]]
> Sent: Thursday, August 22, 2013 9:07 AM
> To: Romascanu, Dan (Dan); [email-masked]
> Subject: [xrblock]RE: 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe
> 
> Hi,
> 
> This is the second WGLC for the Internet-Draft 'RTP Control Protocol
> (RTCP) Extended Report (XR) Blocks for MoS Metric Reporting' previously
> known as 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks for
> QoE Metric Reporting'. Please send your comments, questions, and
> concerns to the WG list before Wednesday 9/4. If you have no comments or
> questions and you believe that this document is ready for submission to
> the IESG for consideration as a Proposed Standard, please send a message
> stating this.
> 
> The latest version of the document can be retrieved from
> http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-qoe-10.txt.
> 
> Thanks and Regards,
> 
> Dan
> 
> 
>

---

From: Qin Wu <[email-masked]>
Date: Thu, 22 Aug 2013 09:37:24 +0000

Hi,Dan:
Thanks for your proposed change.
The change looks good to me.

Regards!
-Qin
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: Thursday, August 22, 2013 5:23 PM
To: Huangyihong (Rachel); [email-masked]
Subject: Re: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

OLD: 

Reserved: 6 bits

NEW: 

Reserved: 6 bits

Regards,

Dan

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Thu, 22 Aug 2013 09:37:14 +0000

Hi Dan,

The change clears my concern. Thanks.

Best regards,
Rachel

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]] 
Sent: Thursday, August 22, 2013 5:23 PM
To: Huangyihong (Rachel); [email-masked]
Subject: RE: [xrblock]RE: 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

OLD: 

Reserved: 6 bits

NEW: 

Reserved: 6 bits

Regards,

Dan

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 22 Aug 2013 09:40:27 +0000

It's up to the editor actually, from my perspective the text was correct, maybe not very clear. 

Dan

> Sent: Thursday, August 22, 2013 12:37 PM
> I think that what is meant here is that until a future usage of this
> field is defined the transmitter and receiver must have the behavior
> indicated in section 4.2 of RFC 6709 for reserved fields.
> 
> > (RTCP) Extended Report (XR) Blocks for MoS Metric Reporting'
> > previously known as 'RTP Control Protocol (RTCP) Extended Report (XR)
> > Blocks for QoE Metric Reporting'. Please send your comments,
> > questions, and concerns to the WG list before Wednesday 9/4. If you
> > have no comments or questions and you believe that this document is
> > ready for submission to the IESG for consideration as a Proposed
> > Standard, please send a message stating this.
> >

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Thu, 22 Aug 2013 10:33:00 +0000

Yes. I didn't read it carefully. And after your clarification, I understand the authors' intention. I think the changes you made are clearer than the old one. But I'm also okay to keep the old one.

Best regards,
Rachel

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]] 
Sent: Thursday, August 22, 2013 5:40 PM
To: Huangyihong (Rachel); [email-masked]
Subject: RE: [xrblock]RE: 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

Dan

---

From: "Roni Even" <[email-masked]>
Date: Thu, 22 Aug 2013 13:38:02 +0300

Hi,
I reviewed the document looks OK. Two editorial comments

1. In section 4.1, mosref has three values l, m, h yet the text 
" the syntax element "mosref" is referred to the media resolution  relative
reference (e.g., Narrowband (3.4kHz) Speech and Standard  Definition (SD)
Resolution Video with lower resolution, Wideband
   (7kHz) Speech and High Definition (HD) Resolution Video with higher
resolution).  MOS scores reported in the mos metrics block might vary with
the MoS reference; For example, MOS values for narrowband,
   wideband codecs occupy the same range but SHOULD be reported in different
value.  For video application, MoS scores for SD resolution, HD resolution
video also occupy the same ranges and
   SHOULD be reported in different value."
I think a better example and division should be for audio l = narrow band
3.5 Khz, m should be wideband 7 Khz and h should be super wideband or higher
>14kHz. For video l should be CIF or lower and h should  be 1080HD or higher
and m should be betwee CIF and 1080HD.

2.  section 4.1 defines mapentry =  "calg:" 1*5 DIGIT ["/" direction].  From
section 4.2 I got the impression that the range [4096-4351] is not to be
used as valid value. I think it should be clarified in section 4.1 what are
the valid range.

Thanks
Roni

> Sent: 21 August, 2013 4:36 PM
> To: [email-masked]
(RTCP)
> Extended Report (XR) Blocks for MoS Metric Reporting' previously known as
> 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks for QoE Metric
> Reporting'. Please send your comments, questions, and concerns to the WG
> list before Wednesday 9/4. If you have no comments or questions and you
> believe that this document is ready for submission to the IESG for
> consideration as a Proposed Standard, please send a message stating this.
>

---

From: Qin Wu <[email-masked]>
Date: Thu, 22 Aug 2013 12:02:09 +0000

Hi,Roni:
Thank for your valuable comments to the update. Please see my rely inline below.

Regards!
-Qin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Thursday, August 22, 2013 6:38 PM
To: 'Romascanu, Dan (Dan)'; [email-masked]
Subject: Re: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

Hi,

[Qin]: Good suggestion. Yes, in the syntax of mosref, we distinguish mosref with 3 vlaues so does text description.
Here is my proposed change:
OLD TEXT:
"
The syntax element "mosref" is referred to the media resolution  relative
reference (e.g., Narrowband (3.4kHz) Speech and Standard  Definition (SD)
Resolution Video with lower resolution, Wideband
   (7kHz) Speech and High Definition (HD) Resolution Video with higher
resolution).  MOS scores reported in the mos metrics block might vary with
the MoS reference; For example, MOS values for narrowband,
   wideband codecs occupy the same range but SHOULD be reported in different
value.  For video application, MoS scores for SD resolution, HD resolution
video also occupy the same ranges and
   SHOULD be reported in different value.
"
NEW TEXT:
"
   The syntax element "mosref" is referred to the media resolution
   relative reference and has three valules 'l','m','h'.(e.g., Narrowband (3.4kHz) Speech and Standard
   Definition (SD) Resolution Video have 'l' resolution, Super Wideband
   (>14kHz) Speech and High Definition (HD) Resolution Video have 'h'
   Resolution,Wideband speech(7khz) and Video with resolution between SD and HD has 'm' resolution). 
    MOS scores reported in the mos metrics block might vary
   with the MoS reference; For example, MOS values for narrowband,
   wideband,super wideband codecs occupy the same range but SHOULD be reported in
   different value.  For video application, MoS scores for SD
   resolution, HD resolution video also occupy the same ranges and
   SHOULD be reported in different value.
"

[Qin]:Good point, we can add some annotation to mapentry syntax in the section 4.1 as follows:
OLD TEXT:
"
mapentry =  "calg:" 1*5 DIGIT ["/" direction]
"
NEW TEXT:
"
mapentry =  "calg:" 1*5 DIGIT ["/" direction];Value 0~4095 are valid
"

Thanks
Roni

(RTCP)

---

From: "Roni Even" <[email-masked]>
Date: Thu, 22 Aug 2013 16:55:05 +0300

Hi Qin,
See inline
Roni

> From: Qin Wu [mailto:[email-masked]]
> Sent: 22 August, 2013 3:02 PM
> To: Roni Even; 'Romascanu, Dan (Dan)'; [email-masked]
> Subject: RE: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe
> 
> Hi,Roni:
> 1. In section 4.1, mosref has three values l, m, h yet the text " the
syntax
> element "mosref" is referred to the media resolution  relative reference
> (e.g., Narrowband (3.4kHz) Speech and Standard  Definition (SD) Resolution
> Video with lower resolution, Wideband
>    wideband codecs occupy the same range but SHOULD be reported in
> different value.  For video application, MoS scores for SD resolution, HD
> resolution video also occupy the same ranges and
> [Qin]: Good suggestion. Yes, in the syntax of mosref, we distinguish
mosref
> with 3 vlaues so does text description.
>    wideband codecs occupy the same range but SHOULD be reported in
> different value.  For video application, MoS scores for SD resolution, HD
> resolution video also occupy the same ranges and

[Roni Even]  I have no problem with having SD as 'l' but it should say SD or
lower. And for 'h' HD or higher.
For audio 'h' should be super wideband or higher since the next term is
fullband (20Khz)

> section 4.2 I got the impression that the range [4096-4351] is not to be
used
> as valid value. I think it should be clarified in section 4.1 what are the
valid
> range.
> mapentry =  "calg:" 1*5 DIGIT ["/" direction] "
> NEW TEXT:
> "
> mapentry =  "calg:" 1*5 DIGIT ["/" direction];Value 0~4095 are valid "
[Roni Even] What about values higher than 4351, since it is 1-5 digits, are
they valid?
> > Extended Report (XR) Blocks for MoS Metric Reporting' previously known
> > as 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks for QoE
> > Metric Reporting'. Please send your comments, questions, and concerns
> > to the WG list before Wednesday 9/4. If you have no comments or
> > questions and you believe that this document is ready for submission
> > to the IESG for consideration as a Proposed Standard, please send a
> message stating this.
> >

---

From: Qin Wu <[email-masked]>
Date: Fri, 23 Aug 2013 01:26:13 +0000

Hi,Roni:
Thank for your followup comments, see my reply inline below.

Regards!
-Qin

-----Original Message-----
From: Roni Even [mailto:[email-masked]] 
Sent: Thursday, August 22, 2013 9:55 PM
To: Qin Wu; 'Romascanu, Dan (Dan)'; [email-masked]
Subject: RE: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

Hi Qin,
See inline
Roni

syntax
mosref

[Qin]: Okay, I agree, here is the proposed new change:
OLD TEXT:
 "
 The syntax element "mosref" is referred to the media resolution  relative
 reference (e.g., Narrowband (3.4kHz) Speech and Standard  Definition (SD)
 Resolution Video with lower resolution, Wideband
    (7kHz) Speech and High Definition (HD) Resolution Video with higher
 resolution).  MOS scores reported in the mos metrics block might vary with
 the MoS reference; For example, MOS values for narrowband,
    wideband codecs occupy the same range but SHOULD be reported in
 different value.  For video application, MoS scores for SD resolution, HD
 resolution video also occupy the same ranges and
    SHOULD be reported in different value.
 "
NEW TEXT:
"
   The syntax element "mosref" is referred to the media resolution
    relative reference and has three valules 'l','m','h'.(e.g., Narrowband
 (3.4kHz) Speech and Standard
    Definition (SD) or lower Resolution Video have 'l' resolution, Super Wideband
    (>14kHz) Speech or higher and High Definition (HD) or higher Resolution Video have 'h'
    Resolution, Wideband speech(7khz) and Video with resolution between SD
     and HD has 'm' resolution).
     MOS scores reported in the mos metrics block might vary
    with the MoS reference; For example, MOS values for narrowband,
    wideband,super wideband codecs occupy the same range but SHOULD be
 reported in
    different value.  For video application, MoS scores for SD
    resolution, HD resolution video also occupy the same ranges and
    SHOULD be reported in different value.
"

used
valid

[Qin]: Good point, how about the following change:
"
mapentry =  "calg:" 1*5 DIGIT ["/" direction];Value other than 4095~4351 are valid
"

---

From: "Roni Even" <[email-masked]>
Date: Fri, 23 Aug 2013 05:35:28 +0300

Hi Qin,
It looks OK to me
Roni

> Sent: 23 August, 2013 4:26 AM
> > element "mosref" is referred to the media resolution  relative
> > reference (e.g., Narrowband (3.4kHz) Speech and Standard  Definition
> > (SD) Resolution Video with lower resolution, Wideband
> > resolution).  MOS scores reported in the mos metrics block might vary
> > different value.  For video application, MoS scores for SD resolution,
> > HD resolution video also occupy the same ranges and
> > relative reference (e.g., Narrowband (3.4kHz) Speech and Standard
> > Definition (SD) Resolution Video with lower resolution, Wideband
> > resolution).  MOS scores reported in the mos metrics block might vary
> > different value.  For video application, MoS scores for SD resolution,
> > HD resolution video also occupy the same ranges and
>     (>14kHz) Speech or higher and High Definition (HD) or higher
Resolution
> Video have 'h'
> [Roni Even] What about values higher than 4351, since it is 1-5 digits,
are they
> valid?
> mapentry =  "calg:" 1*5 DIGIT ["/" direction];Value other than 4095~4351
are
> valid "
> > > Extended Report (XR) Blocks for MoS Metric Reporting' previously
> > > known as 'RTP Control Protocol (RTCP) Extended Report (XR) Blocks
> > > for QoE Metric Reporting'. Please send your comments, questions, and
> > > concerns to the WG list before Wednesday 9/4. If you have no
> > > comments or questions and you believe that this document is ready
> > > for submission to the IESG for consideration as a Proposed Standard,
> > > please send a
> > message stating this.
> > >

---

From: Colin Perkins <[email-masked]>
Date: Mon, 26 Aug 2013 21:34:46 +0100

> The latest version of the document can be retrieved from http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-qoe-10.txt.

The first paragraph of Section 3.1 talks about various flag bits in the header indicating which fields are present. However, the header doesn't appear to contain any such flags.

The header described in Section 3.2 includes an Interval Metric Flag. I understand how MOS scores can be calculated for an interval, and perhaps how they could be sampled, but not how a cumulative MOS can be measured. Should cumulative, and possibly sampled, metrics be prohibited?

Should Section 3.2.1 be titled "Single channel..." rather than "Single stream..."?

Section 3.2.1 says "For RTP sessions where multiple payload formats can be negotiated or the payload format changes during the mid-session), the value of this field will be used to indicate what payload format was in use for the reporting interval", but doesn't the payload type indicate this whether or not multiple payload formats are used in the session?

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Tue, 27 Aug 2013 01:51:39 +0000

Hi,Colin:
Thank for your valuable comments, see my reply inline below.

Regards!
-Qin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Colin Perkins
Sent: Tuesday, August 27, 2013 4:35 AM
To: Romascanu, Dan (Dan)
Cc: [email-masked]
Subject: Re: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

[Qin]: Good catch, I propose to remove the 1st useless paragraph that describes various flag bits as follows:
OLD TEXT:
"
3.1.  Metric Block Structure

   The report block contents are dependent upon a series of flag bits
   carried in the first part of the header.  Not all parameters need to
   be reported in each block.  Flags indicate which are and which are
   not reported. The fields corresponding to unreported parameters MUST
   be present, and MUST be set to zero.  The receiver MUST ignore any
   MoS Metrics Block with a non-zero value in any field flagged as
   unreported.  The encoding of MoS Metrics block payload consists of a
   series of 32 bit units called segments that describe payload Type,
   MoS algorithm and MoS value.

   The MoS Metrics Block has the following format:
"
NEW TEXT:
"
3.1.  Metric Block Structure

   The MoS Metrics Block has the following format:
"

[Qin]: I agree that MoS metrics can only be measured at each interval duration, and can not be sampled. Also it doesn't look reasonable to report the Mos value in cumulative fashion, therefore I propose the following change:
OLD TEXT:
"
In this document, the value I=00 is reserved for future use.
Senders MUST NOT use the values I=00.  If a block is received with
I=00, the receiver MUST discard the block.

"
NEW TEXT:
"
In this document, MoS Metrics can only be measured at
each interval duration, and cannot be sampled or cumulated. 
Also the value I=00 is reserved for future use.
Senders MUST NOT use the values I=00 or I=01 or I=11.  
If a block is received with
I=00 or I=01 or I=11, the receiver MUST discard the block.
"

[Qin]: looks good to me.

[Qin]:Yes it indicates multiple payload format can be used in one RTP session or in RTP sessions. 
So I propose the following change:
OLD TEXT:
"
MoS Metrics reporting depends on the payload format in use.  This
field identifies the format of the RTP payload.  For RTP session
where multiple payload formats can be negotiated or the payload
format changes during the mid-session), the value of this field
will be used to indicate what payload format was in use for the
reporting interval.
"
NEW TEXT:
"
MoS Metrics reporting depends on the payload format in use.  This
field identifies the format of the RTP payload.  For RTP session(s)
where multiple payload formats can be negotiated or the payload
format changes during the mid-session), the value of this field
will be used to indicate what payload format was in use for the
reporting interval.
"
Let me know if this addresses your comment.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Tue, 27 Aug 2013 23:11:30 +0100

Qin,

Thank you, most of this looks good. I just one comment:

On 27 Aug 2013, at 02:51, Qin Wu <[email-masked]> wrote:
...
> Let me know if this addresses your comment.

I don't think this is correct. A better new text might be something like:

"MoS Metrics reporting depends on the payload format in use.  This
field identifies the RTP payload type in use during the reporting interval. The binding between RTP payload types and RTP payload formats is configured via a signalling protocol, for example an SDP offer/answer exchange. If the RTP payload type used is changed during an RTP session, separate reports SHOULD be sent for each RTP payload type, with corresponding measurement information blocks indicating the time period to which they relate"

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Wed, 28 Aug 2013 01:59:09 +0000

-----Original Message-----
From: Colin Perkins [mailto:[email-masked]] 
Sent: Wednesday, August 28, 2013 6:12 AM
To: Qin Wu
Cc: Romascanu, Dan (Dan); [email-masked]
Subject: Re: [xrblock] 2nd WGLC for draft-ietf-xrblock-rtcp-xr-qoe

Qin,

[Qin]: okay, your proposed change looks good to me. Thanks.

-- 
Colin Perkins
http://csperkins.org/