Subject: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

From: Alissa Cooper <[email-masked]>
Date: Mon, 8 Dec 2014 16:55:29 -0800

I have reviewed this draft in preparation for IETF LC and I have some comments and questions before proceeding with the LC announcement.

= Section 3 =
"When comparing this metric
  with pre-repair loss metric of RTCP SR/RR, ambiguity may occur as
  noted in [RFC5725]: Some packets will not be repaired in current RTCP
  interval. Thus it is RECOMMENDED that this report block should be
  generated for those source packets that have no further chance of
  being repaired.”

The analogous text in RFC 5725 specifies both the recommended behavior and the behavior that is not recommended:

"To avoid any ambiguity in the evaluation, it is RECOMMENDED that the post-repair
  Loss RLE be generated for the source packets that have no further
  chance of being repaired.  If the loss-repair method(s) may still
  recover one or more missing source packets, the post-repair Loss RLE
  SHOULD NOT be sent until the loss-recovery process has been
  completed.”

It seems like this document should also specify what is not recommended, otherwise it seems a little vague. Another approach could be to say "Thus it is RECOMMENDED that this report block should be generated only for those source packets that have no further chance of being repaired and not for any other packets."

"But a potential ambiguity may result from sequence
  number range inconsistent. The sequence number range reported by RTCP
  SR/RR may contain some sequence numbers of packets for which repair
  might still be possible. To address this issue, we use begin sequence
  number and end sequence number to explicitly indicate the actual
  sequence number range that this RTCP XR report block reports on as
  the measurement timing.”

I don’t really understand what the stated ambiguity is here, or how it is addressed by including begin_seq and end_seq. As long as the “unrepaired loss count” field is defined as number of packets lost for which repair is no longer possible, what is inconsistent or ambiguous, and how does specifying the sequence number range change anything?

"Note that this metric must be measured in the primary source RTP packets.”

I’m not sure what this means.

= Appendix A =
Doesn’t it sort of defeat the purpose of the 6390 template to fill out a template with a bunch of links back to the definitions of the fields in other parts of the document, rather than putting those definitions directly in the template? I mean, isn’t the point of the template to have all of the information concisely reported in one place?

== Nits to be resolved with any IETF LC comments ==

= Section 1 =
RTCP SR/RR — expand SR/RR on first use

s/cumulative number of packet lost/cumulative number of packets lost/

s/a range of RTP application/a range of RTP applications/

OLD
The metrics defined in this document belongs to the class of
  transport-related metrics defined in [RFC6792]. And it is in
  accordance with the guidelines in [RFC6390] and [RFC6792].

NEW
The metrics defined in this document belong to the class of
  transport-related metrics defined in [RFC6792] and are specified in 
  accordance with the guidelines in [RFC6390] and [RFC6792].

= Section 3 =
s/in current RTCP interval/in the current RTCP interval/

= Section 4.2 =
s/for unilateral parameter/for unilateral parameters/

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Tue, 9 Dec 2014 03:23:58 +0000

Hi Alissa,

Thanks for the comments. Please see inline.

BR,
Rachel

-----邮件原件-----
发件人: xrblock [mailto:[email-masked]] 代表 Alissa Cooper
发送时间: 2014年12月9日 8:55
收件人: [email-masked]
主题: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

= Section 3 =

[Rachel]: All right. It will be addressed in the new version.

[Rachel]: If we don't use begin sequence number and end sequence number, this report block must rely on the Measurement Information Block [RFC6776] to indicate the measurement period, which is also shared with other metrics that may also be pre-pair metrics. For a pre-pair measurement period, number of packets lost = "unrepaired loss count" + "repaired loss count" + to be repaired loss count. But we have recommended that our report block should be only generated for those post-repair packets, because we have to compare the post-repair metrics with pre-pair metrics to show how good the repair mechanisms is. That means the lost packets to be repaired later are not counted. Thus we have to use different measurement period (the separate begin and end sequence numbers) from those who use the Measurement Information Block. 

[Rachel]: Section 2 have specified the primary source RTP packet. The primary source RTP packets are original RTP packets sent from the RTP sender for the first time. A lost primary source RTP packet can be repaired by other non-primary source RTP packets, like retransmission RTP packets or FEC packets. The metrics in this report block only measure the primary source RTP packets, and don’t' consider other RTP packets.

= Appendix A =

[Rachel]: I referenced previous RFCs, like [RFC7243] [RFC7294]. But it's okay for me to have those definitions directly copied in the template if you want. 

= Section 1 =
RTCP SR/RR ― expand SR/RR on first use

OLD

NEW

= Section 3 =

= Section 4.2 =
s/for unilateral parameter/for unilateral parameters/ _______________________________________________

[Rachel]: All are accepted^^.

xrblock mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/xrblock

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 9 Dec 2014 17:04:13 -0800

Hi Rachel,

Thanks, responses inline.

> [Rachel]: If we don't use begin sequence number and end sequence number, this report block must rely on the Measurement Information Block [RFC6776] to indicate the measurement period, which is also…

Ok. I would suggest the following change to clarify:

OLD
But a potential ambiguity may result from sequence
   the measurement timing.

NEW
   This block needs to specify its own measurement period to avoid ambiguity in calculating the post-repair loss count. The sequence number range reported by RTCP
   SR/RR may contain some sequence numbers of packets for which repair
   might still be possible. To avoid the ambiguity, we use begin sequence
   number and end sequence number to explicitly indicate the actual
   sequence number range that this RTCP XR report block reports on as
   the measurement timing.

> [Rachel]: Section 2 have specified the primary source RTP packet. The primary source RTP packets are original RTP packets sent from the RTP sender for the first time. A lost primary source RTP pack…

Ok. I would suggest:

"Note that this metric must measure only primary source RTP packets."

Also, does that “must” need to be a normative MUST for interoperability?

> [Rachel]: I referenced previous RFCs, like [RFC7243] [RFC7294]. But it's okay for me to have those definitions directly copied in the template if you want. 

Well, I guess I did not ask about this for those RFCs, but I would like to understand better what the point of the template is if all the templates end up linking the reader back to various sections of the document.

Alissa

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Wed, 10 Dec 2014 03:06:37 +0000

Hi Alissa,

Thanks for the advices. Please see inline.

BR,
Rachel

-----邮件原件-----
发件人: Alissa Cooper [mailto:[email-masked]] 
发送时间: 2014年12月10日 9:04
收件人: Huangyihong (Rachel)
抄送: [email-masked]
主题: Re: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

Hi Rachel,

On Dec 8, 2014, at 7:23 PM, Huangyihong (Rachel) <[email-masked]> wrote:

> "But a potential ambiguity may result from sequence  number range 
> inconsistent. The sequence number range reported by RTCP  SR/RR may 
> contain some sequence numbers of packets for which repair  might still 
> be possible. To address this issue, we use begin sequence  number and 
> end sequence number to explicitly indicate the actual  sequence number 
> range that this RTCP XR report block reports on as  the measurement 
> timing.”
> 

OLD

NEW

[Rachel]: Okay. Will do.

Ok. I would suggest:

[Rachel]: Yes. It should be "MUST".

[Rachel]: I didn't well involved in the 6390 template usage discussion. So others please correct me if I'm wrong. But as far as I remember, it is raised by the OPS AD who thinks we are defining new metrics while without following the 6390 template. But the drafts of xrblock are more focus on the RTCP XR report block definition. So Compromise is made that we use the 6390 template at the appendix for the metrics. Since most of the definitions in the template have been specified in the draft, we just simply reference them in the template.

Alissa

> _______________________________________________
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 10 Dec 2014 11:56:12 +0000

> -----Original Message-----
> From: xrblock [mailto:[email-masked]] On Behalf Of Huangyihong
> (Rachel)
> 
> > = Appendix A =
> 
> > Doesn't it sort of defeat the purpose of the 6390 template to fill out a
> template with a bunch of links back to the definitions of the fields in other
> parts of the document, rather than putting those definitions directly in the
> template? I mean, isn't the point of the template to have all of the
> information concisely reported in one place?
> 
> >
> 
> > [Rachel]: I referenced previous RFCs, like [RFC7243] [RFC7294]. But it's okay
> for me to have those definitions directly copied in the template if you want.
> 
> 
> 
> Well, I guess I did not ask about this for those RFCs, but I would like to
> understand better what the point of the template is if all the templates end
> up linking the reader back to various sections of the document.
> 
> 
> 
> [Rachel]: I didn't well involved in the 6390 template usage discussion. So
> others please correct me if I'm wrong. But as far as I remember, it is raised by
> the OPS AD who thinks we are defining new metrics while without following
> the 6390 template. But the drafts of xrblock are more focus on the RTCP XR
> report block definition. So Compromise is made that we use the 6390
> template at the appendix for the metrics. Since most of the definitions in the
> template have been specified in the draft, we just simply reference them in
> the template.
> 

Hi,

Rachel's recollection is correct. This discussion happened when the first XRBLOCK documents hit the IESG table after RFC 6390 was published. Qin  was the main editor, Benoit was the OPS AD, Gonzalo was the RAI AD, and I was the document shepherd. Benoit asked to re-use the 6390 template, so that the metrics defined by XRBLOCK are formatted in a similar manner with metrics defined by other WGs in the IETF. The WG wanted that the RFC text keeps compatibility with previous XRBLOCK / RTCP-XR definitions. So we arrived at the solution to include the RFC 6390 style of definition in Annexes, using references in order to avoid duplication. 

Regards,

Dan

---

From: Benoit Claise <[email-masked]>
Date: Wed, 10 Dec 2014 14:17:21 +0100

> Rachel's recollection is correct. This discussion happened when the first XRBLOCK documents hit the IESG table after RFC 6390 was published. Qin  was the main editor, Benoit was the OPS AD, Gonzalo…
Correct.
The primary reason was that the RFC 6390 template definitions in the 
XRBLOCK documents could be reused when populating the performance metric 
registry (see http://tools.ietf.org/html/draft-ietf-ippm-metric-registry)

Regards, Benoit
> .

---

From: Qin Wu <[email-masked]>
Date: Thu, 11 Dec 2014 02:53:05 +0000

-----邮件原件-----
发件人: Benoit Claise [mailto:[email-masked]] 
发送时间: 2014年12月10日 21:17
收件人: Romascanu, Dan (Dan); Huangyihong (Rachel); Alissa Cooper
抄送: [email-masked]; [email-masked]; Qin Wu
主题: Re: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

On 10/12/2014 12:56, Romascanu, Dan (Dan) wrote:
>> template with a bunch of links back to the definitions of the fields 
>> in other parts of the document, rather than putting those definitions 
>> directly in the template? I mean, isn't the point of the template to 
>> have all of the information concisely reported in one place?
>>
>> Well, I guess I did not ask about this for those RFCs, but I would 
>> like to understand better what the point of the template is if all 
>> the templates end up linking the reader back to various sections of the document.
>>
>>
>>
>> [Rachel]: I didn't well involved in the 6390 template usage 
>> discussion. So others please correct me if I'm wrong. But as far as I 
>> remember, it is raised by the OPS AD who thinks we are defining new 
>> metrics while without following the 6390 template. But the drafts of 
>> xrblock are more focus on the RTCP XR report block definition. So 
>> Compromise is made that we use the 6390 template at the appendix for 
>> the metrics. Since most of the definitions in the template have been 
>> specified in the draft, we just simply reference them in the template.
>>
> Hi,
>
Correct.
The primary reason was that the RFC 6390 template definitions in the XRBLOCK documents could be reused when populating the performance metric registry (see http://tools.ietf.org/html/draft-ietf-ippm-metric-registry)

[Qin]: Agree with What both Dan and Benoit said here.
I think it might be helpful to optimize some places for the metrics represented using RFC6390 in the annex, and also it is no harm to add some clarification text in the annex.
e.g., for Units of Measurement, instead of referencing metric definition in some section, you can say something like:
"
This metric is expressed as a 16-bit unsigned integer value.
"
Also for measurement timing, besides referencing to specific section, you can also add some text to say:
"
This metric relies on the sequence number
interval to determine measurement timing.
"
For measurement method of " Unrepaired RTP Packet Loss Count Metric ", you can replace the original text with the following text:

"
* Method of Measurement or Calculation: See section 3, Unrepaired RTP Packet Loss Count Metric definition. It is directly measured and must be measured for
 the primary source RTP packets with no further chance of repair.
"
For measurement point, you can say something like:
"
It is measured at the receiving end of the RTP stream.
"
Instead.

Regards, Benoit

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Thu, 11 Dec 2014 04:52:37 +0000

Hi Qin,

Thanks for your suggestions. It's really helpful to improve the appendix.

BR,
Rachel

-----邮件原件-----
发件人: Qin Wu 
发送时间: 2014年12月11日 10:53
收件人: Benoit Claise; Romascanu, Dan (Dan); Huangyihong (Rachel); Alissa Cooper
抄送: [email-masked]; [email-masked]
主题: RE: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

-----邮件原件-----
发件人: Benoit Claise [mailto:[email-masked]]
发送时间: 2014年12月10日 21:17
收件人: Romascanu, Dan (Dan); Huangyihong (Rachel); Alissa Cooper
抄送: [email-masked]; [email-masked]; Qin Wu
主题: Re: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

Correct.

This metric relies on the sequence number interval to determine measurement timing.

"
* Method of Measurement or Calculation: See section 3, Unrepaired RTP Packet Loss Count Metric definition. It is directly measured and must be measured for  the primary source RTP packets with no further chance of repair.
"
For measurement point, you can say something like:
"
It is measured at the receiving end of the RTP stream.
"
Instead.

Regards, Benoit

---

From: Alissa Cooper <[email-masked]>
Date: Thu, 11 Dec 2014 13:55:26 -0800

Rachel, if you can incorporate Qin’s suggestions and generally think about whether adding a descriptive sentence or two to each field in the appendix would help, that will work for me.

Thanks,
Alissa

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Fri, 12 Dec 2014 01:04:51 +0000

Sure. Will do it.

BR,
Rachel

-----邮件原件-----
发件人: Alissa Cooper [mailto:[email-masked]] 
发送时间: 2014年12月12日 5:55
收件人: Huangyihong (Rachel)
抄送: Qin Wu; Benoit Claise; Romascanu, Dan (Dan); [email-masked]; Gonzalo Camarillo
主题: Re: [xrblock] AD evaluation: draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06

Thanks,
Alissa

On Dec 10, 2014, at 8:52 PM, Huangyihong (Rachel) <[email-masked]> wrote:

> 主题: RE: [xrblock] AD evaluation: 
> draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06
> 
> -----邮件原件-----
> 主题: Re: [xrblock] AD evaluation: 
> draft-ietf-xrblock-rtcp-xr-post-repair-loss-count-06
> 
>>> in other parts of the document, rather than putting those 
>>> definitions directly in the template? I mean, isn't the point of the 
>>> template to have all of the information concisely reported in one place?
>>> 
>>> discussion. So others please correct me if I'm wrong. But as far as 
>>> I remember, it is raised by the OPS AD who thinks we are defining 
>>> new metrics while without following the 6390 template. But the 
>>> drafts of xrblock are more focus on the RTCP XR report block 
>>> definition. So Compromise is made that we use the 6390 template at 
>>> the appendix for the metrics. Since most of the definitions in the 
>>> template have been specified in the draft, we just simply reference them in the template.
>>> 
>> Hi,
>> 
> XRBLOCK documents could be reused when populating the performance 
> metric registry (see 
> http://tools.ietf.org/html/draft-ietf-ippm-metric-registry)
>

---

From: Gonzalo Camarillo <[email-masked]>
Date: Fri, 12 Dec 2014 09:21:49 +0200

Hi,

> Qin  was the main editor, Benoit was the OPS AD, Gonzalo
> was the RAI AD, and I was the document shepherd. 

yes... those were the good old days :-)

Cheers,

Gonzalo