Subject: [xrblock] Usage of RFC 6390 in XRBLOCK documents

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 3 Dec 2012 12:56:31 +0000

Most if not all participants in the work of XRBLOCK are probably familiar with BCP 170 / RFC 6390 - ' Guidelines for Considering New Performance Metric Development'. This RFC describes a framework and a process for developing Performance Metrics of protocols and applications transported over IETF-specified protocols and makes a number of recommendations about the definition of new metrics. 

Specifically section 5.4.2 of the RFC defines the normative part of a Performance Metric definition, which MUST include the following: 

(i) Metric Name
(ii) Metric Description 
(iii) Method of Measurement or Calculation
(iv) Units of Measurement
(v) Measurement Point(s) with Potential Measurement Domain
(vi) Measurement Timing

In XRBLOCK documents we are re-using definitions of metrics already defined somewhere else and normatively refer these documents. We do not intent to change this, but we would like to use the list above as a checklist that all mandatory information does exist in the definitions of the metrics that we refer. Authors can chose whether to use the exact format or terminology used by 6390 or not, provided that all necessary information is present. 

In documents that define new metrics (if there will be such document) it will be required to include all normative parts of the performance metric definition as specified in section 5.4.2 of RFC 6390. 

Please let us know if you see any problem with starting to apply this policy immediately. 

Thanks and Regards,

Dan

---

From: Glen Zorn <[email-masked]>
Date: Tue, 04 Dec 2012 13:19:37 +0700

On 12/03/2012 07:56 PM, Romascanu, Dan (Dan) wrote:
> Most if not all participants in  the work of XRBLOCK are probably
 > familiar with BCP 170 / RFC 6390 - ' Guidelines for Considering New
 > Performance Metric Development'. This RFC describes a framework and a
 > process for developing Performance Metrics of protocols and
 > applications transported over IETF-specified protocols and makes a
 > number of recommendations about the definition of new metrics.
 >
 > Specifically section 5.4.2 of the RFC defines the normative part of a
 > Performance Metric definition, which MUST include the following:
 >
 > (i) Metric Name (ii) Metric Description (iii) Method of Measurement
 > or Calculation (iv) Units of Measurement (v) Measurement Point(s)
 > with Potential Measurement Domain (vi) Measurement Timing
 >
 > In XRBLOCK documents we are re-using definitions of metrics already
 > defined somewhere else and normatively refer these documents. We do
 > not intent to change this, but we would like to use the list above as
 > a checklist that all mandatory information does exist in the
 > definitions of the metrics that we refer. Authors can chose whether
 > to use the exact format or terminology used by 6390 or not, provided
 > that all necessary information is present.
 >
 > In documents that define new metrics (if there will be such document)
 > it will be required to include all normative parts of the performance
 > metric definition as specified in section 5.4.2 of RFC 6390.
 >

One question: what does one do if one or more of the items listed above 
is missing  from the definition (e.g. an ISO or IEEE spec) of the metric 
we are referencing?  Leave the item(s) out?  Make it up ourselves (thus 
creating an "IETF" version of the metric by definition incompatible with 
the original)?

>
 >
 > Thanks and Regards,
 >
 > Dan
 >
 >
 >
 > _______________________________________________ xrblock mailing list
 > [email-masked] https://www.ietf.org/mailman/listinfo/xrblock

---

From: Qin Wu <[email-masked]>
Date: Tue, 4 Dec 2012 18:21:39 +0800

I have no problem to apply this policy to the XRBLOCK documents that reference to RFC6390 
and check the normative part of performance metric definition in RFC6390 is followed by 
the metrics block. 

I believe this is what we always bear this in mind from the beginning when this WG was 
formed, i.e., see whether the XRBLOCK draft is consistent/compliant with RFC6390 .

Another thing I just want to point out is, for some metrics included in the metrics block, 
especially most of the end system metrics, they are some metrics which can be directly 
measured and don't need us to design special measurement method. However for some
metrics that depend on those metrics which can be directly measured, they do need 
method of measurement or calculation.

Regards!
-Qin
----- Original Message ----- 
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: <[email-masked]>
Cc: "Benoit Claise" <[email-masked]>
Sent: Monday, December 03, 2012 8:56 PM
Subject: [xrblock] Usage of RFC 6390 in XRBLOCK documents

> [email-masked]
> https://www.ietf.org/mailman/listinfo/xrblock

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 4 Dec 2012 10:45:42 +0000

> -----Original Message-----
> From: Glen Zorn [mailto:[email-masked]]
> Sent: Tuesday, December 04, 2012 8:20 AM
> To: Romascanu, Dan (Dan)
> Cc: [email-masked]; Benoit Claise; [email-masked]
> Subject: Re: [xrblock] Usage of RFC 6390 in XRBLOCK documents
> 
> > process for developing Performance Metrics of protocols and  >
> applications transported over IETF-specified protocols and makes a  >
> > or Calculation (iv) Units of Measurement (v) Measurement Point(s)  >
> with Potential Measurement Domain (vi) Measurement Timing  >  > In
> XRBLOCK documents we are re-using definitions of metrics already  >
> defined somewhere else and normatively refer these documents. We do  >
> not intent to change this, but we would like to use the list above as  >
> a checklist that all mandatory information does exist in the  >
> definitions of the metrics that we refer. Authors can chose whether  >
> to use the exact format or terminology used by 6390 or not, provided  >

Hi Glen,

My proposal is that in such cases we need at a minimum discuss in the I-D the missing items. The assumption is that if they were defined in RFC 6390 as a normative part in the definition of any metric in the IETF they are necessary for the metric to be completely defined and ensure interoperability. 

I hardly can see how (i), (ii) or (iv) can be absent. If (iii), (v) or (vi) are missing we should at least discuss in the document what we do in the absence of their definition. One thing we already do in such cases is to add fields that define the different option (e.g. algorithm of calculation) in the xr-block. 

Does this seem reasonable? 

Thanks and Regards,

Dan

---

From: Alan Clark <[email-masked]>
Date: Tue, 4 Dec 2012 07:22:26 -0500

The major risk with incorporating metric definitions for an existing metric
into an XRBLOCK draft is that this would be a more comprehensive
non-normative definition of a metric that was incompletely defined in some
other IETF, ITU, ISO, IEEE or other standard.

I support the idea of using RFC6390 however believe that we should preface
the definition with a statement that the definition is non-normative and may
need to be updated if the normative standard is modified. We should also
send a liaison statement back to the appropriate standards committee to
suggest that they adopt the template based metric definition that we have
drafted.

Alan

On 12/3/12 7:56 AM, "Dan (Dan)" <[email-masked]> wrote:

> Most if not all participants in the work of XRBLOCK are probably familiar with
> BCP 170 / RFC 6390 - ' Guidelines for Considering New Performance Metric
> Development'. This RFC describes a framework and a process for developing
> Performance Metrics of protocols and applications transported over
> IETF-specified protocols and makes a number of recommendations about the
> definition of new metrics.
> 
> In XRBLOCK documents we are re-using definitions of metrics already defined
> somewhere else and normatively refer these documents. We do not intent to
> change this, but we would like to use the list above as a checklist that all
> mandatory information does exist in the definitions of the metrics that we
> refer. Authors can chose whether to use the exact format or terminology used
> by 6390 or not, provided that all necessary information is present.
> 
> In documents that define new metrics (if there will be such document) it will
> be required to include all normative parts of the performance metric
> definition as specified in section 5.4.2 of RFC 6390.
>