Subject: [ippm] Call for Consensus on new charter for IPPM

From: Brian Trammell <[email-masked]>
Date: Tue, 19 Mar 2013 10:34:11 +0800

Greetings, all,

We have updated the draft charter based on discussion during our meeting in Orlando last Friday. We believe the draft charter below addresses all comments received, and would like to issue a call for WG consensus on the charter text. We'll be issuing separate calls for each of the documents under consideration for adoption as WG items under this charter.

If you support the adoption of this charter for IPPM, please reply to this message and/or to the list stating so. 

If you do not support the adoption of this charter, please reply to the list, with your suggestions for improvement and further discussion.

Please reply by Monday 25 March 2013; absent continuing discussion, we intend to evaluate consensus then, and send the charter to the IESG for approval.

Many thanks, best regards,

Brian

IP Performance Metrics (ippm)
-----------------------------

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny
     
 Transport Area Directors:
     Martin Stiemerling
     Martin Stiemerling

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:
     General Discussion: [email-masked]
     To Subscribe:       https://www.ietf.org/mailman/listinfo/ippm
     Archive:            http://www.ietf.org/mail-archive/web/ippm

Description of Working Group:

The IP Performance Metrics (IPPM) Working Group develops and maintains standard
metrics that can be applied to the quality, performance, and reliability of
Internet data delivery services and applications. It also develops and
maintains protocols for the measurement of these metrics. These metrics are
designed such that they can be performed by network operators, end users, or
independent testing groups. Metrics developed by the IPPM WG are intended to
provide unbiased quantitative performance measurements and not a value
judgement.

The IPPM WG has produced documents that define specific metrics and procedures
for accurately measuring and documenting these metrics. The working group will
continue advancing these metrics along the standards track, using the
guidelines stated in RFC 6576. To the extent possible, these metrics will be
used as the basis for future work on metrics in the WG.

The WG will develop the minimum number of new metrics and models needed to more
accurately quantitatively characterize the network path(s) under test and/or
the behavior of transport and application layer protocols on these path(s). New
metric definitions will state how the definition improves on an existing metric
definition, or encompasses a metric not previously defined.

Additional methods will be defined for the composition and calibration of
IPPM-defined metrics, as well as active, passive and hybrid measurement methods
for these metrics. In addition, WG encourages work which describes the
applicability of metrics and measurement methods, especially to improve
understanding of the tradeoffs involved among active, passive, and hybrid
methods.

The WG may update its core framework RFC 2330 as necessary to accommodate these
activities.

The WG has produced protocols for communication among test equipment to enable
the measurement of the one- and two-way metrics (OWAMP and TWAMP respectively).
These protocols will be advanced along the standards track. The WG will further
develop and improve these protocols. The WG may develop new measurement
protocols as necessary to support new metrics. New metric and protocol
development will focus on the automatability of measurements in order to
support large-scale measurement efforts.

The equivalency of metric definitions is crucial to accurate, comparable and
reproducible measurement. The WG will define and maintain a registry of metric
definitions to this end. The WG encourages work which compares IPPM metrics
with metrics developed elsewhere, particularly in regards to equivalency,
similarity and functionality. The WG also encourages work which improves the
availability of information about the context in which measurements were taken.

The IPPM WG seeks cooperation with other appropriate standards bodies and
forums to promote consistent approaches and metrics. Within the IETF process,
IPPM metric definitions will be subject to as rigorous a scrutiny for
usefulness, clarity, and accuracy as other protocol standards. The IPPM WG will
interact with other areas of IETF activity whose scope intersects with the
requirement of these specific metrics. The WG will, on request, provide input
to other IETF working groups on the use and implementation of these metrics.

Milestones:

TBD      - Submit draft on access rate measurement protocol problem statement
           to IESG as Informational
TBD      - Submit draft on RFC 2680 standards-track advancement testing
           to IESG as Informational
           
Additional milestones TBD

---

From: Matt Mathis <[email-masked]>
Date: Mon, 18 Mar 2013 22:47:22 -0700

Look Good to Me (LGTM)

One minor issue:
> "To the extent possible, these metrics will be used as the basis for future work on metrics in the WG."
I don't understand the point here.  Perhaps s/metrics/guidelines/
(first time only)?

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our
services to speak in defiance of unjust governments.   We treat
privacy and security as matters of life and death, because for some
users, they are.

On Mon, Mar 18, 2013 at 7:34 PM, Brian Trammell <[email-masked]> wrote:
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: marcelo bagnulo braun <[email-masked]>
Date: Tue, 19 Mar 2013 12:51:02 +0100

Hi,

Thanks for the work on the charter text.
I support it.

Regards, marcelo

---

From: Steve Baillargeon <[email-masked]>
Date: Tue, 19 Mar 2013 12:50:12 +0000

I support.

-Steve

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: March-18-13 10:34 PM
To: [email-masked]
Subject: [ippm] Call for Consensus on new charter for IPPM

Greetings, all,

Brian

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:

The IP Performance Metrics (IPPM) Working Group develops and maintains standard metrics that can be applied to the quality, performance, and reliability of Internet data delivery services and applications. It also develops and maintains protocols for the measurement of these metrics. These metrics are designed such that they can be performed by network operators, end users, or independent testing groups. Metrics developed by the IPPM WG are intended to provide unbiased quantitative performance measurements and not a value judgement.

The IPPM WG has produced documents that define specific metrics and procedures for accurately measuring and documenting these metrics. The working group will continue advancing these metrics along the standards track, using the guidelines stated in RFC 6576. To the extent possible, these metrics will be used as the basis for future work on metrics in the WG.

The WG will develop the minimum number of new metrics and models needed to more accurately quantitatively characterize the network path(s) under test and/or the behavior of transport and application layer protocols on these path(s). New metric definitions will state how the definition improves on an existing metric definition, or encompasses a metric not previously defined.

Additional methods will be defined for the composition and calibration of IPPM-defined metrics, as well as active, passive and hybrid measurement methods for these metrics. In addition, WG encourages work which describes the applicability of metrics and measurement methods, especially to improve understanding of the tradeoffs involved among active, passive, and hybrid methods.

The WG may update its core framework RFC 2330 as necessary to accommodate these activities.

The WG has produced protocols for communication among test equipment to enable the measurement of the one- and two-way metrics (OWAMP and TWAMP respectively).
These protocols will be advanced along the standards track. The WG will further develop and improve these protocols. The WG may develop new measurement protocols as necessary to support new metrics. New metric and protocol development will focus on the automatability of measurements in order to support large-scale measurement efforts.

The equivalency of metric definitions is crucial to accurate, comparable and reproducible measurement. The WG will define and maintain a registry of metric definitions to this end. The WG encourages work which compares IPPM metrics with metrics developed elsewhere, particularly in regards to equivalency, similarity and functionality. The WG also encourages work which improves the availability of information about the context in which measurements were taken.

The IPPM WG seeks cooperation with other appropriate standards bodies and forums to promote consistent approaches and metrics. Within the IETF process, IPPM metric definitions will be subject to as rigorous a scrutiny for usefulness, clarity, and accuracy as other protocol standards. The IPPM WG will interact with other areas of IETF activity whose scope intersects with the requirement of these specific metrics. The WG will, on request, provide input to other IETF working groups on the use and implementation of these metrics.

Milestones:

---

From: Samita Chakrabarti <[email-masked]>
Date: Tue, 19 Mar 2013 16:05:37 +0000

I support the charter update. Thanks.
-Samita 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: Monday, March 18, 2013 7:34 PM
To: [email-masked]
Subject: [ippm] Call for Consensus on new charter for IPPM

Greetings, all,

Brian

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:

Milestones:

---

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 21 Mar 2013 08:47:44 +0800

The charter is fine to me.

Jiankang Yao

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 20 Mar 2013 21:22:21 -0400

Looks good. One nit below.

> for these metrics. In addition, WG encourages work which describes the

s/WG encourages/the WG encourages/

--Alissa

---

From: Konstantinos Pentikousis <[email-masked]>
Date: Thu, 21 Mar 2013 11:15:07 +0000

Dear Chairs,

The charter looks good, many thanks for the effort! I support it as well.

Best regards,

Kostas

|-----Original Message-----
|From: Brian Trammell [mailto:[email-masked]]
|Sent: Tuesday, March 19, 2013 3:34 AM
|To: [email-masked]
|Subject: [ippm] Call for Consensus on new charter for IPPM
|
|Greetings, all,
|
|We have updated the draft charter based on discussion during our meeting
|in Orlando last Friday. We believe the draft charter below addresses all
|comments received, and would like to issue a call for WG consensus on
|the charter text. We'll be issuing separate calls for each of the
|documents under consideration for adoption as WG items under this
|charter.
|
|If you support the adoption of this charter for IPPM, please reply to
|this message and/or to the list stating so.
|
|If you do not support the adoption of this charter, please reply to the
|list, with your suggestions for improvement and further discussion.
|
|Please reply by Monday 25 March 2013; absent continuing discussion, we
|intend to evaluate consensus then, and send the charter to the IESG for
|approval.
|
|Many thanks, best regards,
|
|Brian
|
|
|IP Performance Metrics (ippm)
|-----------------------------
|
| Charter
|
| Current Status: Active
|
| Chairs:
|     Brian Trammell
|     Bill Cerveny
|
| Transport Area Directors:
|     Martin Stiemerling
|     Martin Stiemerling
|
| Transport Area Advisor:
|     Martin Stiemerling
|
| Mailing Lists:
|     General Discussion: [email-masked]
|     To Subscribe:       https://www.ietf.org/mailman/listinfo/ippm
|     Archive:            http://www.ietf.org/mail-archive/web/ippm
|
|Description of Working Group:
|
|The IP Performance Metrics (IPPM) Working Group develops and maintains
|standard metrics that can be applied to the quality, performance, and
|reliability of Internet data delivery services and applications. It also
|develops and maintains protocols for the measurement of these metrics.
|These metrics are designed such that they can be performed by network
|operators, end users, or independent testing groups. Metrics developed
|by the IPPM WG are intended to provide unbiased quantitative performance
|measurements and not a value judgement.
|
|The IPPM WG has produced documents that define specific metrics and
|procedures for accurately measuring and documenting these metrics. The
|working group will continue advancing these metrics along the standards
|track, using the guidelines stated in RFC 6576. To the extent possible,
|these metrics will be used as the basis for future work on metrics in
|the WG.
|
|The WG will develop the minimum number of new metrics and models needed
|to more accurately quantitatively characterize the network path(s) under
|test and/or the behavior of transport and application layer protocols on
|these path(s). New metric definitions will state how the definition
|improves on an existing metric definition, or encompasses a metric not
|previously defined.
|
|Additional methods will be defined for the composition and calibration
|of IPPM-defined metrics, as well as active, passive and hybrid
|measurement methods for these metrics. In addition, WG encourages work
|which describes the applicability of metrics and measurement methods,
|especially to improve understanding of the tradeoffs involved among
|active, passive, and hybrid methods.
|
|The WG may update its core framework RFC 2330 as necessary to
|accommodate these activities.
|
|The WG has produced protocols for communication among test equipment to
|enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
|respectively).
|These protocols will be advanced along the standards track. The WG will
|further develop and improve these protocols. The WG may develop new
|measurement protocols as necessary to support new metrics. New metric
|and protocol development will focus on the automatability of
|measurements in order to support large-scale measurement efforts.
|
|The equivalency of metric definitions is crucial to accurate, comparable
|and reproducible measurement. The WG will define and maintain a registry
|of metric definitions to this end. The WG encourages work which compares
|IPPM metrics with metrics developed elsewhere, particularly in regards
|to equivalency, similarity and functionality. The WG also encourages
|work which improves the availability of information about the context in
|which measurements were taken.
|
|The IPPM WG seeks cooperation with other appropriate standards bodies
|and forums to promote consistent approaches and metrics. Within the IETF
|process, IPPM metric definitions will be subject to as rigorous a
|scrutiny for usefulness, clarity, and accuracy as other protocol
|standards. The IPPM WG will interact with other areas of IETF activity
|whose scope intersects with the requirement of these specific metrics.
|The WG will, on request, provide input to other IETF working groups on
|the use and implementation of these metrics.
|
|Milestones:
|
|TBD      - Submit draft on access rate measurement protocol problem
|statement
|           to IESG as Informational
|TBD      - Submit draft on RFC 2680 standards-track advancement testing
|           to IESG as Informational
|
|Additional milestones TBD

---

From: KEN KO <[email-masked]>
Date: Thu, 21 Mar 2013 12:09:31 +0000

It looks good to me as well.
- Ken Ko

----- Original Message -----
From: Konstantinos Pentikousis <[email-masked]>
To: "<[email-masked]>" <[email-masked]>, "[email-masked]" <[email-masked]>
Sent: 3/21/2013 7:15 AM
Subject: Re: [ippm] Call for Consensus on new charter for IPPM

Dear Chairs,

Best regards,

Kostas

---

From: Christofer Flinta <[email-masked]>
Date: Thu, 21 Mar 2013 14:58:53 +0000

It looks good to me.

/Christofer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: den 19 mars 2013 03:34
To: [email-masked]
Subject: [ippm] Call for Consensus on new charter for IPPM

Greetings, all,

Brian

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:

Milestones:

---

From: "Guangqing Deng" <[email-masked]>
Date: Fri, 22 Mar 2013 09:27:18 +0800

the charter looks good to me.

Guangqing Deng
CNNIC 

From: Brian Trammell
Date: 2013-03-19 10:34
To: [email-masked]

Brian

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:

Milestones:

---

From: Henk Uijterwaal <[email-masked]>
Date: Fri, 22 Mar 2013 07:00:46 +0100

> the charter looks good to me.

I agree as well, Henk

>  
> --------------------------------------------------------------------------------
> Guangqing Deng
> CNNIC 
>  
> *From:* Brian Trammell <mailto:[email-masked]>
> *Date:* 2013-03-19 10:34
> *To:* [email-masked] <mailto:[email-masked]>
> *Subject:* [ippm] Call for Consensus on new charter for IPPM
> Greetings, all,
>  

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk(at)uijterwaal.nl
                                          http://www.uijterwaal.nl
                                          Phone: +31.6.55861746
------------------------------------------------------------------------------

Read my blog at http://www.uijterwaal.nl/henks_hands.html

---

From: Mach Chen <[email-masked]>
Date: Fri, 22 Mar 2013 08:48:33 +0000

Hi,

The charter looks good to me, I support it!

By the way, does it mean that this draft (http://tools.ietf.org/html/draft-chen-coloring-based-ipfpm-framework-01 ) is in the new charter?  So, please take a look at this draft and give your comments!

Many thanks
Mach

> We have updated the draft charter based on discussion during our meeting in
> Orlando last Friday. We believe the draft charter below addresses all comments
> received, and would like to issue a call for WG consensus on the charter text.
> We'll be issuing separate calls for each of the documents under consideration for
> adoption as WG items under this charter.
> 
> If you do not support the adoption of this charter, please reply to the list, with
> your suggestions for improvement and further discussion.
> 
> Please reply by Monday 25 March 2013; absent continuing discussion, we intend
> to evaluate consensus then, and send the charter to the IESG for approval.
>

---

From: <[email-masked]>
Date: Fri, 22 Mar 2013 16:19:35 +0000

looks good, thanks

could the reference path (measurement points) work be mentioned? or maybe some note about TBD whether this is in ippm or lmap?

possible re-phrasings:

para 1
< These metrics are designed such that they can be performed by network operators>
maybe 'used' or 'measured' instead of 'performed'?

para 2 -could this be merged with the later paragraph < The WG has produced protocols ...> (four paragraphs later)> They say ~ same thing

para 3 - < The WG will develop the minimum number of new metrics and models needed to more accurately quantitatively characterize the network path(s) under test and/or the behavior of transport and application layer>
maybe 'performance' instead of 'behavior'?

penultimate para < The equivalency of metric definitions is crucial to accurate, comparable and reproducible measurement. The WG will define and maintain a registry of metric definitions to this end. The WG encourages work which compares IPPM metrics with metrics developed elsewhere, particularly in regards to equivalency, similarity and functionality.>
are ' equivalency' and 'similarity' the same thing? I'm not sure what 'functionality' means in this context.

suggest
Agreement about the definitions of metrics enables them to be measured accurately, comparably and reproducibly. To this end, the WG will define and maintain a registry of metric definitions. The WG also encourages work which assesses the comparability of measurements of IPPM metrics with metrics developed elsewhere.>

best wishes
phil

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: 19 March 2013 02:34
To: [email-masked]
Subject: [ippm] Call for Consensus on new charter for IPPM

Greetings, all,

Brian

 Charter

 Current Status: Active

 Chairs:
     Brian Trammell 
     Bill Cerveny

 Transport Area Advisor:
     Martin Stiemerling

 Mailing Lists:

Milestones:

---

From: Matt Mathis <[email-masked]>
Date: Sat, 23 Mar 2013 12:13:18 -0700

Oops
> could the reference path (measurement points) work be mentioned? or maybe some note about TBD whether this is in ippm or lmap?

This is a very good point.  This should be in scope for IPPM, and it
would be best if the charter said so explicitly.  The dialog about
reference paths (measurement points) has a venn diagram problem
between LMAP and IPPM.  LMAP is clearly focused on the last mile, so
their reference path model contains no markers corresponding to
interconnects, so it can not be used as a foundation for metrics that
might characterize poor performance due to poorly placed or
insufficient peering capacity.

Although such metrics are not part of IPPM's traditional scope, "cloud
metrics" would be a trivial extension to MBM.  (e.g. Given a pool of
content providers and a pool of users, what is the probability that a
random user will receive random content at better than a specified
performance?)

Thanks,
--MM--

---

From: Brian Trammell <[email-masked]>
Date: Sun, 24 Mar 2013 21:38:33 +1300

Hi, Phil,

Thanks for the comments; comments and questions on comments inline.

> could the reference path (measurement points) work be mentioned? or maybe some note about TBD whether this is in ippm or lmap?

I believe it's in IPPM, since it fits into metric definition, and the reference path would be part of the metric definition for reproducible deployment of observation points. (I also think it would be useful to ensure that the reference path doesn't just address LMAP use cases, but wider IPPM use cases, but that's a separate point.)

I see the reference path work as covered in the penultimate paragraph:

"The WG also encourages work which improves the availability of information about the context in which measurements were taken."

Does this not accurately characterize the aim of the reference path?

> maybe 'used' or 'measured' instead of 'performed'?

'used' sounds good; changed in working copy.

> para 2 -could this be merged with the later paragraph < The WG has produced protocols ...> (four paragraphs later)> They say ~ same thing

I don't follow this comment... Para 2 discussed metrics, Para 5 measurement protocols. We want to keep this work separate, I think.

> maybe 'performance' instead of 'behavior'?

Works for me; changed in working copy.

> Agreement about the definitions of metrics enables them to be measured accurately, comparably and reproducibly. To this end, the WG will define and maintain a registry of metric definitions. The WG…

Good point, countersuggest:

Agreement about the definitions of metrics enables accurate, comparable, and
reproducible measurement of those metrics. To this end, the WG will define and
maintain a registry of metric definitions. The WG encourages work which
assesses the comparability of measurements of IPPM metrics with metrics
developed elsewhere. The WG also encourages work which improves the
availability of information about the context in which measurements were taken.

Cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Sun, 24 Mar 2013 21:38:41 +1300

Hi, Alissa,

Thanks for the comment! Nit fixed in working copy of the charter.

Cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Sun, 24 Mar 2013 21:41:26 +1300

Hi, Mach,

On quick review, as the work described in the draft covers a hybrid performance measurement approach, it would seem to be in scope of the proposed charter. However, as we've had no discussion of the draft on the list or in an IPPM meeting, it's premature to make a call for adoption for the draft as a WG item at this time.

(Others, if you have interest in the described work, please read the draft and discuss with the authors.)

Best regards,

Brian

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Sun, 24 Mar 2013 09:58:11 -0400

Following changes others have proposed, I support the new charter
with a few more suggestions, below:

	...New metric definitions will state how the definition improves on an existing metric definition, or encompasses a metric not previously defined.

"encompass" would seem to refer to an existing metric instead of a new one:
http://oxforddictionaries.com/definition/english/encompass

I suggest:
... or assesses a property of packet transfer not previously defined.

(just as an example, we have loss pattern and loss episode metrics,
but nothing comparable for delay except delay variation...)

-=-=-=-=-=-

further on:
... New metric and protocol development will focus on the 
automatability of measurements in order to support large-scale measurement efforts.

suggest:
s/automatability/automation/ 
(it's more straightforward, and my spell-checker doesn't accept "automatability" 

-=-=-=-=-=-
2nd to last para:
The equivalency of metric definitions is crucial to accurate, comparable and reproducible measurement.

I suggest:
The clarity of metric definitions and methods of measurement is crucial to accurate, reproducible, and equivalent results across implementations.

(following approval RFC 6576, "equivalence" has a special meaning in IPPM,
and the synonym for interoperability among results from implementations of metrics and methods).

++++++++++++
OR, with the alternate text suggested by Brian and Phil:
	Agreement about the definitions of metrics enables accurate, comparable, and
	reproducible measurement of those metrics.

suggest:
Agreement about the definitions of metrics and methods of measurement enables accurate, 
reproducible, and equivalent results across different implementations.

-=-=-=-=-=-
last para:

... Within the IETF process, IPPM metric definitions will be subject to as rigorous a scrutiny for usefulness, clarity, and accuracy as other protocol standards.

suggest:
s/definitions/definitions and measurement protocols/

regards,
Al

> Sent: Sunday, March 24, 2013 4:39 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> > could the reference path (measurement points) work be mentioned? or
> maybe some note about TBD whether this is in ippm or lmap?
> 
> I believe it's in IPPM, since it fits into metric definition, and the
> reference path would be part of the metric definition for reproducible
> deployment of observation points. (I also think it would be useful to
> ensure that the reference path doesn't just address LMAP use cases, but
> wider IPPM use cases, but that's a separate point.)
> 
> "The WG also encourages work which improves the availability of
> information about the context in which measurements were taken."
> 
> > para 2 -could this be merged with the later paragraph < The WG has
> produced protocols ...> (four paragraphs later)> They say ~ same thing
> 
> I don't follow this comment... Para 2 discussed metrics, Para 5
> measurement protocols. We want to keep this work separate, I think.
> 
> > para 3 - < The WG will develop the minimum number of new metrics and
> models needed to more accurately quantitatively characterize the network
> path(s) under test and/or the behavior of transport and application layer>
> > penultimate para < The equivalency of metric definitions is crucial to
> accurate, comparable and reproducible measurement. The WG will define and
> maintain a registry of metric definitions to this end. The WG encourages
> work which compares IPPM metrics with metrics developed elsewhere,
> particularly in regards to equivalency, similarity and functionality.>
> > Agreement about the definitions of metrics enables them to be measured
> accurately, comparably and reproducibly. To this end, the WG will define
> and maintain a registry of metric definitions. The WG also encourages work
> which assesses the comparability of measurements of IPPM metrics with
> metrics developed elsewhere.>
> 
> > We have updated the draft charter based on discussion during our meeting
> in Orlando last Friday. We believe the draft charter below addresses all
> comments received, and would like to issue a call for WG consensus on the
> charter text. We'll be issuing separate calls for each of the documents
> under consideration for adoption as WG items under this charter.
> >
> > If you support the adoption of this charter for IPPM, please reply to
> this message and/or to the list stating so.
> >
> > If you do not support the adoption of this charter, please reply to the
> list, with your suggestions for improvement and further discussion.
> >
> > Please reply by Monday 25 March 2013; absent continuing discussion, we
> intend to evaluate consensus then, and send the charter to the IESG for
> approval.
> >
> > The IP Performance Metrics (IPPM) Working Group develops and maintains
> standard metrics that can be applied to the quality, performance, and
> reliability of Internet data delivery services and applications. It also
> develops and maintains protocols for the measurement of these metrics.
> These metrics are designed such that they can be performed by network
> operators, end users, or independent testing groups. Metrics developed by
> the IPPM WG are intended to provide unbiased quantitative performance
> measurements and not a value judgement.
> >
> > The IPPM WG has produced documents that define specific metrics and
> procedures for accurately measuring and documenting these metrics. The
> working group will continue advancing these metrics along the standards
> track, using the guidelines stated in RFC 6576. To the extent possible,
> these metrics will be used as the basis for future work on metrics in the
> WG.
> >
> > The WG will develop the minimum number of new metrics and models needed
> to more accurately quantitatively characterize the network path(s) under
> test and/or the behavior of transport and application layer protocols on
> these path(s). New metric definitions will state how the definition
> improves on an existing metric definition, or encompasses a metric not
> previously defined.
> >
> > Additional methods will be defined for the composition and calibration
> of IPPM-defined metrics, as well as active, passive and hybrid measurement
> methods for these metrics. In addition, WG encourages work which describes
> the applicability of metrics and measurement methods, especially to
> improve understanding of the tradeoffs involved among active, passive, and
> hybrid methods.
> >
> > The WG has produced protocols for communication among test equipment to
> enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
> respectively).
> > These protocols will be advanced along the standards track. The WG will
> further develop and improve these protocols. The WG may develop new
> measurement protocols as necessary to support new metrics. New metric and
> protocol development will focus on the automatability of measurements in
> order to support large-scale measurement efforts.
> >
> > The equivalency of metric definitions is crucial to accurate, comparable
> and reproducible measurement. The WG will define and maintain a registry
> of metric definitions to this end. The WG encourages work which compares
> IPPM metrics with metrics developed elsewhere, particularly in regards to
> equivalency, similarity and functionality. The WG also encourages work
> which improves the availability of information about the context in which
> measurements were taken.
> >
> > The IPPM WG seeks cooperation with other appropriate standards bodies
> and forums to promote consistent approaches and metrics. Within the IETF
> process, IPPM metric definitions will be subject to as rigorous a scrutiny
> for usefulness, clarity, and accuracy as other protocol standards. The
> IPPM WG will interact with other areas of IETF activity whose scope
> intersects with the requirement of these specific metrics. The WG will, on
> request, provide input to other IETF working groups on the use and
> implementation of these metrics.
> >
> > Milestones:
> >

---

From: Brian Trammell <[email-masked]>
Date: Mon, 25 Mar 2013 14:18:58 +1300

hi, Al, all,

(Taking my chair hat off, to discuss charter wordsmithing as a contributor.)

> but nothing comparable for delay except delay variation...)

"a property of packet transfer" is, I think, too limiting. The behavior of transport layer protocols or applications atop them are not really packet transfer properties.

Suggest "or assesses a property of network performance not previously defined."

We're not specifically automating metrics, though, that gets into LMAP territory IMO. How about "Suitability of measurements for automation?"

Best regards,

Brian (as contributor)

---

From: Brian Trammell <[email-masked]>
Date: Mon, 25 Mar 2013 14:24:06 +1300

Hi, Al, all,

(And with my chair hat on...)

Looks good to me, changed in working copy.

Also changed in working copy.

Best regards,

Brian (as chair)

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Mon, 25 Mar 2013 03:15:07 -0400

Hi Contributor Brian,
Both of your alt.suggestions work for me,
Al

> From: Brian Trammell [mailto:[email-masked]]
> Sent: Sunday, March 24, 2013 9:19 PM
> To: MORTON JR., ALFRED C (AL)
> Cc: [email-masked]; [email-masked]; [email-masked]
> > 	...New metric definitions will state how the definition improves on
> an existing metric definition, or encompasses a metric not previously
> defined.
> >
> "a property of packet transfer" is, I think, too limiting. The behavior of
> transport layer protocols or applications atop them are not really packet
> transfer properties.
> 
> We're not specifically automating metrics, though, that gets into LMAP
> territory IMO. How about "Suitability of measurements for automation?"
> 
> Best regards,
> 
> Brian (as contributor)

---

From: Mach Chen <[email-masked]>
Date: Mon, 25 Mar 2013 07:20:03 +0000

Hi Brian,

Sure, thanks for your reply!

As you said, we'd appreciate that the IPPMers can read and comment on the draft. 

Best regards,
Mach

> Sent: Sunday, March 24, 2013 4:41 PM
> To: Mach Chen
> Cc: <[email-masked]>; [email-masked]
> On quick review, as the work described in the draft covers a hybrid performance
> measurement approach, it would seem to be in scope of the proposed charter.
> However, as we've had no discussion of the draft on the list or in an IPPM
> meeting, it's premature to make a call for adoption for the draft as a WG item at
> this time.
> 
> > By the way, does it mean that this draft
> (http://tools.ietf.org/html/draft-chen-coloring-based-ipfpm-framework-01 ) is in
> the new charter?  So, please take a look at this draft and give your comments!
> >
> > Many thanks
> > Mach
> >
> >> If you support the adoption of this charter for IPPM, please reply to this
> message
> >> and/or to the list stating so.
> >>
> intend
> >> standard
> >> procedures
> >> to more
> >> The WG may update its core framework RFC 2330 as necessary to
> accommodate
> >> these
> >> activities.
> >>
> >> enable
> further