Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task Capabilities and ma conditions back into the information model

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 23 Sep 2015 16:52:29 +0200

On Tue, Jul 28, 2015 at 02:52:53PM +0000, Carey, Timothy (Timothy) wrote:
> Juergen,
> 
> In draft 05 we removed the ability for a measurement agent to report its metrics that it supports.
> We need to add
> 
> *         ma-task-capability-obj  ma-supported-tasks<0..*>;
> 
> *         [ma-condition-obj        ma-conditions<0..*>;]
> 
> They were in the ma-status-obj.

You are correct. There currently is no 'list' of tasks that are
supported by a device anymore and we may have to put this back
in.

We changed the reporting of runtime failures to make them more
useful. See the ma-task-status-obj.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Fri, 25 Sep 2015 12:51:00 +0000

Juergen,

But the ma-status-object doesn't provide all the conditions of the MA might be experiencing (which might be implementation specific).

BR,
Tim

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Wednesday, September 23, 2015 9:52 AM
To: Carey, Timothy (Timothy)
Cc: [email-masked]
Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task Capabilities and ma conditions back into the information model

> They were in the ma-status-obj.

You are correct. There currently is no 'list' of tasks that are supported by a device anymore and we may have to put this back in.

We changed the reporting of runtime failures to make them more useful. See the ma-task-status-obj.

/js

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sat, 26 Sep 2015 10:12:22 -0400

Hi Tim and Juergen,

> > *         ma-task-capability-obj  ma-supported-tasks<0..*>;

It's necessary for all non-measurement tasks to have a registry
providing the details of each task (and someone needs to start that
development).

But for measurement Tasks, there can be multiple Performance Metric
Registry entries involved now, and each task will be some combination
of registered metrics. 

So we also need ma-supported-registry-entries<0..*>; 
and the text below (from 3.9) needs some revisions to keep-up with
the latest agreements (if I've got this right):

   A Measurement Task Configuration is the same in information terms to
   any other Task Configuration.  Both measurement and non-measurement
   Tasks have registry entries to enable the MA to uniquely identify the
   Task it should execute and retrieve the schema for any parameters
   that may be passed to the Task.  Registry entries are specified as a
   URI and can therefore be used to identify the Task within a namespace
   or point to a web or local file location for the Task information.
   As mentioned previously, these URIs may be used to identify the
   Measurement Task in a public namespace
   [I-D.ietf-ippm-metric-registry].

Al

> From: lmap [mailto:[email-masked]] On Behalf Of Carey, Timothy
> (Timothy)
> Sent: Friday, September 25, 2015 8:51 AM
> To: Juergen Schoenwaelder
> Cc: [email-masked]
> Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task
> Capabilities and ma conditions back into the information model
> 
> Juergen,
> 
> But the ma-status-object doesn't provide all the conditions of the MA
> might be experiencing (which might be implementation specific).
> 
> BR,
> Tim
> 
> Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task
> Capabilities and ma conditions back into the information model
> 
> supported by a device anymore and we may have to put this back in.
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Sat, 26 Sep 2015 16:16:12 +0000

Al,

Indeed we do need that to be a list as you suggest but what is relationship with the role to the list of registry entries? 

BR,
Tim

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Saturday, September 26, 2015 9:12 AM
To: Carey, Timothy (Timothy); Juergen Schoenwaelder
Cc: [email-masked]
Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task Capabilities and ma conditions back into the information model

Hi Tim and Juergen,

> > *         ma-task-capability-obj  ma-supported-tasks<0..*>;

It's necessary for all non-measurement tasks to have a registry providing the details of each task (and someone needs to start that development).

But for measurement Tasks, there can be multiple Performance Metric Registry entries involved now, and each task will be some combination of registered metrics. 

So we also need ma-supported-registry-entries<0..*>;
and the text below (from 3.9) needs some revisions to keep-up with the latest agreements (if I've got this right):

Al

> Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert 
> Task Capabilities and ma conditions back into the information model
> 
> Juergen,
> 
> Subject: Re: [lmap] draft-ietf-lmap-information-model-06: Reinsert 
> Task Capabilities and ma conditions back into the information model
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sat, 26 Sep 2015 12:51:39 -0400

Hi Tim,  
good question (we're having Tuesday's meeting on Saturday :-(  )

I think we need some approximate hierarchy like this:

Measurement Capability [
   Measurement Protocol [
       Protocol Roles [ ]
   ]
   Registry URI  [
       Method Roles [ ]
   ]
]
so, an example would be

Measurement Capability [
    TWAMP [
       Control-Client; Session-Sender; Server; Session-Reflector;
       ]
    Prefix:Act_IP_UDP_Round-trip_Delay_95th-percentile_Poisson [
       Src; Dst;
    ... more URIs and Roles...
       ]
]
for a fully capable MA.

I think you can easily conjure-up how to indicate a system
with less than full capabilities, or how a different Meas. Protocols could
be substituted in a different meas. capability that measures the *same* 
registered metric.

Al

> From: Carey, Timothy (Timothy) [mailto:[email-masked]]
> Sent: Saturday, September 26, 2015 12:16 PM
> To: MORTON, ALFRED C (AL); Juergen Schoenwaelder
> Cc: [email-masked]
> Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task
> Indeed we do need that to be a list as you suggest but what is
> relationship with the role to the list of registry entries?
> 
> BR,
> Tim
> 
> Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task
> Registry entries involved now, and each task will be some combination of
> registered metrics.
> 
> and the text below (from 3.9) needs some revisions to keep-up with the
> latest agreements (if I've got this right):
>

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Sat, 26 Sep 2015 19:29:11 +0000

Al,

I am not sure where the Measurement Capability and Protocol Roles would come from given a set of registry entries. 
I do think the correlation of the IM to the Registry around Tasks and Actions is what is causing (at least my) confusion.

Summary
-------------------------------
ID | Name | URI | Description |
Metric Definition
-----------------------------------------
Reference Definition | Fixed Parameters |
Method of Measurement
---------------------------------------------------------------
Reference | Packet | Traffic | Sampling | Run-time | Role |
Method | Generation | Filter | dist. | Param | |
| Stream |
Output
-----------------------------
| Type | Reference | Units |
| | Definition | |

I 
BR,
Tim

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Saturday, September 26, 2015 11:52 AM
To: Carey, Timothy (Timothy); Juergen Schoenwaelder
Cc: [email-masked]
Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task Capabilities and ma conditions back into the information model

Hi Tim,

Measurement Capability [
   Measurement Protocol [
       Protocol Roles [ ]
   ]
   Registry URI  [
       Method Roles [ ]
   ]
]
so, an example would be

Measurement Capability [
    TWAMP [

I think you can easily conjure-up how to indicate a system with less than full capabilities, or how a different Meas. Protocols could be substituted in a different meas. capability that measures the *same* registered metric.

Al

> From: Carey, Timothy (Timothy) 
> [mailto:[email-masked]]
> Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert 
> Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert 
> > But the ma-status-object doesn't provide all the conditions of the 
> > MA might be experiencing (which might be implementation specific).
> >
> > BR,
> > Tim
> >
> > > In draft 05 we removed the ability for a measurement agent to 
> > > report
> > its metrics that it supports.
> > > We need to add
> > >

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sat, 26 Sep 2015 18:12:31 -0400

Hi Tim,

The Measurement Protocol and Protocol Roles can't come from the 
IPPM  metrics registry - just like setting a single method of measurement role
would cause entirely duplicate entries in the registry just to change the role,
adding each possible protocol (TWAMP, OWAMP, ICMP, IPSLA,...)
adds even more multiplication of a single entry.

So, that's why we agreed not to add protocol in the registry at
the workshop with BBF last year.

If an MA has TWAMP capability, it certainly knows and can say so
in a capabilities message. My guess is that the MAs will have
one of the above measurement protocols, so it's really just a 
matter of revealing the single choice of the developers and
any limitations (e.g., this MA can only perform as a TWAMP Server&Reflector).

hope this helps,
Al

> Sent: Saturday, September 26, 2015 3:29 PM
> I am not sure where the Measurement Capability and Protocol Roles would
> come from given a set of registry entries.
> I do think the correlation of the IM to the Registry around Tasks and
> Actions is what is causing (at least my) confusion.
> 
> Summary
> Reference Definition | Fixed Parameters | Method of Measurement
> Reference | Packet | Traffic | Sampling | Run-time | Role | Method |
> Generation | Filter | dist. | Param | |
> | Stream |
> Output
> I think you can easily conjure-up how to indicate a system with less
> than full capabilities, or how a different Meas. Protocols could be
> substituted in a different meas. capability that measures the *same*
> registered metric.
> 
> Al
> 
>

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Sun, 27 Sep 2015 13:09:20 +0000

Al,

Hmmm -  I think there might be the gap in this thinking.
If the metrics are defined in the Registry where would the "Tests" be defined such that they can be ultimately translated into an IM Task or Action - These tests have to have knowledge the parameters and results of the "containing metrics" along with their own elements.

For example where/how would the ICMP Ping test be defined that validates a latency metric? There are RFCs (e.g., 792, 4443) that describe the ICMP Ping message but not the Ping test. Also the ICMP RFC doesn't talk about elements of the test of the message (e.g., repetitions) as well as how the elements relate to the metric's parameters and how both the metric's parameters and test elements are mapped to the LMAP IM task/action structure.

What was the conclusion to where these tests are defined if not the Registry?

BR,
Tim

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Saturday, September 26, 2015 5:13 PM
To: Carey, Timothy (Timothy); Juergen Schoenwaelder
Cc: [email-masked]
Subject: RE: [lmap] draft-ietf-lmap-information-model-06: Reinsert Task Capabilities and ma conditions back into the information model

Hi Tim,

The Measurement Protocol and Protocol Roles can't come from the IPPM  metrics registry - just like setting a single method of measurement role would cause entirely duplicate entries in the registry just to change the role, adding each possible protocol (TWAMP, OWAMP, ICMP, IPSLA,...) adds even more multiplication of a single entry.

So, that's why we agreed not to add protocol in the registry at the workshop with BBF last year.

If an MA has TWAMP capability, it certainly knows and can say so in a capabilities message. My guess is that the MAs will have one of the above measurement protocols, so it's really just a matter of revealing the single choice of the developers and any limitations (e.g., this MA can only perform as a TWAMP Server&Reflector).

hope this helps,
Al

> I am not sure where the Measurement Capability and Protocol Roles 
> would come from given a set of registry entries.
> > Registry entries involved now, and each task will be some 
> > combination of registered metrics.
> >
> >    that may be passed to the Task.  Registry entries are specified 
> > as
> a

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 27 Sep 2015 15:07:02 -0400

Hi Tim,

This is a very useful discussion, because it makes clear to me 
some possible gaps in understanding between the LMAP and IPPM
communities (since we are discussing IPPM-specific examples
in this thread, and again during the Interim meeting on Tuesday).
We need to close these gaps, and examples provide the catalyst
for more discussion and learning for all of us.

please replies see below,
Al

> Sent: Sunday, September 27, 2015 9:09 AM
> If the metrics are defined in the Registry where would the "Tests" be
> defined such that they can be ultimately translated into an IM Task or
> Action - These tests have to have knowledge the parameters and results
> of the "containing metrics" along with their own elements.

[ACM] 
The largest category of columns in the Performance Metrics Registry contains
the details on "How to Perform Tests" - what we call Methods of Measurement.
For Active Metrics, the critical columns are:

   Method of Measurement
   ----------------------------------------------
   Reference | Packet     |.| Run-time   | Role |
   Method    | Generation |.| Parameters |      |
             | Stream     |.|

So, the RFCs that define Active Metrics in IPPM literature
also define the Methods of Measurement, or "Tests".
Further, where the authors of a Registry Entry for a Metric
want to augment the Reference Method with more detail, 
they are able to do so in the Reference Method column.

Thus, it's very important to get feedback from folks who may
be implementing the registered metrics: is the level of detail
in the Reference Method sufficient that independent implementations
will produce equivalent results? Or, are there questions that 
we still need to answer to avoid ambiguity?

> 
> For example where/how would the ICMP Ping test be defined that validates
> a latency metric? There are RFCs (e.g., 792, 4443) that describe the
> ICMP Ping message but not the Ping test. Also the ICMP RFC doesn't talk
> about elements of the test of the message (e.g., repetitions) as well as
> how the elements relate to the metric's parameters and how both the
> metric's parameters and test elements are mapped to the LMAP IM
> task/action structure.

[ACM] 
Great question. This reminds me of a conversation between Marcelo, Phil and
me almost three years ago, when we were discussing the apparent lack 
of a DNS response time metric. I suggested that we could rely on the
RFC 2681 Round-trip Delay Metric, and specify that the packet sent
would have a "Type-P" of a UDP DNS Message (Standard Query). The current
version of this DNS example is included in the draft proposing initial contents:
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-00#section-6
This entry contains a Reference Method of Measurement based on RFC 2681 and RFC 6673:
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-00#section-6.3.1

Clearly, we could do the same thing with ICMP Echo Request/Reply packets,
and specify a test equivalent to the connectivity assessment performed by ping apps.

> What was the conclusion to where these tests are defined if not the

[ACM] 
I think another area of knowledge transfer between LMAP and IPPM is the 
role of IPPM's test protocols (and the limits on their scope of operation).

For example, TWAMP is a protocol to coordinate, generate, and control Active
Measurement Streams in the context of Test Sessions between two hosts on the Internet.
TWAMP does its job independent from the metrics that the system intends to measure.

TWAMP entities have to get some testing details from the information model 
(Run-time Parameters) and the Registered Metrics.  A single TWAMP Test Session
would correspond to an LMAP measurement Task. 

The TWAMP protocol does not define how to make measurements, or discuss the 
process to generate specific streams of packets, 
provide accurate timestamps where/when needed,
or other details of the Method of Measurement.  That's where another process
implementing the Registered Metric and its Method of Measurement comes-in. 

A TWAMP entity could be configured by the info model/data model. In
https://tools.ietf.org/html/draft-cmzrjp-ippm-twamp-yang-01
another author team has created a YANG model for TWAMP.

hope this helps!

> The Measurement Protocol and Protocol Roles can't come from the IPPM
> metrics registry - just like setting a single method of measurement role
> would cause entirely duplicate entries in the registry just to change
> the role, adding each possible protocol (TWAMP, OWAMP, ICMP, IPSLA,...)
> If an MA has TWAMP capability, it certainly knows and can say so in a
> capabilities message. My guess is that the MAs will have one of the
> above measurement protocols, so it's really just a matter of revealing
> the single choice of the developers and any limitations (e.g., this MA
> can only perform as a TWAMP Server&Reflector).
> 
> hope this helps,
> Al
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 29 Sep 2015 15:42:49 +0200

> providing the details of each task (and someone needs to start that

Why is this needed? Our code can run tasks that are not registered. In
fact, it does not know how to work with the registry at the moment. Why
would I have to have a registry for a housekeeping task?
 
> the latest agreements (if I've got this right):

How would that be different from ma-task-registry-entries<1..*> in
ma-task-status-obj (except that you allow an empty list, which seems
reasonable)?

/js