Subject: Re: [ippm] framework-compagg-05 comments

From: Al Morton <[email-masked]>
Date: Mon, 11 Feb 2008 09:01:55 -0500

Loki, Reza, and all,

Version 06 addresses all the comments below (Loki and Reza's

messages sent just prior to IETF-70). See

http://tools.ietf.org/wg/ippm/draft-ietf-ippm-framework-compagg/

for text and diffs (which will help folks find the new
material).

Specifically, comments on

- clarifications on forecasting

- the definition of the term, Index

- IPR and Viral Licensing

- time varying metrics (diff terminology from "transient"
below)

have all been addressed.

There's a new term:  Complete Time Interval

(added because the temporal counterpart to Complete Path was
omitted).

If the concept of "forecasting" is now clear, we might
decide

whether to include/exclude this from the framework.

If we agree on the definition of "Index", then we might

further discuss whether this term really fits in the current

scope of the framework.

regards,

Al

At 08:29 PM 11/30/2007, Loki Jorgenson wrote:

Some comments below to Reza's comments (in BLOCKS):

Also, some other general comments:

Section 3.2 - there is a reference to "true path" with no entry
under

Terminology to define it.  Is it related to "ground truth"
(which is

defined)?  If it is not intended to be a unique term but only a

descriptor, I would suggest changing it to something more like
"actual"

(as used in "ground truth").

Section 4.1 - with regard to weighting function and transient vs.

intransient measures (see comments further down).

Section 4.5 - second sentence has "would be a the
delay"

Section 5.

As I was dealing with IP issues today and aspects of viral licensing,
is

it appropriate to include in any composition metric description IP

references to embedded proprietary metrics and/or use of
standards?

Otherwise, I note, most of the requirements in section 5. are
technical.

On a more general note:

I would suggest that there are "ground truths" that fall into
two

categories - I will term them "transient" and
"intransient" - that

reflect their variability (primarily with respect to traffic).
For

example, (minimum) delay is intransient w.r.t. traffic - it may vary
due

to route change.

Most measures and metrics tend to involve transient effects and so

averaging is more typical for those (instead of minima).  There
are

underlying temporal constraints that possibly should be noted with

regard to transience.  That is, aggregate temporal measures
over

times-scales exceeding known variation lengths may be misleading
(i.e.

aggregating over a day when transient traffic effects are known to
vary

on 24 cycle).  At least describing these might be
appropriate.

Perhaps this kind of thing is implicitly part of any weighting
function

(as described in 4.2 regarding spatial aggregation)?

It seems in any case that there are measures that assess the network

with respect to its design (how big, how long), and with regard to
the

traffic and related performance (how much, how well).
Intransient

features, independent of traffic, are then aspects of the network

itself, while transient features are about the traffic using it.

If this were considered applicable, I am not yet sure how this would

appear in the document.  Except possibly as a Section 5.
requirement

with regard to temporal aggregates than span time periods that are

potentially problematic.    However, this might be
subsumed into one of

the other attributes listed there concerning relationship,

justification, or inaccuracy.

Comments?

Loki Jorgenson

Apparent Networks

t   604 433 2333 ext 105

m   604 250-4642

----------------------------------------------------------------------

Message: 1

Date: Wed, 28 Nov 2007 19:27:47 -0800

From: "Fardid, Reza" <[email-masked]>

Section 3.6

R-factor is a valid example of an index as defined here, however, it
is

composed of both IP performance metrics and analog metrics.

On this basis, is it within scope of this draft?

Loki> SUPPOSEDLY NOT 2. Scope INDICATES THAT LOSS, LATENCY, AND
DELAY

VARIATION ARE THE LIMITS.  BUT THIS IS NOT ENTIRELY CLEAR, I CONCUR,
AND

SHOULD BE FURTHER DELIMITED.

Another relevant index composed of IP performance metrics is MDI,
per

rfc4445.

Loki> NOTE THAT THIS ON THE OTHER HAND IS NOT AN IETF STANDARD -
RFC4445

IS INFORMATIONAL AND MDI A PROPRIETARY METRIC (US PATENT APPLICATION

10/604,997).  OF COURSE, COMPAGG DOES NOT EXCLUDE SUCH METRICS (AS
FAR

AS I SEE).

IS THERE ANY EXPLICIT REQUIREMENT WITHIN COMPAGG TO IDENTIFY
CONFORMANCE

TO ACKNOWLEDGED STANDARDS (WHATEVER THOSE ARE)?  I DON'T SEE ANY SO
I

WOULD ASSUME NOT.

At 10:27 PM 11/28/2007, Fardid, Reza wrote:

Content-class: urn:content-classes:message

Content-Type: multipart/alternative;

boundary="----_=_NextPart_001_01C83237.D01BA00C"

Hi Al and Steven,

I have read this draft with interest and have a number of questions and
comments:

Section 3.6

R-factor is a valid example of an index as defined here, however, it is
composed of both IP performance metrics and analog metrics.

On this basis, is it within scope of this draft?

Another relevant index composed of IP performance metrics is MDI, per
rfc4445.

Section 4.1

Is the comment referring to forecasting of one metric from another, whose
temporally aggregated measurements are available?

If so, then I fail to see how this is feasible given that the primary
metrics have been defined to be orthogonal.

Section 4.5

Another example will be helpful here.

In particular, if a composed metric is derived from dissimilar metrics,
e.g., R-factor or MDI, then can one make a general statement or provide
guidance

with respect to the order of composition to minimize measurement
error?

For instance, is it preferred to compose sub-path R-factor or MDI metrics
(assuming their composition functions are defined) derived from primary
metrics, or compose the sub-path primary metrics first, followed by
R-factor or MDI composition?

Is there statistical or empirical evidence to support either order in
general?

Regards,

Reza Fardid

_______________________________________________
ippm mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ippm