Subject: [AVT] Suggestion for how to handle derived/computed metric in draft-avt-rtcpxr-*

From: David R Oran <[email-masked]>
Date: Mon, 23 Jul 2007 17:51:25 -0400

At the Mic during the AVT session I sketched a vague idea for how to  
resolve the conflict between the desire on the part of many folks to  
only carry raw/primitive instrumented measurements in RTCP-XR, and  
other equally vocal, who argued for carrying the derived/computed  
metrics as well.

I won't repeat the pro/con arguments here. Let's assume we want to  
carry both.

The chairs asked me to write up my vague thoughts as the beginnings  
of a concrete proposal.

Some particular technical difficulties with the derived metrics that  
makes them different from the primitive measurements is:
- there are competing standards for how to compute them
- the results are not generally aggregatable, comparable, or self- 
describing.
- the set of interesting derived metrics is not small and seems to be  
growing, so it will be difficult to nail these down for stable RTCP  
reporting.

This makes it problematic to mix primitive and derived metrics in the  
same metric blocks in RTCP.

So, my suggestion is to do the following.

- A standards body who defines one or more consistent derived metrics  
may create (by bringing a draft to IETF) a separate metric block for  
those metrics.
- The block and its contained derived/computed metrics would be  
completely self-describing in terms of exactly what version of what  
standard defines the block.
- The definition of the metric would explicit contain the input  
variables, in order to ensure that those inputs are separately  
available as primitive metrics in the RTCP-XR. This increases the  
probability that any metric that is computable by the endpoint is  
also computable by a central element receiving and processing RTCP-XR  
reports.
- If there are competing/overlapping blocks, an endpoint need not do  
all of them and there would be no confusion about exactly which  
(e.g."video MOS") calculation was done.

As an aside, I would rather not see further updates to the existing  
drafts until we resolve this issue, since this is in fact major  
surgery to the documents that is best done before polishing the text  
and individual metric definitions.

Dave Oran.

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Alan Clark" <[email-masked]>
Date: Mon, 23 Jul 2007 18:08:18 -0400

Dave

I agree with the general tenor of your thoughts.  I also agree that we
should not mandate that endpoints have to implement any particular set of
derived metrics - these should be optional and should be implemented based
on their usefulness.

For cases in which there is a fairly well defined metric class (e.g. MOS for
speech) then I suggest that we do define a separate draft for that with a
registry of algorithms that could be (optionally) used - it is easier to
update a registry with information that a new version of ITU-T G.xxx is
available than it would be to generate a new draft.

Regards

Alan

-----Original Message-----
From: David R Oran [mailto:[email-masked]] 
Sent: Monday, July 23, 2007 5:51 PM
To: AVT WG
Subject: [AVT] Suggestion for how to handle derived/computed metric
indraft-avt-rtcpxr-*

At the Mic during the AVT session I sketched a vague idea for how to resolve
the conflict between the desire on the part of many folks to only carry
raw/primitive instrumented measurements in RTCP-XR, and other equally vocal,
who argued for carrying the derived/computed metrics as well.

I won't repeat the pro/con arguments here. Let's assume we want to carry
both.

The chairs asked me to write up my vague thoughts as the beginnings of a
concrete proposal.

Some particular technical difficulties with the derived metrics that makes
them different from the primitive measurements is:
- there are competing standards for how to compute them
- the results are not generally aggregatable, comparable, or self-
describing.
- the set of interesting derived metrics is not small and seems to be
growing, so it will be difficult to nail these down for stable RTCP
reporting.

This makes it problematic to mix primitive and derived metrics in the same
metric blocks in RTCP.

- A standards body who defines one or more consistent derived metrics may
create (by bringing a draft to IETF) a separate metric block for those
metrics.
- The block and its contained derived/computed metrics would be completely
self-describing in terms of exactly what version of what standard defines
the block.
- The definition of the metric would explicit contain the input variables,
in order to ensure that those inputs are separately available as primitive
metrics in the RTCP-XR. This increases the probability that any metric that
is computable by the endpoint is also computable by a central element
receiving and processing RTCP-XR reports.
- If there are competing/overlapping blocks, an endpoint need not do all of
them and there would be no confusion about exactly which (e.g."video MOS")
calculation was done.

As an aside, I would rather not see further updates to the existing drafts
until we resolve this issue, since this is in fact major surgery to the
documents that is best done before polishing the text and individual metric
definitions.

Dave Oran.