Subject: [xrblock] General comments on xrblock metrics drafts

From: Alan Clark <[email-masked]>
Date: Sun, 30 Oct 2011 18:51:03 -0400

1. Cumulative, Interval and Sample metrics
The drafts current have one bit assigned for Cumulative/Interval indication.
Some metrics are however sampled and don¹t have validity for an interval or
a call/session; an example of this is PPDV, which is a sampled value of a
running average and only has validity for the last 200-300ms before the
value is sampled.
Proposal - increase this to two bits to allow for the indication of a
Sampled value. 

2. Bursts and Gaps (burst-gap-discard and burst-gap-loss drafts)
The original Burst/Gap concept addressed four issues:-

(a) Packet loss and discard have similar effect on the quality of a decoded
voice, audio or video stream.

(b) Packet loss is often (not always) caused by congestion, which also leads
to higher jitter levels and hence packet discard.  This means that loss and
discard are often closely related in time.  From a QoE perspective it is
important to look at the combined effect as the impact on quality may be
quite dependant on the timing of loss and discard relative to each other.

(c) Prior work had focused on the idea of bursts being periods of
consecutive loss - and the duration of such bursts is typically very short.
Loss (and discard) does however occur in ³sparse² bursts which can be
seconds in length.  This was the motivation behind the use of a Markov model
with the Gmin parameter being used to discriminate bursts and gaps.

(d) IP impairments tend to be transient, hence average packet loss rates are
not too helpful in understanding QoE. Given a 1% loss rate there is a major
difference between losing one packet in every one hundred, which PLC could
handle, and losing a burst of ten packets in every one thousand. There are
two approaches to reporting transient problems - either send frequent
interval reports OR define metrics that can describe transients. The
Burst/Gap metrics defined in RFC3611 provide an efficient way of describing
transient bursts of lost and discarded packets.

Having implemented the approach described in RFC3611 in 2000 and used these
metrics for over ten years, we have found that the basic premise was sound
and does allow for very efficient reporting and accurate QoE metrics.  The
feedback from those using these metrics was that they would also like to be
able to correlate bursts with other network events and hence it would be
helpful to have some form of ³time since last burst² metric.

With regard to the current burst-gap-discard and burst-gap-loss drafts, my
opinion is they should be either combined to a single burst-gap-loss-discard
draft OR there should be burst-gap-loss-discard and burst-gap-loss drafts.
A combined loss/discard draft is more useful for QoE assessment however
there may be some cases in which reporting only loss may be useful.

The drafts contain sufficient metrics to calculate standard deviations of
bursts however I¹m personally not convinced that standard deviation is more
useful than mean and max values, together with a time since last burst
value.  Max burst length is important as this is the period that impacted
the user most.  

3. Discard draft

It would be useful to identify early vs late discards, as this can help to
determine what the cause was.

4. PDV draft

As mentioned above - jitter values could be averaged over an interval or
potentially sampled.

5. QoE draft

Separate comments will be provided
Alan