Subject: [xrblock] Transport of I.D-xrblock-rtcp-xr-qoe style quality data in IPFIX

From: Hendrik Scholz <[email-masked]>
Date: Mon, 16 Jul 2012 14:26:31 +0200

Hi!

I've published draft-scholz-ipfix-rtp-audio-quality-00
which contains Information Elements for IPFIX to transport audio
quality information.
The scope is audio only and the multiple channel issue
which was discussed in Paris is also present in this.
My intention is to keep the format in sync with the xrblock format
so that it is easy to transport the existing data in IPFIX or
even take it from RTCP reports.
The metric block part is taken care of in
draft-akhter-opsawg-perfmon-ipfix which will be renamed/updated
before Vancouver.

So far this is just informational and I'll dedicate some more
time to synchronize the content and check for potential pitfalls.

Just FYI,
  Hendrik

-- 
Hendrik Scholz <[email-masked]>

---

From: Qin Wu <[email-masked]>
Date: Tue, 17 Jul 2012 11:30:44 +0800

Hi, Hendrik:
That is an interesting draft you published. Thank for sharing.
I am wondering what the goal of this draft is? Is this draft an part of 
draft-akhter-opsawg-perfmon-ipfix or will be incorporated into draft-akhter-opsawg-perfmon-ipfix?
Would you like to clarify the status of this draft?

Just have a quick look at this draft, I think the 1st reference has been outdated
and should be replaced with I.D-xrblock-rtcp-xr-qoe.

Also it seems you give a range for each RTPMOSClass as follows:

rtpMOSClass 1: MoS value < 3.10
rtpMOSClass 2:  3.10<=MoS value<3.60
rtpMOSClass 3:  3.60<=MoS value<4.03
rtpMOSClass 4:  4.03<=MoS value< 4.34
rtpMOSCalss 5: 4.34<= MoS value<=5

I am wondering what the rationale is for such range allocation. which reference are you based on?

Lastly why should the rtpMoSClass choose 'number of seconds' as unit?

Regards!
-Qin
----- Original Message ----- 
From: "Hendrik Scholz" <[email-masked]>
To: <[email-masked]>
Sent: Monday, July 16, 2012 8:26 PM
Subject: [xrblock] Transport of I.D-xrblock-rtcp-xr-qoe style quality datain IPFIX

> _______________________________________________
> xrblock mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xrblock

---

From: Hendrik Scholz <[email-masked]>
Date: Wed, 18 Jul 2012 11:36:06 +0200

Hi,

> Would you like to clarify the status of this draft?

The intention was to not merge the drafts.
Aamer will publish an updated version of his drafts with lots of 
additional elements from my side.
The reason not to merge was that the scope is somewhat different and it 
would (and already is ;)) quite complicated to find the right working 
group for the documents. Adding a narrow scope RTP audio quality draft
doesn't make it easier.

> and should be replaced with I.D-xrblock-rtcp-xr-qoe.

Thanks. Will do!

> I am wondering what the rationale is for such range allocation. which reference are you based on?

This is based on ITU-T G.107 (table B.1) referring to the perceived user 
experience.

> Lastly why should the rtpMoSClass choose 'number of seconds' as unit?

In order to calculate a MOS value multiple seconds worth of data have
to be present to begin with (TM Forum claims 8-10 seconds).
The data format is flexible in regards to as many seconds worth of
data are transported in a single IPFIX record.
Systems may use fixed time slices (e.g. one record for 30 seconds
worth of stream data) or one record per (variable length) stream/call.
An IPFIX mediator/collector or any data warehouse should be able
to aggregate the data a) from multiple records belonging to a
single streams and b) multiple records belonging to multiple streams.

What unit were you thinking of?

Thanks for your feedback,
  Hendrik

---

From: Qin Wu <[email-masked]>
Date: Mon, 23 Jul 2012 10:43:45 +0800

> This is based on ITU-T G.107 (table B.1) referring to the perceived user 

[Qin]: Make sense.
Table B.1 is a good example to see translation between MoS value and Rating value.
 
> What unit were you thinking of?

[Qin]: I thought rtpMoSClass has no unit and 'Number of seconds the monitored stream'
should be regarded as a new information element in parallel with rtpMoSClass rather than
one attribute of rtpMoSClass. am I wrong?

---

From: Michael Krüger <[email-masked]>
Date: Mon, 23 Jul 2012 16:19:36 +0200

Hi,

> one attribute of rtpMoSClass. am I wrong?

This draft is suppose to support any measurement interval and that is
the reason for having seconds as the unit. The measurement interval
could be a few seconds of a flow or the complete call. To be able to
collect these classes from different sources, which may use different
measurement intervals, the seconds are needed. So the classes do not
contain a count if segments of a flow had a MOS value within a specific
class, but rather how many seconds of the flow have been in that class.
That way it is possible to combine measurements from different sources,
which is the idea behind the draft.
The application of this can then be a report to show how many minutes of
transported VoIP traffic for a day/week/month have been in any of the 5
classes.

Regards,
Michael

---

From: Qin Wu <[email-masked]>
Date: Tue, 24 Jul 2012 11:20:05 +0800

Hi,Michael:
Thank for your clarification.
One thing I am not clear to your idea  is how to calculate MoS value or Rating value based on stream data from multiple sources?
Currently in QoE XR draft, we support MoS calculation per SSRC or MoS calculation per audio channel?
It seems you claim you can calculate MoS values based on audio data from multi-channels? 
Is that MoS value is the weighted mean of MoS value of each channel? If in that case, what the weight for each 
channel MoS calculation?
Maybe this is just implemenation specific issue, however I am interested to listen to that.

Regards!
-Qin

---

From: Michael Krüger <[email-masked]>
Date: Tue, 24 Jul 2012 09:08:19 +0200

Hi, Qin,

> Maybe this is just implemenation specific issue, however I am interested to listen to that.
Thank you for your interest.
Of course we have not reinvented MOS. Our approach is related to ITU-T
P.564. The audio test samples used in that standard have a minimum
length of 8-seconds. We therefore assume that one must evaluate at least
this period of time, to determine a reliable MOS value. What we try to
achieve with monitoring live traffic, is to determine for every
individual flow of a VoIP call MOS values for every 10-second segment.
This provides us with many MOS values over the duration of an individual
call. Most network impairments are of transient nature, so for every
call the result will be a distribution of MOS scores falling into one of
the 5 classes. Depending on the number of seconds in each of the
classes, it is possible to assess how many seconds of a call have been
with which quality on the MOS scale 1-5. If this data is available for a
single flow, call, etc. the seconds of the 5 classes can be aggregated
over all flows/calls performed between VoIP operators, operators and
customers, or simply SIP-trunks. This will e.g. allow to come up with
new SLA agreements where the agreement could possibly be that 99% of all
VoIP minutes must be in the highest MOS class. The accuracy of this is
pretty high, since one determines MOS values for only a few seconds of a
call, bringing every impairment to attention via a lower MOS value for
the segment in which the impairment occurred. No averaging over a call
is done, so the distribution of MOS values like this, truly reflects the
monitored quality.

Does this make sense to you as well?

Regards,
Michael

---

From: Qin Wu <[email-masked]>
Date: Tue, 24 Jul 2012 18:00:40 +0800

I tend to agree, thanks again for your clarification.

Regards!
-Qin