Subject: [Sipping] Perceptual objective media quality reporting

From: "Ravi Raviraj" <[email-masked]>
Date: Thu, 1 Mar 2007 17:13:07 -0000

Kaynam.

There are advantages and disadvantages in calculating MOS centrally.

You mentioned an advantage. A disadvantage is network traffic.

If you want to monitor the quality continuously, the end point could do
the work and send an alert when quality drops below certain level for a
period of time (for example).

In this way, there is no regular additional traffic on the network.
Whereas with continuous monitoring centrally technique, there is regular
additional traffic all the time.

Either way could be suitable depending on the policy of network/service
provider and information accuracy requirement.

So, we don't want to limit MOS calculation to either way. A quality
assessment/collection/distribution framework may recommend alternative
methods and policies.

If/when we calculate MOS at the end point, we need an efficient flexible
way to send this information without any ambiguity. Initially we could
develop an event package for this purpose and then move onto a framework
development afterwards.

Ravi. 

  _____  

From: Hedayat, Kaynam [mailto:[email-masked]] 
Sent: 01 March 2007 16:13
To: Ravi Raviraj; [email-masked]
Subject: RE: [Sipping] Perceptual objective media quality
reporting[Scanned]

Generally speaking I believe there is room for improvement in the
current event package. The main shortfall of the current package is
limiting the calculation of MOS by the endpoint only and not a central
system, thereby introducing the existing problem of correlating MOS
scores in networks where different endpoints use different algorithms.
Unless everyone agrees on one algorithm, which is absolutely the wrong
thing to do, there is no other way to correlate scores but to compute
MOS at a central location where the operator has choice on the
appropriate algorithm.

Regards,

Kaynam

  _____  

	From: Ravi Raviraj [mailto:[email-masked]] 
	Sent: Thursday, March 01, 2007 8:46 AM
	To: [email-masked]
	Subject: [Sipping] Perceptual objective media quality reporting

	Gordon,

	You said: But isn't MOS really a perceptual score, based on how
people really perceive the quality?

	Yes, MOS is the perceptual quality measure of media for both
audio and video.

	MOS measure types are broadly categorized into three types.

	MOS-subjective: this is where humans listen/watch the media and
give their score. Most accurate, but not practical for in-service
prediction.

	MOS-objective: this is where computer algorithm objectively
predicts the subjective value. Very practical and accurate in-service
perceptual quality measure for media.

	MOS-estimate: used with network planning models. This is not
recommended (ITU-T) for actual customer quality opinion prediction.

	Within each one of these categories, there are three further sub
types, listening quality, conversational quality and talker quality.

	Thus, we have MOS-LQO (listening quality objective), MOS-CQO
(conversational quality objective) and MOS-TQO (talker quality
objective), etc.

	(Other types: MOS-LQS, MOS-LQE, MOS-CQS, MOS-CQE, MOS-TQS,
MOS-TQE).

	Each one of these in practice could have four further
combinations based of media bandwidth.

	Please see ITU-T P.800.1 for further details.

	You said:

	As such, isn't this the reference for MOS, and when applied to
voice/audio, it would be acoustic...certainly not packet. The
packet-based measurement models have been devised to provide an
approximating low-processing way to obtain a MOS score.

	What you are thinking of is reference media used with
full-reference models (models used to predict objective scores), such as
PESQ for speech and J.144 for video.

	What I am referring to is reference point (as opposed to
reference media). For example in the ingress direction, MOS could be for
ear (acoustical) reference point, electrical reference point or packet
input reference point. There are other variations too.

	Yes, packet based measurement models are low in CPU usage.
However, there are such models out there which can objectively predict
MOS pretty well.

	Users select between reference points, methods and models to
identify the problem and then to pin point the location (e.g. end point
device problem as opposed to transport problem). 

	Currently, users run a low CPU-power-requirement method most
times, and when problems occur they turn on other techniques at
different points to pin-point the problem/location
automatically/dynamically.

	This is where number of people wants a framework to handle the
situation. 

	All above comments applies equally to video (with variations).
For example with media bandwidth, read as video frame-size/resolution.

	Ravi.

  _____  

	From: Beith, Gordon [mailto:[email-masked]] 
	Sent: 28 February 2007 17:47
	To: Ravi Raviraj; [email-masked]
	Subject: RE: [Sipping] Perceptual objective media quality
reporting[Scanned]

	But isn't MOS really a perceptual score, based on how people
really perceive the quality?

	As such, isn't this the reference for MOS, and when applied to
voice/audio, it would be acoustic...certainly not packet. The
packet-based measurement models have been devised to provide an
approximating low-processing way to obtain a MOS score.

	If a MOS-scale is applied to video, it would be a reference of
optics, I would think.

	Or did I miss what you are saying?

	/Gordon

This email and attachment are confidential. If you have received this message in error, please notify the sender immediately and delete the message from your computer. Psytechnics Ltd has taken steps to ensure that this email is virus free but the recipient should take necessary steps to make certain.

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Beith, Gordon" <[email-masked]>
Date: Sun, 4 Mar 2007 22:32:14 -0500

They are not alone......

e.g.

www.empirix.com <http://www.empirix.com/> 

________________________________

From: Robins, Ellen [mailto:[email-masked]] 
Sent: Sunday, March 04, 2007 10:09 PM
To: Ravi Raviraj; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [Sip] Perceptual objective media quality reporting

Hi all,

There are few steps in this direction.

http://www.ixiacom.com/

Thanks and kind regards / Ellen Robins

ViG Technology Manager

Services Enablers

Wireless Engineering & Operations
Telstra Operations
Telstra Corporation Limited
  
Ph 03 8627 8981
Mob 0428 309 250
Fax 03 9620 7864
Mail [email-masked] <mailto:[email-masked]>  

The information contained in this email message may be confidential. If
you are not the intended recipient, any use of, interference with,
disclosure or copying of this material is unauthorised and prohibited.
If you have received this message in error, please notify me by reply
email and then delete the message.

________________________________

From: Ravi Raviraj [mailto:[email-masked]] 
Sent: Thursday, 1 March 2007 2:03 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: [Sip] Perceptual objective media quality reporting

Hi all,

Over the last few years, a number of perceptual media quality objective
measures and assessment techniques have been standardized by ITU-T. For
example, MOS-LQO, MOS-CQO, etc. Each one of these parameters could have
number of variations like narrow-band/wide-band, electrical/acoustical,
full reference/no-reference, audio/video, etc. When applied within a
network end-point, each media direction (ingress/egress) could have one
or more of these values associated with it. We need a way to report
these values efficiently without any ambiguity. These MOS values could
be sent/distributed as perceptual objective quality SIP events. In this
way, all the advantages of SIP like mobility, etc. will be pulled in.

I would like to propose a work/discussion item on an event package for
this purpose. Even define/specify a quality reporting framework. Is
there any interest out there?

I presume that, an event package work will be within SIPPING WG, where
should we do the framework work? SIP/AVT?

Thanks.

Ravi.

This email and attachment are confidential. If you have received this
message in error, please notify the sender immediately and delete the
message from your computer. Psytechnics Ltd has taken steps to ensure
that this email is virus free but the recipient should take necessary
steps to make certain.