Subject: Re:

From: "Reinaldo Penno Filho" <[email-masked]>
Date: Wed, 1 Dec 1999 13:32:47 -0200

Another approach...

"In spite of the apparent simplicity of MOS tests, there are a number of
important issues that must be carefully addressed. In order to avoid biasing
when designing the test, you must keep in mind issues such as test results;
number and type of speakers; number, type and training of listeners; and
presentation of the listening material. Issues that need attention during
processing of the results are: score distributions, standard deviation, and
confidence intervals.

Unfortunately, in spite of the care exercised in the design of a MOS test,
MOS numbers tend to vary significantly from test to test. As such, the
current wisdom suggests that MOS numbers should not be seen as absolute
values, but should be used for relative ranking of various coders used in a
given test. In the pursuit of speech quality measures that would not vary
much from test to test, an alternate method called MNRU has recently become
popular. In this approach, clean speech is degraded by incorporating known
amounts of "waveform distortion." These speech samples are evaluated in a
MOS test along with candidate coders. By comparing the MOS of the degraded
speech material and the candidate coded speech material, the equivalent
distortion level is determined for the candidate coder. Often, this
distortion is expressed in dB and is seen to be less variable from test to
test.

A drawback of MOS tests is that the listener is forced to rate all types of
signal degradations (such as sounding nasal, fluttering, muffled,
interrupted, thin, and rasping) as well as noise types (hissing, buzzing,
babbling, and rumbling) in a single measure. This can lead to a large
variance in the scores due to the variances in each listener's perceptual
sensitivity and training. Thus a test called the dynamic acceptability (DAM)
test was developed that allows listeners to score on several signal quality
and noise effects and then averages the results. DAM scores vary from 0 to
100, with "good" coders having a score of about 50. While this test does
provide a more scientific measure of speech quality, it is also more
involved to administer.

Although speech quality measurement strictly requires subjective tests such
as those described above, having quantitative measurements of speech quality
can be useful in the development of new algorithms, and in comparative
analysis of two different coders. Two of the more commonly used objective
measures are time domain segmental SNR and spectral distance measure. Both
these tests compare transcoded speech to the original speech and are not
absolute quality measures. The segmental SNR is computed by splitting the
signal into 20ms to 30ms segments, and SNR is computed for each segment,
then averaged. Segments with no speech are not used in calculations. The
spectral distance measures are based on the theory that human hearing is
more sensitive to the magnitude spectrum of the signal and less to the phase
spectrum. Thus time domain SNR measures can be misleading, and therefore
spectral measures have been developed. "

Multimedia Systems Design 02/98

Best Regards,

_________________________________
Reinaldo Penno Filho
Consultor  Tecnico
SAGA Sistemas e Computadores S.A.
tel: 55-21-5508506
e-mail :  [email-masked]

Universidade  Federal do Rio de Janeiro
Nucleo de Computacao Eletronica / Instituto de  Matematica
Grupo de Redes de Alta Velocidade
e-mail : [email-masked]

ACM Member
[email-masked]
________________________________

-----Original Message-----
From: [email-masked] <[email-masked]>
To: Noah, Matt <[email-masked]>
Cc: [email-masked]' <[email-masked]>;
[email-masked] <[email-masked]>
Date: 31 de Dezembro de 1998 09:23
Subject: RE:

>
>
>Hello,
>
>The most commonly accepted measure of Voice Quality associated
>with VoIP is the Mean Opinion Score. In this measurement system,
>POTS (regular phone) is considered a 4. G.723.1 comes in at 3.98.
>Anything above 3.5 is considered acceptable.
>
>More information can probably be found at sites of companies that
>license DSP code for encoding voice ... e.g. the DSP Group at
>http://www.dspg.com
>
>Best Regards,
>
>- Greg Boop
>   Mgr - VoIP Development
>   IBM - NHD
>   RTP, NC, USA
>   Phone: +1 919 254 0319
>   [email-masked]
>

---------
This message came from the IETF IPTEL Working Group Mailing List.