Subject: RE: [speechsc] VoiceXML Compatibility Issues with Synthesis

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Wed, 7 Dec 2005 22:39:10 -0800

________________________________

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of Andrew Wahbe
	Sent: Wednesday, December 07, 2005 1:51 PM
	To: IETF SPEECHSC (E-mail)
	Subject: [speechsc] VoiceXML Compatibility Issues with Synthesis

	The VoiceXML Forum MRCP Liaison committee is currently
evaluating the latest MRCP v2 draft to (a) evaluate the compatibility
between MRCP v2 and VoiceXML and (b) generate test assertions for MRCP
v2 based VoiceXML browsers and MRCP v2 based media resources. We are
currently examining the Speech Synthesis portion of the specification
and have found the following issues:

	1.	Both the Fetch Hint header (8.4.10) and Audio Fetch Hint
header (8.4.11) say that they apply to audio files. Which one is it?
These descriptions should be specific about what resource types they
apply to.
		[Sarvi>>] Fetchj-Hint applies for all documents
including audio files. If the Audio Fetch Hint is specified it applies
specifically for audio files only. 
		
	2.	Audio Fetch Hint header (8.4.11) supports a "stream"
value that is not supported by VoiceXML 2.x. Moreover, this value says
nothing about when to fetch the audio file; it determines when it should
start playing. You could have a "safe" fetch that was either streaming
or non-streaming. The same applies to a "prefetched" audio file.  A
separate header should control streaming if required. It's worth noting
that VoiceXML leaves streaming as a platform-specific optimization. 
		[Sarvi>>] I think this should be fine. "safe" and
"pre-fetched" should cover the VoiceXML needs. This is just indicate to
the Media server if it should fecth SSML documents or audio fiels ahead
of time or just in time before play back. The streaming model, tells the
media server to stream the audio as it fetches the URI and not wait for
the audio file to finish downloading into the media server. Which is
anothe rpossible option the VoiceXML allows for. 
	3.	The maxage and maxstale cache control headers are global
in MRCP while VoiceXML breaks this down by resource type (e.g.
audiomaxage, audiomaxstale, grammarmaxage, grammarmaxstale, etc.). This
results in compatibility issues between VoiceXML and MRCP. Is there a
way to apply these parameters to specific resource types in MRCP?
		[Sarvi>>] I'd like to hear some other opinions on this.
There are different times of URI that will be neeeded to be
loaded/cached. This includes audio files, grammar files, SSML,
voice-prints etc. I don't believe we want to have separate headers for
each. The best I might suggest is to leave the current Cache parameters
to covere all types of documents including Audio, SSML, Grammar etc.
Then if possible implement separate audio-maxstale etc which if
specified apply to audio files only.
		
	4.	It isn't clear if the  "003 uri-failure" SPEAK
completion cause applies to referenced SSML files, or audio files or
both. If it applies to audio files then this is in conflict with the
VoiceXML/SSML behavior of 
		[Sarvi>>]  

		003 uri-failure was meant for cases where the SSML was
fetched through a URI and it failed to fetch it. So we should we leave
this alone.

		 "continuing processing if audio cannot be successfully
rendered. The SSML Recommendation says "If the audio
<http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/#edef_audio>
element is not successfully rendered, a synthesis processor
<http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/#term-processor
>  should continue processing and should notify the hosting
environment." To truly meet this requirement it would seem that we need
some way to indicate that audio was not played while continuing to play
the other audio and SSML specified in the speak request. A
completion-cause of 003 could perhaps be used to indicate this (if we
allow the notification of the host environment to occur after the
request completes), though the completion cause is documented as
representing the "reason of request completion" which wouldn't be
accurate in this case. This definitely needs some clarification. 
		[Sarvi>>]  

		This definitely needs some clarification We might want
add a new "success-partial" completion cause code that says the SSML
commpleted but with some audio failures. 

		An alternate more extensive suggestion is to use create
AUDIO-EXCEPTION event similar to SPEECH-MARKER event to be generated
when such audio elements fail to loaded.

		Or we could use the SPEECH-MARKER event or rename to
something generic and differentiate based on headers.
		 
		Sarvi  

	If you would like to participate in the VoiceXML Forum MRCP
Liaison committee please email me.
	Thanks,
	
	Andrew Wahbe

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: Andrew Wahbe <[email-masked]>
Date: Thu, 08 Dec 2005 13:53:30 -0500

On #2: My concern was that if "stream" is a possible value for audio 
fetch hint, then streaming won't be done when you specify "safe" or 
"prefetch". i.e. It would seem that "safe" and "prefetch" imply that 
streaming should not be done. However, these are two separate concerns:
- safe and prefetch determine when the audio should be _fetched_
- stream determines when the audio should be _played_

So imagine for a second that Audio Fetch Hint only had two possible 
values: safe and prefetch; and we had another header called Audio Play 
Hint that had two possible values: "stream" and "no-stream". This gives 
us 4 scenarios:

1) safe and no-stream: only start downloading when referenced and wait 
for it be completely downloaded before starting playback.
2) safe and stream: only start downloading when referenced and stream it 
(ie. play it as it is downloading)
3) prefetch and no-stream: allow audio downloading to start "early" and 
wait for dowloading to be complete before starting playback
4) prefetch and stream:  allow audio downloading to start "early" and 
stream it

Scenario 4 may seem strange at first glance but note that even if you 
started fetching early you may not have all the audio dowloaded when it 
comes time to start playback. So the issue was that if audio fetch hint 
mixed the 2 concerns into 3 values then they would have the following 
semantics:

"safe" would be scenario 1? Or is it 2?
"prefetch" would be scenario 3
"stream" would be scenario 2? Or is it 4?

Another concern was that if a voiceXML page sets its audio fetch hint to 
"safe" or "prefetch" (and the values are simply passed through to MRCP) 
then streaming is forbidden even though a VoiceXML browser is allowed to 
stream as an optimization.

Now having said all that, I just checked the VoiceXML spec and it reads 
as follows:

/Note that it is a platform optimization to stream audio: i.e. the 
platform may begin processing audio content as it arrives and not to 
wait for full retrieval. The "prefetch" fetchhint can be used to request 
full audio retrieval prior to playback.
/
(Also note that "stream" was a valid audio fetch hint value in earlier 
versions of VoiceXML 2 but was removed.)

So it seems VoiceXML assumes that if you prefetch then you aren't going 
to stream. So it would seem that the following would work:
If you specifiy prefetch in VXML then specify prefetch in MRCP
If yo specify safe in VXML and the platform decides not to stream then 
specify safe in MRCP
If you specify safe in VXML and the platform decides to stream then 
specify stream in MRCP

While perhaps a bit confusing I guess this does work. Sorry for the long 
email to just agree with you but I thought it would be valuable to think 
this through step by step.

Andrew

Shanmugham, Saravanan wrote:

>  
>
>     ------------------------------------------------------------------------
>     *From:* [email-masked]
>     [mailto:[email-masked]] *On Behalf Of *Andrew Wahbe
>     *Sent:* Wednesday, December 07, 2005 1:51 PM
>     *To:* IETF SPEECHSC (E-mail)
>     *Subject:* [speechsc] VoiceXML Compatibility Issues with Synthesis
>
>     The VoiceXML Forum MRCP Liaison committee is currently evaluating
>     the latest MRCP v2 draft to (a) evaluate the compatibility between
>     MRCP v2 and VoiceXML and (b) generate test assertions for MRCP v2
>     based VoiceXML browsers and MRCP v2 based media resources. We are
>     currently examining the Speech Synthesis portion of the
>     specification and have found the following issues:
>
>           header (8.4.11) say that they apply to audio files. Which
>           one is it? These descriptions should be specific about what
>           resource types they apply to.
>           [Sarvi>>] Fetchj-Hint applies for all documents including
>           audio files. If the Audio Fetch Hint is specified it applies
>        2. Audio Fetch Hint header (8.4.11) supports a "stream" value
>           that is not supported by VoiceXML 2.x. Moreover, this value
>           says nothing about when to fetch the audio file; it
>           determines when it should start playing. You could have a
>           "safe" fetch that was either streaming or non-streaming. The
>           same applies to a "prefetched" audio file.  A separate
>           header should control streaming if required. It's worth
>           noting that VoiceXML leaves streaming as a platform-specific
>           optimization.
>           "pre-fetched" should cover the VoiceXML needs. This is just
>           indicate to the Media server if it should fecth SSML
>           documents or audio fiels ahead of time or just in time
>           before play back. The streaming model, tells the media
>           server to stream the audio as it fetches the URI and not
>           wait for the audio file to finish downloading into the media
>           server. Which is anothe rpossible option the VoiceXML allows
>           for.
>        3. The maxage and maxstale cache control headers are global in
>           MRCP while VoiceXML breaks this down by resource type (e.g.
>           audiomaxage, audiomaxstale, grammarmaxage, grammarmaxstale,
>           etc.). This results in compatibility issues between VoiceXML
>           and MRCP. Is there a way to apply these parameters to
>           specific resource types in MRCP?
>           loaded/cached. This includes audio files, grammar files,
>           SSML, voice-prints etc. I don't believe we want to have
>           separate headers for each. The best I might suggest is to
>           leave the current Cache parameters to covere all types of
>           documents including Audio, SSML, Grammar etc. Then if
>           possible implement separate audio-maxstale etc which if
>        4. It isn't clear if the  "003 uri-failure" SPEAK completion
>           cause applies to referenced SSML files, or audio files or
>           both. If it applies to audio files then this is in conflict
>           with the VoiceXML/SSML behavior of
>           [Sarvi>>] 
>
>           fetched through a URI and it failed to fetch it. So we
>           should we leave this alone.
>
>           rendered. The SSML Recommendation says "/If the audio
>           <http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/#term-processor>
>           environment./" To truly meet this requirement it would seem
>           that we need some way to indicate that audio was not played
>           while continuing to play the other audio and SSML specified
>           in the speak request. A completion-cause of 003 could
>           perhaps be used to indicate this (if we allow the
>           notification of the host environment to occur after the
>           request completes), though the completion cause is
>           documented as representing the "reason of request
>           completion" which wouldn't be accurate in this case. This
>           definitely needs some clarification.
>           [Sarvi>>] 
>
>           This definitely needs some clarification We might want add a
>           new "success-partial" completion cause code that says the
>           SSML commpleted but with some audio failures.
>
>           AUDIO-EXCEPTION event similar to SPEECH-MARKER event to be
>           generated when such audio elements fail to loaded.
>
>------------------------------------------------------------------------
>