Subject: RE: [Sipping] Application Interaction Requirements

From: "Fairlie-Cuninghame, Robert" <[email-masked]>
Date: Sat, 28 Sep 2002 02:59:11 -0700

David, 

Thank you for your comments. See inline responses.

> > R1:	For user activity indications, the framework must 
> provide some form
> > of indication of occurrence in time relative to one or more 
> media streams.
> >
> > I don't believe that this should be a requirement - IF you 
> need tight
> > media synchronization then transporting stimulus indications through
> > "out-of-band" mechanisms is not the correct solution for 
> your problem.
> 
> Uh, where do you leap from "indication of time relative to the media 
> stream" to "tight media synchronization". There are lots of 
> situations 
> which require the former but not the latter. Examples:
> - barge-in control in ASR systems
> - stream markers in TTS and other playout situations
> - speaker change annotations in recording systems

David, how do you expect us to get list discussion without making outlandish
claims? :) My statement "IF you need tight media synchronization then you
should be using RFC2833" still stands. However, you are right in saying that
the requirement does not imply *tight* media synchronization.

We are building the application interaction requirements and framework from
a set of usage cases. This application interaction framework is NOT designed
to solve every application problem in existence - indeed we are trying to
keep the framework simple whilst covering a reasonable set of usage cases.
Currently none of the considered usage case require loose or tight media
synchronization.

If you feel you can provide us with a concrete and compelling usage case
that requires loose media synchronization, please do so and we will consider
it.

> > R2:	For key-based indications, the framework must provide 
> the ability
> > for the network element requesting the indications to also 
> request that
> > the indications be suppressed from delivery to [all] other 
> end points on
> > the media path.
> >
> I frankly don't understand what this means. For one thing, it seems 
> inadvisable for one control element to be mandating what 
> other endpoints 
> see or don't see. If there is a requirement lurking here, 
> it's for each 
> participant to control what they see or don't see of the 
> control stream as 
> well as the media stream(s).

David in my email I provided a link where Eric originally raised and
described the requirement and its motivation. The requirement wording was
provided by Eric and its intent seemed reasonably clear to me, however
please refer to this email if you are still in doubt. 

> > There is first the conceptual question: should an 
> application be able to
> > modify the media streams without being a B2BUA.
> I'm not sure what this means, because I could modify a stream 
> and then 
> re-originate it with a different control protocol other than 
> SIP, in shich 
> case I'm modifyng the media stream but I'm not a B2BUA. The important 
> property is that an application not be permitted to modify a 
> stream without 
> being a UA. Otherwise it will run afoul of OPES sooner or later.

Ok, ok, B2BUA or UA - it's what I was meaning. My point is that I don't
think its a good idea for an application to have direct control over another
UA or gateway's media (as opposed to peer-to-peer negotiation). On the other
hand, we could write a protocol to do this; for instance, we could call it a
"Protocol for Controlling Gateway Media" (PCGM) but I think that some people
might realise that this just spells MGCP backwards. :)
 
> > Secondly you quoted a
> > usage case such as "* Transfer". The problems I see here is 
> that you need
> > to know what numbers the user will pressed before they 
> actually press
> > them in order to squelch them.
> Huh? There are lots of ways to eliminate control signals from 
> the media 
> path, assuming you know what's going on. I even have a patent on one 
> technique that avoids introducing extra delay into the media path.

And should it be a requirement that everybody implements it? The framework
will be extensible, however, we are talking are interested in core
requirements. Once again, if you have a compelling usage case that requires
this functionality that is not solvable in other ways, please present it.

I think there are better ways of solving this particular problem.

Regards,

Robert.

> > If you simply want you application to stop
> > all DTMF digits from being passed through (such as a 
> conference server)
> > then is should not have added RFC2833 to start with.
> >
> > So, I am inclined to say that these requirements are 
> out-of-scope for this
> > framework but we wish to open the issue to further discussion.
> >
> Since I frankly don't understand anything about the second 
> requirement, I 
> have no opinion (other than it had better get restated so we 
> can understand 
> it). My opinion on the first requirement is that we 
> definitely need this 
> (speechsc depends on it), but not with anything like the 
> interpretation you 
> have.
> 
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Steve Fisher" <[email-masked]>
Date: Sat, 28 Sep 2002 09:49:05 -0400

I would like to elaborate on Eric's example to help motivate "R2", the
ability for the network element requesting indications (such as an App
Server) to also be able to request that tones be suppressed from the
media stream.

Consider a "transfer connect" feature, in which one user on a call wants
to transfer the call to another device (e.g. from a cell phone to a work
phone), and does this by pressing "*T", followed by the phone number for
the new device. While it may be necessary to pass "*T" along to the
other user, once the user has pressed "*T" the "transfer connect"
application server knows that the next string of digits is intended for
that application, and the application server in that case would request
that the user's UA (which may be a gateway, SIP phone, etc.) suppress
the remaining digits.

In fact, this is exactly how AT&T's existing "Transfer Connect" service
works, which is typically used to transfer caller's from one call center
agent to another (you may have heard the two tones played when you call
an 800 number and are subsequently transferred. Note that you don't hear
the destination phone number, since those tones are suppressed).

Given that, as you said, "we are building the application interaction
requirements and framework from a set of usage cases", we hope that you
will agree that this is a reasonable usage case, since it reflects one
of the most popular "mid-call tone detection" usage cases in use today.

Thanks!
Steve

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Fairlie-Cuninghame, Robert
Sent: Saturday, September 28, 2002 5:59 AM
To: 'David R. Oran'; [email-masked]'
Cc: [email-masked]'
Subject: RE: [Sipping] Application Interaction Requirements

David, 

> - speaker change annotations in recording systems

David, how do you expect us to get list discussion without making
outlandish claims? :) My statement "IF you need tight media
synchronization then you should be using RFC2833" still stands. However,
you are right in saying that the requirement does not imply *tight*
media synchronization.

We are building the application interaction requirements and framework
from a set of usage cases. This application interaction framework is NOT
designed to solve every application problem in existence - indeed we are
trying to keep the framework simple whilst covering a reasonable set of
usage cases. Currently none of the considered usage case require loose
or tight media synchronization.

If you feel you can provide us with a concrete and compelling usage case
that requires loose media synchronization, please do so and we will
consider it.

David in my email I provided a link where Eric originally raised and
described the requirement and its motivation. The requirement wording
was provided by Eric and its intent seemed reasonably clear to me,
however please refer to this email if you are still in doubt. 

Ok, ok, B2BUA or UA - it's what I was meaning. My point is that I don't
think its a good idea for an application to have direct control over
another UA or gateway's media (as opposed to peer-to-peer negotiation).
On the other hand, we could write a protocol to do this; for instance,
we could call it a "Protocol for Controlling Gateway Media" (PCGM) but I
think that some people might realise that this just spells MGCP
backwards. :)
 
> technique that avoids introducing extra delay into the media path.

And should it be a requirement that everybody implements it? The
framework will be extensible, however, we are talking are interested in
core requirements. Once again, if you have a compelling usage case that
requires this functionality that is not solvable in other ways, please
present it.

Regards,

Robert.

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

---

From: Jonathan Rosenberg <[email-masked]>
Date: Sat, 28 Sep 2002 12:02:16 -0400

Ultimately, the problem here is the determination of "focus".

Focus is the selection of an application with which the user is 
currently interacting. The proper way to solve this is to provide the 
user with some kind of user interface by which they can select a focus 
in which their stimulus is interpreted. As an example, a web-based 
calling card application would provide the user with an html form that 
they can use to enter their credit card and pin. The calling card app 
gets the focus when the user selects that window. It is clear that the 
stimulus is targeted at that specific application. The client would 
provide its form post directly to the credit card app. There is not even 
a hint of a feature interaction problem here.

As another example, consider a phone with buttons on a display. A 
transfer connect app might "own" one of those buttons, such that its 
label is "transfer connect", and once its pressed, the user is told to 
enter the target number and press pound. Once they do that, the 
resulting number is sent ONLY to the transfer connect app, since it had 
the focus for the input.

Now, our problem is that the legacy telephone network has (by todays 
standards) a horrible user interface. One of the problems is that when 
there are multiple applications running, there is no explicit way that 
the user can provide focus. For example, if a voice recorder app is 
running while I am speaking to an answering machine, there are two apps, 
each of which is expecting DTMF stimulus. When I press #, who receives 
the pound? Generally speaking, how is focus determined, and by whom?

It is my view that focus can ONLY be determined by the element which 
knows about all the applications which are requesting user input. Thats 
because the focus will need to determined by looking at all of the 
stimulus inputs that have been requested, so as to determine which 
one(s) should and should not receive it. That is the client, NOT an 
individual application. Focus determination is **purely a client 
issue**. Applications should never have any visibility or control of it.

So, how does the client determine focus? If we generalize Steve's 
transfer-connect application, the idea is that each application, 
presumably, has some kind of digit map (in mgcp-speak) which determiens 
the stimulus they are looking for. For the transfer connect app, its 
[*Txxxxxxxxxx]. So, if we assume that there is a set of such digit maps, 
Steve's proposal is that focus is determined by a matching process to 
the digit maps of each application. The one whose digit map matches 
first, receives the focus. As a policy, this clearly has problems - 
multiple digit maps may match, and the matching may require too much input.

So, I am not convinced that this is the right policy. I'm on the fence 
about whether we even need to specify the policy. Dave, for example, 
might like to apply his patented technique to his phones. I might like 
to build phones with the tabbed display approach I describe below. Can't 
he do what he wants, and I, what I want? Why do the applications care?

We may want to recommend or describe a baseline algorithm, though. As 
such, some requirements for it would make sense. Here are the ones I 
would propose:

R-JR-1: The framework must specify a way to determine focus in a client 
when the client cannot explicitly determine focus

R-JR-2: The focus determination algorithm should attempt to minimze the 
delay in providing stimulus to applications

R-JR-3: The focus determination algorithm should attempt to minimize 
false positives, that is, it should aim to provide stimulus to a single 
application at a time

The recipient of the media streams can be considered an application for 
purposes of these requirements.

I don't know what the right algorithm is, and I think we need to work on 
it. I am sure that some of the existing work on feature interaction will 
be helpful here.

Let me make an additional comment on R-JR-1. The problem at hand here is 
strictly a CLIENT problem. An application can still expect pure DTMF 
input, yet the client can provide focus. One can imagine a client 
display with tabs for each app. When you select a tab, any keypad input 
goes only to that app. In this case, the client does have a means of 
explicitly determining focus. Thus, the problem Steve is having is a 
client problem, not an application problem. It therefore cannot be 
solved by an application telling the client to suppress input. The 
client must determine which app(s) to send it to based on information 
from the apps.

Finally, I would like to emphasize that I believe the aim of the 
application interaction work is NOT to design for these dumb phones, but 
rather, to design for the smart ones, with UI capabilities, and deal 
with the dumb ones as a special case. I would prefer not to profilerate 
these awful problems into the future.

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 28 Sep 2002 17:25:48 -0400

I'm not a UI guy, but this is apparently a very old UI problem. Indeed, 
the primary job of a windowing system is to manage keyboard and mouse 
focus. The only exception are "modal" inputs, such as pop-up 
confirmation boxes, where an application can grab focus from the current 
application. From all I've heard, modal input is considered a very bad 
idea except in very limited circumstances ("Your PC is about to explode. 
Shut down?").

Windowing systems use many different means of changing focus, from 
Alt-TAB to mouse-over-window to click-on-window. Many systems, such as 
the X-based window managers, make this a user-configurable option. 
Emphasizing what Jonathan said: all modern systems explicitly leave this 
to the windowing system, rather than the application. Anything else 
makes the system too unpredictable. Adding another application should 
not break what used to work before. I believe that's a fundamental 
principle of system design and rules out most matching algorithms except 
those where false switches are extremely unlikely (e.g., using 
wordspotting with a strange word that's unlikely to occur in normal 
conversation is probably ok.)

I'm not up-to-date on audio operating systems, but I suspect that some 
of the speech interfaces for the blind have to deal with the same 
issues. Systems like Wildfire use word spotting to detect when they are 
being addressed, using the keyword to switch focus. The "windowing 
system" does not necessarily have to reside on the end system, as in the 
example of Wildfire.

---

From: "Steve Fisher" <[email-masked]>
Date: Sun, 29 Sep 2002 10:03:31 -0400

Jonathan,

I agree with everything you say, except your conclusions... In my view,
our work will be most successful if it enables services that help people
migrate into the world of rich devices that provide the most compelling
user experience, and that the most effective approach to achieve this
goal is to provide access to these services from all devices, even those
that connect to the legacy telephone network. We would hope that, over
time, people would increasingly adopt the new devices and abandon the
legacy devices (and networks). In fact, this is why I'm such a big fan
of your "markup" draft -- it provides a framework for applications to
support rich devices while maintaining "backwards compatibility" with
the touch tone keypad.

I also agree that determining focus is a client problem, and I believe
the requirement that Eric proposed is simply a mechanism to enable an
application server to dynamically "register" its application with the
client, and to enable the client to provide the richest user experience
it is capable of. In some cases that client, unfortunately, will have no
inherent interface for determining application focus, and thus requires
assistance to mediate among the various applications available to that
device (after all, one of the most exciting aspects of a SIP network is
the relative ease with which new capabilties are introduced in the form
of application servers).

Steve

-----Original Message-----
From: Jonathan Rosenberg [mailto:[email-masked]] 
Sent: Saturday, September 28, 2002 12:02 PM
To: Steve Fisher
Cc: 'Fairlie-Cuninghame, Robert'; 'David R. Oran';
[email-masked]; [email-masked]
Subject: Re: [Sipping] Application Interaction Requirements

Steve's proposal is that focus is determined by a matching process to 
the digit maps of each application. The one whose digit map matches 
first, receives the focus. As a policy, this clearly has problems - 
multiple digit maps may match, and the matching may require too much
input.

be helpful here.

-Jonathan R.

> Consider a "transfer connect" feature, in which one user on a call 
> wants to transfer the call to another device (e.g. from a cell phone 
> to a work phone), and does this by pressing "*T", followed by the 
> phone number for the new device. While it may be necessary to pass 
> "*T" along to the other user, once the user has pressed "*T" the 
> "transfer connect" application server knows that the next string of 
> digits is intended for that application, and the application server in

> that case would request that the user's UA (which may be a gateway, 
> SIP phone, etc.) suppress the remaining digits.
> 
> In fact, this is exactly how AT&T's existing "Transfer Connect" 
> service works, which is typically used to transfer caller's from one 
> call center agent to another (you may have heard the two tones played 
> when you call an 800 number and are subsequently transferred. Note 
> that you don't hear the destination phone number, since those tones 
> are suppressed).
> 
> requirements and framework from a set of usage cases", we hope that 
> you will agree that this is a reasonable usage case, since it reflects

> one of the most popular "mid-call tone detection" usage cases in use 
> today.
> 
> Thanks!
> Steve
> 

>>stream" to "tight media synchronization". There are lots of situations

---

From: Henry Sinnreich <[email-masked]>
Date: Mon, 30 Sep 2002 09:09:29 -0500

>In some cases that client, unfortunately, 
>will have no inherent interface for determining application focus,

Yes, we will, just think of a PC or PDA GUI.

Henry

> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of Steve Fisher
> Sent: Sunday, September 29, 2002 9:04 AM
> To: 'Jonathan Rosenberg'
> I agree with everything you say, except your conclusions... 
> In my view, our work will be most successful if it enables 
> services that help people migrate into the world of rich 
> devices that provide the most compelling user experience, and 
> that the most effective approach to achieve this goal is to 
> provide access to these services from all devices, even those 
> that connect to the legacy telephone network. We would hope 
> that, over time, people would increasingly adopt the new 
> devices and abandon the legacy devices (and networks). In 
> fact, this is why I'm such a big fan of your "markup" draft 
> -- it provides a framework for applications to support rich 
> devices while maintaining "backwards compatibility" with the 
> touch tone keypad.
> 
> I also agree that determining focus is a client problem, and 
> I believe the requirement that Eric proposed is simply a 
> mechanism to enable an application server to dynamically 
> "register" its application with the client, and to enable the 
> client to provide the richest user experience it is capable 
> of. In some cases that client, unfortunately, will have no 
> inherent interface for determining application focus, and 
> thus requires assistance to mediate among the various 
> applications available to that device (after all, one of the 
> most exciting aspects of a SIP network is the relative ease 
> with which new capabilties are introduced in the form of 
> application servers).
> 
> Steve
> 
> > From: [email-masked] 
> [mailto:[email-masked]] On Behalf
> 
> >>inadvisable for one control element to be mandating what other 
> >>endpoints see or don't see. If there is a requirement lurking here,
> >>it's for each 
> > we could call it a "Protocol for Controlling Gateway Media" 
> (PCGM) but
> I

---

From: Eric Cheung <[email-masked]>
Date: Mon, 30 Sep 2002 12:04:19 -0400

Hi Jonathan

Thanks for the detailed discussion of the "focus" approach and I agree
that this is a good way to solve the feature interaction where multiple
applications need to receive their own separate inputs from the user via
a single client user interface.

I would be happy with R2 being replaced by R-JR-1, 2, & 3, except that
R-JR-3 weakens the requirement from "must" to "should".  It seems to me
that it is important that user inputs only go to the application that
requests it.

I would propose the following rewrite:

R-JR-3': The focus determination algorithm must not produce false
positives, that is, it must provide a stimulus only to the single
application that is in focus at the time.  This may involve suppressing
DTMF tones from the audio stream when the media endpoint is not in
focus.

Thanks
Eric