Subject: [Sipping] App Interact Reqs discussion [Was: ID-ACTION: draft-culpepper-... ]

From: "Fairlie-Cuninghame, Robert" <[email-masked]>
Date: Sun, 28 Apr 2002 05:10:44 -0700

Hi Jonathan,

Responses inline [two emails combined.]

> > > > R5:  A network entity desiring user indications must be able to
> > > >         request user indications from another network 
> entity.  The
> > > >         entity receiving a request must be able to 
> respond with its
> > > >         capability/intent to transmit user indications.
> > >
> > > We should be more specific here. We have a caller and a
> > > callee. I think
> > > that by "entity" you mean one of those two.
> > 
> > No, we are purposefully not restricting the mechanism to 
> only caller and
> > callee (or even proxy). So, yes, I agree we need to re-word to
> > explicitly
> > define. Further discussion later.
> 
> That doesn't make sense to me. By definition, user input indication is
> possible only from a human, NOT from an automata. An automata can try
> (web screen scraping...) but that is a known bad thing.
> 

Admittedly, yes normally one network entity will normally be an application
and the other will normally be a user but I do not see that this should be a
requirement. Sure, it is not feasible for automatas to analyze voice prompts
or HTML presentations but who's to say that people will not invent
automata-friendly presentation (and stimulus) methods. Why add this
artificial restriction?

I am however fine with adding the restriction that at least *one* of the
network entities must be a UA of the associated session (given that we are
now saying that all interaction MUST be session associated). It means that a
proxy or co-operating device cannot act on a user's behalf but I can live
with that restriction.

> > > > R15: The receiving application must be able to detect 
> user activity
> > > >         indication loss due to packet loss from received
> > > user activity
> > > >         indications.
> > >
> > > I don't know what this means.
> > 
> > Detecting missing indications.  For example, a PIN is 
> transmitted one
> > digit
> > at a time.  If one message (network packet) gets lost, we want to be
> > able to
> > determine this at the receiver. Will re-word.
> 
> There is an implicit assumption here of UDP based notifications and/or
> notifications that are not stop-and-wait. I think the real requirement
> you want here is ordering - events are delivered to the 
> observer in the
> same order they occurred at the reporter.
> 

Yes I came to same conclusion after reading your draft. 
 
> > > * The mechanism should support devices that possess a UI, 
> and those
> > > which do not possess a UI
> > 
> > Jonathan can you please explain exactly what you mean by this. My
> > feeling is
> > that this is the crux of your new requirements, yet I don't 
> have a clear
> > idea of what you mean by "The mechanism should support 
> devices ... which
> > do
> > not possess a UI."
> 
> Hopefully the recent draft on markup I just submitted explains better.
> The basic idea is that there are many devices that have the ability to
> interact with the user. There could, for example, be a display with
> buttons. The display would present the user with choices, and the user
> would press a button, causing a notification to be sent into the
> network. This is in contrast with devices, like a traditional 
> dumb black
> phone, which have no ability to interact with the user. There is no
> display. There is no way to know, when a user presses a DTMF key, what
> application they were trying to interact with. 
> 

I disagree here but I believe this is partially due to the need to
standardize terms.

By "device with no user interface" it seems like you are referring to
devices possessing no physical, visual, presentation-based UI components. 

In my mind, a dump phone _does_ possess a user interface (speaker,
microphone & dialpad) and does even possess a UI component capable of
presentation - the speaker - its just that presentation is audio not visual.
A user-interface for me refers to all components: input or presentation,
session-initiated or external markup (or other).

I also believe we need to define a "virtual user interface" more clearly.
For me this represents the notional user interface consisting of all user
interface components (presentation- or input-based) of co-operating
applications that *explicitly* desire to be on the same logical user
interface. 

So now non-co-operating applications will refer to separate virtual user
interfaces but it is up to the device to decide how to map those virtual
user interfaces onto the physical interface. This gets down to an device
design issue raised later.

Additionally I feel, just because a device does not support visual,
presentation-based application interaction, this does not preclude the
ability to select and identify different virtual user interfaces. These are
simply separable issues. [The device may have display but perhaps it is not
available for application use.] 

> > > * The mechanism should support user interfaces which require a new
> > > content to be presented to the user upon input from the 
> user (i.e.,
> > > change the screen when a user presses a button)
> > >
> > 
> > Hmm, this goes beyond "getting indications a device supports" and is
> > more to
> > do with "application action in response to input". I'm not 
> sure it is in
> > scope.
> 
> I think it has to be - that's the whole problem! I think we've been
> thinking far too narrowly on this problem. Its not just about
> notifications of user input. We are talking about supporting stimulus
> signaling for applications, where stimulus signaling is defined in the
> broadest sense. 
> 

Okay I've done a U-turn on this one. I agree with you that requirements need
to address application interaction as a whole and after more thought I think
I would actually reword and strengthen these requirements.

* The mechanism must allow user interface components created through this
mechanism to be updated and terminated by the creating application.

[Remove any reference to "in response to stimulus input"]

> > > * The mechanism should allow the user to interface with 
> multiple user
> > > interfaces for different applications within the same call
> > > (for example,
> > > a single call may involve a call recording application 
> and a prepaid
> > > calling application, both of which might support a UI)
> > 
> > Good one, although I'm not sure how this is a requirement on the
> > mechanism
> > rather than simply how the user locally selects their user interface
> > (which
> > would be a requirement on how to design a user device, i.e., not in
> > scope).
> > Each relationship between an application and a user 
> interface would be a
> > separate subscription.
> 
> No. We have a fundamental disconnect here. There are two radically
> different approaches for interacting with applications. In approach 1,
> there are a set of applications listening for user input. The 
> user input
> is not directed; that is, the user has no ability to say 
> "this input is
> for app 1" or "this input is for app 2". Directing user input 
> generally
> requires a user interface of sorts, and in approach 1, there is no UI.
> In approach 2, input can be directed. The device has a user interface
> that allows the user to send feedback to specific applications. 
> 
> Let me give concrete examples. A black phone is an example of approach
> 1. Multiple apps can listen for DTMF, but the user can't 
> direct DTMF to
> a specific application. This, BTW, is the source of countless feature
> interaction problems, and is an artifact of the legacy 
> interface of the
> phone. An example of approach 2 is a PC. Lets say I make a 
> call from my
> PC, and there are two applications involved in processing 
> that call. One
> is a prepaid calling app. One of the things this app allows me, the
> user, to do, is to hang up the call and make another without 
> re-entering
> my PIN. The other application is a voice recorder app, which allows me
> to record the audio of the call. This app needs my input to start and
> stop the recording. Once I make the call, two windows pop up. One is a
> control panel for the voice recorder (with start and stop buttons) and
> another is a control panel for the prepaid app. When I click "start
> recording" on the control panel for the voice recorder, this 
> starts the
> recording process. That happens through user input 
> indications, but they
> are directed towards the voice recorder app. They can be 
> directed since
> they occur within a context - a user interface - which allows 
> them to go
> to the right place. Approach 2 has MUCH better feature interaction
> properties, and a much more compelling user experience.
> 
> Approach 1 is a subset of approach 2. We should be seeking to solve
> approach 2, and a result, knock off approach 1. The worst thing we can
> do is continue to develop specifications that proliferate the broken
> mode of user interface which the PSTN has provided us! We 
> should support
> it, but it is not the design goal. 

Ok I agree then - we add a requirement that Approach 2 is possible.
However, we also then need to add a requirement that Approach 1 is possible,
that is, 2 (co-operating) applications can share the same virtual user
interface (which I disagree is inherently a subset of Approach 2). 
 
> > > * The mechanism must allow a user to have assurances that the
> > > user input
> > > it is providing is only to applications/servers on the call
> > > path between
> > > caller and callee.
> > 
> > I think this is covered by R16 & R17:
> > 
> > R16:    The mechanism must allow for end-to-end 
> security/privacy between
> > source and destination.
> > 
> > R17:    Both entities must be able to authenticate each other.
> 
> No - these are not the same at all. I don't know what end-to-end
> security is. Security encapsulates a lot of functions. I am 
> saying that
> the mechanism should support a specific authorization model - that the
> system can provide guarantees that only servers on the call 
> path can ask
> for my user input. 
> 

I disagree. There should be no requirement to restrict application servers
to entities on the call path of a particular dialog. What about when there
are multiple co-operating application modules/entities - must they all exist
on the same server? This goes against the distributed model and ties
"sessions" to dialogs - something that we've agreed is bad. 

Besides, how do you authenticate a downstream proxy on the call path any
better than you can authorize any other network entity?

Responses to the other email ...
 
> >  I've given some more thought to a couple of your requirements:
> > 
> > > * The mechanism should allow a device to indicate support 
> for multiple
> > > user interfaces, and allow the application to select which
> > > one is to be
> > > used
> > >
> > > * The mechanism should allow a device to indicate 
> relative preferences
> > > amongst the various user interfaces
> > >
> > 
> > I don't think your requirement adequately answers the 
> questions: what is
> > a
> > "user interface" and what does it mean to have "multiple user
> > interfaces"?
> > Assumedly a device normally has only one *physical* interface (be it
> > keys or
> > display or whatever). Now we all agree that the mechanism 
> must not force
> > the
> > device's interface to be "taken over" by an application.
> > 
> > One logical way to define multiple user interfaces or 
> rather "virtual
> > user
> > interfaces" is to create a virtual instance of the physical instance
> > that is
> > active when the application/session is in focus to the user 
> (much in the
> > way
> > a multiline SIP phone may work). 
> 
> THis seems reasonable. 
> 
> > There are other approaches but most of
> > the
> > ones I can think of are diminuitive cases of this general 
> approach. In
> > any
> > case, how this interface is selected or created will 
> largely depend on
> > the
> > user device and not this mechanism.
> 
> There is a subtle problem here. You indicated that the user 
> needs to put
> a particular virtual UI into focus. Doing that is only 
> possible if there
> is some display to the user that allows them to know which one it is
> they need. This means there has to be a way for the application to
> provide some kind of display to the user. THus, it is an issue for the
> mechanism. The mechanism has to have a way to provide a display to the
> user.
 
Not at all. The mechanism must simply allow a user interface component to be
associated with the application. [Or a virtual user interface with the set
of participating (co-operating) applications]. We cannot dictate that the
device must support some mechanism to display that identification or must
support multiple switchable user interfaces - these are device design
decisions or capability limitations. We can only require that it is
*possible* to identify the application that created the UI component and
that it is possible to identify components within the same virtual user
interface.

Once again, how a device display, maps or switches between virtual user
interfaces is, I contend, largely a device design decision - the requirement
is that the required information is available.

Perhaps I am missing your point?

> > 
> > It is my contention therefore, that it should not be a 
> requirement on
> > the
> > network application interaction mechanism to present multiple
> > (prioritized)
> > permutations or instances of user interfaces to an 
> application. Rather,
> > the
> > relevant requirement should be:
> > 
> > * The mechanism should support the identification of a requesting
> > application identity [e.g. application name+version @ 
> creator's domain]
> > and
> > application instance [e.g. instance@ service perovider's domain].
> 
> OK.
> 
> > * The mechanism should the ability for a user interface 
> instance to be
> > shared between subscriptions (and probably applications 
> when multiple
> > applications are co-operating).
> 
> In my model, there is a one-to-one mapping of "subscriptions" to
> applications.

I disagree. Your model does not create subscriptions or sessions at all. The
markups exist in their own little world. I think this is one area that needs
to be addressed. 
 
> > 
> > This allows a device to create and offer virtual user 
> interfaces as it
> > sees
> > fit. The application always operates on the "virtual" user interface
> > instance. This is also a MUCH, MUCH simpler problem - merely one of
> > identification - not presentation or negotiation (this is a highly
> > complicated area already). [This approach also allows the device to
> > store
> > customized application-specific interfaces that can be recalled as
> > required
> > whenever interacting with the same application.]
> 
> No - how can the phone create custom UI for an app when it 
> doesn't know
> what the app is ahead of time? The app presents the user interface. In
> the case of a phone with buttons, that means the app provides labels
> that should be associated with some of the buttons. In the case of a
> voice recording app, there could be one button that is labelled "start
> recording" and another "stop recording". The app needs a way 
> to tell the
> phone "label one button with 'start recording' and another with 'stop
> recording'. Tell me when one of these is pressed, and tell me which".
> The phone can decide whether to use the two top right buttons or the
> bottom left buttons; I agree on that. But, the key idea is 
> that the app
> is telling the phone information it must render to the user 
> in order to
> provide context for the button.
 
Okay now we are getting to a different discussion - how to do key-based
input. I'm sure we can fill another email with this discussion alone (and
this email is long enough already).

Anyway, I think we are thrashing out a few key model assumptions in building
these requirements - which is a very good thing. 

Regards,

Robert.

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Bert Culpepper" <[email-masked]>
Date: Tue, 30 Apr 2002 22:02:53 -0400

Hi all,

Seeing that "Key Events/Stimulus Markup" has been added to the Interim
Meeting's agenda, Robert and I have modified the requirements I-D.  The
document has been significantly updated since the last revision (based on
list comment so far).  Hopefully this version will be more useful during the
meeting than the previous version.

I've submitted the I-D, until it appears in the archives a copy can be
gotten from
http://home.cfl.rr.com/bculpepper/draft-culpepper-sipping-app-interact-reqs-
01.txt.

Comments, suggestions, etc. are very welcome.

Regards,
Bert