Subject: Re: [REMOTEUI] Widex charter proposal draft

From: Dave Raggett <[email-masked]>
Date: Thu, 1 Sep 2005 11:09:32 +0100 (BST)

This is a good start, but I agree with Eric about the first few
paragraphs and consider the protocol as being of broader use than 
the GUI widget scenario you describe. The terminology being used in 
the W3C Multimodal Interaction Working Group is that of "Modality 
Components" which provide a means to interact with users via one or 
more modalities, and which vary from simple to complex (e.g. an 
XHTML or VoiceXML web page). This is described in: 
http://www.w3.org/TR/mmi-arch/

How about the following:

With the Internet reaching out to more and more devices, people are 
increasingly expecting to have access to services at anytime, from 
anywhere and using any device. Such services are being developed 
using Web technologies such as XML and distributed across the 
network rather than resident on any one device.

An example is a service to access flight arrival times, where the 
user interface is expressed in XHTML and the application logic runs 
on a remote server. In this case, synchronization takes place in 
terms of a request/response protocol (HTTP) for Web pages. What
is currently lacking is a convenient means for finer grained 
synchronization that occurs in between page loads. This would allow 
the user interface to reflect changes in application state and offer 
greater flexibility for applications to respond to user input 
events.

This Working Group seeks to define a light weight synchronization
protocol for distributed user interfaces where the user interface
is represented in XML, and synchronization involves DOM events
and DOM mutation/update operations.

On Thu, 25 Aug 2005 [email-masked] wrote:

> Hi,
>
> Please find below the charter proposal for the WiDeX WG.
>
> Regards,
> Vlad
>
> ----------------------------------------------------------------
>
> Description of Working Group
>
> UI remoting is a technique whereby a user interface is rendered
> on other devices than those that run the application logic.
>
> A widget is an element of a user interface (UI) that displays 
> information or provides a specific way for a user to interact with 
> the operating system and application. Widgets may for instance 
> include icons, pull-down menus, buttons, selection boxes, progress 
> indicators, on-off checkmarks, scroll bars, windows, window edges 
> (that let you resize the window), toggle buttons, forms, and many 
> other devices for displaying information and for inviting, 
> accepting, and responding to user actions.
>
> Widget-level UI remoting provides a mechanism that enables a 
> client device to process widget descriptions received from a 
> server device, keeping the client UI synchronized with the 
> application logic.
>
> The WiDeX (Widget Description Exchange Service) WG will define a 
> standard mechanism that can be used in an IP-based network for 
> conveing initialization, update and eventing information about 
> user visible objects, i.e. widgets.
>
> The WG will strive to preserve an extensible architecture so that 
> the work possibly be useful in the future with other types of 
> descriptive user interfaces beyond those specifically considered 
> by the group.
>
> Specific topics that are NOT goals of this WG are:
>
>  - XML representation of user interface objects.
>  - Means to establish sessions and support for device 
>    coordination.
>
> The WiDeX service definition will define:
>
> - A mechanism for synchronizing distributed XML DOM objects by 
>   propagating DOM mutations/updates and DOM events.
> - A set of parameters that need to be negotiated by a service 
>   discovery and session setup mechanism in order to start the UI 
>   remoting session.
> - A framework that enables a widget-level UI remoting 
>   mechanism using distributed XML DOM synchronization and a 
>   service discovery and session setup mechanism as building 
>   blocks.
>
> Deliverables:
>
>  - Requirements document.
>  - Document specifying the stateless protocol for synchronizing 
>    XML DOM objects.
>  - Document specifying the framework for widget-level UI 
>    remoting.
>
> It is possible that work undertaken in other working groups and 
> even other standards bodies (i.e. W3C) will be referenced by this 
> working group. It is even possible that entire deliverables could 
> be satisfied by the work of other working groups (i.e. discovery 
> protocols). This working group will seek to maximize the use of 
> existing specifications where applicable.
>
> Goals and Milestones:
>
> TBD.
>
> Internet-Drafts:
>
> LRDP: The Lightweight Remote Display Protocol, 
> http://www.ietf.org/internet-drafts/draft-stirbu-lrdp-00.txt
>

- --
  Dave Raggett <[email-masked]>  W3C lead for multimodal interaction
  http://www.w3.org/People/Raggett +44 1225 866240 (or 867351)

_______________________________________________
REMOTEUI mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/remoteui

---

From: Dave Raggett <[email-masked]>
Date: Thu, 1 Sep 2005 15:42:10 +0100 (BST)

Some further questions.

To make the protocol more concrete we might want to bind it to
SIP and to mention this in the charter.

SIP is an obvious choice because of its use for establishing
sessions and for distributing events (represented in XML),
although the focus has hitherto been on presence information.
SIP is also widely deployed on desktop and mobile devices.
Web Services and the widespread deployment of HTTP make the
latter another candidate worthy of discussion.

We could also use the charter to constrain the work by mentioning
technical assumptions. Here are two that I have thought of:

First, when an event is raised in the user interface, the DOM is 
responsible for passing it to event handlers according to the
DOM event capture/bubble model. If an event is passed to a remote 
handler, the User interface should be able to pass the event to
the next handler without having to worry about whether or not the
remote handler calls the stopPropagation method. A similar concern
relates to the preventDefault method. I therefore suggest that
these two methods have no effect when called by a remote handler
of a DOM event. This ensure that events can be asynchronously
passed off to remote handlers.

Second, if the user interface wants to modify the document, it
is required to pass the request as an event to the document owner
which is then responsible for making the corresponding changes.
Note that the user interface may involve more than one document
and that these may have different owners. Moreover, this only
applies to documents that are under synchronization. Other
documents may be freely updated by default event handlers or
local scripts.

We may also want to constrain what we intend for the framework
for session initiation. For instance, in:

   http://whatwg.org/specs/web-apps/current-work/#server-sent

section 6.1.2 defines an extension to the DOM for a 
RemoteEventTarget interface that should be supported by any
object that implements the DOM EventTarget interface. The idea
is to use URIs to specify the means to subscribe to an event
or list of events and correspondingly to unsubscribe:

    // src is used for a URI in both cases
    interface RemoteEventTarget {
       void addEventSource(in DOMString src);
       void removeEventSource(in DOMString src);
    };

This is fine for sourcing events from a remote device, but 
inadequate for our purposes where the UI device needs to
send selected events to a sink on the application server,
and to receive DOM mutation/update operations from that
server. Is this information expressed in the UI markup and
acted upon by the markup language interpreter, or is it set
up via an out of band mechanism such as the SIP/SDP etc.
where the information is passed separately to the markup.

- --