Subject: PP8: Asynchronous change notification for HTTP-based resources

From: Lisa Dusseault <[email-masked]>
Date: Fri, 18 Jan 2008 11:00:57 -0800

Asynchronous change notification for HTTP-based resources
Rohan Mahy

see also: http://svn.resiprocate.org/rep/ietf-drafts/rohan/web-notify/ 
draft-mahy-http-change-notification-via-sip-00.html

Many applications are interested in timely notification about changes
of HTTP-based resources. For example:
- appointments changing in a set of CalDAV calendars
- web-based news or blog-articles updated (the typical domain of Atom  
and RSS)
- my photo directory changing
- a subdirectory changing in a large software project's source code  
repository
- a change to the state of an Internet Draft in the tracker.

Current practices in change notification involve polling individual
HTTP resources, or polling some kind of HTTP "catalog" resource to see
what resources have changed. Both of these methods inherently suck.
The "catalog" approach at least substantially reduces the number of
resources that need to be polled, but the amount of time to wait
between polls is always one-sided compromise and rarely optimal for
response time, network usage, and battery life.

In many applications it is useful to listen for changes for whole
subtrees starting from a specific collection.  For example,
registering interest once for all photos in my "family" photo
directory or all source code for the "foo" project.  In other
applications it is useful to find out only when a small part of a
single large XML document changes.

I believe we need async notifications for these types of events.  SIP
and XMPP both have notification mechanisms (SIP Events and XMPP
PubSub, respectively). Both of these protocols are appropriate.  I
used SIP to make a concrete draft [1] because I am more familiar with
it. There is also a WebDAV-specific notification scheme described for
XMPP in [2].

In some cases details about what resources have changed needs to be
filtered for privacy reasons.  For example, assume Alice has
publicly-readable photos and photos viewable only by a handful of her
friends. If Alice adds (or deletes) a new private photo, we want to
notify those who were authorized to fetch the photo only, not everyone
who asks to be notified when Alice's photos change.  This could result
in different notifications based on who is requesting them.

We usually care about the state or status or these resources, rather
than a detailed blow-by-blow about how a resource ended up in a
particular state.  However, sometimes we may want to track a version
history for some resources.

Regardless, the native representation of an HTTP-resource could be
anything, not just files and directories in a filesystem. Changes to
these resources could be caused by non-HTTP operations (such as
modifying a database), using standard HTTP methods, or using WebDAV.

---

From: Simon Leinen <[email-masked]>
Date: Sun, 20 Jan 2008 19:31:52 +0100

> Asynchronous change notification for HTTP-based resources
[...]
> HTTP resources, or polling some kind of HTTP "catalog" resource to
> see what resources have changed. Both of these methods inherently
> suck.  The "catalog" approach at least substantially reduces the
> number of resources that need to be polled, but the amount of time
> to wait between polls is always one-sided compromise and rarely
> optimal for response time, network usage, and battery life.

FYI, there are a few interesting techniques to make server-to-client
notifications over HTTP more efficient.  The buzzword for this is
"Comet", and the Wikipedia article has good descriptions on some of
these techniques, e.g. "long polling":

http://en.wikipedia.org/wiki/Comet_(programming)
-- 
Simon.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 21 Jan 2008 08:58:54 -0700

> http://en.wikipedia.org/wiki/Comet_(programming)

In the XMPP community we have developed a technology called BOSH, which 
has some similarities to the Comet approach. BOSH was originally 
developed as a method for XMPP servers to offer an HTTP transport for 
XMPP traffic instead of the TCP transport defined in RFC 3920, but we 
have generalized it so that it is no longer specific to XMPP:

http://www.xmpp.org/extensions/xep-0124.html

We've talked about contributing this to the Internet Standards Process 
but haven't gotten around to it yet...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/