Subject: [Sipping] SIP Subscription for SCADA/Stock quotes

From: Benjamin Carlyle <[email-masked]>
Date: Fri, 17 Nov 2006 07:34:05 +1000

G'day,

I work for Westinghouse Rail Systems Australia, and am investigating
standard subscription protocols for use in our SCADA product line. SCADA
systems (centralised monitoring and control of remote equipment such as
power distribution systems) tend to rely a lot on subscription to push
data to users and to distributed services. SIP has come up as a possible
way to do this, however at present given my reading of rfc3265 I suspect
it is not appropriate and would like to confirm with this mailing list.

rfc3265 indicates that it is not a general purpose subscription
mechanism. Specific weaknesses I see include
* There is no obvious way to subscribe to a HTTP resource. Subscriptions
are sent to the combination of a SIP url and event package identifier.
* Proxies are used for transport through firewall-intensive networks,
but do not participate in the subscription. A restful architecture is
constrained to support layering, whereby muliple clients who share a
proxy might enjoy a multicasting effect through the proxy
* Data flow is limited by explicit timers. In a SCADA or stock quote
system old data degrades rapidly in value with time, and the objective
is to get as close as possible to immediate notifications. The maximum
notification rate should be related to available bandwidth, or at worst
to network latency
* Subscriptions are refreshed individually at a client-specified rate.
In a SCADA environment we are likely to have thousands of subscriptions
active between a client and server. The client-specified rate may be
used as a keep-alive so that the client can detect server death and
recreate its subscriptions on another cluster member. I believe it is
better for the client to send exactly no renewals. Keep-alive should be
sent from the server only, and at a long reliable rate when no data has
been transmitted over the subscription during the keep-alive interval.

Before I started my quest for an appropriate protocol, I wrote up a
strawman which may be viewed here:
http://soundadvice.id.au/blog/draft-carlyle-sena-00.txt

If I am unable to resolve the issues I have with existing standards I
intend to refine and propose the straw-man as a potential
standards-track rfc. I hope to find that an appropriate subscription
mechanism already exists. I see defining a new one as something of a
last resort.

Security is still a weakness of my protocol when NOTIFY and PATCHNOTIFY
are used. HTTP may only support the EXPIRE notification model without
additional trust mechanisms becoming better understood and deployed.

I believe I have already ruled out the use of XMPP's XEP-0060 given the
way that it makes use of an intermediatary to manage subscriptions but
does not summarise subscription data for slow clients:
http://mail.jabber.org/pipermail/standards-jig/2006-November/012971.html
http://mail.jabber.org/pipermail/standards-jig/2006-November/012974.html
http://mail.jabber.org/pipermail/standards-jig/2006-November/013042.html

Benjamin.

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 16 Nov 2006 17:21:30 -0500

I don't think you should quite give up yet. Some of the constraints are 
general pieces of advice, not protocol constraints. They are meant to 
protect certain general-purpose call routing proxies, say, which isn't a 
concern in your case.

> are sent to the combination of a SIP url and event package identifier.

I don't know what you mean by 'subscribe to HTTP resources'. Clearly, if 
you want to monitor if an HTTP resource changes, that resource has to 
PUBLISH or NOTIFY when the change occurs.

> proxy might enjoy a multicasting effect through the proxy

I don't understand this comment.

> notification rate should be related to available bandwidth, or at worst

See my comment at the top. This may not apply in your case.

> been transmitted over the subscription during the keep-alive interval.

There is nothing that prevents the client from specifying a long 
(infinite) expiration interval or the server from changing it. Thus, you 
can easily implement a model where the client detects server liveness. 
The reason for the client renewal was to prevent "memory leakage" on the 
server, where clients no longer interested in subscriptions would get 
NOTIFYs.

---

From: "Samir Srivastava" <[email-masked]>
Date: Thu, 16 Nov 2006 16:46:38 -0600

Do we want to cater the different verticals now ? More in-line.

Thx
Samir

> -----Original Message-----
> From: Benjamin Carlyle [mailto:[email-masked]]
> Sent: Thursday, November 16, 2006 1:34 PM
> To: [email-masked]
> Subject: [Sipping] SIP Subscription for SCADA/Stock quotes
> 
> G'day,
> 

We can transport different URL schemes over SIP network e.g. Tel, URN
etc,
It will require enhancements to current PUBLISH etc.

> proxy might enjoy a multicasting effect through the proxy

As far as I know, maddr was meant for that originally. 3261 talks about
the multicasting at several places. But I don't know, who has
implemented that.  ICE is nice solution within MMUSIC for firewall/NAT
traversal. 

> notification rate should be related to available bandwidth, or at
worst

Overload/congestion etc in this regard is the work in progress.

> been transmitted over the subscription during the keep-alive interval.

Server has a mechanism with Min-Expire header with 423 response to
enforce the refresh interval, it is comfortable with. You can use
OPTIONS from server to do that with a long subscription duration.  

> additional trust mechanisms becoming better understood and deployed.

We are heavily involved in ironing out security issues.

---

From: "Brian Stucker" <[email-masked]>
Date: Thu, 16 Nov 2006 19:03:44 -0600

Inline comments below. 

> From: Srivastava, Samir (SC100:8826) 
> Sent: Thursday, November 16, 2006 4:47 PM
> To: Benjamin Carlyle; [email-masked]
> Subject: RE: [Sipping] SIP Subscription for SCADA/Stock quotes
> 

Huh? This is a request to evaluate part of SIP framework to see if it is
fit for handling a large scale, highly distributed event driven
environment. It had better handle it or SIP is in major trouble trying
to handle presence. 

Ben, you may wish to take a look at some of the scaling work out of the
SIMPLE WG around presence. Although the service involved is different, I
think you will find some useful information and mechanisms for your
problem.

> > Subject: [Sipping] SIP Subscription for SCADA/Stock quotes

[snip]

> We can transport different URL schemes over SIP network e.g. 
> Tel, URN etc, It will require enhancements to current PUBLISH etc.

The point of that statement in 3265 and elsewhere is not to limit the
URL schemes that can be used, but to point out that there are many ways
of pushing data around a network. The intent of the framework wasn't to
solve world-event-subscription-hunger. There was a lot of discussion at
the time around using (PUBLISH in particular) for all sorts of things
and we wanted to constrain the problem space so we could finish off the
RFC sometime in our lifetime.

[snip]

> Server has a mechanism with Min-Expire header with 423 
> response to enforce the refresh interval, it is comfortable 
> with. You can use OPTIONS from server to do that with a long 
> subscription duration.  

Samir--

Min-Expires is only used to ask the client to extend the expire timer
from what it requested. It does not enforce the refresh interval alone.
There's quite a bit involved in managing the problem of
highly-distributed, soft-state, state machines. Probably the best
example of this space is the complexity of TCP congestion algorithms. 

OPTIONS isn't intended to be used as a generic pinging mechanism and has
no semantics for keepalive. It is sent to inquire as to the capabilities
of an endpoint, and when the linger timer for the transaction pops after
sending a response, there is no semantic for keeping track of
who/what/when/why an OPTIONS was sent. All an OPTIONS response will
typically tell you is that the endpoint's SIP stack is alive. It could
be brain-dead above that, and there's no way to know that an OPTIONS
failure (for instance a 408 time-out response) is due to a transient
network error. Even if you do get a response, it doesn't mean that the
subscription is still active on the device.

Ben--

RFC-3265 does not have a server-initiated refresh mechanism because the
server handling the subscriptions should be spending it's time handling
incoming subscription requests, generating notifications, and processing
the events that trigger notification. A typical 3265 server
implementation will not, for instance, have a timer running for the
duration of a subscription. It may simply timestamp the subscription
when it is created and then passively garbage collect it later after it
expires. Having the much larger set of clients keep track of their
timers rather than centralizing the problem is viewed as a better
scaling solution because it fans out the work to the largest source of
capacity (ie. the clients).

IMHO- The best way to solve the dead server problem is to have neither
end generating keep-alive traffic (which, in an ideal environment in
which no node fails, is always useless). Solve the problem at the server
by utilizing a high-availability scheme there so that a server failure
does not interrupt service.

> We are heavily involved in ironing out security issues.

Ben--

I've done some work in the past with SCADA, and typically the payload in
the event notifications does not require encryption. What you need to
typically prevent is someone trying to corrupt your central database
with garbage data (indicating faults where there are none, or masking
faults where they do exist). This can be handled by request
authentication. I believe there are a number of ways to solve this
problem depending on the level of assurance that you require for your
particular application. You could use HTTP Digest authentication to deal
with this, or something heavier such as TLS. Without understanding the
security requirements needed, it's going to be hard to evaluate.
However, there are a number of models out there already that you can
look at to see if there's a fit for your particular purpose.

Regards,
Brian

---

From: "Samir Srivastava" <[email-masked]>
Date: Mon, 20 Nov 2006 03:34:38 -0600

Hi,

  in line.

Thx
Samir

>Huh? This is a request to evaluate part of SIP framework to 
>see if it is fit for handling a large scale, highly 
>distributed event driven environment. It had better handle it 
>or SIP is in major trouble trying to handle presence. 

If you remember Hennings frustration from the meeting that one bit of
information is going with how many bytes.  SIP presence framework itself
is not that good :-) Though I am not focused on optimizing it currently.

In 2000, there was an attempt on the communication of Intelligent
Appliances...
"SIP Extensions for Communicating with Networked Appliances", H. 
  Schulzrinne, Simon Tsang, Stanley Moyer, Dave Marples, Arjun 
  Roychowdhury, 11/16/2000, <draft-tsang-sip-appliances-do-00.txt>

    A variety of technologies are available to network appliances and 
    provide home automation and control.  However, these do not support 
    wide-area access control and interworking of these Networked 
    Appliances (NA).  This document describes a new SIP method, DO, that

    allows a SIP UA to communicate with NAs. "

If my recollection of arguments on the list is correct, There were
arguments that most of the startups die because of indigestion etc. SIP
is not an infant any more. In the startup mode, you focus on the problem
at the hand. But you keep architecture open such that it can handle
varying needs. It should increase the degree of pervassiveness in the
future seamlessly. I think fathers/god father of SIP were not be having
such a myopic vision on the utility of SIP for just mimicking PSTN and
Presence as you are trying to say. Presence came later and it fit in
well. 

To me, your fear comes from the arguments given by others for weakness
of SIP as "SIP Does many things for many people" . I counter it "IP does
the same", It depends on how you see and design the versatile /
pervassive protocol fulfilling different needs seamlessly with little
extensions where processing intelligence of those extensions is just at
the required end point. Rest nodes are just conduit. To me, Presence in
its current form is just one aspect. SIP in the core is excellent
protocol but with some rough edges coming from mimicking PSTN, etc...

<<SNIP>>

>
>The point of that statement in 3265 and elsewhere is not to 
>limit the URL schemes that can be used, but to point out that 
>there are many ways of pushing data around a network. The 
>intent of the framework wasn't to solve 
>world-event-subscription-hunger. There was a lot of discussion 
>at the time around using (PUBLISH in particular) for all sorts 
>of things and we wanted to constrain the problem space so we 
>could finish off the RFC sometime in our lifetime.

I am not asking world-event-hunger to be fullfilled with this like
X-Motif/JDK event models etc. I am asking only the presence framework to
extend....

<<SNIP>>

>OPTIONS isn't intended to be used as a generic pinging 
>mechanism and has no semantics for keepalive. It is sent to 
>inquire as to the capabilities of an endpoint, and when the 
>linger timer for the transaction pops after sending a 
>response, there is no semantic for keeping track of 
>who/what/when/why an OPTIONS was sent. All an OPTIONS response 
>will typically tell you is that the endpoint's SIP stack is 
>alive. It could be brain-dead above that, and there's no way 
>to know that an OPTIONS failure (for instance a 408 time-out 
>response) is due to a transient network error. Even if you do 
>get a response, it doesn't mean that the subscription is still 
>active on the device.

As per Section 11 of 3261 
   "An OPTIONS request MAY be sent as part of an established dialog to
   query the peer on capabilities that may be utilized later in the
   dialog."

OPTIONS does allow the Allow-Event header etc. There may be sometime
need for a SIP entity to find the state of subscirbe created dialog on
another end, there it will be useful. (Like missing NOTIFY with state
change delta) State indication can be event package specific. 200 OK
with subscription event state tells the querying endpoint what is
happening. If event state is currently not supported, then protocol
might need to extend. Other response codes can be viewed differently.. I
was coining it as he was asking for server driven. I don't understand
completely his problem requirements as of now.

>
>Ben--
>
>RFC-3265 does not have a server-initiated refresh mechanism 
>because the server handling the subscriptions should be 
>spending it's time handling incoming subscription requests, 
>generating notifications, and processing the events that 
>trigger notification. A typical 3265 server implementation 
>will not, for instance, have a timer running for the duration 
>of a subscription. It may simply timestamp the subscription 
>when it is created and then passively garbage collect it later 
>after it expires. Having the much larger set of clients keep 
>track of their timers rather than centralizing the problem is 
>viewed as a better scaling solution because it fans out the 
>work to the largest source of capacity (ie. the clients).

Your point was earlier well understood even before commenting. See the
above.

>
>IMHO- The best way to solve the dead server problem is to have 
>neither end generating keep-alive traffic (which, in an ideal 
>environment in which no node fails, is always useless). Solve 
>the problem at the server by utilizing a high-availability 
>scheme there so that a server failure does not interrupt service.
>

Refer my long time back proposal on the list for the Event based heart
beat 

http://www1.ietf.org/mail-archive/web/sip/current/msg08844.html This was
to avoid the  periodic OPTIONS messages flow when other server was down
to avoid congestion etc.. I was just scratching the congestion at that
time...

Somewhere at the IP layer when packet is sent, it needs to know which IP
server should fill. VRRP just talks about the IP layer connectivity. Can
you describe your scheme...

<SNIP>

>Ben--
>
>I've done some work in the past with SCADA, and typically the 
>payload in the event notifications does not require 
>encryption. What you need to typically prevent is someone 
>trying to corrupt your central database with garbage data 
>(indicating faults where there are none, or masking faults 
>where they do exist). This can be handled by request 
>authentication. I believe there are a number of ways to solve 
>this problem depending on the level of assurance that you 
>require for your particular application. You could use HTTP 
>Digest authentication to deal with this, or something heavier 
>such as TLS. Without understanding the security requirements 
>needed, it's going to be hard to evaluate. However, there are 
>a number of models out there already that you can look at to 
>see if there's a fit for your particular purpose.

There were some postings on the HTTP Digest being prone to brute force
attacks.
BTW, I worked earlier in different event based/driven systems too. When
I have more free time, I will start looking into this.

---

From: Adam Roach <[email-masked]>
Date: Mon, 20 Nov 2006 05:55:25 -0600

> Presence as you are trying to say. Presence came later and it fit in

Having been deeply involved in the whole genesis of events and presence, 
I can confidently say that this is highly inaccurate revisionist bunk.

Forgive me as I indulge in a bit of personal anecdote.

When Jonathan et. al. dropped the first version of the documents that 
would become the presence and instant messaging extensions using SIP, I 
was in Stockholm helping Ericsson's 3GPP delegates cope with the then 
very recent decision by 3GPP to use SIP as their protocol for the IMS 
core. (At that point in time, the only people doing standards work 
within Ericsson who had any familiarity of SIP were me and Sean Olson). 
I recognized the impending train wreck between the use of 
SUBSCRIBE/NOTIFY for PINT and SUBSCRIBE/NOTIFY for what was then IMPP, 
and decided that what we really needed was an extensible framework that 
would allow different usages of these primitives peacefully co-exist. 
Thus was born the draft that eventually became RFC3265. (Ironically, my 
own interest in the problem stemmed partially from the need for a 
call-completion service [1] -- a topic still generating considerable 
consternation).

So, important and relevant point #1: the use of SIP for presence 
preceded a general framework for event notification by several weeks.

(If you would like to verify the timing of a SIP architecture in the IMS 
core, I'd advise starting with S2-00751 [3], dated May of 2000, which I 
wrote during my stint in Stockholm: you will be unable to find anything 
resembling a coherent SIP architecture in any form within 3GPP prior to 
this contribution).

> protocol but with some rough edges coming from mimicking PSTN, etc...

The importance of the 3GPP aspect of this story is as follows: until 
3GPP got into the business of extending SIP and defining network 
architectures for it, it was as decidedly un-PSTN-like as possible.

In fact, there was a rather constant thunder of complaining from the 
ITU-T wonks ("Bell Heads", if you will) that the way SIP did things was 
so very, very different than the way ISUP and H.323 did things, and that 
interworking the networks would be problematic. You can see echoes of 
the SIP community's response to this in 
draft-sparks-sip-service-examples-00 [2] (which the Bell Heads 
completely missed the point of, considering it to be the equivalent of 
H.323's annexes as opposed to a thesis on why service standardization 
was unnecessary) and the PSTN interoperation work which was eventually 
published as RFC 3372 and RFC 3398.

So, important and relevant point #2: no one was trying to make SIP mimic 
the PSTN at the time RFC 3265 was developed; reality was quite the 
opposite: "the SIP way" was so thoroughly anti-PSTN that it received a 
steady stream of criticism for not fitting neatly into the Bell Heads' 
view of the world.

Beyond these points of fact, there are a number of points of philosophy 
that I could raise in objection to a lot of what you say (e.g., yes, a 
lot of things run on IP -- this is a very deliberate design that is 
intended NOT to be mimicked in higher layers; google "Internet 
Hourglass" [4] for myriad explanations), but I'll leave the rest of 
those points to others.

/a

[1] http://www.potaroo.net/ietf/idref/draft-roach-sip-acb/
[2] http://tools.ietf.org/html/draft-sparks-sip-service-examples-00
[3] 
http://www.3gpp.org/ftp/tsg_sa/WG2_Arch/drafting_meetings/00_05_Stockholm/S2_key_issues/tdocs/S2-000751.zip
[4] http://www.google.com/search?q=%22internet+hourglass%22

---

From: "Brian Stucker" <[email-masked]>
Date: Mon, 20 Nov 2006 10:37:02 -0600

Samir,

See below.

> Sent: Monday, November 20, 2006 3:35 AM
> To: Stucker, Brian (RICH1:AR00); 'Benjamin Carlyle'; 
> [email-masked]'
> see if it 
> >is fit for handling a large scale, highly distributed event driven 
> If you remember Hennings frustration from the meeting that 
> one bit of information is going with how many bytes.  SIP 
> presence framework itself is not that good :-) Though I am 
> not focused on optimizing it currently.
> 
> In 2000, there was an attempt on the communication of 
> Intelligent Appliances...
> If my recollection of arguments on the list is correct, There 
> were arguments that most of the startups die because of 
> indigestion etc. SIP is not an infant any more. In the 
> startup mode, you focus on the problem at the hand. But you 
> keep architecture open such that it can handle varying needs. 
> It should increase the degree of pervassiveness in the future 
> seamlessly. I think fathers/god father of SIP were not be 
> having such a myopic vision on the utility of SIP for just 
> mimicking PSTN and Presence as you are trying to say. 
> Presence came later and it fit in well. 
> 
> To me, your fear comes from the arguments given by others for 
> weakness of SIP as "SIP Does many things for many people" . I 
> counter it "IP does the same", It depends on how you see and 
> design the versatile / pervassive protocol fulfilling 
> different needs seamlessly with little extensions where 
> processing intelligence of those extensions is just at the 
> required end point. Rest nodes are just conduit. To me, 
> Presence in its current form is just one aspect. SIP in the 
> core is excellent protocol but with some rough edges coming 
> from mimicking PSTN, etc...
> 

I think Adam already responded to this, and I happen to agree with his
response.

> I am not asking world-event-hunger to be fullfilled with this 
> like X-Motif/JDK event models etc. I am asking only the 
> presence framework to extend....
> 

Presence framework to extend what?

[SNIP]

> OPTIONS does allow the Allow-Event header etc. There may be 
> sometime need for a SIP entity to find the state of subscirbe 
> created dialog on another end, there it will be useful. (Like 
> missing NOTIFY with state change delta) State indication can 
> be event package specific. 200 OK with subscription event 
> state tells the querying endpoint what is happening. If event 
> state is currently not supported, then protocol might need to 
> extend. Other response codes can be viewed differently.. I 
> was coining it as he was asking for server driven. I don't 
> understand completely his problem requirements as of now.

OPTIONS is not intended to carry information that can change over time.
SUBSCRIBE/NOTIFY does this. The protocol has already been extended, and
it does not use OPTIONS. See RFC-3857.

> Refer my long time back proposal on the list for the Event 
> based heart beat 
> 
> http://www1.ietf.org/mail-archive/web/sip/current/msg08844.htm
l This was to avoid the  periodic OPTIONS messages flow when > other
server was down to avoid congestion etc.. I was just 
> scratching the congestion at that time...
> 
> Somewhere at the IP layer when packet is sent, it needs to 
> know which IP server should fill. VRRP just talks about the 
> IP layer connectivity. Can you describe your scheme...
>  
> 

My point here is that within this application, heartbeating between
clients and servers to solve an issue of server availability is a bad
model. Use a high-availability scheme between the servers so that they
can take over for one another. Nothing special happens at the clients. 

[SNIP]

> 
> There were some postings on the HTTP Digest being prone to 
> brute force attacks.
> BTW, I worked earlier in different event based/driven systems 
> too. When I have more free time, I will start looking into this.
> 

Please do.

---

From: "Samir Srivastava" <[email-masked]>
Date: Mon, 20 Nov 2006 14:45:14 -0600

Hi Brian,

   I will address the other issues separately. 

> model. Use a high-availability scheme between the servers so that they
can
> take over for one another. Nothing special happens at the clients.
> 
> 

What IP address client will fill in when it needs to send the packet to
the SIP Server ? VRRP in the front or something else. If it is VRRP -IP,
there is still a chance that SIP message is delivered to the server
whose IP layer is good but SIP application server is down/stuck etc...

Thx
Samir

---

From: "Michael Procter" <[email-masked]>
Date: Wed, 22 Nov 2006 02:17:31 -0800

Comments inline:

> are sent to the combination of a SIP url and event package identifier.

This is true.  A SIP SUBSCRIBE subscribes to a SIP resource.

> proxy might enjoy a multicasting effect through the proxy

This is also true.  But proxies in SIP have a different role to those in
HTTP.  State Agents may be a closer fit to this requirement, which I
discuss later in this email.

> notification rate should be related to available bandwidth, or at
worst

I don't agree.  Notification rate is to be defined by the event package
in question.  Since you haven't defined an event package yet, this is
open to some flexibility.

> been transmitted over the subscription during the keep-alive interval.

Actually, the client specifies the maximum duration that it can support,
but the server determines the interval used.  Ranges for durations can
be defined in the event package (see RFC3265 Sec 4.4.4), so you can pick
suitable ones for your application.

> http://soundadvice.id.au/blog/draft-carlyle-sena-00.txt

I glanced through this document, and picked up on your list of desirable
characteristics:

     Summarisation. This is the organised discarding of older
     information to ensure that slow clients recieve as much fresh data
     as their connection characteristics permit.

     Differential flow control. A slow client should not prevent fast
     ones from getting updates.

     Localised resynchronisation. A client need not reach back to the
     origin server for the current resource status if its immediate
     server is already handling the subscription.

     Patch updates. For large resources (especially lists), the ability
     to deliver a message that indicates the change from last time,
     only. Not the whole state.

     Dial-back. Pub/Sub can be an amplifier of denial of service
     attacks. The subscription mechanism must be able to detect
     when its notifications are being treated as spam and end the
     subscription.

The third item in particular suggests to me that you might benefit from
inserting a layer of State Agents (RFC3265 Sec 4.4.11) between your
clients and servers.  The servers can PUBLISH their data to the State
Agents, which accept subscriptions from your clients.  This ensures that
your servers are not troubled by subscription refreshes, and that
clients can obtain the latest information from the State Agents rather
than the servers directly.  

The other items can be dealt with in the event package itself, and have
no direct bearing on RFC3265.

There are, of course, other benefits and complications associated with
this approach, but you need to weigh up whether it is appropriate for
your application.

Regards,

Michael Procter

---

From: Benjamin Carlyle <[email-masked]>
Date: Sun, 26 Nov 2006 17:08:56 +1000

Sorry to all of you whose replies I originally missed. They were filed
in a different mail folder than I had expected.

So at the moment I'm still looking at SIP as a possible way to transfer
notifications that a resource has changed. I am also still considering
trying to push a cut-down version of my own draft somewhere. What do you
guys think?

To achieve a SIP-based version I think I would need:
* An event package that allows a http resource to be specified somehow
* A way to locate the appropriate SIP url to send the SUBSCRIBE request
to
* Either to follow along the current NOTIFY+partial NOTIFY path that SIP
seems to be taking, or look at how to mirror the EXPIRE approach from my
draft
* Better understand state agents, and how/whether they could be used to
layer subscriptions for transparent proxying and scalability
* Better understand how a state agent with bounded buffers can
participate in the subscription. Can they summarise, do they need
unbounded buffers, do they refuse new messages until slow clients catch
up, do they unsubscribe slow clients, or is the event package completely
responsible for defining their behaviour?
* Better understand the Rate of notifications section of rfc3265. Can I
design an event package that sets the rate at "all available bandwidth"
without harming the SIP network?

My cut-down draft (EXPIRE only, no NOTIFY/PATCHNOTIFY support) is
accessable here:
http://soundadvice.id.au/blog/draft-carlyle-sena-01.txt

More in-line.

> think you will find some useful information and mechanisms for your

Thanks, I'll take a look. Is this technical report
<http://mice.cs.columbia.edu/getTechreport.php?techreportID=426> the
main place to look?

My quick first impressions of the TR for my use cases:

RLS is similar to something I've thought about in the past, and might
minimise keep-alive traffic when a large number of subscriptions are
made to a real or apparent small set of servers. The resource lists do
have to be built up on the server-side, though, limiting the application
of this techinque for ad hoc subscriptions.

Partial PUBLISH and NOTIFY are approaches I have listed in my draft as
particularly useful when synchronising list content. There is some
tension between partial notification support and summarisation by state
agents (proxies in my draft). The systems I work with are built
fundamentally not to melt down under extreme change rates relative to
the rate at which changes can be processed. This means that state agents
have a bounded (often no more than 1-space) buffer size allocated for
each subscription, and that something needs to happen when that buffer
limit is hit.

My approach is to follow Rohit Khare's model from his paper on
Decentralizing REST: To summarise. This means that the state agent may
be discarding content or even rewriting it to preserve as much
information as possible while reducing the number of bytes in a lossy
stream compression. Building a security model around this may be
impossible, so my draft probably has to drop it for use in insecure
environments.

I hadn't fully come to terms with the place of state agents in SIP
subscription. Can they be layered transparently to users in a similar
way to HTTP proxies? For example, could an ISP run a state agent that
aggregates subscriptions to all of its users... or could a corporation
set up a state agent at each of its offices?

> RFC sometime in our lifetime.

I'm not sure I understand this. Can I specify a http url somehow as the
target for my subscription under SIP's rfc3265? I thought that the url
had to be a sip one, and that the event package did the rest. Are we
talking about a parameter being applied to the event package?

> ...

My suggestion is that at the time of garbage collection the server
attempts to send keep-alives (ie duplicate notifications) back to any
client that hasn't recieved a notification since the last collection
interval. If the request is treated as spam or is unable to get through
based on normal timeouts, the subscription terminates. If the request
does get through and recieves an OK response, the subscription
continues. This allows the server to reap old subscriptions without a
client-initiated keepalive. Whether the gc is driven from a timer or
from a high watermark wouldn't be too relevant so long as the watermark
isn't hit and the gc performed too often.

I haven't implemented this exact scenario myself as yet. In fact, the
implementation I'm working with presently ties subscription lifetime to
that of a TCP/IP connection. I am hoping to move to a more standard
approach.

> does not interrupt service.

I agree. This is the gist of my draft's musings on the topic. The client
never sends keepalives because the server should ensure that the
subscription is persistent and keeps operating despite outages. Clients
are more numerous and less reliable, so keepalive is still required in
the reverse direction. I feel that the client should not normally be
involved in choosing the interval. It really doesn't concern the client,
and the interval should be long enough that any client can cope with the
traffic. I suggested the same keepalive interval as TCP/IP suggests: 2
hours minimum, and only when notifications haven't been sent recently.

> look at to see if there's a fit for your particular purpose.

When I started thinking about the security problem it became clear to me
that it isn't worth putting too much effort into encryption as part of
an internet-scale mechanism as the concept of secrecy itself doesn't
scale. You might as well just use SSL or some other direct mechanism.
Secret data doesn't benefit from increases in scale, because only a
small set of users are ever going to access the secret data.

Authentication for NOTIFY requests is an interesting area, though.
Normally HTTP authenication is based around a client who has an account
on the server. With NOTIFY what you want to determine is that the client
is acting on behalf of the server you subscribed to. You could ensure
this by authenticating the server during your SUBSCRIBE, and passing it
some kind of token to pass back. However that is subject to replay
attacks and the like. Probably the most sustainable solution is to
obtain a public key from the server that is signed by a trusted
certificate authority.

If the notify resource recieves a notification signed by the server's
public key, and the sequence number (part of the signed portion of the
document) is greater than the last one, then it knows it has a valid
notification. If we are talking about pure nofication and not parital
publication this is enough to continue to update to the latest state
despite summarisation by state agents. Parital publication with
summarising state agents requires that the signature be for the state
produced by applying the patch, and can similarly be summarised by state
agents.

Unfortunately, I don't think there is any hope of getting something with
such a new and complex security mechanism standardised in HTTP as part
of a subscription protocol :) This pretty much leaves me with EXPIRE
functionality only, i.e. someone claims something has changed but might
not be trustworthy: You had better to a GET and see if they are right.
The veracity of EXPIREs requests are only tested by going back and
making that GET request. Easy, and probably standardisable.

As such, I wonder if SIP might still be the way to go. The first hurdle
would pretty-much be figuring out which sip URL to send your SUBSCRIBE
request to when you have http://example.com/circuit-breakers/001 in-hand
and want to subscribe to it.

I think I'm reading that dialogs have to be initiated by the subscriber,
as opposed to the resource being subscribed to.. and that a dialog needs
to be established for a conversation. I think that would rule out using
only the SUBSCRIBE part of my draft, and specifying a sip address as a
callback. Or would that work?

Benjamin.