Subject: RE: [Sipping] Update to event throttles

From: [email-masked]
Date: Sun, 13 Mar 2005 18:51:13 +0200

Jonathan,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of ext Jonathan Rosenberg
> Sent: 11 March, 2005 11:39
> To: Niemi Aki (Nokia-NRC/Helsinki)
> Cc: SIPPING WG
> Subject: Re: [Sipping] Update to event throttles
> 
> 
> I'm inclined to say that I don't find this a terribly 
> pressing problem. 
> We don't have any real networks for whom subscribers are being 
> overwhelmed with notifications, for which this would be the 
> ideal way to 
> deal with it (compared to compression or filters, both of 
> which reduce 
> the amount of data).
> 

The overall problem of bandwidth consumption is extremely pressing.

We have been talking to many mobile operators (GPRS, EDGE, WCDMA) about deploying SIP-based presence in their current networks, and it turns out many of them are saying it is impossible. The reason they are giving is that presence would generate too much data transmitted over the air, that it would make the service commercially viable. You might blame the operators' business models, too expensive radio technology, etc. but the situation is what it is.

So, we have to pursue the reduction of traffic on several fronts:
- Compression (SigComp, GZIP for the payloads)
- Filtering
- Partial notifications
- Throttling

Each of these are useful components and help in different ways. Throttling may not be the most effective of the methods, but it is definitely the simplest, and useful in certain circumstances. 

If someone does not need throttling, they don't need to implement it.

So far people seem to have argued either that
- the problem does not exist: I have just argued that it does
- the problem scope is too narrow: Well, the particular component to address the overall problem field happens to be simple, and there are no known requirements beyond that
- the problem can be solved in a different way: Yes, but the ways speculated have been complementary, not better than what is specified in the draft.

Regards,
	Markus

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 13 Mar 2005 18:40:31 -0500

During the discussion at the meeting it struck me that perhaps what is 
needed is a bandwidth sensitive filter specification. The basic throttle 
mechanism doesn't necessarily reduce bandwidth, while the basic filter 
mechanism isn't time or size sensitive. A richer filter mechanism could 
have more flexibility.

	Paul

---

From: Aki Niemi <[email-masked]>
Date: Mon, 14 Mar 2005 12:40:05 +0200

> mechanism isn't time or size sensitive. A richer filter mechanism could 

Wouldn't using a throttle along with a filter achieve exactly what 
you're after?

Cheers,
Aki

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 14 Mar 2005 08:43:02 -0500

Aki Niemi wrote:
>> needed is a bandwidth sensitive filter specification. The basic 
>> throttle mechanism doesn't necessarily reduce bandwidth, while the 
>> basic filter mechanism isn't time or size sensitive. A richer filter 
>> mechanism could have more flexibility.
> 
> 

Ho, I don't think that would be sufficient. The trick is that the filter 
has to be able to make decisions about what to let through that vary 
based on the resulting impact to required bandwidth. E.g. indicating 
that you would like geoloc info but are willing to give it up to achieve 
a desired bandwidth.

I don't have a clear idea how one would express this. But I can imagine 
applying one filter, then checking if the result fits a throttle 
rate/bandwidth restriction, then if not applying a more restrictive 
filter, etc.

This puts the decision in the hands of the watcher about what to give up 
to achieve the desired objective.

	Paul

---

From: David R Oran <[email-masked]>
Date: Mon, 14 Mar 2005 08:48:49 -0500

> mechanism could have more flexibility.
I tend to agree with Paul, however what we really need to accomplish is 
to trade off three things:
a) timeliness of notifications
b) usage of bandwidth
c) cost to the user

So far the throttle draft doesn't quite address any of these directly. 
My suggestion is to allow the user to express  (a) and (c) and let the 
presence server decide if the constraints can be met within the 
bandwidth (b) the service provider is willing to consume. If the 
constraints can't be met, the SUBSCRIBE would be rejected. The service 
provider could take an optimistic or pessimistic authorization approach 
by varying their assumptions about the presence update rate.

Dave.

>>> I'm inclined to say that I don't find this a terribly pressing 
>>> problem. We don't have any real networks for whom subscribers are 
>>> being overwhelmed with notifications, for which this would be the 
>>> ideal way to deal with it (compared to compression or filters, both 
>>> of which reduce the amount of data).
>>>
>> We have been talking to many mobile operators (GPRS, EDGE, WCDMA) 
>> about deploying SIP-based presence in their current networks, and it 
>> turns out many of them are saying it is impossible. The reason they 
>> are giving is that presence would generate too much data transmitted 
>> over the air, that it would make the service commercially viable. You 
>> might blame the operators' business models, too expensive radio 
>> technology, etc. but the situation is what it is.
>> Each of these are useful components and help in different ways. 
>> Throttling may not be the most effective of the methods, but it is 
>> definitely the simplest, and useful in certain circumstances. If 
>> someone does not need throttling, they don't need to implement it.
>> - the problem scope is too narrow: Well, the particular component to 
>> address the overall problem field happens to be simple, and there are 
>> no known requirements beyond that
>> - the problem can be solved in a different way: Yes, but the ways 
>> speculated have been complementary, not better than what is specified 
>> in the draft.
>> Regards,
>> 	Markus
David R. Oran
Cisco Fellow
Cisco Systems
7 Ladyslipper Lane
Acton, MA 01720 USA
Tel: +1 978 264 2048
Email: [email-masked]

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 15 Mar 2005 15:13:03 -0500

inline.

Paul Kyzivat wrote:

> 
> 
> Aki Niemi wrote:
> 
>>
>> ext Paul Kyzivat wrote:
>>
>>> During the discussion at the meeting it struck me that perhaps what 
>>> is needed is a bandwidth sensitive filter specification. The basic 

Right. Its a hard problem, though. Let us say that the rate R of 
notifications generated is a function of the filter F. The problem is to 
  adjust F over time such that R(F) is below some maximum rate 
threshold. Telling the server to do that amounts to providing a function 
that, for any unfiltered notification rate, generates a filter that 
reduces the rate to the threshold. No idea how that could be done.

Though I am still queasy about it, I will back down from my position and 
agree that folks do appear to have a demonstrated need for this, that it 
doesnt break anything, and that filters alone would be heard to achieve 
the effect of reducing the notification rate to below a specific threshold.

-Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Director, Service Provider VoIP Architecture   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 16 Mar 2005 11:31:39 -0500

Jonathan Rosenberg wrote:
>> Ho, I don't think that would be sufficient. The trick is that the 
>> filter has to be able to make decisions about what to let through that 
>> vary based on the resulting impact to required bandwidth. E.g. 
>> indicating that you would like geoloc info but are willing to give it 
>> up to achieve a desired bandwidth.
>>
>> I don't have a clear idea how one would express this. But I can 
>> imagine applying one filter, then checking if the result fits a 
>> throttle rate/bandwidth restriction, then if not applying a more 
>> restrictive filter, etc.
>>
>> This puts the decision in the hands of the watcher about what to give 
>> up to achieve the desired objective.
> 
>  adjust F over time such that R(F) is below some maximum rate threshold. 
> Telling the server to do that amounts to providing a function that, for 
> any unfiltered notification rate, generates a filter that reduces the 
> rate to the threshold. No idea how that could be done.

Agree that this may not be easy.

But that doesn't mean a way can't be found. The question is whether the 
throttle mechanism otherwise solves the problem.

As an off-hand suggestion, how about just a sequence of progressively 
more agressive filters? The server would apply them one by one until the 
size of the message was sufficiently small that sending it will conform 
to the bandwidth constraint.

> agree that folks do appear to have a demonstrated need for this,

Yeah, I too have accepted the need.

> that it 

I would still like to see the use cases where this is sufficient to 
solve the problem. I understand that the situation may differ based on 
the semantics of different event packages. If the major point of concern 
is with presence, then I'd like to see the use case for that.

I'm really wondering what happens when somebody on a cellphone has a 
buddylist with a 100 people in it, and they are all doing stuff that 
changes there presence regularly.

I don't have any intrinsic objection to this - just an objection to 
defining mechanisms that aren't useful.

	Paul

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 16 Mar 2005 13:47:43 -0500

The problem of event throttling to reduce inbound bandwidth for 
bandwidth-limited (mobile) devices seems inherently difficult to solve 
in a distributed environment. Even if you had the most sophisticated 
control on notifications for a single presentity, a typical buddy list 
is likely to have presentities from many different domains, none of whom 
know about each other's existence, let alone their rate of sending.

It is also likely that the notification rate from a single source will 
vary significantly. If we take geoloc, it will have a spurt of activity 
during the morning and afternoon commute, and then be essentially 
quiescent, except if trips to the bathroom, water cooler and cafeteria 
are being reported on a foot-by-foot basis. If the receiver is worried 
about exceeding their data allotment for the month, short-term 
throttling during those busy times is unlikely to be particularly 
productive in ensuring this outcome, without being overly conservative.

Thus, this would only work if you had, as somebody mentioned, an 
aggregator that would somehow squeeze all of these presentity 
notifications through a single filter and maintain overall rate control, 
probably over long time horizons.

Thus, expending tremendous energy on single-source control seems 
unlikely to achieve the goal of strictly limiting inbound rates. If all 
you want is some per-source averaging, the simple spacing in time is 
about as likely to achieve that than some sophisticated content-size 
dependent leaky bucket algorithm.

I see the value of the throttle as mainly being a "fuse" mechanism - it 
keeps things from getting out of hand, e.g., due to some 
misconfiguration on the sender side such as a program that continuously 
toggles between open and closed state due to a feedback loop, but won't 
do much for aggregate rate control.

Henning

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 17 Mar 2005 01:22:38 -0500

inline.

> to the bandwidth constraint.

I'm not sure if this kind of feedback system can work. There is too much 
latency from when the filter is applied to when you can measure the 
rate, compared with the potential rate of variation of the unfiltered 
stream. I suspect the system would heavily oscillate.

> changes there presence regularly.

The interaction of this mechanism and buddy lists is a huge open issue I 
think. Are the filters passed downstream? Do you try to enforce any kind 
of fairness so that the notifications that are discarded are distributed 
  uniformly across buddies? This may also interact badly if you are 
using partial notifications AND buddy lists.

-Jonathan R.

---

From: Aki Niemi <[email-masked]>
Date: Thu, 17 Mar 2005 09:46:26 +0200

Inline.

ext Jonathan Rosenberg wrote:
>> more agressive filters? The server would apply them one by one until 
>> the size of the message was sufficiently small that sending it will 
>> conform to the bandwidth constraint.
> 

I agree. Optimally, the feedback loop should also extend all the way to 
the watcher (i.e., slow-read), which is not a possibility here.

>  uniformly across buddies? This may also interact badly if you are using 
> partial notifications AND buddy lists.

I agree. The interaction between presence lists and throttles needs more 
work. My suspicion is that it might turn out to be the key ingredient 
that solves people's issues with the draft.

Like you say, implementing fair scheduling for the notifications becomes 
criticalhere, like avoiding starvation of a single buddy etc. Hmm, 
perhaps each buddy needs a 'nice' value (sort of like "who is your best 
buddy?") that determines the scheduling priority? ;-)

Cheers,
Aki

---

From: Aki Niemi <[email-masked]>
Date: Thu, 17 Mar 2005 09:46:33 +0200

Inline.

> know about each other's existence, let alone their rate of sending.

Agreed. There is also no guarantee that all of these domains would 
deploy the throttle mechanism. This is a problem if the watcher depended 
on it being available.

> productive in ensuring this outcome, without being overly conservative.

I did envision some sort of adaptive use of the throttle, e.g., 
increasing the value when either the number of subscriptions increased, 
or during periods of low activity by the user (at night for example).

This becomes a much more palatable approach if an aggregator is used, 
since then the watcher only needs to adapt the throttle for a single or 
small set of subscriptions. (Having to do 100 re-subscribes in itself is 
a burden.)

> dependent leaky bucket algorithm.

This is also true; however, we have a limited set of tools available. 
Doing rate control at the watcher end, e.g., using a slow-read scheme 
like what Jabber seems to be doing, would be a better alternative.

Cheers,
Aki

---

From: Hans Persson <[email-masked]>
Date: Thu, 17 Mar 2005 09:15:45 +0100

> small set of subscriptions. (Having to do 100 re-subscribes in itself is 

And probably a large number of these SUBSCRIBEs will go to the same
place (people have lots of buddies within their own company). Perhaps it
would be nice to perform them in a batch?

Hans

-- 
Hans Persson <[email-masked]>    Ingate - Firewalls with SIP & NAT
Ingate Systems AB  +46 13 210877   http://www.ingate.com/

Private: <[email-masked]>  http://www.lysator.liu.se/~unicorn/

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 17 Mar 2005 10:07:28 -0500

> small set of subscriptions. (Having to do 100 re-subscribes in itself is 

Do we have the PIDF mechanisms to subscribe to a group of people and get 
everything delivered in one message?

> like what Jabber seems to be doing, would be a better alternative.

I'm not sure if this is the same thing, but if you want the watcher to 
have tight control, you do end up with polling, where the watcher can 
spread its attention around as it sees fit. (This is another problem - a 
watcher may well care more about certain presentities than others.)

For many applications, continuous presence awareness is actually 
overkill. I might call or IM a particular person once in a week. I don't 
really have to know their presence status the remaining 167 hours of 
that week.

Henning

---

From: Sam Ganesan <[email-masked]>
Date: Mon, 21 Mar 2005 04:09:08 -0500

If the end point(UA) is band-width challenged, and that it is bandwidth 
challenged is a known datum, and these events that require throttling 
are all subscribed events.... wouldnt NOT subscribing to the number of 
events that could flood the UA make this mechanism that is going to be 
difficult to implement moot?  Or am I missing something here

Sam

__
Sam Ganesan
Systems Engineering
Iperia Inc.

Henning Schulzrinne wrote:
>> since then the watcher only needs to adapt the throttle for a single 
>> or small set of subscriptions. (Having to do 100 re-subscribes in 
>> itself is a burden.)
> 
>

---

From: Adam Roach <[email-masked]>
Date: Mon, 21 Mar 2005 19:27:46 -0600

Henning Schulzrinne wrote:

> Do we have the PIDF mechanisms to subscribe to a group of people and 
> get everything delivered in one message?

 If you design your network for that kind of behavior, yes. That was 
kind of the motivation behind the event-list work.

/a

---

From: Aki Niemi <[email-masked]>
Date: Tue, 22 Mar 2005 11:06:06 +0200

Hi,

> difficult to implement moot?  Or am I missing something here

Naturally not subscribing in the first place would be the best way to 
avoid receiving the NOTIFYs. But I don't know what value such a presence 
system would have. The point is you'd probably want a mechanism that 
stands somewhere in between the flood and the drought.

Cheers,
Aki

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 22 Mar 2005 07:31:19 -0600

Dear Clued (not just Aki),

>> If the end point(UA) is band-width challenged, and that it is 
>> bandwidth challenged is a known datum, and these events that 
>> require throttling are all subscribed events.... wouldnt NOT 
>> subscribing to the number of events that could flood the UA make 
>> this mechanism that is going to be difficult to implement moot?  Or 
>> am I missing something here
>
> Naturally not subscribing in the first place would be the best way 
> to avoid receiving the NOTIFYs. But I don't know what value such a 
> presence system would have. The point is you'd probably want a 
> mechanism that stands somewhere in between the flood and the 
> drought.

My understanding is that with either throttling or filtering we are 
talking about making the subscription mechanism less reliable (because 
we are increasing the number of state changes that the watcher doesn't 
see), and increasing the chances of synchronization loss between the 
watcher and the presence agent.

Is this of any concern, or are people really going to write 
applications that poll anyway? (Would you tell a user that a resource 
is unavailable, instead of attempting to contact it, if you know you 
requested filtering/throttling?)

(and, if low-bandwidth devices end up polling to recover from 
filtered/throttled notifications, are we sure filtering and throttling 
will be a win? but please answer the first question first!)

Unclued

---

From: Arjun Roychowdhury <[email-masked]>
Date: Tue, 22 Mar 2005 08:49:29 -0500

> requested filtering/throttling?)

The concept of the events framework is just that - to generate a
trigger when an event occurs. A subscriber cannot guess the
frequency/nature of an event it subscribes to as effectively as the
presentity - therefore polling becomes ineffective in general.
Regarding deployment, almost all such applications are using the sub/nfy 
package - which by architecture, is notification related, not polling.
As I indicated before, the events framework gives everyone an open
license to create what they want as 'information' to the subscriber.
The unfortunate part is that services, that I personally believe can
(and should ) be done by HTTP is also being done by SUB/NFY (Yahoo
stock quotes and such).
Therefore, this problem is of serious concern in terms of potential
for network clogging.
I liked Aki's approach simply because it allows the device and/or
network to throttle a rate to a package that the "user" may have
subscribed to - and depending on what device the user is using at a
point of time (and the network she is in) - this rate can be modified.

regds
arjun

-- 
Arjun Roychowdhury

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 22 Mar 2005 09:14:52 -0500

As noted earlier, PUB/SUB is rather bandwidth-wasteful for the typical 
"can-I-call/IM-her-now" application since the vast majority of updates, 
filtered or not, will be blissfully ignored by the recipient, except a 
compulsive presence watcher that stares at the mobile screen all day to 
see the red and green dots change. Thus, trying to develop elaborate 
bandwidth management systems seems rather besides the point.

Therefore, I also agree that the simplest possible spacing mechanism, as 
proposed, is about all that is useful, and only with some strong caveats.

---

From: Adam Roach <[email-masked]>
Date: Tue, 22 Mar 2005 17:46:02 -0600

> watcher and the presence agent.

I wouldn't characterize it as a loss of synchronization as much as a delay.

/a

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 22 Mar 2005 23:17:25 -0600

Hi, Adam,

>> talking about making the subscription mechanism less reliable 
>> (because we are increasing the number of state changes that the 
>> watcher doesn't see), and increasing the chances of synchronization 
>> loss between the watcher and the presence agent.
>
>

I think I understand this - I'm trying to figure out if the delay 
turns into an effective loss of synchronization (if you run behind 
reality often enough, it would be better to run without notifications 
and just see what works when you need it to work).

Thanks,

Spencer