Subject: RE: [Sipping] How to control drop algorithms for throttling

From: [email-masked]
Date: Thu, 20 Nov 2003 12:23:55 +0200

Hi Jonathan,

Inline.

 > -----Original Message-----
 > From: [email-masked] 
 > [mailto:[email-masked]]On Behalf Of
 > ext Jonathan Rosenberg
 > Sent: 14 November, 2003 01:30
 > To: sipping
 > Subject: [Sipping] How to control drop algorithms for throttling
 > 
 > 
 > This mail is a followup to some comments I made during sipping today 
 > on the event throttling work.
 > 
 > Once we add throttling capabilities to the SIP event 
 > framework, we end 
 > up with a lossy system. Some notifications will be dropped, 
 > and others 
 > will be transmitted. The problem is that all notifications are not 
 > created equally. If the notififer drops notifications at 
 > random, will 
 > the resulting stream of notifications meet the needs of the watcher? 
 > Can we do better than simply dropping notifications randomly?

I think dropping notifications randomly is almost always what we don't want. The notifications generally have ordering, so having for example a simple FIFO buffer with like a tail drop packet treatment would really mess up the state.
 
 > Some possible things we might do:
 > 
 > 1. instead of dropping notifications, the notifier buffers 
 > them. If it 
 > receives another notification within the next M seconds, it 
 > drops the 
 > older one, and sends the newer one. This would allow you to control 
 > the rate through delays. I suspect that many applications 
 > will have a 
 > better sense of the utility of the presence data as a 
 > function of age. 
 > Do we want the mechanism to support this?

This is exactly the sort of treatment I can see value in. And my assumption is that many notifiers today implement something close to this. So if I fall asleep on top of my keyboard and start making presence updates in 10ms intervals, the notifier can cope and not exceed its rate limits by mirroring this activity to the watchers.

In practice, this would mean that a notification arriving in the buffer replaces any notification still there (a LIFO with buffer size of 1). For those events that support partial state notifications, it is a bit more complicated, because the notifier would have to resolve the interdependency of the buffered notifications, e.g., by merging their state.
 
 > 2. the watcher could provide a filter type of document in the 
 > subscription. This filter could indicate that certain 
 > transitions are 
 > more important than others. For example, in the case of presence, it 
 > might say that transitions of the <basic> status are more important 
 > than transitions of the <placetype> from RPID. The notifier 
 > could use 
 > these to preferentially drop certain notifications over 
 > others. Do we 
 > want the mechanism to support this?

It really seems to me that this falls into the filtering work more than throttling. And even there, while this is quite interesting, I don't think it is a first order of priority. I think we could live without it in the first phases.
 
 > 3. If we don't use the concepts of (1) and (2), is there any 
 > value in 
 > applying algorithms like RED to the systems (I dont think 
 > so, but its 
 > worth considering).

This is an interesting idea, but I suspect applying things like the RED algorithm to the notifier buffer would need to take into ocnsideration the ordering of notifications. And in partial notification case, also the interdependency of notifications. 

At least in the first phase of throttling, this might not be such a high priority.
 
 > 4. Do we want to specify that each package should have a 
 > considerations section about how to use throttling? In particular, 
 > what will be the typical impact on service for various loss 
 > rates? Or, 
 >   which components of a document should a client ask to have dropped 
 > (assuming the filtering of (2)).

I think this is worth having. 

Cheers,
Aki
 
 > 
 > Thanks,
 > Jonathan R.
 > -- 
 > Jonathan D. Rosenberg, Ph.D.                600 Lanidex Plaza
 > Chief Technology Officer                    Parsippany, NJ 07054-2711
 > dynamicsoft
 > [email-masked]                     FAX:   (973) 952-5050
 > http://www.jdrosen.net                      PHONE: (973) 952-5000
 > http://www.dynamicsoft.com
 > 
 > 
 > 
 > _______________________________________________
 > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
 > This list is for NEW development of the application of SIP
 > Use [email-masked] for questions on current sip
 > Use [email-masked] for new developments of core SIP
 > 

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Eric Burger" <[email-masked]>
Date: Sun, 23 Nov 2003 22:00:24 -0500

I said it before, and I'll say it again:

    Rate limiting / throttling is a function of the semantics of the 
    event package.  There CANNOT be a sensible, generic rate limiting
    method.

> From: Henning Schulzrinne [mailto:[email-masked]]
> Sent: Thu, November 20, 2003 9:47 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Sipping] How to control drop algorithms for throttling
> 
> 
> 
> 
> [email-masked] wrote:
> 
> > Hi Henning,
> > 
> > Inline.
> > 
> >> Indeed, I wonder if users will really want to get continuous 
> >> updates on their buddy list members. I may want some general sense
> >> of how many of my colleagues are around and I may want to quickly
> >> determine whether a particular colleague is around right now, but
> >> clicking on the name seems perfectly fine for that, without getting
> >> the detailed updates on the person's cell phone idle time every
> >> five minutes.
> > 
> > I think you just spelled out a key use case for filtering. If I have
> > 500 buddies in my buddy list, plus restricted screen real estate,
> > there might only be room for a simple icon to show per buddy.
> > However, without being able to control the content of notifications,
> > I would still get the full presence of each buddy at all times.
> 
> My earlier message goes further than that: in many cases, 
> even filtered 
> delivery is too much useless information. Having a quick 
> SUBSCRIBE/unsubscribe on demand will work better, with no additional 
> filtering effort.
> 
> This obviously does not negate the need for filtering.
> 
> We do seem to agree that pure blind notification dropping 
> according to 
> some arbitrary rate threshold is probably the method least 
> likely to be 
> useful in practice, except for the continuous-update cases that I 
> mentioned before.
> 
> > 
> > Throttling would be able to lower the aggregate rate of those 500
> > subscriptions, but what you really want to do is control the amount
> > of content, focusing on the key things you can show to the user in
> > this buddy list view. The mechanism for that is filtering.
> 
>

---

From: [email-masked]
Date: Mon, 24 Nov 2003 10:24:20 +0200

Hi Eric,

 >     event package.  There CANNOT be a sensible, generic rate limiting

I remember you saying this about filtering. Can you explain a bit further how you see a generic throttling method go against the semantics of an event package? Do you not agree with the model in:

http://www.ietf.org/internet-drafts/draft-ietf-sipping-event-throttle-reqs-00.txt ?

Jonathan pointed out that a throttled system is a lossy one, and that we should look at how notifications are dropped. Since there is possibility for a wide variety of different event packets carrying different types of event state, the drop algorithm of the notifier should be well defined, or even configurable by the watcher, or specified event package respectively.

My argument has been that an event system is inherently a lossy one, and that the drop algorithms that are applied to notifications occurring more frequent than the default notification rate of the event package are no different from the ones applied when notifications occur more frequent than the throttled rate limit.

What the throttle ultimately does is increase the loss probability of the system. 

Regards,
Aki

 > > Sipping mailing list  
 > https://www1.ietf.org/mailman/listinfo/sipping