Subject: RE: [Sipping] Event notification rate limiting requirements

From: [email-masked]
Date: Mon, 27 Jan 2003 11:45:42 +0200

We have to have a number here that doesn't jeopardise a service but also doesn't wake watchers susceptible to DoS attacks.

Regards,
Hisham

> -----Original Message-----
> From: ext Paul Kyzivat [mailto:[email-masked]]
> Sent: Monday, January 27, 2003 3:01 AM
> To: Jonathan Rosenberg
> Cc: Niemi Aki (NMP/Helsinki); [email-masked]
> Subject: Re: [Sipping] Event notification rate limiting requirements
> 
> 
> Jonathan Rosenberg wrote:
> > 
> > Is the assumption that the rate is a limit on the peak transmission 
> > rate, or is it a leaky-bucket style limit which provides an average 
> > maximum but allows for small bursts?
> 
> This is important! For instance, I believe the limit on presence 
> notifications is currently 5 seconds. Now suppose you have a 
> phone that 
> changes presence state to indicate "On the Phone". This means 
> that there 
> are two presence changes per call. Now limiting calls to one every 10 
> seconds may be reasonable. But saying that you must wait 5 
> seconds after 
> completing one call before answering another (or reporting 
> it) isn't so 
> reasonable.
> 
> 	Paul
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP
> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments of core SIP
> 
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: [email-masked]
Date: Mon, 27 Jan 2003 13:33:57 +0200

Hi,

Comments inline.

> It seems to me that the main issue is, as you point out, how this 
> relates to filters. The question really is, how is it that 
> the notifier 
> reduces the notification rate to the desired limit? I can see a few 
> choices here:
> 
> 1. the notifier implements a simple, package-independent 
> algorithm that 
> merely drops notifications that would cause the notification 
> rate to be 
> exceeded.
> 
> 2. the notifier implements a vendor-proprietary algorithm that 
> selectively discards notifications in an intelligent, 
> package-specific 
> way, meeting the desired notification rate limit.
> 
> 3. the notifier implements a subscriber-provided filter, and 
> its up to 
> the definition of this filter to make sure that the rate is 
> kept low enough
> 
> 4. the notifier implements a subscriber-provided filter, and if the 
> resulting notification rate is still not below the limit, it 
> implements 
> 1 or 2 above.
> 
> 
> Approach 1 is a mistake, I think. You need to intelligently filter in 
> order to still provide a useful service (approach 2). In the case of 
> buddy list presence, for example, the PA may choose an 
> algorithm which 
> round-robins through the buddy list, selecting the next buddy whose 
> state has changed since the last notification to the watcher, 
> and sends 
> their state to the watcher. THis guarantees that no buddy is 
> "starved" 
> in terms of getting its updates through to the watcher.

I think this sounds reasonable. And I agree, option 1 doesn't sound usable. So in my opinion, option 4 combined with option 2 is the preferred approach.
 
> I would tend to think that approach 2 (having the client provide just 
> the rate, and having the server implement a proprietary algorithm to 
> meet it) is useful, as is supplying a filter to limit the rate, along 
> with the orthogonal combination of the two.

I agree.
 
> A few comments on the requirements:
> 
> >  REQ3: The subscriber SHOULD be allowed to indicate support for the
> >          throttle mechanism without requiring it.
>
> what is the point of this requiremetn?

This means that the client provides the rate but doesn't require that the server abide by it. This is less strict than REQ2 in which a client provided rate is required to be used by the notifier. So REQ3 tries to lay out the leaky-bucket style limit you mention below.
 
> >  REQ8: The notifier MUST be allowed to use a maximum rate lower than
> >          the one given by the subscriber.
> 
> is this implying some kind of rate negotiation mechanism, akin to the 
> Expires value in registrations? If so, there are some more 
> requirements 
> you need, such as conveying the final maximum rate back to the client.

This tries to say that the notifier is always allowed to send less frequent notifications than what the throttle indicates, e.g., based on some local policy decision. Since the client is only really interested in limiting the rate to a specific maximum, it doesn't really matter what the actual rate is as long as it stays below the given maximum.

In any case, with a very low maximum rate, say once per subscription expiration, the net effect would actually be equivalent to polling -- a throttle might limit the notifications to exactly two per subscription.

So I'm open to suggestions on whether there is a requirement here to have a way for the notifier to at least decline a client provided rate, if it is unacceptable.

> maximum but allows for small bursts?

Both, I guess. In one model, the throttle would be given as advice to the notifier, and in the other, its use would be strictly required. In other words, a throttle could be given two ways <solution_space>with "Require" and without it.</solution_space> ;)

Overall, the model I had in mind is the one that the MWI uses. The throttle would set a minimum quarantine period for the notifications. Like in the mwi, the way notifications would be handled in this quarantine (e.g., does state get replaced and how) would have to be package specific, just like you say in option 2 above.

I'm beginning to think maybe some of this discussion should already be present in the requirements I-D?

Cheers,
Aki

---

From: [email-masked]
Date: Mon, 27 Jan 2003 14:03:30 +0200

> From: ext [email-masked] [mailto:[email-masked]]
> Sent: Monday, January 27, 2003 1:34 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: [Sipping] Event notification rate limiting requirements
> 
>
>  
> This tries to say that the notifier is always allowed to send 
> less frequent notifications than what the throttle indicates, 
> e.g., based on some local policy decision. Since the client 
> is only really interested in limiting the rate to a specific 
> maximum, it doesn't really matter what the actual rate is as 
> long as it stays below the given maximum.
> 
> In any case, with a very low maximum rate, say once per 
> subscription expiration, the net effect would actually be 
> equivalent to polling -- a throttle might limit the 
> notifications to exactly two per subscription.
> 
> So I'm open to suggestions on whether there is a requirement 
> here to have a way for the notifier to at least decline a 
> client provided rate, if it is unacceptable.
> 
> Both, I guess. In one model, the throttle would be given as 
> advice to the notifier, and in the other, its use would be 
> strictly required. In other words, a throttle could be given 
> two ways <solution_space>with "Require" and without 
> it.</solution_space> ;)
> 

I viewed this requirement from another angle; the notifier must be also able to increase the rate limit, or simply reject the request if the rate is too low. One reason for this is that local policies at servers, say Presence server, may have a limit on the frequency of notifications for security as well as bandwidth reasons. In this case, you don't want the subscriber to override this limit, hence the requirement I state above.

To be in line with registration, one would suggest rejecting the request if rate limit is too low compared to local server policy. But is it needed? Can a server just increase the limit? Do the same reasonings applied to registration apply here? (I can't remember them anymore). Certainly a require-header type solution means rejecting the request, but lets examine the requirements first.

> I'm beginning to think maybe some of this discussion should 
> already be present in the requirements I-D?

I guess.

Regards,
Hisham

---

From: [email-masked]
Date: Mon, 27 Jan 2003 14:32:37 +0200

Hi Hisham,

Inline.

> I viewed this requirement from another angle; the notifier 
> must be also able to increase the rate limit, or simply 
> reject the request if the rate is too low. One reason for 
> this is that local policies at servers, say Presence server, 
> may have a limit on the frequency of notifications for 
> security as well as bandwidth reasons. In this case, you 
> don't want the subscriber to override this limit, hence the 
> requirement I state above.

"Rate" may be confusing me here. In terms of quarantine period, or time between two consecutive notifications, Are you saying that the notifier should be able to increase the quarantine time, or decrease it?

If you mean increasing the time, I think that's what the requirements already say (a lower rate means higher time between notifications). And this is similar to the registrar declining with a min-expires - it's saving its own resources, by sending less frequent notifications.

If it's decrease, then I agree we would need some way for the notifier to either reject, or reject and adjust the client-given rate. I just don't see a huge need for this. If we assume more frequent notifications require more resources - this behavior would make things "worse" for the notifier.

Having said that, I believe rejecting (or making recommendations to that effect) throttles with ridiculously low rates may be a good idea, e.g., when the throttle achieves the same effect as doing a fetch every few hours or once a week...

> To be in line with registration, one would suggest rejecting 
> the request if rate limit is too low compared to local server 
> policy. But is it needed? Can a server just increase the 
> limit? Do the same reasonings applied to registration apply 
> here? (I can't remember them anymore). Certainly a 
> require-header type solution means rejecting the request, but 
> lets examine the requirements first.

To be in line with registrations, I don't think we necessarily need the ability to reject a throttle - with the above exceptions.
 
> > already be present in the requirements I-D?

I'll fire up my emacs then. ;)

Cheers,
Aki

---

From: [email-masked]
Date: Mon, 27 Jan 2003 14:40:06 +0200

Hi,

> completing one call before answering another (or reporting 

I agree. But isn't then the use of a throttle mechanism ill-adviced, if one requires more frequent updates than what a throttled subscription would give?

Cheers,
Aki

---

From: [email-masked]
Date: Mon, 27 Jan 2003 14:49:57 +0200

I guess I confused myself as well as others.

Yes, lower rate means higher period of time between notifies.

I was talking about subscribers increasing the rate to a number that is higher than the server's allowed maximum. If the server has a rate limit of the typical once per 5 seconds, the subscriber must not be able to increase this rate to, say, 5 times per 5 seconds. 

So the requirement is: The notifier must be able to reject a subscription containing a rate limit higher that is local configured maximum.

You are talking about the subscriber lowering that rate to a ridiculously low rate as to once per hour. I think this is ok and no need to reject those types of requests.

Regards,
Hisham

> From: Niemi Aki (NMP/Helsinki) 
> Sent: Monday, January 27, 2003 2:33 PM
> To: Khartabil Hisham (NMP/Helsinki); [email-masked]'
> Cc: [email-masked]'
> "Rate" may be confusing me here. In terms of quarantine 
> period, or time between two consecutive notifications, Are 
> you saying that the notifier should be able to increase the 
> quarantine time, or decrease it?
> 
> If you mean increasing the time, I think that's what the 
> requirements already say (a lower rate means higher time 
> between notifications). And this is similar to the registrar 
> declining with a min-expires - it's saving its own resources, 
> by sending less frequent notifications.
> 
> If it's decrease, then I agree we would need some way for the 
> notifier to either reject, or reject and adjust the 
> client-given rate. I just don't see a huge need for this. If 
> we assume more frequent notifications require more resources 
> - this behavior would make things "worse" for the notifier.
> 
> Having said that, I believe rejecting (or making 
> recommendations to that effect) throttles with ridiculously 
> low rates may be a good idea, e.g., when the throttle 
> achieves the same effect as doing a fetch every few hours or 
> once a week...
> 
> To be in line with registrations, I don't think we 
> necessarily need the ability to reject a throttle - with the 
> above exceptions.
>

---

From: [email-masked]
Date: Mon, 27 Jan 2003 14:59:04 +0200

I'm still confused... Hey, it's Monday so bear with me ;)

> I was talking about subscribers increasing the rate to a 
> number that is higher than the server's allowed maximum. If 
> the server has a rate limit of the typical once per 5 
> seconds, the subscriber must not be able to increase this 
> rate to, say, 5 times per 5 seconds. 
> 
> So the requirement is: The notifier must be able to reject a 
> subscription containing a rate limit higher that is local 
> configured maximum.

My point was that there is no need to reject such a limit, as long as the notifier MUST be allowed to use a lower rate. So the notifier may accept, and still abide by its own internal default policy of 1 notification per 5 seconds. The subscriber is still happy, since its max rate is not exceeded. Such a throttle will be utterly useless, but won't break anything...
 
> You are talking about the subscriber lowering that rate to a 
> ridiculously low rate as to once per hour. I think this is ok 
> and no need to reject those types of requests.

Except if the effects of such a filter become worse than for example, polling at the same frequency.

Cheers,
Aki

---

From: [email-masked]
Date: Mon, 27 Jan 2003 16:06:36 +0200

One would think that there is only one notify in the queue that gets replaced by an update to, for example, presence of a presentity.

If I am "on the phone" for 20 mins (which results in another presence state change to "available" after that), but my watcher is notified every 30 mins. Then I would expect the presence service to send the newest notify which carries the info that I'm available.

-5 mins - "available"
<-- NOTIFY to watcher "available"
0 mins - "on the phone"
20 mins - "available"
30 mins - "available"
<-- NOTIFY to watcher "available"

You don't want to queue and send all notifies, just the most recent one at the time of sending a notify.

/Hisham

> From: ext Henning Schulzrinne [mailto:[email-masked]]
> Sent: Monday, January 27, 2003 3:41 PM
> To: Paul Kyzivat
> Cc: Jonathan Rosenberg; Niemi Aki (NMP/Helsinki); [email-masked]
> Complexity can get out of hand pretty quickly here. Once you have a 
> leaky bucket with queueing, you have to start worrying about removing 
> pending announcements that cancel each other. No point 
> sending 'on the 
> phone' 30 minutes later if the person has long since ceased talking, 
> particularly since the user interface may not be able to tell whether 
> this notification is still current or not, particularly after you 
> gateway into other systems.
> 
> If you just drop notifications, you can get 'stuck' in the 
> wrong state, 
> like 'talking on the phone', for very long periods of time.
> 
> Paul Kyzivat wrote:
> > notifications is currently 5 seconds. Now suppose you have 
> a phone that 
> > changes presence state to indicate "On the Phone". This 
> means that there

---

From: [email-masked]
Date: Mon, 27 Jan 2003 16:18:29 +0200

If the presence info sent to the watcher is the most recently available, then fetching is no more accurate than what I described. If the location entity, that reports one's position to the presence server, only sends updates every 30 mins, and the presence server sends the watcher that last presence update when the rate limit allows it to, then its as accurate as it can get, stateless or not.

Regards,
Hisham 

> Sent: Monday, January 27, 2003 3:52 PM
> To: Khartabil Hisham (NMP/Helsinki)
> Cc: Niemi Aki (NMP/Helsinki); [email-masked]; 
> [email-masked]
> > I used this example in a separate discussion and I'll use it again
> > here: If Aki was driving down from Lapland to Helsinki, I'm
> > interested to get his presence info once every hour or so. By
> > subscribing to his presence and limiting the rate of 
> notifications to
> > 1 per hour is a good thing. I get the notifications automatically. I
> > don't have to manually subscribe every hour and waste bandwidth.
> 
> There are two classes of notifications: stateless and 
> stateful. Presence 
> (closed/busy) is stateful, in that it persists at the 
> receiver until the 
> next announcement. Location is stateless - the accuracy of 
> estimation of 
> the variable just decreases as the rate of updates decreases. In many 
> cases, physical constraints limit the error. If I know you 
> are in a land 
> vehicle, I can draw a circle around a last position report that 
> increases its radius at around 70 mph and have high 
> confidence that the 
> entity hasn't suddenly gone into hyperspace.
> 
> It's relatively easy to ratelimit the latter, much harder to 
> ratelimit 
> the former.
> 
> > 
> > The only scenario where I think a subscription with rate 
> limit may be
> > rejected is when the rate allows 0 notifies to be sent with the
> > subscription duration. I.e. Subscription duration is 1 hour and rate
> > limit is once per 2 hours. I think this scenario is worse than
> > fetching (It might be ok to accept the subscription and 
> only generate
> > the first notify).
> > 
> 
>