Subject: Re: API Rate Limits and HTTP Code [#255]

From: Bryce Nesbitt <[email-masked]>
Date: Fri, 21 Jan 2011 12:54:09 -0800

Max-age is a different concept.
I'm controlling a robot that is sequentially retrieving data... say records
50,000-60,000 out of a data set of 500,000 elements.

On Thu, Nov 11, 2010 at 7:01 PM, Mark Nottingham <[email-masked]> wrote:

> Good to hear you're using 503 + Retry-After.
>
> Instead of setting Retry-After on 200, why not just set appropriate
> Cache-Control: max-age?
>
> Regards,
>
>
> On 12/11/2010, at 9:41 AM, Bryce Nesbitt wrote:
>
> > Berkeley Electronic Press (http://www.bepress.com/) is using 503 plus
> retry_after to rate limit:
> >
> > out $cgiutils->header(-status=>"503 Excessive OAI queries from IP address
> ($ip)", -retry_after => $throttle);
> > out $cgiutils->start_html(-title=>"OAI database busy");
> >
> > But more importantly we're using (and this is an extension) a 200 error
> code for success, but also including retry_after:
> >
> > out $cgiutils->header(MIME_TYPE => 'text/xml', -retry_after =>
> $throttle);
> >
> > The goal is to say "you got this reply, but if you ask back in less than
> 5 seconds, we're going to block you.  It saves an entire server round trip.
>  Going directly by the standard our clients would need to do:
> >
> >     Request->200
> >     Request->503 retry after 10 seconds
> >     Wait ten seconds
> >     Request->200
> >     Request->503 retry after 10 seconds
> >     Wait ten seconds
> >
> > What would it take to make retry_after an officially list for 2xx?  Then
> you'd get:
> >
> >     Request->200 retry after 10 seconds
> >     Wait ten seconds
> >     Request->200 retry after 10 seconds
> >     Wait ten seconds
> >     Request->200 retry after 10 seconds
> >     Wait five seconds
> >     Request->503 retry after 5 seconds
> >     Wait five seconds
> >     Request->200 retry after 10 seconds
> >     Wait ten seconds
> >
> > We have this in production use, with a particular high volume client, who
> in fact halved the number of transactions hitting our server.
> >
> > We use 5xx because we want the client to come back later, not think the
> resource is gone.
> > We added retry_after to 2xx codes reduce the number of server round
> trips, for clients that are crawling
> > the heck out of us.
> >
> >
> > On Thu, Oct 28, 2010 at 5:33 PM, Mark Nottingham <[email-masked]> wrote:
> > Hi Karl,
> >
> > That's a very timely question; I've had similar discussions privately
> with a number of folks, and have created an issue to track it:
> >  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/255
> >
> > My personal feeling is that we should clarify one of the existing status
> codes to include this use case, allowing space for someone to define an
> additional header or two giving more information. I'm happy to be argued
> into a new status code, but don't think it's necessary at this point.
> >
> > Based on the links below, it looks like common practice is to use either
> 403 or 503.
> >
> > The commonly cited case for 4xx codes (also explained in the blog you
> link to) is that it's a client error, not a server error. While I agree
> generally, it's important to keep in mind the effects on clients. 403 (and
> any 4xx error that a client doesn't recognise) will result in a conservative
> client believing that they're not allowed to resubmit the request, whereas
> 503 + Retry-After results in the correct client behaviour.
> >
> > That said, I'm more interested in coming to an agreement and assuring
> that existing software doesn't get broken than in coming up with "perfect"
> semantics. To that end, it looks like our options are:
> >
> > a) Clarify that 503 can be used for rate limiting, or
> >
> > b) Clarify that 403 can be used for rate limiting, and allow Retry-After
> to appear there, or
> >
> > c) Define a new status code (4xx or 5xx TBD).
> >
> >
> >
> > Some other folks doing similar things:
> >
> > Github uses 403:
> >
> http://support.github.com/discussions/site/1151-api-returns-403-when-rate-limiting
> >
> > BaseCamp uses 503:
> >  http://developer.37signals.com/basecamp/
> >
> > Google seems to use 404:
> >
> http://groups.google.com/group/google-base-data-api/browse_thread/thread/cb5752b43b030fed
> >
> > Amazon appears to use 503:
> >
> http://developer.amazonwebservices.com/connect/message.jspa?messageID=142896
> >
> > Apache mod_limitipconn uses 503:
> >  http://dominia.org/djao/limit/contrib/sbarta/mod_limitipconn.c
> >
> > Lighttpd mod_evasive uses 403:
> >
> http://redmine.lighttpd.net/projects/lighttpd/repository/entry/trunk/src/mod_evasive.c
> >
> > ... as does Apache mod_evasive:
> >  http://www.zdziarski.com/blog/?page_id=442
> >
> > Nginx uses 503:
> >  http://wiki.nginx.org/NginxHttpLimitZoneModule
> >
> > Yahoo uses 403:
> >  http://developer.yahoo.com/search/rate.html
> >
> > I see references to Facebook using 341, e.g.,:
> >  http://forum.developers.facebook.net/viewtopic.php?pid=245966
> >
> > Mozilla Weave doesn't specify a status code, but re-specifies
> Retry-After:
> >  https://wiki.mozilla.org/Labs/Weave/Sync/1.0/API#X-Weave-Backoff
>
> --
> Mark Nottingham   http://www.mnot.net/
>
>
>
>
>

-- 
Bryce Nesbitt
The Berkeley Electronic Press
bepress: sustainable scholarly publishing

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 26 Jan 2011 12:37:36 +1100

The effect of a max-age on a 503 will be to tell clients they can treat that response as authoritative for the stated number of seconds. I think this will have the desired effect -- i.e., clients that are aware of this will avoid making requests for the given period.

The bonus, of course, is that caches are already widely deployed, so it's easy to leverage this in.

You'll either need to send CC: private or an appropriate Vary header to assure that the 503 doesn't "spill" to other users of a shared cache.

What am I missing? 

Cheers,

On 22/01/2011, at 7:54 AM, Bryce Nesbitt wrote:

> I'm controlling a robot that is sequentially retrieving data... say records 50,000-60,000 out of a data set of 500,000 elements.
> 
> 
> Instead of setting Retry-After on 200, why not just set appropriate Cache-Control: max-age?
> 
> Regards,
> 
> 
> > Berkeley Electronic Press (http://www.bepress.com/) is using 503 plus retry_after to rate limit:
> >
> > out $cgiutils->header(-status=>"503 Excessive OAI queries from IP address ($ip)", -retry_after => $throttle);
> > But more importantly we're using (and this is an extension) a 200 error code for success, but also including retry_after:
> >
> > out $cgiutils->header(MIME_TYPE => 'text/xml', -retry_after => $throttle);
> >
> > The goal is to say "you got this reply, but if you ask back in less than 5 seconds, we're going to block you.  It saves an entire server round trip.  Going directly by the standard our clients would need to do:
> >
> >     Request->200
> > What would it take to make retry_after an officially list for 2xx?  Then you'd get:
> >
> > We have this in production use, with a particular high volume client, who in fact halved the number of transactions hitting our server.
> >
> > We use 5xx because we want the client to come back later, not think the resource is gone.
> > We added retry_after to 2xx codes reduce the number of server round trips, for clients that are crawling
> > the heck out of us.
> >
> >
> > That's a very timely question; I've had similar discussions privately with a number of folks, and have created an issue to track it:
> > My personal feeling is that we should clarify one of the existing status codes to include this use case, allowing space for someone to define an additional header or two giving more information. I'm happy to be argued into a new status code, but don't think it's necessary at this point.
> >
> > Based on the links below, it looks like common practice is to use either 403 or 503.
> >
> > The commonly cited case for 4xx codes (also explained in the blog you link to) is that it's a client error, not a server error. While I agree generally, it's important to keep in mind the effects on clients. 403 (and any 4xx error that a client doesn't recognise) will result in a conservative client believing that they're not allowed to resubmit the request, whereas 503 + Retry-After results in the correct client behaviour.
> >
> > That said, I'm more interested in coming to an agreement and assuring that existing software doesn't get broken than in coming up with "perfect" semantics. To that end, it looks like our options are:
> >
> > b) Clarify that 403 can be used for rate limiting, and allow Retry-After to appear there, or
> >
> > Mozilla Weave doesn't specify a status code, but re-specifies Retry-After:

---

From: Bryce Nesbitt <[email-masked]>
Date: Fri, 28 Jan 2011 11:19:42 -0800

Mark,
What you're missing is the proper use case.  I agree with you for a
single resource -- max-age on the 503 is great for a resource that has
freshness associated.

I'm talking about incremental retrieval of a large data set.  Each
resource is new to the client, and will not be cacheable:

GET /oai2.cgi?token=Az123Z&offset=0&count=100&query='Obama'
HTTP/1.1 200
Retry-After: 10
Cache-Control: no-store, no-cache, private

GET /oai2.cgi?token=Az123Z&offset=100&count=100
HTTP/1.1 503 Excessive queries from this IP address.  Please respect
the Retry-After header.
Retry-After: 10
Cache-Control: no-store, no-cache, private

GET /oai2.cgi?token=Az123Z&offset=100&count=100
HTTP/1.1 200
Retry-After: 5
Cache-Control: no-store, no-cache, private

GET /oai2.cgi?token=Az123Z&offset=100&count=100
HTTP/1.1 200
Retry-After: 5
Cache-Control: no-store, no-cache, private

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 29 Jan 2011 16:04:27 +1100

It seems like you want the scope of the 503 on the Retry-After to be the entire server, correct? I.e., on a 200, its semantic is roughly "here's your answer, but don't ask me (the server) another question for n seconds."

---

From: Karl Dubost <[email-masked]>
Date: Sat, 29 Jan 2011 08:41:52 -0500

> It seems like you want the scope of the 503 on the Retry-After to be the entire server, correct? I.e., on a 200, its semantic is roughly "here's your answer, but don't ask me (the server) another q…

Given the example below I would say for a group of URIs pattern more than the full server. you could set for sure api.example.com but if the server is done with a 
example.com/api
example.com/profile
or even
example.com/api2

And there are different limits based on your subscription levels: Premium for api and silver for api2.

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 30 Jan 2011 00:50:31 +1100

That's the concern underlying my question; scoping metadata in HTTP headers -- especially existing ones -- is often an exercise in "I meant what I said, and I said what I meant." 

While it's fine for Bryce to document what he means for interested developers to figure out, it's less possible for automated, generic software (e.g., a robot, spider, cache) to understand what concepts of user identity and metadata scope he has in mind. And, that's the point of defining it as a standard. 

Furthermore, if we define a scope that's sufficiently precise to be good for Bryce, it's very likely that it won't be good for someone else who's minted a different API.

At this point I'd observe that HTTP doesn't disallow Retry-After on a 2xx response; only that it doesn't define what it means. As such, Bryce is free to define additional semantics for it (or if that is distasteful, another header) so that people who read his documentation will know what to do with it. However, as per above I don't see how we can explicitly allow it without falling into this trap (beyond perhaps having some general introductory text about what is and isn't permissible in cases like this).

Those are my thoughts at the moment; it's late here, so I'll stop there.