Subject: Retry-After header on 20X response -- HTTP/1.1 spec extension?

From: "Bryce Nesbitt" <[email-masked]>
Date: Mon, 22 Dec 2008 20:51:45 -0800

Dear Working Group Folks,

I am not a member of the working group.  But I have recently been tempted to
"stretch" the HTTP spec, and I'm writing to inquire if what I'm doing is
reasonable enough to eventually fold into the spec.

Basically I'm sending a Retry-After header on a 20x HTTP response.

I'm working with a "throttled" data service which rate limits connections.
Clients are harvesting a huge volumes of data over time.  Presently clients
get some data with a 200 result, ask again right away and get a 503
response, then wait out the proper Retry-After time.

If I can return Retry-After with the 20x result, it will cut the total
requests in half.  Clients can ask for data, and know immediately how long
to wait before they ask again.  Only a client that violates the timeout
would ever see a 503.

The HTTP/1.1 spec is pretty clear (in section 14.37) that Retry-After is for
503 and 3xx return codes only. Your thoughts?  Where would I go to suggest
an expansion of the Retry-After header, to be inclusive of 20x results?  Is
this a reasonable extension in your view?

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 24 Dec 2008 00:52:38 +1300

Hi Bryce

I couldn't find a browser that honoured Retry-After even on a 3xx 
response.  So I wouldn't hold out much hope about widespread support for 
honouring it on a 2xx.  It looks like it hasn't really been picked up by 
the browser vendors as far as I can tell.

I was testing it for rate-limiting connections, redirecting the client 
back to the URL they just requested if the rate was over threshold.

Interesting behaviour from different clients.

IE went into a retry loop, I clocked over 3500 req/s
Chrome and FF just showed an error page complaining about a retry loop.
Didn't test much after that due to unusable results of the main browsers.

Regards

Adrien

Bryce Nesbitt wrote:
> I am not a member of the working group.  But I have recently been 
> tempted to "stretch" the HTTP spec, and I'm writing to inquire if what 
> I'm doing is reasonable enough to eventually fold into the spec.
>
> I'm working with a "throttled" data service which rate limits 
> connections.   Clients are harvesting a huge volumes of data over 
> time.  Presently clients get some data with a 200 result, ask again 
> right away and get a 503 response, then wait out the proper 
> Retry-After time.
>
> requests in half.  Clients can ask for data, and know immediately how 
> long to wait before they ask again.  Only a client that violates the 
> timeout would ever see a 503.
>
> The HTTP/1.1 spec is pretty clear (in section 14.37) that Retry-After 
> is for 503 and 3xx return codes only. Your thoughts?  Where would I go 
> to suggest an expansion of the Retry-After header, to be inclusive of 
> 20x results?  Is this a reasonable extension in your view?

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: "Bryce Nesbitt" <[email-masked]>
Date: Tue, 23 Dec 2008 08:30:35 -0800

But, browser support is not pertenant to this particular need.  This is a
need 100% in the world of robots.
And in this case I'm dealing with robots that are already required to
properly handle the 503 response.

For an example, see
http://www.openarchives.org/OAI/openarchivesprotocol.html

So what's the procedure for proposing a spec change like this, aimed at
robot efficiency, and understanding that browsers don't care now and won't
care in the future?  Keep in mind that if a 20X Retry-After header is
ignored, the robot will simply get a 503 later.  So the compatible fallback
is perfect.  And if browser authors really wanted to get into the game, they
could play also, as long as they fix their 503 implementation first.

On Tue, Dec 23, 2008 at 3:52 AM, Adrien de Croy <[email-masked]> wrote:

>
>
> Hi Bryce
>
> I couldn't find a browser that honoured Retry-After even on a 3xx response.
>  So I wouldn't hold out much hope about widespread support for honouring it
> on a 2xx.  It looks like it hasn't really been picked up by the browser
> vendors as far as I can tell.
>
>