Subject: HTTP status code for "response too large"

From: Andreas Maier <[email-masked]>
Date: Wed, 18 Apr 2012 07:41:24 -0400

Hi Mark,
I am working for IBM in the DMTF standards org. We are defining a RESTful
protocol for CIM based directly on HTTP ("CIM-RS"), and we want to be as
truthfully RESTful and HTTP compliant as possible.

We want to support an "expand" query parameter that causes references to
resources in the result to be expanded to the resources that they
reference. One of the error situations in this context is that the
expansion can lead to a result that is too large to handle for the server
(e.g. in cases of high mutiplicities on CIM associations). The recovery for
this situation is that the client specifies less expansions in the first
request issues subsequent requests for separate expansion (one per
reference, which can then be paged into multiple responses in case of high
multiplicities).

So it is not the typical server-side recovery, where the client waits for
less load on the server or the server admin needs to add more resources to
the server. The recovery attempt can be immediately decided upon by the
client without any change in server configuration or workload.

We'd like to have a HTTP status code that allows the client to detect this
situation so it can recover from it. In order to make it easy for the
client, I think that should not be a use of status code 500, but its own
status code. I checked all status codes on the IANA HTTP status code
registry and found no one that matched this situation. But I found that the
HTTP WG is working on an RFC for additional HTTP status codes that is
currently in draft :-)

So my questions to you and the WG are:
   - Do you have a recommendation on how to handle this situation?
   - How is your view on the idea to add an HTTP status code for "response
   too large"?

PS: If you're interested in CIM-RS, what you find currently on the DMTF
site or when googling it are informational specs from an incubator that do
not yet have the "expand" query parameter. The real specs from the CIM-RS
WG are about to be released as a Work in Progress in the next weeks.

Andy

Andreas Maier
IBM Senior Technical Staff Member, Systems Management Architecture & Design
IBM Research & Development Laboratory Boeblingen, Germany
[email-masked], +49-7031-16-3654
________________________________________________________________________
IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Apr 2012 13:57:14 +0200

> HTTP WG is working on an RFC for additional HTTP status codes that is

a) It's not the HTTP WG, and b) it has been approved already.

> ...

First of all, it's not really a server error; it's the client request 
that needs to change; thus I believe it should be a 4xx.

Also, WebDAV (RFC 4918) has a similar case where servers may choose not 
to honor too complex PROPFIND requests. In this case, the server just 
sends a 403 Forbidden, and the response body contain sufficient 
additional information for a protocol-aware client to understand what 
happened. Maybe that would be an alternative.

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 18 Apr 2012 10:40:13 -0400

Hi Julian,

--On April 18, 2012 1:57:14 PM +0200 Julian Reschke <[email-masked]> 
wrote:

>>     - How is your view on the idea to add an HTTP status code for
>>     "response too large"?
>> ...
>

Also WebDAV defines 507 (Insufficient Storage) as:

11.5.  507 Insufficient Storage

   The 507 (Insufficient Storage) status code means the method could not
   be performed on the resource because the server is unable to store
   the representation needed to successfully complete the request.  This
   condition is considered to be temporary.  If the request that
   received this status code was the result of a user action, the
   request MUST NOT be repeated until it is requested by a separate user
   action.

Seems like that would be a good fit (with perhaps a response body with some 
more specific information related to the nature of the failure). We already 
have various WebDAV operations making use of that to signal truncation of 
responses and other such things.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Apr 2012 16:49:11 +0200

On 2012-04-18 16:40, Cyrus Daboo wrote:
> Hi Julian,
>
> --On April 18, 2012 1:57:14 PM +0200 Julian Reschke
> <[email-masked]> wrote:
>
> Seems like that would be a good fit (with perhaps a response body with
> some more specific information related to the nature of the failure). We
> already have various WebDAV operations making use of that to signal
> truncation of responses and other such things.

I believe you're reading something into the definition which isn't there :-)

Note:

"...means the method could not be performed on the resource because the 
server is unable to store the representation needed to successfully 
complete the request."

So that's about the payload sent with the request (such as PUT), not a 
payload generated for a response. (Otherwise 5xx wouldn't make sense 
here...)

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 18 Apr 2012 10:59:37 -0400

Hi Julian,

> payload generated for a response. (Otherwise 5xx wouldn't make sense

Except that WebDAV SEARCH, sync REPORT, and CardDAV all re-use that code to 
indicate a truncation/limit applied to the response (albeit as a status 
element inside a multi-status rather than an overall response status code). 
So in that sense the cat is already out of the bag when it comes to using 
507 to indicate limits related to the response.

That said, 403 with a sensible body indicating the nature of the error is a 
good way around having to mint specific status codes for every possible 
type of error that could occur. I wonder if that should be more formally 
defined for HTTP as it is with WebDAV (the DAV:error element response).

-- 
Cyrus Daboo

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 18 Apr 2012 08:32:01 -0700

On 18/04/2012, at 7:59 AM, Cyrus Daboo wrote:

> Hi Julian,
> 
> --On April 18, 2012 4:49:11 PM +0200 Julian Reschke <[email-masked]> wrote:
> 
> Except that WebDAV SEARCH, sync REPORT, and CardDAV all re-use that code to indicate a truncation/limit applied to the response (albeit as a status element inside a multi-status rather than an overall response status code). So in that sense the cat is already out of the bag when it comes to using 507 to indicate limits related to the response.

This is a good illustration of How Not To Do It. 507 should have been more generic, focusing on the interface rather than the implementation (as Roy constantly reminds us).

Cheers,

--
Mark Nottingham
http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Apr 2012 17:48:32 +0200

> This is a good illustration of How Not To Do It. 507 should have been more generic, focusing on the interface rather than the implementation (as Roy constantly reminds us).

I think the problem is the reverse one; the definition of 507 isn't 
precise enough, leading people to think it's about responses as well 
(and yes, me too).

Best regards, Julian

---

From: Jan Algermissen <[email-masked]>
Date: Wed, 18 Apr 2012 19:40:43 +0200

> This is a good illustration of How Not To Do It. 507 should have been more generic, focusing on the interface rather than the implementation (as Roy constantly reminds us).

FWIW, I read about 507 the first time just now and I would never have thought it applies to the response. IMHO, the definition seems to actually work quite well.

Jan

---

From: Jan Algermissen <[email-masked]>
Date: Wed, 18 Apr 2012 21:38:02 +0200

> reference, which can then be paged into multiple responses in case of high

[snip]

>   - Do you have a recommendation on how to handle this situation?

Sounds like a response *entity* issue more than anything else. Use paging links[1] to allow the server to split the response as it sees fit.  

Jan

[1] http://www.ietf.org/rfc/rfc5005.txt