Subject: Fwd: [httpbis] #432: Review Cachability of Status Codes WRT "Negative Caching"

From: Mark Nottingham <[email-masked]>
Date: Mon, 11 Feb 2013 17:28:10 +1100

... and this is the ticked I just promised:

Begin forwarded message:

> From: "httpbis" <[email-masked]>
> Subject: [httpbis] #432: Review Cachability of Status Codes WRT "Negative Caching"
> Date: 11 February 2013 5:27:44 PM AEDT
> To: [email-masked]
> Reply-To: [email-masked]
> 
> #432: Review Cachability of Status Codes WRT "Negative Caching"
> ----------------------------+-----------------------------
> Reporter:  [email-masked]  |      Owner:
>     Type:  design          |     Status:  new
> Priority:  normal          |  Milestone:  unassigned
> Component:  p6-cache        |   Severity:  In WG Last Call
> Keywords:                  |     Origin:  #223
> ----------------------------+-----------------------------
> Currently, the following status codes are defined as cacheable -- that is,
> able to be stored without any explicit freshness information:
> 
> - 200 (OK)
> - 203 (Non-Authoritative Information)
> - 206 (Partial Content)
> - 300 (Multiple Choices)
> - 301 (Moved Permanently)
> - 410 (Gone)
> 
> However, many caches store other status codes (often called "Negative
> Caching")
> 
> For example, both Squid and Traffic Server (which have considerable market
> share, and form the basis of many other implementations) negatively cache
> the following status codes:
> 
> - 204 (No Content)
> - 400 (Bad Request)
> - 403 (Forbidden)
> - 404 (Not Found)
> - 405 (Method Not Allowed)
> - 414 (Request URI Too Long)
> - 500 (Internal Server Error)
> - 501 (Not Implemented)
> - 502 (Bad Gateway)
> - 503 (Service Unavailable)
> - 504 (Gateway Timeout)
> 
> While some of these may be bad to cache by default (in particular, 400 and
> 500), others may make sense: for example, 204 seems straightforward, and
> 404 seems high-value.
> 
> The major concern here is making semantic changes to the protocol.
> 
> -- 
> Ticket URL: <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/432>
> httpbis <http://tools.ietf.org/wg/httpbis/>
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Feb 2013 11:12:00 +1100

I haven't seen any discussion, and this is our last ticket (at least for the moment).

So, I'll make a proposal; we should identify the following additional status codes as cacheable (i.e., eligible for using a heuristic to determine freshness, in the absence of explicit information);

	• 204 (No Content)
	• 404 (Not Found)
	• 405 (Method Not Allowed)
	• 414 (Request URI Too Long)
	• 501 (Not Implemented)
	• 502 (Bad Gateway)
	• 503 (Service Unavailable)
	• 504 (Gateway Timeout)

Note that I'm *not* proposing the following, even though they are negatively cached by some implementations, as I suspect doing so may cause interop problems:

	• 400 (Bad Request)
	• 403 (Forbidden)
	• 500 (Internal Server Error)

Thoughts?

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 18 Feb 2013 15:44:00 +1300

> I haven't seen any discussion, and this is our last ticket (at least for the moment).

Sorry I didn't get to reply to this earlier...

You pointed at Squid as a widely used implementation doing negative 
caching. I would like to point out it in its current form it has been no 
end of trouble and we are actively advising people disable it. The 
usefulness seems to be greatest for reverse-proxy installations where 
the backend generation of errors is much more under control of the proxy 
admin. In general proxy installations it already commonly leads to the 
DoS vulnerability mentioned in the earlier thread.

For now Squid only caches these if it generated the response itself 
(with a short TTL) or if the appropriate caching controls are present 
AND the admin explicitly enabled a negative caching timeout limiting the 
storage.

> So, I'll make a proposal; we should identify the following additional status codes as cacheable (i.e., eligible for using a heuristic to determine freshness, in the absence of explicit information);

+1.  200 is cacheable, therefor all 2xx SHOULD be cacheable or at least 
harmless when cached.

> 	• 414 (Request URI Too Long)

+1. These ones make a lot of sense due to the long-term nature of the 
problems they are reporting. Even heuristic caching would seem to be safe.

> 	• 503 (Service Unavailable)

These ones are often temporary conditions and where we have the most 
trouble with Squid despite the automatic re-try of alternative routes. A 
short glitch in DNS or routing can trigger them and DoS the caches 
entire client base for an overly-large timespan. It often does make 
sense to cache them, but only for very short time (seconds at most) and 
not in caches outside of the device which generated them.

For devices with only one upstream route; the upstream outage is a 
problem for as long as the upstream sets cacheability on the reply.
For devices with multiple upstream routes; the alternative routes may 
have shorter recovery times (mandatory decrease in cached time?) or be 
fully working for this request.

Either way I am against caching these set without explicit Expiry 
information being present.

> 	• 500 (Internal Server Error)

403 Forbidden might be cacheable when a Vary:WWW-Authenticate is also in 
the reply

400 and 500, I agree should be left non-cacheable by default so that 
their future 4xx/5xx sub-codes are not affected.

/2c
Amos

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Feb 2013 14:53:15 +1100

On 18/02/2013, at 1:44 PM, Amos Jeffries <[email-masked]> wrote:
> These ones are often temporary conditions and where we have the most trouble with Squid despite the automatic re-try of alternative routes. A short glitch in DNS or routing can trigger them and DoS the caches entire client base for an overly-large timespan. It often does make sense to cache them, but only for very short time (seconds at most) and not in caches outside of the device which generated them.
> 
> For devices with only one upstream route; the upstream outage is a problem for as long as the upstream sets cacheability on the reply.
> For devices with multiple upstream routes; the alternative routes may have shorter recovery times (mandatory decrease in cached time?) or be fully working for this request.
> 
> Either way I am against caching these set without explicit Expiry information being present.

I'm fine with that; was nervous about including them on that list in the first place.

> 403 Forbidden might be cacheable when a Vary:WWW-Authenticate is also in the reply

Sure, but if you set Vary, you can also set CC; this isn't a common use, and lots of people use 403 to mean "forbidden for ill-defined reasons", so using a heuristic would likely get them in trouble.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Feb 2013 13:33:22 +1100

OK, since there's been no more discussion, I'm going to go ahead and mark the following as cacheable:

	• 204 (No Content)
	• 404 (Not Found)

Cheers,