Subject: Query string cacheability

From: Mark Nottingham <[email-masked]>
Date: Wed, 19 May 2010 22:31:43 +1000

One of the things that I did in the big caching rewrite was to remove the text about the effect of query strings on cacheability:

> Section 13.9
[...]
> 
>    We note one exception to this rule: since some applications have
>    traditionally used GETs and HEADs with query URLs (those containing a
>    "?" in the rel_path part) to perform operations with significant side
>    effects, caches MUST NOT treat responses to such URIs as fresh unless
>    the server provides an explicit expiration time. This specifically
>    means that responses from HTTP/1.0 servers for such URIs SHOULD NOT
>    be taken from a cache.

replacing it with, in p6 2.3.1.1:

>    [[REVIEW-query-string-heuristics: took away HTTP/1.0 query string
>    heuristic uncacheability.]]

Looking at this with somewhat fresh (but also a bit sleepy) eyes, I think we can re-introduce this text, but wonder if we need the last sentence; it's somewhat of a non-sequitor, AFAICT, since RFC1945 had Expires to determine an explicit expiration time, and anyway it should probably say "origin server," which as discussed before is sometimes difficult to tell, given the lack of Via support in many intermediaries.

I propose we address this by changing the beginning of 2.3.1.1 to:

"""
   If no explicit expiration time is present in a stored response that
   has a status code of 200, 203, 206, 300, 301 or 410, a heuristic
   expiration time can be calculated.  Heuristics MUST NOT be used for
   other response status codes. 

   Also, heuristic freshness MUST NOT be used for responses 
   to requests with a query component, because
   some applications have traditionally used queries on URLs to 
   perform operations with significant side effects.

   [ remaining paragraphs as in -09]
"""

Thoughts?

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 19 May 2010 15:48:43 +0200

>     [ remaining paragraphs as in -09]

Sounds good to me.

Maybe replace

"some applications have traditionally used queries on URLs to perform 
operations with significant side effects"

with

"some historic, non-compliant applications have implemented non-safe 
operations in this case"

(points being: what's in error is the server, and it always has been a 
compliance issue, no?)

Best regards, Julian

---

From: Eric Lawrence <[email-masked]>
Date: Wed, 19 May 2010 19:10:03 +0000

FWIW, my research shows that most current version browsers do not meet this requirement.  For a resource which contains no explicit lifetime information but whose URL contains a query string:

- Firefox will conditionally request/revalidate for LINK HREF (e.g. CSS) and SCRIPT SRC tags. For IMG tags, Firefox appears to revalidate the resource only once per browser session.

- Internet Explorer 8 and below revalidate such resources once per browser session, regardless of the context in which the resource is used.

- Chrome and Opera appear to ignore the query string and reuse the cached resource without validation, both during navigation and across browser restarts.

- Safari for Windows does not revalidate a page's resources during hyperlink navigation. However, it does not appear to cache heuristically cacheable content across multiple browser sessions. Safari 4.0.5 always appears to unconditionally re-request the direct target of a navigation, regardless of whether or not the resource was delivered with headers indicating it was still fresh.

Eric Lawrence
IE Program Management

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Julian Reschke
Sent: Wednesday, May 19, 2010 6:49 AM
To: Mark Nottingham
Cc: HTTP Working Group
Subject: Re: Query string cacheability

Sounds good to me.

Maybe replace

"some applications have traditionally used queries on URLs to perform operations with significant side effects"

with

"some historic, non-compliant applications have implemented non-safe operations in this case"

(points being: what's in error is the server, and it always has been a compliance issue, no?)

Best regards, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 19 May 2010 15:09:23 -0700

I would prefer to remove the paragraph from the spec.  It was not true when
it was added, it isn't true now, and it won't be true tomorrow.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 20 May 2010 10:43:03 +1000

Interesting. My tests were purely using XHR, and they found that most caches honoured this requirement (IE being the exception); <http://www.mnot.net/blog/2006/05/11/browser_caching>.

I know that Squid honours it as well (or at least did until very recently).

All things being equal, I agree with Roy that it would be good to get rid of this requirement. However, we can't just remove requirements because we don't like them.

If there really isn't existing interop over this requirement -- i.e., a significant number of caches don't honour it -- then it makes more sense. Eric, do you have anything to help reproduce your results that can be made public?

Taking a completely different tack: It's always bothered me that the heuristic is so ill-defined; someone could say that a heuristic is "if it's a HTTP request, I'll cache it." 

Is anyone aware of an implementation that uses a heuristic that *isn't* based upon the Last-Modified header? Tightening that up might make loosening this easier (because script-served query URIs don't emit LM unless they really mean it).

Cheers,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 20 May 2010 01:32:22 -0700

> I know that Squid honours it as well (or at least did until very recently).

A lot of caches do not cache responses for arbitrary URIs with query string,
but the reason has nothing to do with that requirement.  It is because those
URIs are not efficiently cacheable (unlikely to result in a later hit).

> All things being equal, I agree with Roy that it would be good to get rid of this requirement. However, we can't just remove requirements because we don't like them.

Sure we can.  It was added to the spec without implementation or consensus.
AFAIAC, it is an editorial error.  All cache implementations that I have looked
at on the server-side allow the maintainer to configure caching of specific
query-including URIs.

Browser caches are a different matter and not really the target of that
paragraph.

> Taking a completely different tack: It's always bothered me that the heuristic is so ill-defined; someone could say that a heuristic is "if it's a HTTP request, I'll cache it."

There's nothing wrong with that.  It is only when explicit caching
requirements are given by the server that we have to be concerned
about violating them.  No explicit requirements should mean cache.

> Is anyone aware of an implementation that uses a heuristic that *isn't* based upon the Last-Modified header? Tightening that up might make loosening this easier (because script-served query URIs do…

Yes.  Why are we concerned about that?

....Roy

---

From: Henrik Nordström <[email-masked]>
Date: Thu, 20 May 2010 11:56:09 +0200

> it was added, it isn't true now, and it won't be true tomorrow.

Partly agreed.

I think it would be quite unexpected for a shared cache to enforce
caching of such URLs by a expiry time set in the cache.

I technically see no problem with using the general heuristics
calculations based on Last-Modified. If a query response includes
Last-Modified time then it should be safe to cache.

The tricky query responses is those having neither an explicit expiry
time or Last-Modified and there you can find both old and new web
applications not expecting the response to get cached. But I do not have
any data on how common such applications are, nor do I have data on how
common it is to administratively configure caches to heuristically
assign a freshness interval to responses without Last-Modified.

But I also do think that practically nothing will break if this rule is
removed from the specifications, and the HTTP world will adjust with
little or no fuzz about it. The requirements on servers to clearly
indicate when responses are not intended to be cached has been around
for quite some time now, even long before the text in question was
added.

As Squid has already been mentioned I can note that current Squid
implementation implements this 2616 rule purely in the recommended
configuration of heuristic cache rules. The code as such do not make any
distinction for query URLs.

Regards
Henrik

---

From: Henrik Nordström <[email-masked]>
Date: Thu, 20 May 2010 11:59:48 +0200

tor 2010-05-20 klockan 10:43 +1000 skrev Mark Nottingham:

> Is anyone aware of an implementation that uses a heuristic that
> *isn't* based upon the Last-Modified header?

The Squid refresh_pattern min-age field is one.

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 20 May 2010 16:09:30 +0200

> it was added, it isn't true now, and it won't be true tomorrow.

How about removing that, but including a statement that for historic 
reasons, some caches will default to non-cacheable, thus it might be 
needed to explicitly specify an expiration time?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 May 2010 16:06:53 +1000

If we mention anything, it should be for content producers who believe that URIs with query strings won't be cached.

On 21/05/2010, at 12:09 AM, Julian Reschke wrote:

> How about removing that, but including a statement that for historic reasons, some caches will default to non-cacheable, thus it might be needed to explicitly specify an expiration time?
> 
> Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 May 2010 16:16:24 +1000

Now <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/211>.

---

From: David Morris <[email-masked]>
Date: Fri, 21 May 2010 07:39:59 -0700 (PDT)

As one of the strong proponents of the querystring rule in the orginal 
HTTP WG, my thinking at the time was that web application authors expected
that query string identified content would not be cached. At that time, 
there was no standard way to specify the cachability of responses. I think
this rule/suggestion was considered a bridge to the future when there was
an alternative.

While I agree, there is a suitable alternative, I think removing the 
restriction without an express deprecation note will not call sufficient
attention to the change.

Something to the effect:
  "This restriction was specified historically because it was a known
   expectation by web content developers that such content wouldn't
   be cached, the lack of consistent cache-control alternatives and
   the tendancy of some caches of the time to treat the lack of an
   expires header as doesn't expire vs. expires immediately. Cache
   controls specified for HTTP/1.1 are sufficent and should be used
   to express caching expectations.:

Dave Morris

On Fri, 21 May 2010, Mark Nottingham wrote:

> If we mention anything, it should be for content producers who believe
> that URIs with query strings won't be cached.
> 
>

---

From: Eric Lawrence <[email-masked]>
Date: Fri, 21 May 2010 22:04:50 +0000

My test case entails JS, CSS, and IMAGE resources whose references are directly into the HTML. I wouldn't be surprised at all to learn that some browsers have different behavior for XHR. It's worth noting that I only checked the Windows versions of the browsers mentioned.

Unfortunately, my test case might not be easily accessible to non-Windows users, as it's built as a Meddler Script. Meddler (www.fiddler2.com/meddler/) is a trivial little program which allows you to package a web site repro into a single file (the sort of thing that I'd imagine most folks would do in Perl). 

The test script is here: www.debugtheweb.com/dl/heuristicexpiration.ms. Basically, you need only load the script in Meddler, then visit http://127.0.0.1:8088/lm/ and then watch your HTTP traffic.

As to the question of whether anything will "break" without this requirement: I agree that for the browser world the answer is "probably not", largely because the majority of clients never implemented it. 

I only bring up the topic now because some webdevs have complained that IE6,7 and 8 do not enforce the requirement and thus they consider IE "broken" because they have to put a randomized query string on requests to avoid caching. Obviously, these webdevs don't really understand how response headers containing cache directives are meant to work, but it would be great to get the spec clarified on this particular issue.

Thanks!

-Eric

-----Original Message-----
From: Mark Nottingham [mailto:[email-masked]] 
Sent: Wednesday, May 19, 2010 5:43 PM
To: Eric Lawrence; Roy Fielding
Cc: Julian F. Reschke; HTTP Working Group
Subject: Re: Query string cacheability

Cheers,