Subject: dont-revalidate Cache-Control header

From: Ben Maurer <[email-masked]>
Date: Thu, 9 Jul 2015 15:25:20 -0700

It is considered a best practice for websites to use "long caching" for
serving images, javascript and CSS. In long cacing if a website has a
resource X.js which might change from time to time, rather than referencing
X.js and giving the endpoint a short expiration date, they reference
X-v1.js with a nearly infinite expiration date. When X.js changes, the
website uploads X-v2.js and changes any references to use the new version.
This has the benefit that the browser never needs to revalidate resources
and that it sees changes instantly. [1]

At Facebook, we use this method to serve our static resources. However
we've noticed that despite our nearly infinite expiration dates we see
10-20% of requests (depending on browser) for static resource being
conditional revalidation. We believe this happens because UAs perform
revalidation of requests if a user refreshes the page. Our statistics show
that about 2% of navigations to FB are reloads -- however these requests
cause a disproportionate amount of traffic to our static resources because
they are never served by the user's cache.

A user who refreshes their Facebook page isn't looking for new versions of
our Javascript. Really they want updated content from our site. However UAs
refresh all subresoruces of a page when the user refreshes a web page. This
is designed to serve cases such as a weather site that says <img
src="/weather.php?zip=94025">. If that site had a 10 minute expiration on
the image, the user might be able to refresh the page and get more up to
date weather.

10-20% additional requests is a huge impact for a site's performance. When
a user presses refresh, they want to quickly see the latest updates on the
site they are on. Revalidating all resources on the site is a substantial
drain. In discussing this with Ilya Grigorik from the Chrome team, one
suggestion that came up was an explicit cache control option to tell UAs
not to revalidate. The proposed header would be Cache-Control:
dont-revalidate

This header would tell a UA that the resource in question is unquestionably
valid until it's expiration time. A UA MUST NOT send a conditional request
for a resource with a dont-revalidate header prior to the resource's
expiration. In most cases the UA SHOULD simply treat the resource as valid.
If the UA is not willing to treat the resource as valid, it should send a
full request with no revalidation. The UA MAY do this in cases where the
user has explicitly requested to use a clean cache -- for example a refresh
via ctrl+shift+r, or using developer tooling. Such functionality SHOULD be
targeted at advanced users rather than the average user.

Without an additional header, web sites are unable to control UA's behavior
when the user uses the refresh button. UA's are rightfully hesitant in any
solution that alters the long standing semantics of the refresh button (for
example, not refreshing subresources).

[1]
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#invalidating-and-updating-cached-responses

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 10 Jul 2015 22:29:36 +1200

On 10/07/2015 10:25 a.m., Ben Maurer wrote:
> serving images, javascript and CSS.

Sadly true. This myth is very common amongst web application developers
and leads to a lot of broken websites and annoyance amongst customers.

Speaking with my dual hats of Web Designer and Software Engineer
developing caching proxy software;

Best Practice is actually to put *appropriate* caching times and
revalidation limits on objects regardless of original file type. What is
appropriate depends on the predicted volatility of the individual
objects. Object type is secondary to that volatility criterion.

> In long cacing if a website has a
> X-v1.js with a nearly infinite expiration date.

I know where this started... An old tutorial written back in the late
1990's when HTTP/1.0 expiration was the only type of caching available
and the first Browser War was in full swing. "Archaic" is the best word
to describe it.

It applies badly to HTTP/1.1 caching situations and can actually
*reduce* cacheability of objects if applied indiscriminantly. Also
introducing the possibility of nasty human errors via the manual version
control system.

There are a few edge cases where it applies well. But Best Practice it
certainly is NOT in the current web environmant.

> When X.js changes, the

These days we have HTTP/1.1 revalidation. Where the object ETag is
derived from either stored mtime value, a hash of the object, or both
the HTTP/1.1 software out there today can take care of version control
easily and fast without any manual assistance needed from the web dev or
duplicated copies of things hanging around.

> they are never served by the user's cache.

That tells me that 10-20% of your traffic is probably coming from a
HTTP/1.1 proxy cache. Whether it reveals itself as a proxy or not.

Speaking for Squid, we limit caching time at 1 year**. After which
objects get revalidated before use. Expires header in HTTP/1.1 only
means that objects are stale and must be revalidated before next use.
Proxy with existing content does that with a synthesized revalidation
request even if the client that triggered it did a plain GET. Thereafter
the proxy has a new Expires value to use*** until that itself expires.

** Plan is to extend that to 68 years as per new RFC7234 spec when the
compliance updates get done in that area.

*** Assuming the server actually did change the Expires header. There
are a number that send back the same old already-expired value for years
on end.

> valid until it's expiration time.

That meaning is already provided by the Expires header and
Cache-Control:max-age=N value.

I dont see how adding yet another way to signal it will help with the
software revalidating today.

I do think I know where you are coming from with this, and kind of
agree. The UA whose refresh button goes straight to reload-everything
instead of efficient revalidate-only-as-needed behaviour is broken IMHO.
However that is a UA bug, not a protocol problem.

> A UA MUST NOT send a conditional request
> expiration.

Placing that criteria on objects will guarantee that your bandwidth bill
becomes rather large when it need not. Why would you bother sending any
Cache-Control at all in the case you wanted such a costly bill? "Just
send a 200 reply with new object in response to revalidation requests."
does the same thing and I hope you understand how that effects the
efficiency of caching.

> In most cases the UA SHOULD simply treat the resource as valid.

This statement goes for all responses not actively labeled as already
stale. Again no need for a new control signal.

> full request with no revalidation.

As I mentioned above the revalidation may not have come from the UA. It
may be requesting full complete copies of all objects on the page. A
middleware cache may be revalidating its own content to optimize the
cache->server connection bandwidth costs.

> The UA MAY do this in cases where the

IMHO this control is unnecessary and an intrusion onto physical User
expectations of their GUI behaviour. What we need is a standard
behaviour from the button that works identically across all browsers.

The Ctrl-r (revalidate-as-needed) and Ctrl+Shift+r / Ctrl-R (reload
completely from scratch) is a good defacto standard already. It just
needs to be consistently followed by all browsers and UA. The selective
'as-needed' part may be new right now [hint, hint] and some older UA
needing bug fixes to realign, but thats code not protocol.

Today I had to help a user with an app unfamiliar to both of us on an OS
I'd never heard of before recover some accidentally deleted text
(uh-oh). Of course per Murphys Law it was an urgent recovery with
seconds literally ticking up an invoice cost. Trying Ctrl-Z on the off
chance it worked ... one happy user.
Consistent basic UA behaviour is good UA design.

HTH
Amos

---

From: Ben Maurer <[email-masked]>
Date: Fri, 10 Jul 2015 10:14:26 -0700

> introducing the possibility of nasty human errors via the manual version

Nobody is recommending people do this manually, just as we don't recommend
people write javascript without whitespace. A build process should take
resources such as JS and CSS, minify them and create uniquely named versions

> certainly is NOT in the current web environmant.

This practice seems to be one that is widely recommended -- eg the Google
guide in my original email, Steve Souders's book
https://books.google.com/books?id=jRVlgNDOr60C&lpg=PA23&ots=pbw_DA5ce0&dq=cache%20expiration%20steve%20souders&pg=PA27#v=onepage&q=cache%20expiration%20steve%20souders&f=false.

At Facebook we've found that this technique dramatically increases user
performance. A number of other large websites also deploy it.

> duplicated copies of things hanging around.

ETag validation is exactly what we want to avoid. Validating an etag still
requires a round trip from the client to the server to revalidate the tag.
When we serve a resource X.js on our page, if the user has the current
version of X.js in their cache we want them to use it without a round trip
to the server. Naming the file X-<version>.js accomplishes this, as if the
server references X-v20.js the client knows that v20 is the current
preferred version and that if it has cached v20 it can use it without an
extra RTT.

> HTTP/1.1 proxy cache. Whether it reveals itself as a proxy or not.

Facebook is served exclusively over HTTPS meaning that we see relatively
few proxies. We are fairly sure that we're not seeing this level of proxy
traffic and that the revalidations are from UA triggered refreshes.

> However that is a UA bug, not a protocol problem.

This is the crux of the problem -- as a website that carefully manages its
cache control headers, we want UAs to treat a reload as a normal navigation
and to respect normal caching rules. However the de facto behavior of UAs
is that a refresh causes revalidation of all resources on the page.

There are two possible solutions:

1) Ask UAs to change their behavior. This will silently change the behavior
of websites that do nothing. Maybe there's a website out there that says
"please hit the refresh button on your website to see the latest weather".
That site will no longer work if it depends on the revalidation behavior.
In my discussions with UA implementers, they seem unwilling (or at least
extremely hesitant) to take such a risk

2) Create a new behavior that websites can opt in to. Ensure that UAs
implement it consistently. This has less risk of breaking existing sites,
though I understand the hesitance to have a header that says "no *REALLY*
trust my expiration times". Perhaps the header is poorly advertising the
functionality that we wish to achieve. A better name/behavior might be
Cache-control: content-addressed. content-addressed would signal that the
contents of the current URL is a pure function of the URL itself. IE, that
the contents will never change. It would take priority over a max-age
header and signal to the browser that the resource should be permanently
cached.

---

From: Zhong Yu <[email-masked]>
Date: Fri, 10 Jul 2015 16:57:06 -0500

Questions:

1. MUST NOT seems too harsh. It'll put UAs that don't yet understand the
directive as non-compliant. And we can't be sure whether there could be
legitimate reasons for UAs to send conditional GET.

2. If the user views a subresource, e.g. an embedded image, directly as a
top resource, and the user refreshes UA, what should UA do? It's unlikely
that UA will refuse to oblige the refresh request.

3. If UA requests an older version of a resource that the server no longer
keeps, how should the server respond? If the server returns the latest
version, it'll compromise the semantics of the URI. This requires clearer
specification.

Zhong Yu
bayou.io

---

From: Ben Maurer <[email-masked]>
Date: Fri, 10 Jul 2015 15:15:13 -0700

> legitimate reasons for UAs to send conditional GET.

What about something like:

The UA MUST re-use the request in response to normal page loading or reload
requests discoverable by the average user. The UA SHOULD make
non-conditional requests when specifically requested via advanced browser
functionality (eg, developer tools, shift+refresh)

My ultimate goal here is to set very clear expectations for UAs in handling
this content.

> that UA will refuse to oblige the refresh request.

Yes, I think it would be very reasonable for UAs to only implement this on
subresources. This prevents a situation where a website accidentally sends
down a dont-revalidate page in response to GET / and then the user can
never recover from the error except for by clearing cache. If the
accidental header occurs on a subresource the website can simply change the
resource.

> version, it'll compromise the semantics of the URI. This requires clearer

What do you mean by request an old version? If the browser requests
/X-v2.js but the current version os /X-v3.js, the server should respond to
/X-v2.js with v2 of the javascript. If the developer has deleted that
javascript, the server should return a 404. The developer should never
re-use the /X-v2.js URI for other content. They could choose to re-upload
/X-v2.js to the same URL in the future if they wanted to so long as it had
the same content.

Basically, if you have a URI /X-vY.js once you serve a dont-revalidate
response on that URI you must forever either (1) serve the same content (2)
serve an error.

That said, your content doesn't necessarily have to be byte-for-byte
identical, it just needs to be logically the same. For example, you might
have a CDN that gzip encodes the file. If two nodes on the CDN gziped the
file slightly differently that would be OK. The key is that you have
absolutely no expectation that browsers will ever re-fetch a
dont-revalidate URI once they have fetched it.

-b

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 11 Jul 2015 11:33:30 +1000

Hi Ben,

> On 11 Jul 2015, at 3:14 am, Ben Maurer <[email-masked]> wrote:
> 
> 2) Create a new behavior that websites can opt in to. Ensure that UAs implement it consistently. This has less risk of breaking existing sites, though I understand the hesitance to have a header that says "no *REALLY* trust my expiration times". Perhaps the header is poorly advertising the functionality that we wish to achieve. A better name/behavior might be Cache-control: content-addressed. content-addressed would signal that the contents of the current URL is a pure function of the URL itself. IE, that the contents will never change. It would take priority over a max-age header and signal to the browser that the resource should be permanently cached. 

This seems like the crux of the matter. A CC extension is one way to do this, but I wonder if a more appropriate place might be in HTML, since this is really about how the browser behaves in reaction to user input, not how the cache behaves.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Ben Maurer <[email-masked]>
Date: Sat, 11 Jul 2015 10:58:00 -0700

Hey Mark,

This suggestion also came up in prior discussions of the problem. One major
issue with this solution is that it doesn't address situations where
content is embedded in a third party site. Eg, if a user includes an API
like Google Maps or the Facebook like button those APIs may load
subresources that should fall under this stricter policy. This issue cuts
both ways -- if 3rd party content on your site isn't prepared for these
semantics you could break it.

I believe UAs were worried about the potential security implications of
such an extension -- eg what if you could alter the caching semantics of a
SWF that had privileged cross origin access so that you could extend the
lifetime of a vulnerable version (granted this extension would only make
the situation worse in the case where the user refreshed the page, so it
might not be that big of a deal).

A third design I had discussed would be a heuristic. For example:

If a UA would normally revalidate a fresh resource due to a user action
expressing interest in requesting a fresh page (eg if the user presses the
reload button) it SHOULD use the following heuristic to avoid excessive
revalidations. If the difference between freshness_lifetime and current_age
is less than 7 days, then the UA SHOULD NOT perform the revalidation and
serve the resource.

The idea here is that reloading is such a rare action (2% of page views
according to our stats) that no reasonable site would design their
experience around the user using the reload button. On the other hand it
might be reasonable for the site to have something with a 5 minute
freshness lifetime and for the user to want an up-to-date version of the
resource

Pros:
- No changes to existing headers
- Would provide a speedup for people who are currently following best
practices

Cons:
- Feels like magic
- Theoretically changes the behavior of a site which actively encourages
it's users to use the reload button (I've never seen such a site)
- Forces a developer who actively edits their code to use short lifetimes
or ctrl+shift+r
- UAs are hesitant to do anything which changes existing behavior.

Do you have any more thoughts on preferences between these options?

On Fri, Jul 10, 2015 at 6:33 PM, Mark Nottingham <[email-masked]> wrote:

> This seems like the crux of the matter. A CC extension is one way to do
> this, but I wonder if a more appropriate place might be in HTML, since this
> is really about how the browser behaves in reaction to user input, not how
> the cache behaves.
>
> Cheers,
>
>
> --

---

From: Adam Rice <[email-masked]>
Date: Mon, 13 Jul 2015 17:23:02 +0900

> it's users to use the reload button (I've never seen such a site)

https://www.google.com/search?q="click+reload+to"

---

From: Ben Maurer <[email-masked]>
Date: Mon, 13 Jul 2015 10:57:40 +0100

Hard to gather if "click reload to" in these cases is trying to get the
user to simply re-load the current URL or if they actually want the user to
change cache semantics. Maybe there's concrete data we could gather here.
For example, it would be amazing if we could get a UA to measure what % of
revalidations it sends result in not modified vs new content and if
resources with a distant expiration date that are forcibly revalidated via
a refresh are particularly unlikely to be refreshed. We could also study
this in the HTTP Archive -- if I took all resources that had a 30 day or
greater max age and send their servers revalidation requests 1 week from
today, what % of them return a 304 vs other responses.

That said, I completely understand why there would be hesitance to change
the semantics of this button given that it is implemented consistently
today across UAs -- it may be that the only way to implement this type of
behavior is a new CC header or an HTML tag to alter the behavior.

---

From: Ilya Grigorik <[email-masked]>
Date: Mon, 13 Jul 2015 15:31:04 -0700

> the proxy has a new Expires value to use*** until that itself expires.

Amos, not sure I follow the proxy conclusion.. I'm reading this correctly,
it sounds like if I specify a 1 year+ max-age, then Squid will revalidate
the object for each request? If so, ouch. However, unless that gotcha
accounts for all of the extra revalidations, why would the proxy cause more
revalidations? Intuitively, shouldn't it reduce the number of revalidations
by collapsing number of requests to FB origin?

(also, as Ben noted, due to HTTPS, I doubt that's the culprit...)

On Sat, Jul 11, 2015 at 10:58 AM, Ben Maurer <[email-masked]> wrote:

> One major issue with this solution is that it doesn't address situations
> where content is embedded in a third party site. Eg, if a user includes an
> API like Google Maps or the Facebook like button those APIs may load

Hmm, I think a markup solution would still work for the embed case:
- you provide a stable embed URL with relatively short TTL (for quick
updates)
- embedded resource is typically HTML (iframe) or script, that initiates
subresources fetches
-- said resource can add appropriate attributes/markup on its subresources
to trigger the mode we're discussing here

^^ I think that would work, no? Also, slight tangent.. Fetch API has notion
of "only-if-cached" and "force-cache", albeit both of those are skipped on
"reload", see step 11:
https://fetch.spec.whatwg.org/#http-network-or-cache-fetch.

On Mon, Jul 13, 2015 at 2:57 AM, Ben Maurer <[email-masked]> wrote:

> We could also study this in the HTTP Archive -- if I took all resources
> that had a 30 day or greater max age and send their servers revalidation
> requests 1 week from today, what % of them return a 304 vs other responses.

Not perfect, but I think it's should offer a pretty good estimate:
http://bigqueri.es/t/how-many-resources-persist-across-a-months-period/607

- ~48% of resource requests end up requesting the same URL (after 30 days).
Of those...
-- ~84% fetch the same content (~40% of all request and ~33% of total bytes)
-- ~16% fetch different content (~8% of all requests and ~9% of total bytes)

ig

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 14 Jul 2015 21:12:41 +1200

On 14/07/2015 10:31 a.m., Ilya Grigorik wrote:
> the object for each request?

No, for the first year you get normal caching behaviour. Then from the
1yr mark you get one-ish revalidation, and the new copy is used the
resets the 1yr counter. So instead of getting things cached forever /
68yrs (possibly by error). You get at least one revalidation check per
year per object.

> If so, ouch. However, unless that gotcha

It would reduce the total requests yes. But in doing so I would expect
some ratio of total requests (relative to the amount of proxy usage) to
become revalidation instead of full fetches. Since a proxy does not have
a reload button, and its quite popular to configure proxy caches to
convert client reload into an IMS request.

What I mean is; you may find those 10% of revalidations instead of
disappearing, could turn into full fetches. That would raise the total
latency cost by the bandwidth transfer time without reducing the server
CPU latency to identify object versions.

> (also, as Ben noted, due to HTTPS, I doubt that's the culprit...)

FWIW: Judging by the user queries and Debian/Ubuntu install statistics
in the past year some 10,000+ Squid devices (minimum. Lots of CentOS,
RHEL and BSD unaccounted for) have been converted to HTTPS MITM due to
administrative and legal requirements (specifically for filtering
Facebook and Google traffic). Use of HTTPS is not avoiding the proxy
interaction.

(This conversion of previously perfectly standards compliant installs to
MITM is depressing).

> 
> On Sat, Jul 11, 2015 at 10:58 AM, Ben Maurer wrote:
> 
> On Mon, Jul 13, 2015 at 2:57 AM, Ben Maurer wrote:
> 

Pretty much inline with what we see in average proxy cache HIT rates.
Specific ISP situations vary from 20%-60% caching depending on customer
counts vs storage space size. Those who can cache only a weeks traffic
get lower rates than those caching a months, etc.

I expect to see similar numbers from browser caches.

Amos

---

From: Ben Maurer <[email-masked]>
Date: Tue, 14 Jul 2015 11:03:10 +0100

> 68yrs (possibly by error). You get at least one revalidation check per

We are seeing this behavior on objects that have not existed for 1 year.
So I don't think we are triggering that behavior.

> > https://fetch.spec.whatwg.org/#http-network-or-cache-fetch.

One version of the "markup option" could be allowing us to explicitly
override the Fetch attributes of a subresource. Eg, set the cache-mode to
force-cached. This option does present the challenge of ensuring that we
address every possible subresource (think url()'s in css). Being able to
control fetch options for sub resources is extremely useful.

That said, this doesn't feel like a great thing for us to promote as a web
performance best practice. "If you use long cache lifetimes for your static
content, the dont-revalidate cache control header will reduce the cost of
client reloads" seems like a piece of advice folks might take, as would
"Use the <meta> tag 'dont-reload-non-expired-resources' to avoid browsers
revalidating your content when the user presses reload". On the other hand
"you should find every image, script, stylesheet, etc and set the fetch
option on each to say force-cached" feels more tedious and unlikely to be
used.

> > -- ~16% fetch different content (~8% of all requests and ~9% of total

Is it possible to limit this to only resources which claimed a > 30 day
max-age in the first request?

-b

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 14 Jul 2015 14:01:57 +0200

Hey Ben,

> On 12 Jul 2015, at 3:58 am, Ben Maurer <[email-masked]> wrote:
> 
> Hey Mark,
> 
> This suggestion also came up in prior discussions of the problem. One major issue with this solution is that it doesn't address situations where content is embedded in a third party site. Eg, if a user includes an API like Google Maps or the Facebook like button those APIs may load subresources that should fall under this stricter policy. This issue cuts both ways -- if 3rd party content on your site isn't prepared for these semantics you could break it.

Yes. That problem (and similar ones) is coming up in a number of contexts, so there are quite a few headers being minted at the moment :-/

Maybe a HTML control for first-party and near-party control, and a header for third-party? (just thinking out loud; not a big fan of duplication of effort in general, but it might be worthwhile here).

> I believe UAs were worried about the potential security implications of such an extension -- eg what if you could alter the caching semantics of a SWF that had privileged cross origin access so that you could extend the lifetime of a vulnerable version (granted this extension would only make the situation worse in the case where the user refreshed the page, so it might not be that big of a deal).

Sorry - you lost me at "secure" and "swf" :)

> If a UA would normally revalidate a fresh resource due to a user action expressing interest in requesting a fresh page (eg if the user presses the reload button) it SHOULD use the following heuristic to avoid excessive revalidations. If the difference between freshness_lifetime and current_age is less than 7 days, then the UA SHOULD NOT perform the revalidation and serve the resource.
> 
> The idea here is that reloading is such a rare action (2% of page views according to our stats) that no reasonable site would design their experience around the user using the reload button. On the other hand it might be reasonable for the site to have something with a 5 minute freshness lifetime and for the user to want an up-to-date version of the resource
> 
> Pros:
> - Would provide a speedup for people who are currently following best practices
> 
> Cons:
> - Feels like magic
> - Theoretically changes the behavior of a site which actively encourages it's users to use the reload button (I've never seen such a site)
> - Forces a developer who actively edits their code to use short lifetimes or ctrl+shift+r

I agree with your dislike of magic…

From time to time, we've had people ask for "Cache-Control: Infinity-I-really-will-never-change-this." I suspect that often they don't understand how caches work, and that assigning a one-year lifetime is more than adequate for this purpose, but nevertheless, we could define that so that it worked and gave you the semantics you want too.

To keep it backwards compatible, you'd need something like:

Cache-Control: max-age=31536000, static

(or whatever we call it)

Cheers,

---

From: Ben Maurer <[email-masked]>
Date: Tue, 14 Jul 2015 13:36:11 +0100

On Tue, Jul 14, 2015 at 1:01 PM, Mark Nottingham <[email-masked]> wrote:

> From time to time, we've had people ask for "Cache-Control:
> Infinity-I-really-will-never-change-this." I suspect that often they don't
> understand how caches work, and that assigning a one-year lifetime is more
> than adequate for this purpose, but nevertheless, we could define that so
> that it worked and gave you the semantics you want too.
>

Static is a fairly reasonable name. Static does imply that the resource
will *never* be revalidated (ever) vs the dont-reload which implies no
revalidations prior to expiration. I don't have any preferences between the
two but wanted to call that out.

What's the next step here?

---

From: Ilya Grigorik <[email-masked]>
Date: Tue, 14 Jul 2015 15:33:11 -0700

> option on each to say force-cached" feels more tedious and unlikely to be

To this point, the HTTP mechanism is something that FEO / optimization
proxies can do on your behalf - e.g. rewrite and/or bundle resources, add
version fingerprint, append the HTTP header we're discussing here. By
comparison, rewriting markup (HTML, CSS, JS) is significantly harder and
very expensive. Which is to say.. +1 for HTTP directive over markup.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 14 Jul 2015 17:19:13 -0700

> On Jul 14, 2015, at 3:33 PM, Ilya Grigorik <[email-masked]> wrote:
> 
>> That said, this doesn't feel like a great thing for us to promote as a web performance best practice. "If you use long cache lifetimes for your static content, the dont-revalidate cache control header will reduce the cost of client reloads" seems like a piece of advice folks might take, as would "Use the <meta> tag 'dont-reload-non-expired-resources' to avoid browsers revalidating your content when the user presses reload". On the other hand "you should find every image, script, stylesheet, etc and set the fetch option on each to say force-cached" feels more tedious and unlikely to be used.
> 
> To this point, the HTTP mechanism is something that FEO / optimization proxies can do on your behalf - e.g. rewrite and/or bundle resources, add version fingerprint, append the HTTP header we're discussing here. By comparison, rewriting markup (HTML, CSS, JS) is significantly harder and very expensive. Which is to say.. +1 for HTTP directive over markup.

No, it would be managed in the CMS along with all of the other decisions that led to a static version. Sane folks don't manage their content in an optimization proxy.

This is an attribute of the reference, not the resource. It belongs in the content.

....Roy

---

From: Ilya Grigorik <[email-masked]>
Date: Tue, 14 Jul 2015 19:53:55 -0700

On Tue, Jul 14, 2015 at 5:19 PM, Roy T. Fielding <[email-masked]> wrote:

>> That said, this doesn't feel like a great thing for us to promote as a
>> web performance best practice. "If you use long cache lifetimes for your
>> static content, the dont-revalidate cache control header will reduce the
>> cost of client reloads" seems like a piece of advice folks might take, as
>> would "Use the <meta> tag 'dont-reload-non-expired-resources' to avoid
>> browsers revalidating your content when the user presses reload". On the
>> other hand "you should find every image, script, stylesheet, etc and set
>> the fetch option on each to say force-cached" feels more tedious and
>> unlikely to be used.
>>
>
> No, it would be managed in the CMS along with all of the other decisions
> that led to a static version. Sane folks don't manage their content in an
> optimization proxy.

Most every CDN has an FEO product that performs resource optimization
(minification, obfuscation, bundling, fingerprinting + cache extension, and
more). PageSpeed modules [1] alone, which I'm most familiar with myself,
power many hundreds of thousands of sites. Which is to say, "sane folks" do
deploy such tools and with great success.

[1] https://developers.google.com/speed/pagespeed/module/

---

From: Guille -bisho- <[email-masked]>
Date: Wed, 15 Jul 2015 10:13:54 -0700

On Tue, Jul 14, 2015 at 5:36 AM, Ben Maurer <[email-masked]> wrote:

> static

If corruption is still a concern (not sure if it is because https will give
us better integrity guarantees), what about an optional checksum?
static=<type>:<hash> like static=SHA1:###... ?

That could help preventing corruption for items that are going to stay in
cache forever. It doesn't need to be mandatory neither on origin nor
clients to validate it, but intermediate proxies/browsers can be more sure
that the content they are storing is non-corrupted.

-- 
Guille -bisho- <[email-masked]>
:wq

---

From: Martin Thomson <[email-masked]>
Date: Wed, 15 Jul 2015 10:28:59 -0700

> static=<type>:<hash> like static=SHA1:###... ?

At that point you are reinventing SRI.  Which suggests that SRI is a
signal that user agents could use, and might be all that is necessary.

---

From: Guille -bisho- <[email-masked]>
Date: Wed, 15 Jul 2015 10:49:52 -0700

But SRI is for sub resources, meant to be used in an html file. If a
resource is cached in a intermediate proxy, is corrupted, and follows this
new proposal to avoid revalidations, how the clients noticing the
corruption with SRI will force the re-fetch? We might need to ensure the
content is valid before permanente storing it.

To simplify thing we might prefer not to add the hash, but then
specifically mention that static should be used only over https channels
that provide better integrity guarantees.

Or alternatively elaborate in the SRI proposal that the resources cached
that match the hash should never be revalidated, the TTL must be obeyed.
This could be nice, especially given SRI is still a draft and accepting
changes.

-- 
Guille -ℬḭṩḩø- <[email-masked]>
:wq

---

From: Ben Maurer <[email-masked]>
Date: Wed, 15 Jul 2015 23:39:52 +0100

SRI is difficult to use in many of these cases. For example, at Facebook,
we often dynamically construct thumbnails of photos in order to avoid
having to store both high-res and low-res versions of a photo. We might
link you to /profilepics/64x64/4.jpg which is a 64x64 version of 4.jpg.
Since this file is computed on the fly, we don't know the hash of the file.
But we'd still like to serve it with a "static" tag so that it can avoid
situations.

-b

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 16 Jul 2015 12:38:36 +1200

> But we'd still like to serve it with a "static" tag so that it can avoid

This is where simply doing chunked encoding is designed to prevent
corruption. Transfers lacking the terminal 0-byte chunk are visibly
corrupt to all recipients of the message.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Jul 2015 11:26:42 +0200

> On 14 Jul 2015, at 2:36 pm, Ben Maurer <[email-masked]> wrote:
> 
> Static is a fairly reasonable name. Static does imply that the resource will *never* be revalidated (ever) vs the dont-reload which implies no revalidations prior to expiration. I don't have any preferences between the two but wanted to call that out.
> 

Writing a small spec. Is there anyone who has experience doing this that's interested in helping out here? E.g., Ilya?

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Jul 2015 11:25:45 +0200

> On 15 Jul 2015, at 7:13 pm, Guille -bisho- <[email-masked]> wrote:
> 
> 
> If corruption is still a concern (not sure if it is because https will give us better integrity guarantees), what about an optional checksum? static=<type>:<hash> like static=SHA1:###... ?
> 
> That could help preventing corruption for items that are going to stay in cache forever. It doesn't need to be mandatory neither on origin nor clients to validate it, but intermediate proxies/browsers can be more sure that the content they are storing is non-corrupted.

At the least, we should have language to the effect that this directive (however conveyed) is only valid if the response is complete, and connection close was NOT used to delimit the response.

---

From: Ben Maurer <[email-masked]>
Date: Thu, 16 Jul 2015 10:56:49 +0100

To take this a step further, it's really critical that UAs do not cache
resources in any situation where the connection is closed or aborted prior
to receiving the full Content-Length number of bytes or the terminal chunk
in chunked encoding. There's no good way for a user to fix this kind of
situation other than clearing their cache -- if they refresh with ctrl+r,
they would generally get a 304 on the resource. We actually went through a
round of testing this behavior at FB while we were tracking down a bug, but
I'm struggling to find our results since it was a long time ago.

Servers should probably not send long expiration headers on a connection
that is delimited by connection close unless they are able to send a
Content-Length header. In practice we haven't seen this be an issue because
nearly all UAs are able to get chunked responses.

-b

On Thu, Jul 16, 2015 at 10:25 AM, Mark Nottingham <[email-masked]> wrote:

> > If corruption is still a concern (not sure if it is because https will
> give us better integrity guarantees), what about an optional checksum?
> > That could help preventing corruption for items that are going to stay
> in cache forever. It doesn't need to be mandatory neither on origin nor
> At the least, we should have language to the effect that this directive
> (however conveyed) is only valid if the response is complete, and
> connection close was NOT used to delimit the response.
>
>
>
> --

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Jul 2015 12:00:14 +0200

Note:
  http://httpwg.github.io/specs/rfc7234.html#incomplete.responses

Cheers,

> On 16 Jul 2015, at 11:56 am, Ben Maurer <[email-masked]> wrote:
> 
> To take this a step further, it's really critical that UAs do not cache resources in any situation where the connection is closed or aborted prior to receiving the full Content-Length number of bytes or the terminal chunk in chunked encoding. There's no good way for a user to fix this kind of situation other than clearing their cache -- if they refresh with ctrl+r, they would generally get a 304 on the resource. We actually went through a round of testing this behavior at FB while we were tracking down a bug, but I'm struggling to find our results since it was a long time ago.
> 
> Servers should probably not send long expiration headers on a connection that is delimited by connection close unless they are able to send a Content-Length header. In practice we haven't seen this be an issue because nearly all UAs are able to get chunked responses.
> 
> -b
>

---

From: Ben Maurer <[email-masked]>
Date: Thu, 16 Jul 2015 12:00:50 +0100

Here's a sketch of what I think a spec might look like, modeled off the
stale-while-revalidate spec.

- I wasn't sure how specific we should get about the handling of browser
reload behavior.
- The static resource must stay "semantically" the same in this spec. This
allows for static resources that could change on a byte-by-byte basis --
for example if they are recompressed, etc.
- The main security risk I can think of here is defacement (there's also a
similar operational risk of accidentally returning a static document on the
root page of your domain). As I noted, I think a reasonable mitigation here
is to only apply the static semantics to sub-resources. Personally as a
site owner, I'd prefer that UAs always revalidate content that the user
directly navigates to.

Abstract

This document defines a Cache-Control extension formalizing of HTTP
resources with long expiration times where the URI of the resource is
changed when the content of the resource is changed.

1. Introduction

RFC7234 states "The goal of caching in HTTP/1.1 is to significantly improve
performance by reusing a prior response message to satisfy a current
request" Highly performant websites seek to maximize the efficiency of HTTP
caches by changing the URI of their resources every time the resource
changes then giving each such URI an extremely distant expiration date.

The static HTTP Cache-Control extension clarifies that a resource is
guaranteed never to change and allows caches to optimize based on these
semantics. For example, it allows user agents to avoid revalidating static
resources when a user presses the reload button. It also signals to caches
that the expiration date of the object may be set further in the future
than the actual expected lifetime of the object.

2. The static Cache-Control extension

When present in an HTTP response, the static Cache-Control extension
indicates that the semantic content of the response will never change in
the future. A server MUST NOT either in the past or future serve different
semantic content for the same URI. If a server accidentally serves
different content on the URI, it MUST alter all resources that reference
that URI to reference a different URI. A server MAY either in the past or
future serve an error response for the URI. The static cache-control header
MUST be used with either the "public" or "private" cache-control directive.
It MUST NOT be used in combination with "no-cache", "no-store", or
"must-revalidate". The server MUST send a max-age directive and SHOULD use
a delta-age of at least 30 days.

A cache MUST treat a response with the static Cache-Control extension as
having the maximum allowable lifetime for that cache. The cache SHOULD NOT
attempt to revalidate the response. Operations that would normally cause
the cache to revalidate the resource SHOULD result in the reuse of the
cached response. The cache MAY make an unconditional request for the
resource in response for an end-to-end reload. A user agent SHOULD NOT
generate an end-to-end reload in response to prominent user-facing
functionality such as a reload button.

2.1 Example

A HTTP response might have the header:

Cache-Control: public,static,max-age=31536000

While normally this resource would only be considered cacheable for 1 year,
a cache may choose to store it for as long as it wished. If the page using
this resource was refreshed, the user agent would not revalidate the
response. If the server wishes to change the contents of the resource in a
semantically meaningful way, it would place the resource on new URI.

3.  Security Considerations

User agents already have the capability of caching resources for long
periods of time. The static header alters the behavior of the reload button
so that it no longer triggers revalidation. The static Cache-Control
extension is designed to be used for sub-resources where the user does not
see the URI of the request. If an attacker were to compromise a directly
used URI (say the root document of a domain) and serve a response with the
static extension, it could deface the URI in a way that would not easily be
reversed by the refreshing the page. User Agents MAY  ignore the static
extension when a URI is directly navigated to by a user rather than
referenced by another page.

On Thu, Jul 16, 2015 at 10:26 AM, Mark Nottingham <[email-masked]> wrote:

> Writing a small spec. Is there anyone who has experience doing this that's
> interested in helping out here? E.g., Ilya?
>
> Cheers,
>
> --

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 17 Jul 2015 00:35:05 +1200

> stale-while-revalidate spec.

With initial feedback inline...

> than the actual expected lifetime of the object.

I dont think that last statement is correct. "may be set" implies that
heuristic lifetimes are applicable. But this control is explicitly
setting maximum lifetime when Expires/max-age/s-maxage are absent. That
is not a heuristic estimation but the absolute "infinity" value for the
cache.

> MUST be used with either the "public" or "private" cache-control directive.

Why? content is always either public or private no matter what
Cache-Controls are used.

> "must-revalidate".

Or proxy-revalidate, or stale-while-revalidate, ... and the as yet
undefined ones?

Also, what if it does happen? Effectively any combination of cache
controls can be sent.

IMHO its probably best to say that when this control is present in
responses any other controls causing revalidation MUST NOT be generated
by senders, and recipients must ignore such revalidation controls. With
the list of named controls just an example set.

> The server MUST send a max-age directive and SHOULD use

Why the MUST? "static" by itself could mean caching for maximum lifetime
permitted. (ie ~68 years).

The SHOULD and delta-age seems arbitrary. I thought the intent of
"static" was to prevent heuristic cache expiry/revalidation limits being
applied anyways.

> having the maximum allowable lifetime for that cache.

There you go. :-) the max-age bit conflicts here.

> The cache SHOULD NOT
> attempt to revalidate the response.

s/SHOULD NOT/MUST NOT/ and this one line encompasses almost all the
requirements about revalidation controls.

> Operations that would normally cause
> periods of time. The static header alters the behavior of the 

... "web browser" ...

> reload button
> so that it no longer triggers revalidation

... "for this resource."

> The static Cache-Control

 s/by the refreshing/ by reloading/

> referenced by another page.

Lots of MUST criteria, then a giant loophole of MAY ignore it all is a
bit rough. All the non-browser agents including middleware/shared caches
either cannot identify a "directly navigated" URL (or consider
*everything* as directly navigated) anyways so the MAY is just setting
up a worse problem of conflicting cache behaviour between software.

Probably best to leave the client sent Cache-Control:max-age=0 (aka
force-reload) control operational as a non-conditional fetch. This is
already implied by the text at the end of section 2.

HTH
Amos

---

From: Ben Maurer <[email-masked]>
Date: Thu, 16 Jul 2015 16:32:33 +0100

Thanks for the feedback!

> is not a heuristic estimation but the absolute "infinity" value for the

What I was trying to say here is that if I have a resource foo-v1.js with a
lifetime of 1 year, I'm not actually expecting the resource to be useful
for 1 year, merely that the value of the resource won't change. "static"
might help signal this and change cache ranking algorithms.

That said, this was a side note, and I don't know of anybody who uses
expiration dates in this way, and could probably be deleted.

> Why? content is always either public or private no matter what

Yeah, this may be better removed.

> the list of named controls just an example set.

Personally, I'd prefer to have static overridden by by other headers --
better to be more conservative and and make the browser refresh more.

> "static" was to prevent heuristic cache expiry/revalidation limits being

I guess it doesn't need to be a must, but it seems like a poor decision for
one to have a short lifetime that would apply to UAs that did not implement
this extension. Maybe it should be "SHOULD specify a long max-age (eg, 30
days or more) for the sake of caches that have not implemented this
extension".

> There you go. :-) the max-age bit conflicts here.

The idea is that static should take precedence over max-age. You need to
support max-age for legacy caches. I could have been more clear here.

> requirements about revalidation controls.

I debated about making this MUST NOT. Using MUST NOT would conflict with
any type of heuristics (in the security considerations section)

> already implied by the text at the end of section 2.

You're right that I hadn't been thinking about this with the perspective of
browser with an intermediate cache. I guess the behavior I'm suggesting
here is that a browser should treat a refresh on the main resource the same
as it does today (send a max-age=0 request), however it should *not* do
this for sub-resources that have the static extension. Open to other
suggestions about behavior here -- mainly I want to provide a safety net if
a hack/mistake were to make www.foo.com/ return a corrupt document with a
static caching flag.

---

From: Guille -bisho- <[email-masked]>
Date: Thu, 16 Jul 2015 10:38:52 -0700

>
>
> You're right that I hadn't been thinking about this with the perspective
> of browser with an intermediate cache. I guess the behavior I'm suggesting

The risk is still the typical ga.js url embedded in all websites. If a
bug/hack makes that static, you will need to ask all site owners to go and
change the url to something else, which will take ages.

Perhaps we need to change this header to the page owner? For example
facebook.com embedding a policy no never revalidate sub-resources feched
from static.facebook.com? A website wont be allowed to self-reference, so
we have this security issue solved. The page people navigate to is still
working as normal, subresources can be marked as no revalidate needed, in
case of bug, the policy can be removed and sub-resource urls changed.

---

From: Ben Maurer <[email-masked]>
Date: Thu, 16 Jul 2015 18:45:32 +0100

If ga.js were served with a max-age of 1 year, the product would still be
pretty screwed. Users would cache the resource in their browser for a long
time and it would only be refreshed if the user happens to ctrl+r on a site
that has ga.js. Same is true of www.foo.com/, though the user is more
likely to reload if they see an entire web page break.

-b

On Thu, Jul 16, 2015 at 6:38 PM, Guille -bisho- <[email-masked]> wrote:

>> a hack/mistake were to make www.foo.com/ return a corrupt document with
>> a static caching flag.
>>
>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 16 Jul 2015 11:01:09 -0700

> change the url to something else, which will take ages.

Shift-reload is a tool we provide our users to get around this class of problem.

Also, we like to engineer security systems that don't have a
point-in-time compromises of a system resulting in a persistent
attack.

Maybe we should look for alternatives.  If Facebook wanted to
construct a service worker that handled fetch events for "static"
resources and manage its own cache, we can't really stop that from
happening.  We can't stop you from blocking your own requests after
all.

Note that this wouldn't work if the resources were requested by
another origin unless you want to support foreign fetch events for
service workers.

---

From: Guille -bisho- <[email-masked]>
Date: Thu, 16 Jul 2015 11:19:09 -0700

> Shift-reload is a tool we provide our users to get around this class of

I know, but you can't ask end-user to shift-reload easily... :/

> another origin unless you want to support foreign fetch events for

I would like to avoid a system like that.

A header or html directive indicating that sub-resources of a given domain
must not be revalidated and their ttl is for good will solve all security
issues, and still provide a really nice improvement.

- If a page adds the header by mistake, can easily remove it.
- A page can't self-reference their domain, so no risk of bricking
yourself.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 16 Jul 2015 11:32:11 -0700

> I know, but you can't ask end-user to shift-reload easily... :/

Like I said, this is not a tool for you (the site operator), it's a
tool for our users.

> I would like to avoid a system like that.

I don't see how using service workers would be bad here.  You wanted
this level of control and now you don't have to ask for something new,
you just have to do some work.

> - If a page adds the header by mistake, can easily remove it.

Who do you mean by "you" in this statement?

> - A page can't self-reference their domain, so no risk of bricking yourself.

But as proposed you would be able to pin index.html.  So I don't get
this either.

---

From: Guille -bisho- <[email-masked]>
Date: Thu, 16 Jul 2015 13:03:05 -0700

> Who do you mean by "you" in this statement?

The domain owner. Imagine www.foo.com adds a header by mistake to not
revalidate from domain not-static.foo.com. If it's a mistake, they can
remove the header easily, and not-static.foo.com will have revalidations
again. The static behavior can be revoked with ease.

> But as proposed you would be able to pin index.html.  So I don't get

No if we don't allow self-reference. www.foo.com can only disable
revalidations on other *different* domains and those are effective for
sub-resources only, no direct page navigations.

Imagine page a.com tries to use maliciously b.com/index.html as a
subresouce and request not revalidation. If b.com/index.html has a cache
control header, it will be obeyed, and client reloads on a.com won't cause
revalidation of b.com/index.html. But people browsing to b.com/index.html
will have the normal behavior, revalidations and so on.

So this will be simply let page-owner define what other domains are used
for static resources and should not be revalidated. The subresources still
need the usual ttls and cache headers. And direct browsing to the
subresources won't follow the non-revalidation policies.

It's exactly what facebook (and others need). Define the reload behavior,
but rather than doing it in the sub-resources, specify the behavior on the
hosting page directly.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 16 Jul 2015 13:07:17 -0700

> sub-resources only, no direct page navigations.

That's a bizarre limitation.  It means that you have to own or use
servers (or two names) in order to use the feature.  That's a pretty
arbitrary limitation.

It also doesn't work, because if you do own the two names, you can use
one to freeze the other.

Like I said, you can implement your proposed solution today, without
writing any standards.  Sure, only Chrome supports it right now, but
that's a whole lot more of the web than none of it.

---

From: Guille -bisho- <[email-masked]>
Date: Thu, 16 Jul 2015 13:25:13 -0700

> servers (or two names) in order to use the feature.  That's a pretty

Well, yes, we can allow the same domain too.

> It also doesn't work, because if you do own the two names, you can use

Again, it will only work for sub-resources, and the resources loaded still
need the cache-headers.

If a.com freeze b.com/index.html, it will be freezed only for reloads on
a.com! and you can unfreeze at any time removing the policy from a.com.
Loading b.com will work as normal, won't be following any policies imposed
by a.com.

To be clear, the policies specified on a page will be only valid during the
load of sub-reources for THAT page. The cached resources won't have any
"static" property associated, will follow the same current cache control
policies in use. We will just allow pages to request the browser not to
revalidate cached content for domains of their choice.

> that's a whole lot more of the web than none of it.

AFAIK CacheStorage for service workers does not reuse the internal browser
cache, it's more limited and I suspect implementing this with service
workers will be far slower in general. It's good for implementing an
offline application, but not as good for implementing a general browser
cache solution.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 16 Jul 2015 15:46:22 -0700

> On Jul 14, 2015, at 7:53 PM, Ilya Grigorik <[email-masked]> wrote:
> On Tue, Jul 14, 2015 at 5:19 PM, Roy T. Fielding <[email-masked] <mailto:[email-masked]>> wrote:
> On Jul 14, 2015, at 3:33 PM, Ilya Grigorik <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> On Tue, Jul 14, 2015 at 3:03 AM, Ben Maurer <[email-masked] <mailto:[email-masked]>> wrote:
> Most every CDN has an FEO product that performs resource optimization (minification, obfuscation, bundling, fingerprinting + cache extension, and more). PageSpeed modules [1] alone, which I'm most familiar with myself, power many hundreds of thousands of sites. Which is to say, "sane folks" do deploy such tools and with great success.
> 
> [1] https://developers.google.com/speed/pagespeed/module/ <https://developers.google.com/speed/pagespeed/module/>
Umm, I don't consider mod_pagespeed to be an optimization proxy, but I guess it can be configured that way
in combination with mod_proxy. Managing configuration of CDNs is a common feature for a CMS.

In any case, the place where an output filter like pagespeed should be adding the static indicator is the same
place it is doing the content modification to modify the URL reference by adding a content hash: the HTML
element attributes. It is making a decision to force the reference to have a static representation and we want that
decision to have an impact on the page rendering algorithm of a browser, so the static association belongs with the
reference so that it can be retained regardless of how many other protocols might be used to deliver, cache,
or otherwise distribute that modified content.

OTOH, if we want to have metadata that indicates a given resource has one and only one representation
for all time, regardless of the use context, that metadata would not belong in Cache-Control either (because
it isn't about controlling a cache -- it is asserting some knowledge about the resource that can be used
by any recipient, regardless of cache behavior). That could be defined as a new header field or as a
relation for Link (where the link could point to the original resource that isn't static).

We would then be left with the question of when does the page rendering process have sufficient
confidence that it has the right representation in order to avoid making a conditional request for a
given static resource.  I think having either some sort of Content-Hash (or similar) [also orthogonal
to cache-control], or properly marking incomplete responses as suggested by HTTP/1, would be
sufficient for browsers to make their own decision to optimize away that request.

....Roy

---

From: Ben Maurer <[email-masked]>
Date: Fri, 17 Jul 2015 10:23:09 +0100

> that's a whole lot more of the web than none of it.

Using unique URIs to define resources is a common behavior to many sites
and is widely recommended in blogs/books/etc. It seems worth creating a
standard way to implement this recommendation that is far simpler than a
service worker.

Shift-reload is a tool we provide our users to get around this class of
> problem.

This draft does not suggest altering the behavior of shift-reload.
Shift-reload should continue to be an absolute refresh of the site -- for
example I believe in Chrome shift-reload even ignores service workers. In
future drafts it may be worth being more clear about this behavior.

-b

---

From: Guille -bisho- <[email-masked]>
Date: Fri, 17 Jul 2015 10:26:52 -0700

> standard way to implement this recommendation that is far simpler than a

Agree!

But again, why not just changing the page reload behavior by some directive
on the page reloaded, rather than changing the caching semantics of the
cached objects? Changing the caching semantics to make a url absolutely
permanent is dangerous as we discussed, you can freeze a page.

My proposal to just specify the reload behavior for subresources (disabling
revalidation) on the page that causes the fetches looks a simple and less
dangerous. Just makes the reload button same as clicking on the url bar and
pressing enter again.

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 18 Jul 2015 05:57:10 +1200

On 18/07/2015 5:26 a.m., Guille -bisho- wrote:
> On Fri, Jul 17, 2015 at 2:23 AM, Ben Maurer wrote:
> 
>> On Thu, Jul 16, 2015 at 9:07 PM, Martin Thomson wrote:
>>>

Because a) this is about revalidation (Ctrl+r) rather than reloading
(Ctrl+Shift+r) and b) revalidation also happens a lot from non-browser
client and middleware. The latter is very unlikely to have even looked
at the payload before trying its revalidation. If you put it in the
payload it effectively becomes an end-to-end feature only of use to
private caches (aka browser and closely related apps).

> dangerous. Just makes the reload button same as clicking on the url bar and

If this were a feature only of use to browsers. I would agree with you.

But its also of potential use to shared/middleware caches for the same
purpose of reducing revalidations. Which means header solutions are much
preferrable over payload ones.

Amos

---

From: Guille -bisho- <[email-masked]>
Date: Fri, 17 Jul 2015 16:10:20 -0700

> preferrable over payload ones.
Yeah, I agree.

What I don't see is how to easily prevent the potential issues if a url
that is not versioned is marked as static by a hack/mistake. The
implications are really dangerous. The idea proposed of limiting to
sub-resources does not apply well for non-browsers and middleware. What
should be the behavior for non browsers in that case? How do you plan to
know in middleware/non-browser clients when things are / aren't
subresources?

If there is a non-browser client that is revalidating content, it's
probably because is configured to do so, it can be changed to not do so if
someone really wants that. There is a must-revalidate header that they
should be obeying, and in theory if its not set and ttl of cache not
expired, they should not be doing those revalidations.

That's why I think that the problem with revalidations mostly affects to
browsers, and why my proposal makes sense.

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 18 Jul 2015 17:27:05 +1200

> know in middleware/non-browser clients when things are / aren't

It doesn't matter. We trust the origin servers to represent their
objects properly. If in doubt the client is trusted to force-reload
appropriately.

If an origin gets hacked and the defaced objects cached, they have the
options of a) distributing a JS that performs a request for the fixed
URL with Cache-Control:max-age=0,no-cache automatically without users
manual involvement, or b) changing references to another URL.

The (a) methodology above is made possible by the proposed semantics
that "static" means clients always do a full fetch to replace the
content whole instead of revalidating what they have.

> expired, they should not be doing those revalidations.

In the absence of explicit expiry or revalidation requirements
heuristics come into effect. That may place any kind of short lifetime
on objects. As I discussed with Ben at the beginning 1 year is the max
Squid allows currently, and 1 week is the default etc.

When that lifetime runs out revalidation is what happens today. With
static as proposed by Ben that lifetime would either never run out, or
if it did a full-fetch would happen instead of revalidate.

> browsers, and why my proposal makes sense.

Mostly, yes.  Always/Only, no.

Amos