Subject: Re: Cache-Control: no-cache

From: Amos Jeffries <[email-masked]>
Date: Thu, 01 Mar 2012 23:35:42 +1300

On 01.03.2012 17:04, Zhong Yu wrote:
> On Wed, Feb 29, 2012 at 4:34 PM, Adrien de Croy wrote:
>
>> On 28/02/2012 2:56 p.m., Amos Jeffries wrote:
>>> Intermediary admin already have to violate the specs to a certain 
>>> degree,
>>> ignoring no-cache to reduce bandwidth costs wrongly imposed by 
>>> broken server
>>> libraries which insist on sending no-cache and no-store on static 
>>> content.
>
>> Last time I sampled Cache-control response headers (over couple 
>> million hits
>> crawling sites), I found a large majority use it to prevent caching. 
>>  Very
>> few to enable it.  It's a shame.
>
> For most modern web sites, I can see "heuristic expiration" is their
> enemy. They need to disable it by "no-cache".

"no-cache" is fine when the revalidation actually works. All too often 
the frameworks send back identical copies of the already cached data. 
Wasting bandwidth and giving cache admin the clear message that they 
need to overrides on the no-cache in order to get good service out of 
that website.
  So, no-cache promptly gets overridden and heuristics kick in anyway. 
Cache admin probably sets them to a long period for extra cost savings 
on the obvious stuff like images and videos.

That is only the first step in the arms-race. Frameworks have moved 
past that to adding no-store, to prevent the no-cache overrides causing 
heuristics. That in turn is overridden, and they moved on to 
must-revalidate, max-age=0, with Pragma: and  Expires: to spice up the 
troubles and recently even "private" came under fire.

And what happens last of all is that the poor web dev who was clueless 
about HTTP header passes his or her course in website creation and 
learns about how to use them properly, or simply cut-n-paste one of the 
functions out there that calculate and sets sensible values. Only to be 
left fighting all of the cache-unfriendly mechanisms in existence sent 
by the framework on their behalf.

>
> A library aiming to simplify web development is justifiable to
> automatically add "no-cache" if "Expires" is not set. Otherwise,
> developers will pull their hair off, no understanding why the browser
> refuses to fetch an updated page.
>
> Ideally, developers should assign proper "Expires" header for
> responses. But it is not a simple task. I would say it is very hard.

Sometimes figuring out good values is hard, sometimes figuring out how 
to disable the framework is harder.

One of the jobs the framework do should be making things simpler, not 
harder. Is it too hard to call a function saying this object will change 
in probably X seconds (or never)? with documentation to say what not 
calling it will do. I think not.

> It may not make economical sense for most sites to spend time 
> thinking
> about it.

Thinking takes minutes. It is very easy when authoring a site to know 
that items of a certain type are static, long-lasting or volatile, which 
can expire in hours, and which need to be real-time. Working around the 
frameworks to make it happen often is the part that takes weeks.

AYJ