Subject: httpbis-p6-cache-06 and no-store response directive

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Mon, 08 Jun 2009 03:08:59 +0200

Hello all,

When reading the p6-06 draft it seemed to me that the new phrasing seem to  
forbid client's own cache from using the received response again under any  
circumstance, which I think is slightly different from my interpretation  
of what RFC 2616 says.

RFC 2616 says:

   14.9.2

    no-store
       The purpose of the no-store directive is to prevent the
       inadvertent release or retention of sensitive information (for
       example, on backup tapes). The no-store directive applies to the
       entire message, and MAY be sent either in a response or in a
       request. If sent in a request, a cache MUST NOT store any part of
       either this request or any response to it. If sent in a response,
       a cache MUST NOT store any part of either this response or the
       request that elicited it. This directive applies to both non-
       shared and shared caches. "MUST NOT store" in this context means
       that the cache MUST NOT intentionally store the information in
       non-volatile storage, and MUST make a best-effort attempt to
       remove the information from volatile storage as promptly as
       possible after forwarding it.

       Even when this directive is associated with a response, users
       might explicitly store such a response outside of the caching
       system (e.g., with a "Save As" dialog). History buffers MAY store
       such responses as part of their normal operation.

       The purpose of this directive is to meet the stated requirements
       of certain users and service authors who are concerned about
       accidental releases of information via unanticipated accesses to
       cache data structures. While the use of this directive might
       improve privacy in some cases, we caution that it is NOT in any
       way a reliable or sufficient mechanism for ensuring privacy. In
       particular, malicious or compromised caches might not recognize or
       obey this directive, and communications networks might be
       vulnerable to eavesdropping.

p6-cache says:

   3.2.2

    no-store

       The no-store response directive indicates that a cache MUST NOT
       store any part of either the immediate request or response.  This
       directive applies to both non-shared and shared caches.  "MUST NOT
       store" in this context means that the cache MUST NOT intentionally
       store the information in non-volatile storage, and MUST make a
       best-effort attempt to remove the information from volatile
       storage as promptly as possible after forwarding it.

       This directive is NOT a reliable or sufficient mechanism for
       ensuring privacy.  In particular, malicious or compromised caches
       might not recognize or obey this directive, and communications
       networks may be vulnerable to eavesdropping.

To me it seems that the new phrasing seem to forbid the client's own cache  
 from using the received response even when the resource is referenced  
multiple time from the same document, which is common for some sites using  
small spacer images or other small icons, or by multiple documents, like  
style sheets and images. (The text also seems to have lost the history  
reference, though Sec. 4 may make up for that)

I agree that for proxies the requirement to discard immediately make sense.

But for client it is IMO not just a waste of bandwidth (particularly on  
performance restricted devices) to reload such resources multiple times,  
even for the same document, but it would probably require significant  
changes in how clients handle resources. It also essentially duplicates  
the no-cache directive in some respects about reuse, although it does go a  
little further ("must not reuse"). I'll remind you of sec 1.1 "Caching  
would be useless if it did not significantly improve performance", and the  
above text will significantly reduce performance in clients if implemented  
according to my current understanding of it, and IMO such a reduction is  
unnecessary even from a security perspective.

Opera's implementation of this directive since we implemented it has been  
"Do not store to filesystem, keep in RAM, discard quickly when it is no  
longer in use". Such resources are re-used just like any other resource in  
the cache that are not specially treated, like POST form results, and if  
necessary re-validated when expired. The only difference is that they are  
not written to the disk cache part of our caching system (this does not  
prevent virtual memory swapping from writing them to disk; other measures  
are being considered for that; but that problem apply to all use of these  
data, also for display).

Another aspect of this is that quite a lot of sites, as well as the  
default configuration of several Wiki packages, in my experience,  
automatically send the no-store directive, along with must-revalidate,  
even when there is no need for it.

A while back MAMA, our structural web search engine, see  
http://dev.opera.com/articles/view/mama/ , did a crawl of the Alexa top  
million sites and other sites, and while the crawl was still underway I  
asked for a list of sites using the no-store directive.

The resulting list contained ~300000 unique sites of over 4 million URLs  
scanned (total), of which ~50000 (5%) were on the Alexa list, some of them  
quite high on the list. As the scan was not complete, the actual numbers  
are probably higher.

Examples included these URLs (checked early April) :

    http://www.mediabox.fr/
    http://wiki.mediabox.fr/
    http://www.tayloryourevent.com/
    http://joomla-wiki.de/doku.php
    http://sourceforge.net/
    http://technorati.com/
    http://secondlife.com/
    http://www.alltheweb.com/
    http://babynames.com/
    http://broadwayworld.com/article/Photo_Coverage_reasons_to_be_pretty_Opening_Night_Celebration_20000101

As you will see, many of these are well known sites, and almost all of  
them are front pages, which are unlikely to be sensitive, or changing very  
frequently (as in: every few seconds or minutes, and that could be handled  
using no-cache).

A point about broadwayworld.com : Their *articles* are using

     Cache-Control: no-store, no-cache, must-revalidate, post-check=0,  
pre-check=0

while the *front* pages (which are the dynamic ones) doesn't.

Additionally, the default in PHP (at least my copy of v5.0.4) seems to be  
"Cache-Control: no-store, no-cache, must-revalidate, post-check=0,  
pre-check=0", and I seem to recall that the MoinMoin wiki had that, too,  
at least last year (but in that case I may be misremembering).

Given the extensive use of no-store in situations where it does not seem  
necessary, I have started wondering if Opera need to start ignoring the  
no-store header in non-HTTPS responses, just like we currently only accept  
must-revalidate (interpreted as re-validate on history navigation) only  
for HTTPS responses. No decision has been reached yet.

My recommendation is that the text describing no-store response directive  
is phrased so that all caches are forbidden from storing the response to  
non-volatile media, and clear away ASAP after use, (as it is currently  
phrased) and that caches that are not part of the client MUST NOT use the  
response in when responding to another request, while allowing *clients*  
to use their locally stored copy as long as it can according to other  
cache policies.

Looking forward, past http-bis, given the apparent amount of  
misunderstanding about the current cache directives (I receive regular  
questions from customers and bug reports claiming that no-cache means "do  
not use again", while it only means "revalidate each time you load the  
document") I am starting to reach the conclusion that no-cache, no-store  
and must-revalidate should be discarded and replaced with more descriptive  
names (which should includes the context of when they are to be used), for  
example, on-load-revalidate, sensitive-content-storage,  
on-navigate-revalidate, respectively, or words to that effect. If a  
must-not-reuse indication is needed, then it should also directly say so,  
e.g. single-use-response or unique-response.

Also, while only distantly related, as I've pointed out earlier, HTTP is  
currently missing a mechanism to let servers invalidate a group of cache  
entries, for example during logout.I have suggested such a cache context  
mechanism in a draft (the most recent version is currently expired, but I  
am planning to refresh it; the most recent version is available at  
http://my.opera.com/yngve/blog/2008/11/06/refreshed-internet-drafts) .

-- 
Sincerely,
Yngve N. Pettersen
 
********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 8 Jun 2009 12:09:16 +1000

Hi Yngve,

I think the your question should be posed as: what compromises the  
cache subsystem?

In a browser, the HTML parser component would dispatch requests to a  
cache, which then either satisfies the requests or forwards them. If  
an HTML page has three references to <http://example.com/foo.gif>, for  
example, there are two ways of approaching this problem;

1) asking the cache for foo.gif once (perhaps by piggybacking the  
callbacks for subsequent images into the first instance), or

2) asking the cache for foo.gif three times.

In my reading, #1 is conformant even if the response contains no- 
store; technically (and probably mostly theoretically), #3 is not.

However, I don't think that matters much.

A much greater concern here is that HTTP authors need stable semantics  
for cache directives. By "creatively" interpreting them, or trying to  
guess what the authors intend based upon a survey of Web sites, a  
great disservice is done; authors will need to begin to second guess  
what cache vendors do when they encounter different directives. This  
already happens to a degree (although it's not as bad as it used to  
be, I think, and hopefully HTTPbis will improve the situation a bit  
more), but no-store has always been one of the more unambiguous  
directives.

Please don't dilute it. If sites choose to set it, the only rational  
choice is to assume that they want you to honour it; indeed, there may  
even be legal implications here (although IANAL). If it makes their  
site appear slower, well, they're reaping the benefits -- or  
detriments -- of what they've done.

As far as establishing new directives go, you're absolutely right to  
characterise this as something "beyond HTTPbis"; assuming you want to  
replace (rather than augment) the current directives, the only effort  
that could do this would be HTTP/2.0.

Cheers,

On 08/06/2009, at 11:08 AM, Yngve N. Pettersen (Developer Opera  
Software ASA) wrote:

> Hello all,
>
> When reading the p6-06 draft it seemed to me that the new phrasing  
> seem to forbid client's own cache from using the received response  
> again under any circumstance, which I think is slightly different  
> from my interpretation of what RFC 2616 says.
>
>
> RFC 2616 says:
>
>  14.9.2
>
>   no-store
> To me it seems that the new phrasing seem to forbid the client's own  
> cache from using the received response even when the resource is  
> referenced multiple time from the same document, which is common for  
> some sites using small spacer images or other small icons, or by  
> multiple documents, like style sheets and images. (The text also  
> seems to have lost the history reference, though Sec. 4 may make up  
> for that)
>
> But for client it is IMO not just a waste of bandwidth (particularly  
> on performance restricted devices) to reload such resources multiple  
> times, even for the same document, but it would probably require  
> significant changes in how clients handle resources. It also  
> essentially duplicates the no-cache directive in some respects about  
> reuse, although it does go a little further ("must not reuse"). I'll  
> remind you of sec 1.1 "Caching would be useless if it did not  
> significantly improve performance", and the above text will  
> significantly reduce performance in clients if implemented according  
> to my current understanding of it, and IMO such a reduction is  
> Opera's implementation of this directive since we implemented it has  
> been "Do not store to filesystem, keep in RAM, discard quickly when  
> it is no longer in use". Such resources are re-used just like any  
> other resource in the cache that are not specially treated, like  
> POST form results, and if necessary re-validated when expired. The  
> only difference is that they are not written to the disk cache part  
> of our caching system (this does not prevent virtual memory swapping  
> from writing them to disk; other measures are being considered for  
> that; but that problem apply to all use of these data, also for  
> display).
>
> automatically send the no-store directive, along with must- 
> revalidate, even when there is no need for it.
>
> A while back MAMA, our structural web search engine, see http://dev.opera.com/articles/view/mama/ 
>  , did a crawl of the Alexa top million sites and other sites, and  
> while the crawl was still underway I asked for a list of sites using  
> the no-store directive.
>
> The resulting list contained ~300000 unique sites of over 4 million  
> URLs scanned (total), of which ~50000 (5%) were on the Alexa list,  
> some of them quite high on the list. As the scan was not complete,  
> the actual numbers are probably higher.
>
> As you will see, many of these are well known sites, and almost all  
> of them are front pages, which are unlikely to be sensitive, or  
> changing very frequently (as in: every few seconds or minutes, and  
> that could be handled using no-cache).
>
> Additionally, the default in PHP (at least my copy of v5.0.4) seems  
> to be "Cache-Control: no-store, no-cache, must-revalidate, post- 
> check=0, pre-check=0", and I seem to recall that the MoinMoin wiki  
> had that, too, at least last year (but in that case I may be  
> misremembering).
>
> Given the extensive use of no-store in situations where it does not  
> seem necessary, I have started wondering if Opera need to start  
> ignoring the no-store header in non-HTTPS responses, just like we  
> currently only accept must-revalidate (interpreted as re-validate on  
> history navigation) only for HTTPS responses. No decision has been  
> reached yet.
>
>
> My recommendation is that the text describing no-store response  
> directive is phrased so that all caches are forbidden from storing  
> the response to non-volatile media, and clear away ASAP after use,  
> (as it is currently phrased) and that caches that are not part of  
> the client MUST NOT use the response in when responding to another  
> request, while allowing *clients* to use their locally stored copy  
> as long as it can according to other cache policies.
>
>
> misunderstanding about the current cache directives (I receive  
> regular questions from customers and bug reports claiming that no- 
> cache means "do not use again", while it only means "revalidate each  
> time you load the document") I am starting to reach the conclusion  
> that no-cache, no-store and must-revalidate should be discarded and  
> replaced with more descriptive names (which should includes the  
> context of when they are to be used), for example, on-load- 
> revalidate, sensitive-content-storage, on-navigate-revalidate,  
> respectively, or words to that effect. If a must-not-reuse  
> indication is needed, then it should also directly say so, e.g.  
> single-use-response or unique-response.
>
> Also, while only distantly related, as I've pointed out earlier,  
> HTTP is currently missing a mechanism to let servers invalidate a  
> group of cache entries, for example during logout.I have suggested  
> such a cache context mechanism in a draft (the most recent version  
> is currently expired, but I am planning to refresh it; the most  
> recent version is available at http://my.opera.com/yngve/blog/2008/11/06/refreshed-internet-drafts) 
>  .
>
> -- 
> Sincerely,
> Yngve N. Pettersen
>

--
Mark Nottingham     http://www.mnot.net/

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 08 Jun 2009 15:01:55 +1200

There is possibly the question about what to do with seemingly 
conflicting cache directives.

Combining no-store with any other directive that implies the result can 
be stored is therefore a contradiction.

If you can't store something, how can you revalidate it?  So any sort of 
revalidation directives imply the result may be stored.

The safe option is to honour the no-store, however that dishonours the 
other directives.

A quick search throws up a few sites expounding the virtues of setting 
no-store, no-cache, pre-check=0 etc on all responses.  Even though 
pre-check seems to be an IE non-standard extension.

Perhaps some wording about which ones should be ignored when they 
conflict could be useful in the spec.

Regards

Adrien

Mark Nottingham wrote:
> In my reading, #1 is conformant even if the response contains 
> no-store; technically (and probably mostly theoretically), #3 is not.
>
>> other resource in the cache that are not specially treated, like POST 
>> form results, and if necessary re-validated when expired. The only 
>> difference is that they are not written to the disk cache part of our 
>> caching system (this does not prevent virtual memory swapping from 
>> writing them to disk; other measures are being considered for that; 
>> but that problem apply to all use of these data, also for display).
>>
>> automatically send the no-store directive, along with 
>> must-revalidate, even when there is no need for it.
>>
>> http://dev.opera.com/articles/view/mama/ , did a crawl of the Alexa 
>> top million sites and other sites, and while the crawl was still 
>> underway I asked for a list of sites using the no-store directive.
>>
>> to be "Cache-Control: no-store, no-cache, must-revalidate, 
>> post-check=0, pre-check=0", and I seem to recall that the MoinMoin 
>> wiki had that, too, at least last year (but in that case I may be 
>> misremembering).
>>
>> (as it is currently phrased) and that caches that are not part of the 
>> client MUST NOT use the response in when responding to another 
>> request, while allowing *clients* to use their locally stored copy as 
>> long as it can according to other cache policies.
>>
>>
>> regular questions from customers and bug reports claiming that 
>> no-cache means "do not use again", while it only means "revalidate 
>> each time you load the document") I am starting to reach the 
>> conclusion that no-cache, no-store and must-revalidate should be 
>> discarded and replaced with more descriptive names (which should 
>> includes the context of when they are to be used), for example, 
>> on-load-revalidate, sensitive-content-storage, 
>> must-not-reuse indication is needed, then it should also directly say 
>> so, e.g. single-use-response or unique-response.
>>
>> Also, while only distantly related, as I've pointed out earlier, HTTP 
>> is currently missing a mechanism to let servers invalidate a group of 
>> cache entries, for example during logout.I have suggested such a 
>> cache context mechanism in a draft (the most recent version is 
>> currently expired, but I am planning to refresh it; the most recent 
>> version is available at 

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 8 Jun 2009 04:07:39 +0100

> callbacks for subsequent images into the first instance), or

I think it might be worth describing under what circumstances a
browser is allowed (and maybe encouraged) to do this with no-store
responses, as well as no-cache and uncachable responses generally.

In other words, when can a browser reasonably use the same response in
multiple ways, and when must it issue new requests?  We can't dictate
browser behaviour, of course, but we might be able to devise a useful
guideline.

There is little point in a web page with multiple references a
no-store/no-cache image from triggering multiple requests to that
image, and doing so makes no-store/no-cache less unusable if there are
hundreds of such references and a browser works that way.

I think I recall some version of IE where a web page containing a
"rollover" changing image would issue a new request every time the
image changed, if it was served with no-cache, making no-cache less
useful.

-- Jamie

---

From: "Yngve Nysaeter Pettersen" <[email-masked]>
Date: Mon, 15 Jun 2009 16:42:35 +0200

On Mon, 08 Jun 2009 04:09:16 +0200, Mark Nottingham <[email-masked]> wrote:

> Hi Yngve,
>
> I think the your question should be posed as: what compromises the cache  
> subsystem?
>
> cache, which then either satisfies the requests or forwards them. If an  
> HTML page has three references to <http://example.com/foo.gif>, for  

Mark, from my point of view, the document parsing code have to do #2  
before it can do #1, as #2 produces the index key used by #1.

> In my reading, #1 is conformant even if the response contains no-store;  
> technically (and probably mostly theoretically), #3 is not.
>
> guess what the authors intend based upon a survey of Web sites, a great  
> disservice is done; authors will need to begin to second guess what  
> cache vendors do when they encounter different directives. This already  
> happens to a degree (although it's not as bad as it used to be, I think,  
> and hopefully HTTPbis will improve the situation a bit more), but  
> no-store has always been one of the more unambiguous directives.
>

Assuming the author even chose it, and didn't have the choice made for  
them, as appears to be the case with PHP and several wikis.

I think it is necessary to distinguish between to uses of no-store (and  
must-revalidate): With unencrypted content, and with encrypted content.

As I said, using no-store with unencrypted content as an indication to  
proxies that they must not re-use the response makes sense, because it  
restricts distribution to other clients using the same proxy, but it does  
not IMO make sense to apply the same restriction to a client cache (or if  
you will, index) for either encrypted or unencrypted, as it will both  
cause a, possibly significant, performance reduction when accessing the  
website, and for unencrypted connections it does not confer any extra  
protection since the information is already sent in the clear, and  
client-side there are other mechanisms at work to prevent any information  
leaks that may be of concern.

My point, though, is that I think the no-store text in the updated text  
goes beyond what 2616 said.

> even be legal implications here (although IANAL). If it makes their site  
> appear slower, well, they're reaping the benefits -- or detriments -- of  
> what they've done.

As I said above: If they made the choice. In many cases I don't think they  
did more than select a development environment that made the choice for  
them, based on what is supposed to provide a "revalidate each time the  
user clicks on a link to this document"-functionality, that is, the same  
as "Cache-Control: max-age=0" and "no-cache".

> On 08/06/2009, at 11:08 AM, Yngve N. Pettersen (Developer Opera Software  
> ASA) wrote:
>
>> Hello all,
>>
>> When reading the p6-06 draft it seemed to me that the new phrasing seem  
>> to forbid client's own cache from using the received response again  
>> under any circumstance, which I think is slightly different from my  
>> interpretation of what RFC 2616 says.
>>
>>
>> RFC 2616 says:
>>
>>  14.9.2
>>
>>   no-store
>> multiple documents, like style sheets and images. (The text also seems  
>> to have lost the history reference, though Sec. 4 may make up for that)
>>
>> performance restricted devices) to reload such resources multiple  
>> significantly reduce performance in clients if implemented according to  
>> my current understanding of it, and IMO such a reduction is unnecessary  
>> even from a security perspective.
>>
>> been "Do not store to filesystem, keep in RAM, discard quickly when it  
>> is no longer in use". Such resources are re-used just like any other  
>> resource in the cache that are not specially treated, like POST form  
>> results, and if necessary re-validated when expired. The only  
>> writing them to disk; other measures are being considered for that; but  
>> that problem apply to all use of these data, also for display).
>>
>> URLs scanned (total), of which ~50000 (5%) were on the Alexa list, some  
>> of them quite high on the list. As the scan was not complete, the  
>> actual numbers are probably higher.
>>
>> them are front pages, which are unlikely to be sensitive, or changing  
>> very frequently (as in: every few seconds or minutes, and that could be  
>> handled using no-cache).
>>
>> Additionally, the default in PHP (at least my copy of v5.0.4) seems to  
>> be "Cache-Control: no-store, no-cache, must-revalidate, post-check=0,  
>> pre-check=0", and I seem to recall that the MoinMoin wiki had that,  
>> too, at least last year (but in that case I may be misremembering).
>>
>> directive is phrased so that all caches are forbidden from storing the  
>> response to non-volatile media, and clear away ASAP after use, (as it  
>> is currently phrased) and that caches that are not part of the client  
>> MUST NOT use the response in when responding to another request, while  
>> allowing *clients* to use their locally stored copy as long as it can  
>> according to other cache policies.
>>
>>
>> questions from customers and bug reports claiming that no-cache means  
>> "do not use again", while it only means "revalidate each time you load  
>> the document") I am starting to reach the conclusion that no-cache,  
>> no-store and must-revalidate should be discarded and replaced with more  
>> descriptive names (which should includes the context of when they are  
>> to be used), for example, on-load-revalidate,  
>> sensitive-content-storage, on-navigate-revalidate, respectively, or  
>> words to that effect. If a must-not-reuse indication is needed, then it  
>> should also directly say so, e.g. single-use-response or  
>> unique-response.
>>
>> cache entries, for example during logout.I have suggested such a cache  
>> context mechanism in a draft (the most recent version is currently  
>> expired, but I am planning to refresh it; the most recent version is  
>> available at  
>> --Sincerely,

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 16 Jun 2009 23:12:34 +0200

> as "Cache-Control: max-age=0" and "no-cache".

All environments I have seen support setting these kind of things if you
care about them, and emit a default "do not cahe this response" header
if the author / site developer using such environment don't care. Most
people who don't care simply do not know, and quite happily try to
accomodate for caching when they learn what it is.

Blaiming the dev environment for emitting a safe low-performance default
cache profile won't get us very far, neither is trying to work around
this in the cache layer. This situation will persist, and any changes we
make to the protocol will only get reflected in those dev environments
using the new names, until the content/site developers gets their acts
together.

This is a case of careless developers making their sites slower than
they need to be, not a specifacion fault, and not causing an error of
any kind, just slow performance due to content author/developer
ignorance, just as the 100 other asoects which makes web content
delivery slower than it need to be and is as frequently ignored by
content developers/authors.

Regarding some of the big sites using this that can only be assumed to
be their choice, quite likely due to browser bugs in dealing with
no-cache or Vary:. Many of these sites uses Cookie based logins or user
identification, with a farily large userbase of "known users" who do get
slightly modified content compared to anonymous readers.

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 18 Jun 2009 10:48:31 +1000

Exactly. This is a problem that can be addressed by evangelisation and  
education.

E.g., Curl until quite recently emitted Pragma: no-cache, but as of  
this January (IIRC) this changed.

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 18 Jun 2009 13:08:09 +1200

has there been much past discussion on precedence for seemingly 
conflicting directives though?

e.g. if you get a no-store plus public, or no-store + any other 
directive that implies it could be stored / revalidated.

IMO a site that really wants to prevent caching should use no-store, and 
not emit Expires, Last-Modified, or ETag.

in the current www environment, honouring no-store over and above other 
directives has a significant impact on cache usefulness.

Regards

Adrien

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 18 Jun 2009 11:54:38 +0200

> conflicting directives though?

Not really.

> directive that implies it could be stored / revalidated.

But no-store is a MUST and anything related to actually caching stuff is
merely a MAY so imho there isn't much room for getting this wrong.

But just as Yngve I don't quite like the new wording for no-store. The
old wording was more suited for it's purpose. There is perfectly valid
situations where a client (user-agent or similar end consumer) MAY
temporarily cache the response in memory, as long as it makes sure there
is no risk the response leaks outside the client (no "permanent"
storage).

Regarding cache validators there is applications where these are very
useful even if the response as such isn't cached as such. Authoring,
conditional fetches (not validations, even if the request at HTTP level
looks the same), etc.

> not emit Expires, Last-Modified, or ETag.

no-store MUST prevent any non-voilatile storage of the response.

But yes, if you are really paranoid then...

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 18 Jun 2009 22:34:42 +1200

Henrik Nordstrom wrote:

Not really.

OK.  that's the approach I've taken, but it leads to much lower cache
utilisation, and it's pretty clear when you see every second site
sending back

Expires: -1

Last-Modified: Thu, 18 Jun 2009 10:21:31 GMT

that actually it should be storable.  There are 4 directives and a
cache validator and freshness information indicating the response can
be stored, and 1 directive indicating it mustn't.  If it were a
democracy.....

Sure this indicates general level of ignorance of site operators etc.
But the users of the sites that do this pay more to keep downloading
invariant content that really should be cached.  It gets tempting to
put an option into the UI like "ignore no-store directive if other
directives indicate cachability".

I think as Mark suggested, if you considered that the renderer simply
uses the resource many times within the 1 page, you can consider it to
be a single retrieval.  no-store doesn't mean you can't re-use the
resource in a client.  You just can't store it.

Scanning such content for viruses could be a real problem though, if
you aren't allowed to spool it to disk, you'd need to set a size limit
on content that could be transferred with no-store to protect from
resource overuse.

Regards

Adrien

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 18 Jun 2009 12:48:09 +0200

tor 2009-06-18 klockan 22:34 +1200 skrev Adrien de Croy:
  
> utilisation,

Sure, but nothing really breaks from it, apart from the users experience
of (and cost of visiting) the affected sites...

If you want to do something productive in this area then try to
investigate WHY these sites sends both "don't ever cache this" together
with valid cache validators. It's possible they all share some common
framework which simply overlooked this fact, and if you can address that
you get the major sites addressed pretty quickly (year, not decades)...

Regards
Henrik

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 24 Jun 2009 21:31:32 +0100

> leaks that may be of concern.

no-store does make an important security difference, even without
encryption.

The difference is, after you have finished accessing the site, with
no-store there is no record in the browser cache of the pages you have seen.

Someone inspecting the computer later will not be able to retrieve
those pages.  Yes in theory they could have been intercepted at the
time they were transmitted, due to lack of encryption, but being able
to find them later is just as much a security/privacy risk.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 24 Jun 2009 21:41:40 +0100

> using the new names, until the content/site developers gets their acts

Absolutely.

I think the main reason I've seen for dev environments disabling HTTP
caching is that occasionally, a developer changes the underlying files
which make up a site, and if caching is enabled, stale pages show in
the browser sometimes which is a _very_ annoying time sink when it's
not expected.

The performance advantage from caching is just not worth the confusion
from accidental stale pages.

This may sound stupid, but you can tell it's important from the
significant number of web pages explaining how to disable HTTP caching
really really thoroughly on all clients and proxies.  (Cache-Control:
no-cache,no-store,max-age=0; Pragma: no-cache; something like that.)

Even I found, in some tests recently with Apache and Firefox that I
was surprised to see the "Reload" button not showing up to date
changes after I'd edited files on the server...  I'd forgotten about
"Shift+Reload" being needed in Firefox.  It was a default Apache
configuration, and that isn't good enough...

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 24 Jun 2009 21:47:09 +0100

> before it can do #1, as #2 produces the index key used by #1.

I think #1 is fine, and it's what I'd expect any good browser to do.

Furthermore, if a page creates Image objects (e.g. for rollovers),
then I'd expect the browser to fetch/revalidate them exactly once, not
every time the image is displayed as some versions of IE did.

An "index key" is just an implementation detail which has nothing to
do with HTTP.  My server-side dependency-structured caches don't use
an index key, for example.

It's not _that_ hard to implement #1, even if you do it by asking the
cache for foo.gif three times like #2.  As an internal implementation
detail, you can associate the internal cache request with the
displayed page which is making it, so piggybacking them like #1.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 24 Jun 2009 22:03:48 +0100

> for HTTPS responses. No decision has been reached yet.

Oh.  My.  Goodness.

You ignore must-revalidate?

Thanks for letting me know that Opera is broken.

      "Recipients MUST NOT take any automated action that
      violates this directive, and MUST NOT automatically provide an
      unvalidated copy of the entity if revalidation fails."

Do you have a suggested workaround for this client bug?  The only one
which comes to mind is redirecting to an HTTPS-equivalent page when
the User-Agent indicates Opera.  Something cheaper would be nice.

If I've set must-revalidate, it's because I want my Etags revalidated,
thank you.  As in, when the user visits the page, they must not see an
out of date page.  It causes confusion if they do.  Web applications,
you know the sort of thing.

Your interpretation that the directive wasn't really intended is just
another step in a silly arms-race, where we have to add yet more
workaround on the server side to compensate for clients who don't
follow simple specifications.

-- Jamie

---

From: Bil Corry <[email-masked]>
Date: Wed, 24 Jun 2009 16:32:18 -0500

> the browser sometimes which is a _very_ annoying time sink when it's

For web applications with dynamic content, it is easier to just prevent caching than it is to implement a system to handle If-Modified-Since.  Expires header isn't an option when the content can change at any given moment.  Of course, that's for HTML content; for media assets (png, jpg, etc), we use the Expires header to prevent any further requests for it.

> no-cache,no-store,max-age=0; Pragma: no-cache; something like that.)

That's the reason you see the shotgun approach to preventing caching; different browsers (and legacy browsers) and proxies respond to different headers.  And even then, I still hear about users behind a corporate proxy that serves cached content from one to another, causing User A to see User B's information.

Is there a resource that gives the "official" way to prevent caching across all browsers and proxies?

- Bil

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 25 Jun 2009 00:05:56 +0200

> Is there a resource that gives the "official" way to prevent caching across all browsers and proxies?

RFC2616?

I think a lot of this headache actually comes from the ignorance
mentioned earlier. Site authors don't care, users / network operators
gets annoyed and start to bend rules trying to optimize their (or their
users) Internet experience, and suddently what should be an easy
condition no longer is trivial with multi level operator error to take
into account..

RFC2616 rules is simple:

Not cached:

Expires: Date header or older
Cache-Control: no-cache

and avoid any cache validator (Last-Modified/ETag)

Not cached by proxies but cached in browsers:

Expires: Date header or older
Cache-Control: private, max-age=NN

The Expires requirement is due to HTTP/1.0 legacy.

In practice using very old date for Expires is probably better as some
cache implementations take the Expires date literally not compensating
for clock skew when origin server clock is ahead of the cache clock...
(implementation bugs..)

Regards
Henrik

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Thu, 25 Jun 2009 00:54:23 +0200

On Wed, 24 Jun 2009 23:03:48 +0200, Jamie Lokier <[email-masked]>  
wrote:

>> be

For HTTP-URLs, yes we do ignore it, but not HTTP*S*. When heeded, history  
navigation, as well as normal load, will trigger a if-modified since load.

And the reason we had to do that was that there was a significant uproar  
among our users when we first added support for the directive, as it  
caused a significant slow down when browsing, even on static wikis. The  
average user won't blame the *site* when their experience is bad, they  
will blame the browser, even when it is the site causing the bad  
experience.

> the User-Agent indicates Opera.  Something cheaper would be nice.

A no-cache will work fine for load events, must-revalidate on HTTPS does  
cause that to be set, but otherwise only affects history navigation.

> you know the sort of thing.

As long as you send a no-cache directive we will send an if-modified since  
for when the resource is used in a *loading* page

> follow simple specifications.

The interpretation (history navigation revalidation + no-cache) was one  
that made sense to us, based on the phrasing in the specification, since  
if it did not imply revalidation during history navigation, then it was  
just an alias for no-cache, which is apparently what a significant number  
of framework developers think it is.

As stated above, the backlash was significant enough when we released it,  
that we had to disable it for HTTP-only sites.

We implemented the history navigation revalidation because a number of  
online banks indicated that they required revalidation when navigating  
history, something which seems to work implicitly in certain other  
browsers, AFAICT because they "always" reload top documents even during  
history navigation, something which Opera does not do since our default is  
history navigation shows the state when we loaded the document, not the  
current state (see also RFC 2616 sec 13.13), and if the server does not  
specifiy policy we use local preferences to determine revalidation (which  
have had to be cut to the bone for documents, again due to the browser  
being blamed when the expirence is "bad" when the cache copy was used on  
going to a page that "should" have been updated, but did not specify any  
expiration policy).

-- 
Sincerely,
Yngve N. Pettersen

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 25 Jun 2009 01:17:29 +0100

> > Is there a resource that gives the "official" way to prevent caching across all browsers and proxies?

No, because it does not cover all browsers and proxies.
Only those conforming to RFC2616.

It probably doesn't even cover most proxies.  When did Squid get
HTTP/1.1 support?

> Cache-Control: private, max-age=NN

If the Expires is older than Date, I see why the proxy can't cache,
but why do think it permits caching in browsers?

> for clock skew when origin server clock is ahead of the cache clock...

Is the common advice to include "Pragma: no-cache" unnecessary?

There's also the Microsoft ones, post-check and pre-check:

The post-check and pre-check cache-control extensions are defined as follows:

    * post-check

        o Defines an interval in seconds after which an entity must be
          checked for freshness. The check may happen after the user
          is shown the resource but ensures that on the next roundtrip
          the cached copy will be up-to-date.

    * pre-check

          o Defines an interval in seconds after which an entity must
          be checked for freshness prior to showing the user the
          resource.

I'm not quite sure how these differ from max-age, but they presumably do.

Some advice on disabling caching includes these.
The whole set I'm aware of is:

    Pragma: no-cache
    Cache-Control: no-cache,max-age=0,must-revalidate,pre-check=0,post-check=0
    Expires: A-VERY-OLD-DATE

But then you have to jump through hoops to make sure some versions of
IE will not fetch <img> files in a single page multiple times, once
per instance.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 25 Jun 2009 01:46:50 +0100

Bil Corry wrote:

> For web applications with dynamic content, it is easier to just
> prevent caching than it is to implement a system to handle
> If-Modified-Since.  Expires header isn't an option when the content
> can change at any given moment.

I use Etags.  The ETags are generated from all the prerequisites to
the dynamic content as accurately as possible.  E.g. a combined result
of all the file timestamps and sizes which are used to make the
content, and database cache tags.

I agree that it's not entirely easy, although good frameworks usually
have a way to encapsulate it.

> Of course, that's for HTML content; for media assets (png, jpg,
> etc), we use the Expires header to prevent any further requests for it.

I do similar.

For dynamically generated images, I set Expires 1 year into the
future, and use a hash of the image, either of the image file itself
or the instructions used to generate it, and use the hash as part of
the URL of the image.  That way it can be dynamic but only fetched
when it changes, by the HTML which references it changing automatically.

It's a bit complicated, but it can be hidden well in the guts of a
server and performs well.

> That's the reason you see the shotgun approach to preventing
> caching; different browsers (and legacy browsers) and proxies respond
> to different headers.  And even then, I still hear about users behind
> a corporate proxy that serves cached content from one to another,
> causing User A to see User B's information.
>

I don't think there is any "official" method which considers all real
implementations.  Official and implementation are in constant tension.

As your example shows, there might not be any way.
Some ancient caches just have a fixed expiry age configured.

The ultimate method is to keep changing the URL.
(Similar to the way some frameworks change the names of form elements,
to prevent browsers from pre-filling forms with wrong sensitive data by
accident).

This is what I've found, for the paranoid:

    Pragma: no-cache
    Expires: VERY-OLD-DATE

The apparently redundant fields are in case of implementations which
don't understand, or don't correctly implement, the other fields.

There's probably a browser out there which doesn't understand
"Cache-Control: no-cache,..." when there's anything else on the same
line.  IE had a reputation for being a bit rigid in how it recognises
some headers.  But I'm pretty sure anything like that will recognise
"Pragma: no-cache" so it doesn't matter.

-- Jamie

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 25 Jun 2009 11:26:02 +1000

I'm going to push back on that a bit. I'd put forth that any browser  
that doesn't understand Cache-Control: no-store is also not going to  
do Ajax (because it'll lack XmlHttpRequest), isn't going to have what  
people today consider a usable JS implementation, etc. CC has been  
around for a *long* time.

I'm in the midst of putting together some tools to get some metrics  
off of implementations ATM...

Cheers,

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 25 Jun 2009 03:51:17 +0200

> It probably doesn't even cover most proxies.  When did Squid get

It hasn't even come there yet (almost there now). But has understood
Cache-Control since about a decade or so...

> but why do think it permits caching in browsers?

Expires older than Date has nothing to do with browser vs proxy. Just
HTTP/1.0 vs 1.1.  It's a safety precaution if you should ever encounter
a HTTP/1.0 client cache (user-agent or proxy) who don't know about
Cache-Control.

For anything reasonably modern Cache-Control max-age takes priority over
Expires.

> Is the common advice to include "Pragma: no-cache" unnecessary?

These days yes. And was in fact never really needed provided you expired
the content...

>           be checked for freshness prior to showing the user the

Sounds similar to some other cache-control headers proposed by mnot some
year ago..

> I'm not quite sure how these differ from max-age, but they presumably do.

post-check is about adding a bit of fuzziness to the process, doing
cache validation asynchronous from client request, showing the client
the cached copy while performing a cache validation in the background.

but I don't get how pre-check differs from max-age...

> Some advice on disabling caching includes these.

I think that's just "because they exists in MSIE so we better set them
to something..." but I don't know. Haven't studied MSIE handling of
pre/post-check..

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 25 Jun 2009 15:50:43 +1000

Yngve, why does must-revalidate affect user experience so much? All  
that it means is that you can't serve the response once it becomes  
stale. Does Opera ignore server-provided expiry as a matter of course?

On 25/06/2009, at 8:54 AM, Yngve N. Pettersen (Developer Opera  
Software ASA) wrote:

> On Wed, 24 Jun 2009 23:03:48 +0200, Jamie Lokier  
> <[email-masked]> wrote:
>
> For HTTP-URLs, yes we do ignore it, but not HTTP*S*. When heeded,  
> history navigation, as well as normal load, will trigger a if- 
> modified since load.
>
> And the reason we had to do that was that there was a significant  
> uproar among our users when we first added support for the  
> directive, as it caused a significant slow down when browsing, even  
> on static wikis. The average user won't blame the *site* when their  
> experience is bad, they will blame the browser, even when it is the  
> site causing the bad experience.
>
>
> A no-cache will work fine for load events, must-revalidate on HTTPS  
> does cause that to be set, but otherwise only affects history  
> navigation.
>
> As long as you send a no-cache directive we will send an if-modified  
> since for when the resource is used in a *loading* page
>
> The interpretation (history navigation revalidation + no-cache) was  
> one that made sense to us, based on the phrasing in the  
> specification, since if it did not imply revalidation during history  
> navigation, then it was just an alias for no-cache, which is  
> apparently what a significant number of framework developers think  
> it is.
>
> As stated above, the backlash was significant enough when we  
> released it, that we had to disable it for HTTP-only sites.
>
> We implemented the history navigation revalidation because a number  
> of online banks indicated that they required revalidation when  
> navigating history, something which seems to work implicitly in  
> certain other browsers, AFAICT because they "always" reload top  
> documents even during history navigation, something which Opera does  
> not do since our default is history navigation shows the state when  
> we loaded the document, not the current state (see also RFC 2616 sec  
> 13.13), and if the server does not specifiy policy we use local  
> preferences to determine revalidation (which have had to be cut to  
> the bone for documents, again due to the browser being blamed when  
> the expirence is "bad" when the cache copy was used on going to a  
> page that "should" have been updated, but did not specify any  
> expiration policy).
>
> -- 
> Sincerely,
> Yngve N. Pettersen
>

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Thu, 25 Jun 2009 12:06:35 +0200

On Thu, 25 Jun 2009 07:50:43 +0200, Mark Nottingham <[email-masked]> wrote:

> Yngve, why does must-revalidate affect user experience so much? All that  
> it means is that you can't serve the response once it becomes stale.  
> Does Opera ignore server-provided expiry as a matter of course?

Opera treats the directive as a no-cache for history-navigation, and that  
means that when you navigate back and forth in history all resources in  
the document with that flag will be revalidated, before they are displayed  
again. That causes a significant delay while navigating.

It *may* be that we should also check expiration first (and that is being  
considered, but may require significant changes), but the purposes for  
which we added it, online banking, requires checking on every view in any  
case. And as mentioned, the "primary" use of the directive in many  
frameworks, such as PHP, combines must-revalidate with no-cache and  
no-store.

> On 25/06/2009, at 8:54 AM, Yngve N. Pettersen (Developer Opera Software  
> ASA) wrote:
>
>>>> to be
>>>> too,
>>>> seem
>> history navigation, as well as normal load, will trigger a if-modified  
>> since load.
>>
>> uproar among our users when we first added support for the directive,  
>> as it caused a significant slow down when browsing, even on static  
>> wikis. The average user won't blame the *site* when their experience is  
>> bad, they will blame the browser, even when it is the site causing the  
>> bad experience.
>>
>>
>> that made sense to us, based on the phrasing in the specification,  
>> since if it did not imply revalidation during history navigation, then  
>> it was just an alias for no-cache, which is apparently what a  
>> significant number of framework developers think it is.
>>
>> As stated above, the backlash was significant enough when we released  
>> it, that we had to disable it for HTTP-only sites.
>>
>> history navigation, something which Opera does not do since our default  
>> is history navigation shows the state when we loaded the document, not  
>> the current state (see also RFC 2616 sec 13.13), and if the server does  
>> not specifiy policy we use local preferences to determine revalidation  
>> (which have had to be cut to the bone for documents, again due to the  
>> browser being blamed when the expirence is "bad" when the cache copy  
>> was used on going to a page that "should" have been updated, but did  
>> not specify any expiration policy).
>>
>> --Sincerely,
>> Yngve N. Pettersen
>>

-- 
Sincerely,
Yngve N. Pettersen

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 26 Jun 2009 16:41:43 +1000

On 25/06/2009, at 8:06 PM, Yngve N. Pettersen (Developer Opera  
Software ASA) wrote:

> Opera treats the directive as a no-cache for history-navigation, and  
> that means that when you navigate back and forth in history all  
> resources in the document with that flag will be revalidated, before  
> they are displayed again. That causes a significant delay while  
> navigating.

RFC2616 specifically states that the cache and history are separate  
(section 13.13).

> It *may* be that we should also check expiration first (and that is  
> being considered, but may require significant changes), but the  
> purposes for which we added it, online banking, requires checking on  
> every view in any case. And as mentioned, the "primary" use of the  
> directive in many frameworks, such as PHP, combines must-revalidate  
> with no-cache and no-store.

If you have users that need to restrict use of browser history (a well- 
worn topic), please come up with extension protocol directives, rather  
than abusing the existing ones. Section 13.13 talks about how  
improperly implemented history lists can persuade Web authors to not  
use must-revalidate, but as you've described your implementation, it's  
doing far worse -- it's eroding what trust authors have in the  
existing ones.

Thanks,

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 26 Jun 2009 09:49:32 +0200

tor 2009-06-25 klockan 12:06 +0200 skrev Yngve N. Pettersen (Developer

> frameworks, such as PHP, combines must-revalidate with no-cache and  

must-revalidate SHOULD only have any effect once the response has become
stale and also disables heuristic expiry calculation. Of course your
users complained very loudly if you implemented it as no-cache.

If forced checking on every view is desired then
max-age=0,must-revalidate is the combination to use. max-age=0 says it's
immediately stale, must-revalidate forces validation before reuse of
stale copies.

Regards
Henrik

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Fri, 26 Jun 2009 21:45:32 +0200

On Fri, 26 Jun 2009 08:41:43 +0200, Mark Nottingham <[email-masked]> wrote:

> On 25/06/2009, at 8:06 PM, Yngve N. Pettersen (Developer Opera Software  
> ASA) wrote:
>

Please tell that to everybody (particularly banks) who *requires* (and  
AFAIK even block clients that does not adhere to it) that the client  
rechecks/reloads content on history navigation. Particularly after a  
logout.

Case in point, I just noticed a bugreport from somebody saying that  
no-store should mean that content is revalidated/reloaded during history  
navigation. He wanted a login page when navigating back in history after  
logging out of a site. We have also received many reports, including from  
major customers, saying that no-cache should have the same effect.

The fundamental issue is that HTTP (as you are also touching on) does not  
specify any directives for history navigation. That leaves both web site  
developers, and browser vendors (and probably other vendors) in a lurch  
about how to facilitate things like secure logout which makes sure history  
navigation is not possible, while still ensuring good display performance,  
and the result is that we use the keywords that are available and that  
seem (or can be extended, maybe even stretched) to indicate the desired  
functionality, like we did with must-revalidate (before we used  
must-revalidate we tried to use the combination "private, no-cache" as a  
kick out of cache mechanism, but that caused too many problems).

I think it would be an idea if the new HTTP RFC give specific examples for  
how the directives should be interpreted in various situations, such as  
when going to a new page, when navigating ing history, full document  
reload, etc. Yes, that may look like something that is rather targeted at  
a specific application, but most uses of HTTP will be close to one of  
these. Such a description should have enough details that it will be  
useful to designers and implementors, even if it just says that "During  
history navigation the client MUST NOT evaluate cache directives, and MUST  
NOT attempt to reload/revalidate content from the server".

> If you have users that need to restrict use of browser history (a well-

Actually, it would be more proper to say that it is web sites that our  
users visit that want those restrictions, and who sometimes use what I  
consider extreme measures to ensure that those policies are implemented.

> worn topic), please come up with extension protocol directives, rather

Please see my draft on cache-context (I am planning to refresh it soon):

    http://files.myopera.com/yngve/blog/draft-pettersen-cache-context-03.txt

Background:

   http://my.opera.com/yngve/blog/2007/02/27/introducing-cache-contexts-or-why-the

Alternatively, new headers and/or directives defining actions in history  
navigation should be defined. Any suggestions?

-- 
Sincerely,
Yngve N. Pettersen

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 28 Jun 2009 02:07:16 +0200

fre 2009-06-26 klockan 21:45 +0200 skrev Yngve N. Pettersen (Developer

> navigation should be defined. Any suggestions?

user agent history buffer behavior is slightly outside of the HTTP
realm, but I would suggest that must-revalidate applies to history
buffer views as well, preventing stale information from being shown
without validation.

And if that's not sufficient then one can always resort to no-store..
preventing any form of caching of the response.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 28 Jun 2009 02:09:38 +0200

> immediately stale, must-revalidate forces validation before reuse of

Forgot to mention that this is in principle the same as no-cache, but
some implementation do not cache no-cache responses at all implementing
it the same as no-store.

Regards
Henrik

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 28 Jun 2009 23:45:22 +0100

> navigation, as well as normal load, will trigger a if-modified since load.

Sigh.  So we have to add a server-side rule "if user-agent == Oper
cache behaviour depends on HTTP vs. HTTPs".

> will blame the browser, even when it is the site causing the bad  

I remember a similar thing with Mozilla/Firefox.  They originally
revalidated on history browsing, but added and advertised a new
feature - showing pages from history without revalidating them :-)

> cause that to be set, but otherwise only affects history navigation.

I don't see how that's a workaround.

no-cache means don't cache at all.  I *do* want pages cached - but I
want them validated on display too.

> for when the resource is used in a *loading* page

If it has no-cache directive, isn't sending if-modified a browser bug too?

That's why all those sites you mentioned send must-revalidate etc. etc.
Because browsers don't obey no-cache by itself.

> of framework developers think it is.

More likely, framework developers found browsers who don't obey
no-cache (like Opera, sending if-modified?), so they add every
directive they can think of, hoping at least one of them causes a
browser to not cache.

> that we had to disable it for HTTP-only sites.

An arms race.

The next step is pages having Javascript to revalidate themselves,
with the whole page inside "<div class=hide-while-validating>".  I
already have the code in mind.

If that becomes a standard part of common frameworks, it'll be
interesting to see your response when users complain about history
speed :-)

> history navigation,

I agree with the banks.  It's reasonable to want to hide user's
personal data after a secure logout or session expiry.

It's just ugly to do it so differently for HTTP and nHTTPS.

> something which Opera does not do since our default is  
> current state (see also RFC 2616 sec 13.13),

It's in interesting problem, as fast history is a good feature, but
non-validating history is wrong for some applications, after a session
has finished.

> and if the server does not specifiy policy we use local preferences
> to determine revalidation (which have had to be cut to the bone for
> documents, again due to the browser being blamed when the expirence
> is "bad" when the cache copy was used on going to a page that
> "should" have been updated, but did not specify any expiration
> policy).

Ok.  As a server author, how do I specify a "please revalidate" policy?

You said no-cache, but that means don't cache at all, not cache but
please always revalidate.  So it won't work with conforming browsers,
and it'll break caching at proxies.

Perhaps a Cache-Control: history-revalidate(=yes/no) is needed, with
the defaults being HTTPS-dependent as you implemented.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 28 Jun 2009 23:53:18 +0100

> around for a *long* time.

I'm not sure what Ajax has to do with this.

Defeating caches is useful for non-Ajax pages too.  In fact, I've only
encountered thorough cache-defeating advice in a non-Ajax context.

(And a side point: Ajax can be done without XmlHttpRequest.  Sometimes
even on modern browsers, due to XmlHttpRequest having stricter source
checks than <iframe src=...>, and alternate domains being needed to
work around the 2 connection per domain limit when doing
streaming/long-polling...)

> off of implementations ATM...

That's great :-)

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 28 Jun 2009 23:56:30 +0100

>"Pragma: no-cache" so it doesn't matter.

We can also add:

If client may be Opera (who knows about others), going from other
messages in this thread:

   - Use HTTPS if you do want caching but you want must-revalidate to
     be honoured in history browsing.  A complicated quirk, yet
     important to anything with sessions revealing personal data.

   - Cache-Control: no-cache doesn't prevent caching.  You may still
     see If-Modified requests, validating a response which was sent
     with no-cache.

-- Jamie

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 29 Jun 2009 12:43:04 +1200

no-cache no longer means don't reply from cache at all.  It only means
don't return a stored representation without revalidating it first.
The stored representation can still be served if you get a 304 from the
server, so you can still save the bulk transfer.  So this is a client
or server override on freshness processing.  You get the same result
from a conformant cache by using max-age=0, which by the way is what
Chrome sends when you Ctrl-F5.  IE and FF send no-cache.

Actually this highlights some problems in the structure of 13.1.1 Cache
Correctness.  The no-store directive shouldn't be only part of option
2.  It's global.  As it's currently written, 13.1.1 allows serving a
stored response that had no-store if it's revalidated.

"... which meets one of the following conditions"

Back when everyone used Pragma: no-cache, no-cache meant don't cache
it.  Now that there is a more explicit distinction in RFC 2616 between

a) retrieving something from a store

b) putting something into a store

c) revalidating something stored

the words "cache" (verb), and "cachable" are no longer clear and
unambiguous.  I'd be keen for some better (more strict) nomenclature
for these cases.

Adrien

Jamie Lokier wrote:

Oh. My. Goodness.

An arms race.

-- Jamie

---

From: David Morris <[email-masked]>
Date: Mon, 29 Jun 2009 14:11:27 -0700 (PDT)

> buffer views as well, preventing stale information from being shown

The issue here is user interaction model design ... I find it really 
offensive when a browser reloads a page or denys same because it is stale.
Until this is done right we'll never get folks to stop printing. As a 
user, I should be able to review and/or print what I'm looking at w/o 
requiring ANY network activity. Without ANY change from what I last read.
It should work just like a printed document, newspaper, etc. Until I 
request an update, I want to flip between pages of what I have already 
received.

I have no problem with the browser adding a non-intrusive notification 
that the page is now stale (e.g., similar to the page secure LOCK used
now to show a page as secure). Likewise, the browser should block 
re-posting already used content or content from a stale page. It typically 
takes 10-60 seconds to revalidate/refresh a page ... flipping back in 
history should be really fast ... as a user I've already paid to see the 
page.

Yeah we've deemed this outside of HTTP's scope and it needs to stay that 
way. History navigation does not constitute a new web request. How well 
the UIModel is implemented is an opportunity for browser differentiation.

Dave Morris