Subject: Re: Questions (errata?) about caching authenticated responses [#174]

From: Mark Nottingham <[email-masked]>
Date: Fri, 24 Jul 2009 11:51:15 +1000

My take:

On 25/06/2009, at 4:12 PM, Mark Nottingham wrote:

> Trolled up from the old list..
>  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/174
>
> Begin forwarded message:
>
>> Resent-From: [email-masked]
>> From: Duane Wessels <[email-masked]>
>> Date: 20 July 2000 3:47:59 PM
>> To: [email-masked]
>> Subject: Questions (errata?) about caching authenticated responses
>>
>> I've been reading RFCs 2616 and 2617 about caching authenticated
>> responses, and have possibly found some inconsistencies.
>>
>> #1.     The very last sentence of Sec 14.9.4 (under proxy-revalidate)
>> 	says: ``...such authenticated responses also need the public
>> 	cache control directive in order to allow them to be cached at
>> 	all''
>>
>> 	Yet, Sec 14.8 lists three cache-control directives that allow a
>> 	shared cache to reuse an authenticatd response: s-maxage,
>> 	must-revalidate, and public.

I can understand s-maxage making an authenticated response public (and  
if that's agreed, it needs to be added to p6 2.1).

However, I think that a shared cache storing an authenticated response  
because it has must-revalidate is going to be surprising and  
undesirable in almost any situation; "must-revalidate" means "don't  
serve this stale", not "shared caches can store this authenticated  
content".

Does anyone know of a cache that implements this? Since there's a  
conflict in the specs here, I'd hope that implementations would be  
conservative. My inclination here is to remove must-revalidate from  
14.8 as a way of indicating that an authenticated response can be  
stored.

>> #2.	If must-revalidate alone is enough to allow an authenticated
>> 	response to be cached, and if proxy-revalidate is the same
>> 	as must-revalidate for a shared cache, is proxy-revalidate
>> 	alone enough to allow an authenticated response to be cached?
>>
>> 	If so, should proxy-revalidate be listed in section 14.8?

As per above, my preference would be not to. proxy-revalidate in an  
authenticated response that doesn't explicitly have a 'public'  
directive should be a no-op.

>> #3.	RFC 2617, Sec 3.2.2.5 says:
>>
>> 	    when a shared cache ... has received a request containing
>> 	    an Authorization header and a response from relaying that
>> 	    request, it MUST NOT return that response as a reply to any
>> 	    other request, unless one of two Cache-Control (see section
>> 	    14.9 of [RFC2616]) directives was present in the response.
>>
>> 	I believe this is referring to section 14.8, rather than 14.9,
>> 	and "two" is not the right number?

Seems like an errata for 2617.

>> Finally, Sec 14.8 doesn't mention if a non-shared cache needs to  
>> treat
>> an authenticated response specially.  I assume that a non-shared
>> cache can store and reuse an authenticated response by default.
>> Should that be made explicit?

I think this is made more explicit by p6 2.1, but if anyone wants to  
propose text...

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 25 Jul 2009 01:19:06 +0200

> 14.8 as a way of indicating that an authenticated response can be  

We (Squid) always require public for authenticated responses. Perhaps
not optimal, but very easy to explain. If there is authentication
respond with public on resources not actually requiring authentication
if you want them cached.

But I can see the point about s-maxage implicit making the response
public even if not explicit stated in the cache-control header.

And given the fact that we do have the "private" cache-control I can
understand why must-revalidate also may implying public. But see below
for why not.

In HTTP/1.0 we did not have any of these, so any authenticated response
without Cache-Control has to be handled as private from the HTTP/1.0
legacy.

When Cache-Control was added both "private" and "public" was added at
the same time (or at least I think so, my history of Cache-Control only
goes as far as RFC2068), enabling servers to clearly express their
intentions whenever there is any doubt. And with HTTP authentication
being optimistic in that clients send the credentials without really
knowning it's needed and servers having a lot of content which does not
require authentication and where therefore authentication quite likely
isn't considered when selecting the proper cache-control settings it
makes some sense that some/many/all Cache-Control directives implicit
makes content public unless private is used. The question there is why
is the text restricted to s-maxage and must-revalidate, why not max-age
or proxy-revalidate or even no-cache for that matter as no-chache is not
much different from max-age=0, must-revalidate or even the mere presence
of a Cache-Control response header.

However, the definition of must-revalidate only speaks about this
directive forcing validation after an object has become stale, not that
it would also enable otherwise private (authenticated) objects to be
cached in a shared cache. As far as I know the Cache-Control definitions
nowhere speak about any other headers than public as the method for
marking authenticated content public. For example the definition of
must-revalidate says nothing about this, just a lot of further
restrictions compared to not having must-revalidate. This places authors
that want to tune their clients private caches in a somewhat bad
situation as they clearly run the risk of doing the wrong thing unless
they also happen to read the cache section and pay careful attention to
the notion of shared caches and that some cache-control headers may have
unexpected effects on authenticated content in such caches, almost
opposite to the effect the same directives have in the context on the
browser private cache.

Making that decision on must-revalidate just feels odd to me as it's not
what must-revalidate is about. In all other aspects must-revalidate
places further restrictions on the cache (shared & private), not
enabling.

So I am going to stick to the current implementation, and require public
for accepting authenticated content as cacheable.

I may change my mind if it can be shown that nearly all authenticated
content with Cache-Control also includes private in the set of controls,
but that's something which I am not in a good position to determine, not
is it something I would like my users to discover by trial and error..

Maybe we may change the implementation regarding s-maxage to implicit
assume public when s-maxage is used as this is an explicit instruction
for a shared cache which does not make sense to see on private content.
But it then begs the question if proxy-revalidate also means public,
which it probably should do by the same reasoning.

Perhaps the intention actually was to use proxy-revalidate in that text
and not must-revalidate. If proxy-revalidate is substituted in that text
then it becomes coherent and makes some sense. But I do not know if that
was the intention as I was not around in the discussions then. I
probably could have been but were not as I was not familiar with the
workings of IETF then.

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 25 Jul 2009 10:34:10 +1000

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/174>

> places further restrictions on the cache (shared & private), not

Agreed.

> which it probably should do by the same reasoning.

Yes. My inclination here would be to make it as simple as possible,  
keeping in mind the behaviour of current implementations.

> probably could have been but were not as I was not familiar with the

I had a quick look through the old caching list and didn't find  
anything, but this list's predecessor had a few relevant bits:

   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1996MayAug/0470.html
   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1996MayAug/0473.html

   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1996SepDec/0578.html
   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1996SepDec/0582.html
   http://ftp.ics.uci.edu/pub/ietf/http/draft-mogul-http-revalidate-01.txt

   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1998MayAug/0133.html
   http://lists.w3.org/Archives/Public/ietf-http-wg-old/1998MayAug/0137.html

(still looking through these to draw my own conclusion)

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 25 Jul 2009 18:06:33 +0200

> anything, but this list's predecessor had a few relevant bits:

a little confusing as some of that predates s-maxage.

unfortunately I could not find anything in that on why must-revalidate
should also imply public, but possibly it was already defined that way
by then...

but there is a clear spirit of trying to optimize cache-control by only
requiring a single directive "in common cases" (or what was assumed to
be "common case") without at the same time considering the effects on
non-common cases combined with several proposed directives
added/removed/modified at the time..

This over optimization can still be seen in the definition of s-maxage
implicitly also meaning proxy-revalidate. I guess it was believed no one
would want to ask shared caches to just TRY to revalidate more often
than private ones and at the same time allow for the shared caches to
respond with stale information if the revalidation wasn't possible for
one reason or another. Which partly seems to come from the view that the
only reason one would want shared caches to revalidate more often is
when there is authentication of some kind.. (to be exact that the only
s-maxage setting that is relevant to care about in context of
authenticated responses is s-maxage=0 and that it therefore should also
imply proxy-revalidate). Unfortunately nothing we can change now and we
will just have to live with it even if it rules out a whole family of
freshness settings.

Going back reading RFC2068 for a while... and there the picture is quite
different when it comes to proxy-revalidate in authenticated responses:

     1. If the response includes the "proxy-revalidate" Cache-Control
        directive, the cache MAY use that response in replying to a
        subsequent request, but a proxy cache MUST first revalidate it with
        the origin server, using the request-headers from the new request
        to allow the origin server to authenticate the new request.
     2. If the response includes the "must-revalidate" Cache-Control
        directive, the cache MAY use that response in replying to a
        subsequent request, but all caches MUST first revalidate it with
        the origin server, using the request-headers from the new request
        to allow the origin server to authenticate the new request.
     3. If the response includes the "public" Cache-Control directive, it
        may be returned in reply to any subsequent request.

So it turns out that what have happened with respect to proxy-revalidate
is that the MAY rule regarding proxy-revalidate was lost from this
section when s-maxage was added in RFC2616. Probably an editorial error
when introducing s-maxage, alternatively (but less likely) there may
have been discussions to remove proxy-revalidate entirely as the
definition of s-maxage makes proxy-revalidate redundant, but then this
text was never added back when it was decided to keep proxy-revalidate.

Proposed resolution to the proxy-revalidate issue: Simply add back the
text from 1. above where it was nly correcting the text to say "shared
cache" where it said "proxy", at least for now. This makes
specifications consistent with what was specified in RFC2068 and what
one would expect from "use as few cache-control verbs as needed".

Regarding must-revalidate it may well be the case that this should be
restricted to say "non-shared caches" instead of "the cache" for the
reason outlined before, but if in doubt one can always use
private/public to specify what you intended so it's not that critical.
It's been specified like this "for ever".

Hmm.. on a second reading I notice "private" is missing in this section.
A shared cache MUST NOT use a response with "private", especially not
when there is authentication involved... Current text kind of implies a
shared cache is allowed to cache those as long as it always revalidates
on every request IF the request was authenticated but not otherwise..
which is nonsense. (with it being nonsense I surely hope noone have
implemented their shared caches in such manner..)

Regards
Henrik