Subject: Re: Questions (errata?) about caching authenticated responses [#174]

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jun 2010 14:54:31 +1000

Picking this issue back up and CC:ing Jeff for his recollections (see <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/174> for background).

On 26/07/2009, at 2:06 AM, Henrik Nordstrom wrote:

> Going back reading RFC2068 for a while... and there the picture is quite
> different when it comes to proxy-revalidate in authenticated responses:
> 
>     1. If the response includes the "proxy-revalidate" Cache-Control
>        directive, the cache MAY use that response in replying to a
>        subsequent request, but a proxy cache MUST first revalidate it with
>        the origin server, using the request-headers from the new request
>        to allow the origin server to authenticate the new request.
>     2. If the response includes the "must-revalidate" Cache-Control
>        directive, the cache MAY use that response in replying to a
>        subsequent request, but all caches MUST first revalidate it with
>        the origin server, using the request-headers from the new request
>        to allow the origin server to authenticate the new request.
>     3. If the response includes the "public" Cache-Control directive, it
>        may be returned in reply to any subsequent request.
> 
> So it turns out that what have happened with respect to proxy-revalidate
> is that the MAY rule regarding proxy-revalidate was lost from this
> section when s-maxage was added in RFC2616. Probably an editorial error
> when introducing s-maxage, alternatively (but less likely) there may
> have been discussions to remove proxy-revalidate entirely as the
> definition of s-maxage makes proxy-revalidate redundant, but then this
> text was never added back when it was decided to keep proxy-revalidate.

Here's what I think happened. RFC2068 included this text in 13.2.1:

> If an origin server wishes to force any HTTP/1.1 cache, no matter how
> it is configured, to validate every request, it should use the
> "must-revalidate" Cache-Control directive (see section 14.9).

even though 14.9 of the same document says:

> When the must-revalidate directive is
> present in a response received by a cache, that cache MUST NOT use
> the entry after it becomes stale to respond to a subsequent request
> without first revalidating it with the origin server.

The former text mis-defining must-revalidate was removed from 2616, but not before it confused the authors of the Auth and Cookie specs, as noted in <http://tools.ietf.org/html/draft-mogul-http-revalidate-00>.

> Proposed resolution to the proxy-revalidate issue: Simply add back the
> text from 1. above where it was nly correcting the text to say "shared
> cache" where it said "proxy", at least for now. This makes
> specifications consistent with what was specified in RFC2068 and what
> one would expect from "use as few cache-control verbs as needed".

There was a conscious decision in 2616 to remove the effects of the misinterpretation of must-revalidate from the spec, including in the Authorization header.   Moving back to the 2068 text would cause yet more confusion.

Additionally, 2616 explicitly says in the definition of proxy-revalidate:

> Note that such authenticated responses also need the public cache
> control directive in order to allow them to be cached at all.

Counter-proposal:

Add a new section to p6:

---8<---
Shared Caching of Authenticated Responses

Shared caches MUST NOT use a cached response to a request with an Authorization [ref] header to satisfy any subsequent request unless a cache directive that allows such responses to be stored is present in the response.

In this specification, the following Cache-Control response directives [ref] have such an effect: must-revalidate, public, s-maxage.

Note that cached responses that contain the "must-revalidate" and/or "s-maxage" response directives are not allowed to be served stale [ref] by shared caches. In particular, a response with either "max-age=0, must-revalidate" or "s-maxage=0" cannot be used to satisfy a subsequent request without revalidating it on the origin server. 
--->8---

... with appropriate changes to p6 2.1, 2.2, as well as the definitions of the Auth header and appropriate CC directives.

> Regarding must-revalidate it may well be the case that this should be
> restricted to say "non-shared caches" instead of "the cache" for the
> reason outlined before, but if in doubt one can always use
> private/public to specify what you intended so it's not that critical.
> It's been specified like this "for ever".

Non-shared caches aren't affected by the Authentication header.

> Hmm.. on a second reading I notice "private" is missing in this section.
> A shared cache MUST NOT use a response with "private", especially not
> when there is authentication involved... Current text kind of implies a
> shared cache is allowed to cache those as long as it always revalidates
> on every request IF the request was authenticated but not otherwise..
> which is nonsense. (with it being nonsense I surely hope noone have
> implemented their shared caches in such manner..)

Are you referring to 2068 text or HTTPbis text here?

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: Jeffrey Mogul <[email-masked]>
Date: Wed, 02 Jun 2010 10:56:23 -0700

Picking this issue back up and CC:ing Jeff for his
   recollections (see <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/174>
   for background).

Sorry, I don't have any clear recollections.  I did find some
relevant emails that I had sent:

(email 1) (Thu, 02 Jan 97 10:55:03 PST) in private (not to the
mailing list) that contains these paragraphs:

   The current issues in the HTTP WG related to caching, for HTTP/1.1
   in particular, are limited to a few issues that seem particularly
   pressing:

        (1) there is an apparent bug (or at least a "poor design
        choice") in the current draft re: the "proxy-revalidate"
        Cache-control directive, probably my fault.  We will likely
        end up defining a new directive to avoid the problem, and
        I've volunteered to write up a brief draft on this.
        
        (2) Several related documents will need to be revised to
        deal with this bug.

(email 2) (Mon, 21 Apr 97 19:19:18 PST) to Larry Masinter and
Jim Gettys (also not to the entire list):

       I thought the http-versions draft was ready, but looking at

       ftp://ietf.org/internet-drafts/draft-mogul-http-revalidate-00.txt

       I see that contains two alternatives, and clearly cannot
       progress until we've reached consensus on which one.

   You didn't read it carefully enough.  Section 3 lays outs
   a number of alternatives, to show that they were considered,
   and includes an analysis to show what their benefits and
   drawbacks are ... but does not propose any of these directly.

   Section 4 precisely defined a specific proposed change.  This
   is the "normative" part of the proposal, i.e., a set of edits
   to be applied to RFC2068.  If Jim finds anything in section
   4 ambiguous, then I need to fix this; otherwise, not.

       Were you intending to release a new draft so I can 'last call' it?

   The only glitch is that Roy, during a hallway chat at the WWW6
   conference, casually suggested that he wanted me to replace the
   term "proxy [cache]" with "shared cache".  He hasn't ever sent
   email to this effect, but I have no objections to making this
   change.  I consider it to be an editorial issue (clarification)
   and it seems pointless to spin another I-D version just to clarify a
   single word.

I don't remember what Jim did as a result of this exchange.
(See http://tools.ietf.org/html/draft-mogul-http-revalidate-00 for
the draft mentioned above.)

Also, Duane sent an email on 19 Jul 2000 to the http-wg list
   Subject: Questions (errata?) about caching authenticated responses
which brings up some questions about proxy-revalidate; see
<http://ftp.ics.uci.edu/pub/ietf/http/hypermail/2000/0142.html>

That's all I can remember at this point, but I think a careful
reading of http://tools.ietf.org/html/draft-mogul-http-revalidate-00
might clarify much of the history.

-Jeff

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 3 Jun 2010 17:03:49 +1000

Thanks, Jeff!

> <http://ftp.ics.uci.edu/pub/ietf/http/hypermail/2000/0142.html>

Yes, that's the source of the issue we're currently trying to resolve.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jun 2010 11:33:34 +1000

I haven't heard any comment on this proposal. I *think* it accurately reflects what's in 2616, and AFAICT from the history, what's in 2616 was intentional. 

Thoughts?

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 7 Jun 2010 19:40:44 -0700

Wouldn't it be easier to just say Authorization implies
"Cache-control: private" unless explicitly given otherwise?

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jun 2010 15:02:35 +1000

Well, most of the proposal is defining what 'explicitly given otherwise' means (and noting the consequences)...

Which is more clear?

> Shared caches MUST NOT use a cached response to a request with an Authorization [ref] header to satisfy any subsequent request unless a cache directive that allows such responses to be stored is pr…

or 

"""
Requests with Authoration [ref] headers MUST have the same effect as Cache-Control: private [ref] on the response.
"""

---

From: David Morris <[email-masked]>
Date: Mon, 7 Jun 2010 22:36:27 -0700 (PDT)

Use the 2nd form and save the extra words to make sure "Cache-Control: 
private" is clear if it isn't already.

---

From: Henrik Nordström <[email-masked]>
Date: Tue, 08 Jun 2010 08:28:20 +0200

> I haven't heard any comment on this proposal. I *think* it accurately reflects what's in 2616, and AFAICT from the history, what's in 2616 was intentional. 

Looks fine to me.

Regards
Henrik

---

From: Henrik Nordström <[email-masked]>
Date: Tue, 08 Jun 2010 09:08:45 +0200

> "Cache-control: private" unless explicitly given otherwise?

What is "explicitly given otherwise"? There is no directive which
explicitly negates private. Cache-Control directives all adds up to the
status of the response, restricting the bounds of caching or softly
extending it. There technically is no conflict in "Cache-Control:
private, public, s-maxage=100000" even if the "public, s-maxage=100000"
part is redundant. (private MUST NOT, public/s-max-age MAY)

I don't see how to get out of this without explicitly stating which
directives overrides the implicit "private". And referencing to the
authenticated state as an implicit "private" only adds confusion I
think, reducing the meaning of MUST NOT unless worded carefully.

Regards
Henrik

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 8 Jun 2010 14:22:57 -0700

> part is redundant. (private MUST NOT, public/s-max-age MAY)

No, if "Cache-control: public" is given then it overrides any
default in the protocol semantics.  I meant that private is the new
default for that response, not that private would appear in Cache-control.

> think, reducing the meaning of MUST NOT unless worded carefully.

And I don't understand how that is confusing.  Implicit == default.
The only reason "public" exists is to override that default when,
for whatever reason, the default is private or no-cache.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 9 Jun 2010 10:37:25 +1000

Actually, that's not a good proposal, because it doesn't cover the 'defaulting' concept Roy has been talking about.

Roy, can you propose some text, if you want to pursue this? I feel like we're approaching diminishing returns here...

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 09 Jun 2010 12:55:00 +1200

I think there's a slight difference between the 2 cases

1. Authenticated response containing Cache-control: public
2. Response (authenticated or not) containing Cache-control: private, public

In the first case it's obvious the server wants to make the response 
cachable in spite of the fact that it's the result of an authenticated 
request.

In the second case, it's not at all obvious what the server wants, and 
an optimist might choose public, and a conservative might choose private.

In most if not all computer languages, there is a concept of defined 
operator precedence.  Maybe such a concept would be useful here, e.g. 
define which tokens have precedence over others.  I'd expect however in 
that case that for security reasons private would take priority over 
public, which then leads us to problems treating authenticated responses 
as if they included the public directive.  So in short, I think it needs 
to be treated as a special case.

Regards

Adrien

---

From: Henrik Nordström <[email-masked]>
Date: Wed, 09 Jun 2010 10:38:41 +0200

> for whatever reason, the default is private or no-cache.

It's not only public which overrides the implicit private. It's a whole
range of directives of which public is one (public, s-maxage,
must-revalidate, proxy-revalidate, but not  max-age). Not specifying
which directives overrides the implicit default private would be
confusing.

Regards
Henrik