Subject: draft-ietf-httpbis-p6-cache-06

From: Adrien de Croy <[email-masked]>
Date: Wed, 27 May 2009 05:17:26 +1200

I've just been working through draft-ietf-httpbis-p6-cache-06, so here 
are a few comments.  all in all it's looking a lot better than the 
sections in 2616

S 2.2 Constructing responses from cache
---------------------------------------
The requirements for whether a stored response may be served require 
that it be (paraphrased)

* matching URI
* compatible method
* Vary used to check selecting request headers.  Presumably no check if 
no Vary (only SHOULD requirement)
* no directives preventing returning from cache
* fresh, allowed to be stale, or validated

This leaves problems if there's no Vary header but content negotiation 
was used.  It's not possible to reliably heuristically determine if 
content-negotiation was used without the Vary header.  Vary is only a 
SHOULD level.  Maybe it should be a MUST level?   There needs to be 
another set of checks, at the minimum that Accept-Encoding matches the 
stored Content-Encoding.  E.g. you can't serve gzip content if there's 
no Accept-Encoding: gzip  Arguably others as well (like Accept-Language 
matching on Content-Language , with q values etc).

It says if there are several stored representations serve the one with 
the most recent Date header (MUST level), but this may not be the 
appropriate one if Vary headers aren't available, and you are say 
selecting based on language.

S 2.5 Request methods that invalidate
-------------------------------------
I don't understand how a URI can be compared to a Content-Location or 
Location header and match yet the host part be different.  Surely to 
match the host part must be the same?  It's not clear to me what's being 
matched with what.

Wrt POST (or any method).  If the response to a POST is marked 
explicitly by the origin server as cachable, why should a subsequent 
POST invalidate that contrary to other Cache-control directives?  Surely 
this should only apply if the original method was not POST?

S 2.6 Caching Negotiated responses
----------------------------------
Should I then be referring to Section 4.1 of [part3] to resolve the 
issues around content negotiation?  If so, maybe a mention in S 2.2 
would be useful.

I also don't understand in para 5 the sentence "If the server responds 
with 304 (Not Modified) and includes an entity tag or Content-Location 
that indicates the entity to be used"  How can Content-Location be used 
to select an entity?  Do you match on previously returned 
Content-Location headers for requests for the same URI?  Is that what 
the final para is getting at?  Maybe the wording could be a bit clearer.

S 3.2 Cache-Control
-------------------
first sentence states that directives MUST be obeyed.   This doesn't fit 
with a strategy of ignoring unhandled directives if you get a mixture of 
request and response directives in a message (which is still allowed in 
the ABNF).  I think it should therefore be explicit that it's not valid 
to mix the directives, else you get a MUST requirement to obey 
nonsensical directives.

Otherwise relax the MUST, or relax it to the extent of nonsensical 
directives.  Also, you can't have a MUST requirement on an extensible 
mechanism.  Extensions need to be optional.

Some cache control directives are confusingly similar, especially for 
response directives.

I can't see anywhere easily where it says what to do in some cases either. 

1. private directive with headers.

It states that these headers then are all that is private, and these 
must not be stored. However it does not state what to do when you get a 
subsequent request for that URI that would match.  Does this mean that 
the stored response must be revalidated with the server, or is it 
appropriate to send the stored response (if still fresh) to the client 
without those headers?

2. no-cache.

It's not clear to me what the point of revalidating an entire entry vs 
just revalidating header fields.  Especially when you consider that a 
conditional request will revalidate both or either.  So I can't see any 
point in having headers specified in a no-cache response directive.  
Unless it's intended that failure to revalidate just headers could still 
result in the entry being served, but that in that case those headers 
would need to be omitted?

S 3.4 Pragma. 
-------------
The BNF for this mentions extension-pragma.  Were there ever any of 
these?  Does it make sense to continue to support an extension mechanism 
on a deprecated header that no-one extended?

I've also seen some responses lately that have multiple Cache-Control 
headers - is this valid?

Thanks

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: "Brian Smith" <[email-masked]>
Date: Tue, 26 May 2009 21:42:19 -0500

Adrien de Croy wrote:
> SHOULD level.  Maybe it should be a MUST level?

As you noted below, this rule applies:

   Caches MUST use the most recent response (as determined by the Date
   header) when more than one suitable response is stored.  They can
   also forward a request with "Cache-Control: max-age=0" or "Cache-
   Control: no-cache" to disambiguate which response to use.

Using this rule, along with other rules for caches, results in predictable
behavior, right?

> There needs to be another set of checks, at the minimum that
> Accept-Encoding matches the stored Content-Encoding.  E.g. you
> can't serve gzip content if there's no Accept-Encoding: gzip.
> Arguably others as well (like Accept-Language

Actually, a server *is* allowed to do that. And, caches don't have to worry
about that because they don't have to interpret Accept-* and Content-*
headers. (See my previous message in the other thread.)

> selecting based on language.

The server needs to provide a Vary header in that case.

> match the host part must be the same?  It's not clear to me what's
> being matched with what.

Example 1:
The Request-URI is  http://example.ORG/foo.
Content-Location in the response is http://example.COM/bar.
In this case, we shouldn't invalidate cached representations of
http://example.COM/foo because the request-URI's host doesn't match the host
of the Content-Location header.

Example 2:
The Request-URI is http://example.ORG/foo.
Content-Location is http://example.ORG/bar.
In this case, we should invalidate both http://example.ORG/foo and
http://example.ORG/bar.

Basically, this is trying to implement a "same origin" policy for cache
invalidation.

> POST invalidate that contrary to other Cache-control directives?
> Surely this should only apply if the original method was not POST?

See the discussion about whether the method is part of the cache key. Caches
really need to be very conservative here (that is, MUST invalid) as there
seems to be a lot of disagreement amongst implementers and standardistas
regarding this issue.

> S 2.6 Caching Negotiated responses

No, see above.

> the final para is getting at?  Maybe the wording could be a bit clearer.

Content-Location is only used by caches for invalidation, and never for any
other reason (by caches). Basically, when choosing which cache entries to
invalidate, you must invalidate all the ones with the same Content-Location,
subject to the "same-origin" restriction explained above. 

> S 3.2 Cache-Control
> -------------------
> first sentence states that directives MUST be obeyed.   This doesn't
> fit with a strategy of ignoring unhandled directives if you get a
> mixture of request and response directives in a message (which is
> still allowed in the ABNF). I think it should therefore be
> explicit that it's not valid to mix the directives, else you get
> a MUST requirement to obey nonsensical directives.

A directive that looks like a cache-response-directive in a request is
actually a cache-extension, not a cache-response-directive. Similarly, a
directive that looks like a cache-request-directive in a response is
actually a cache-extension, not a cache-request-directive. The grammar is
just wrong.

> mechanism.  Extensions need to be optional.

"Unrecognized cache directives MUST be ignored." But, caches must recognize
all directives defined in the HTTP spec.

> Some cache control directives are confusingly similar, especially for

I agree.

> 1. private directive with headers.

I will reply in a separate message.

> these?  Does it make sense to continue to support an extension
> mechanism on a deprecated header that no-one extended?

There are undoubtedly extension-pragmas being used which are not defined in
any standard.

> I've also seen some responses lately that have multiple Cache-Control

Yes. See the rules for repeated header field values in Part 1.

- Brian

---

From: "Brian Smith" <[email-masked]>
Date: Tue, 26 May 2009 22:03:48 -0500

Adrien de Croy wrote:

[snip]

> Unless it's intended that failure to revalidate just headers could
> still result in the entry being served, but that in that case those
> headers would need to be omitted?

I thought I had an explanation of how this could be useful, but my
explanation doesn't really jive with what the spec. says. I, too, would
really like to hear somebody explain how a cache is supposed to handle
private="header1, header2" and no-cache="header1, header2".

Regards,
Brian

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 27 May 2009 15:55:06 +1200

> Using this rule, along with other rules for caches, results in predictable
Forcing revalidation doesn't in my mind solve the problem of which (say) 
ETags to put in an If-None-Match header.  I guess you then put them all 
in, but that's then just getting the origin server to re-process the 
negotiation algorithm.  At least it's correct then though.

> The server needs to provide a Vary header in that case.
right, hence my comment maybe Vary should be a MUST.  But I'm sure there 
would be a multitude of problems if that happened.

 So I guess if there's no Vary, what do we we fall back to?

> Basically, this is trying to implement a "same origin" policy for cache
thanks - that makes more sense now, esp after reading the section in 
Part 3 on Content-Location as well.

> seems to be a lot of disagreement amongst implementers and standardistas
The basis I've been working on is that the method should be part of the 
key as long as it's not GET or HEAD, since otherwise you couldn't reply 
to a HEAD with the cached results of a previous GET as you'd get an 
index miss.

In that case there's no ambiguity about the other methods, so we'd be 
back to arguably needlessly invalidating valid stored representations.

> subject to the "same-origin" restriction explained above. 

If a shared cache however appends a bunch of ETags to an If-None-Match 
header, then the server selects from those using Content-Location, then 
the shared cache / proxy needs to return the resultant selected entity?  
Same as if the origin server returned an ETag.

> actually a cache-extension, not a cache-request-directive. The grammar is
OK, that makes sense.

> all directives defined in the HTTP spec.
Agreed.

> There are undoubtedly extension-pragmas being used which are not defined in
OK

> Yes. See the rules for repeated header field values in Part 1.
thanks - your comments are most helpful.

Regards

Adrien

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 27 May 2009 00:04:10 -0500

Adrien de Croy wrote:
> Brian Smith wrote:
> > Adrien de Croy wrote:
> >> It says if there are several stored representations serve the one
> >> with the most recent Date header (MUST level), but this may not
> >> be the appropriate one if Vary headers aren't available, and
> >> you are say selecting based on language.
> >
> right, hence my comment maybe Vary should be a MUST.  But I'm sure
> there would be a multitude of problems if that happened.
> 

A cached response can always be returned unless its Vary header says it
can't be. If some responses included a Vary header and then a later
cacheable response doesn't include one (for the same resource), then the
later response will effectively replace all the other cached responses
because of the "most recent" rule.

> > See the discussion about whether the method is part of the cache key.
> > Caches really need to be very conservative here (that is, MUST
> > invalid) as there seems to be a lot of disagreement amongst
> > implementers and standardistas regarding this issue.

> back to arguably needlessly invalidating valid stored representations.

It seems you are suggesting that it would be better if there was some way
for a POST request to indicate that no invalidation is necessary. I agree.
However, the Cache-Control of the POST response cannot be overloaded for
that purpose. Some other (new) mechanism would be needed, which would be out
of scope of the HTTPbis effort.

> > Content-Location is only used by caches for invalidation, and never
> > for any other reason (by caches). Basically, when choosing which cache
> > entries to invalidate, you must invalidate all the ones with the same
> > Content-Location, subject to the "same-origin" restriction explained
> > above.
> 

I'm not sure what you mean here. The origin server doesn't use
Content-Location to choose a variant; it is going to use the request to
choose a variant. 

- Brian

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 27 May 2009 17:26:25 +1200

> because of the "most recent" rule.
I can see that a later one would always be served by the cache, however 
existing older ones could still be revalidated if say the client 
requested revalidation, or the freshest one expired.  Any time a 
revalidation is done, the cache could indicate all ETags for the URI.  
At that point, an older one could be selected by the origin server, 
which then would become the newest one for subsequent requests.

Seems to me this would lead to weird results, esp with multiple language 
versions of the same URI.

> of scope of the HTTPbis effort.
so it's invalid to include a Cache-Control: public say on the POST response?

> Content-Location to choose a variant; it is going to use the request to
It uses it to indicate a chosen variant is what I meant sorry.  the 
cache then uses the Content-Location field to choose from amongst a set 
of variants.

Adrien

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 27 May 2009 17:37:47 +1200

Adrien de Croy wrote:
> I can see that a later one would always be served by the cache, 
> however existing older ones could still be revalidated if say the 
> client requested revalidation, or the freshest one expired.  Any time 
> a revalidation is done, the cache could indicate all ETags for the 
> URI.  At that point, an older one could be selected by the origin 
> server, which then would become the newest one for subsequent requests.
>
> Seems to me this would lead to weird results, esp with multiple 
> language versions of the same URI.
>
also, the matching rules for matching on selected headers are quite 
unintuitive.  the rule states that they must be identical except for 
whitespace and combining headers.

However order therefore is still important, so matching between

Content-Language: en-nz, fr
and
Content-Language: fr, en-nz

would fail, even though logically these are identical since order is not 
significant in the spec.  I can understand this makes matching simpler 
(normalise and string compare), and probably shields against arbitrary 
selection algorithms (e.g. order-based) being used by the origin server, 
but it's not clear from the spec as to why, and anyone wanting to do 
things a bit "smarter" may consider doing a different sort of match with 
potentially questionable results.

regards

Adrien

---

From: Yves Lafon <[email-masked]>
Date: Wed, 27 May 2009 04:16:37 -0400 (EDT)

> private="header1, header2" and no-cache="header1, header2".

The same way you will handle "Cache-Control: no-cache, max-age=86400". It 
is an error case, and a good implementation will most probably choose to 
take the more restrictive approach (no-cache), or simply discard the 
response for the cache purpose (so treat it as uncachable, which is the 
same as no-cache, but the intent is different).
Another extreme is to simply reject such responses as it is clearly an 
error.

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 27 May 2009 09:28:24 -0500

Adrien de Croy wrote:
> So it's invalid to include a Cache-Control: public say on the POST
> response?

It's not invalid, but it doesn't mean anything. POST responses are never
cacheable; caches must write POST requests through to the server, and a
cache can't use a POST response to respond to a GET/HEAD request.

> >> If a shared cache however appends a bunch of ETags to an If-None-
> >> Match header, then the server selects from those using
> >> Content-Location, then the shared cache / proxy needs to
> >> return the resultant selected entity?
> > Content-Location to choose a variant; it is going to use the request
> > to choose a variant.
> >

If the server returned cacheable responses with ETags, and then returns a
304 response during validation, then it needs to return an ETag in the 304
response. A cache will only use Content-Location to determine which variant
to return when the server didn't return an ETag (that is, when the
validation is being done using If-Modified-Since instead of If-None-Match).

Regards,
Brian

---

From: Thomas Broyer <[email-masked]>
Date: Wed, 27 May 2009 16:49:01 +0200

> cache can't use a POST response to respond to a GET/HEAD request.

Even when the POST response has a Content-Location?

-- 
Thomas Broyer

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 27 May 2009 09:14:53 -0500

Yves Lafon wrote:
> > explanation doesn't really jive with what the spec. says. I, too,
> > would really like to hear somebody explain how a cache is supposed
> > to handle private="header1, header2" and
> > no-cache="header1, header2".
> 
> The same way you will handle "Cache-Control: no-cache, max-age=86400".
> It is an error case, and a good implementation will most probably
> choose to take the more restrictive approach (no-cache), or simply
> discard the response for the cache purpose (so treat it as
> uncachable, which is the same as no-cache, but the intent is
> different). Another extreme is to simply reject such responses
> as it is clearly an error.

I was unclear. I was asking, simply, what does
Cache-Control:private="header1, header2" mean? How, exactly, should a cache
process subsequent requests differently than if the cache response had just
Cache-Control:private?

And, how can a cache treat Cache-Control:no-cache="header1, header2"
differently than just Cache-Control:no-cache? How can one cache and
revalidate only part of a response?

Regards,
Brian

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 27 May 2009 10:31:35 -0500

Thomas Broyer wrote:
> Brian Smith <[email-masked]> wrote:
> > It's not invalid, but it doesn't mean anything. POST responses are
> > never cacheable; caches must write POST requests through to the
> > server, and a cache can't use a POST response to respond to a
> > GET/HEAD request.
> 

There's nothing in RFC2616 or the latest HTTPbis drafts that says a cached POST response can be returned in response to a GET/HEAD request. Applying Postel's rule, a cache shouldn't return a cached POST response to a GET/HEAD request, and servers shouldn't include Cache-Control/Expires headers in POST responses. That should be explicit in the specification.

- Brian

---

From: David Morris <[email-masked]>
Date: Wed, 27 May 2009 08:26:40 -0700 (PDT)

> cacheable; caches must write POST requests through to the server, and a

A long time ago the was an ID re. 'safe' post responses which was intended
to allow caching. I've lost track of what happened to the ID as well as I 
have no idea re. implementation.