Subject: #290 [was: SHOULD-level requirements in p6-caching]

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 12:37:14 +1000

On 07/04/2011, at 7:35 PM, Poul-Henning Kamp wrote:
>> 
>> In 3.3,
>> 
>>>   A server SHOULD NOT send Expires dates more than one year in the
>>>   future.
>> 
>> Prose.
> 
> Why this policy restriction ?
> 
> Remove entirely ?

Now <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/290>.

>> In 3.4,
>> 
>>>   When the no-cache directive is present in a request message, a cache
>>>   SHOULD forward the request toward the origin server even if it has a
>>>   stored copy of what is being requested.
>> 
>> Prose.
> 
> Have you discussed the future of "Pragma:" before ?

<http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-14#section-3.4>:

>    This mechanism is deprecated; no new Pragma directives will be
>    defined in HTTP.

> I would like to see the text say that if there is a "Cache-Control:"
> header "Pragma:" MUST be ignored, to resolve the possible conflicts
> between them.

We already have:

> A cache SHOULD treat "Pragma: no-cache" as if the client had sent
>    "Cache-Control: no-cache".

so that makes them equivalent. 

Combined with the rules in sections 2.1 and 2.2, that *should* clarify correct behaviour when they occur together, I think.

E.g., if something has both max-age and no-cache in the response, it might be stored, but won't be reused, because it violates the fourth bullet in section 2.2.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 04 May 2011 10:01:14 +0000

In message <[email-masked]>, Mark Nottingham wri

>so that makes them equivalent. 

That is probably a bit more than I would read into it.

But that does not in any way resolve what to do if you have both
and they are in conflict.

I think it is a good idea to state clearly that Cache-Control takes
precedence if both are present.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 20:20:22 +1000

> precedence if both are present.

A couple of things come to mind.

In requests, Pragma: no-cache is defined to be equivalent to CC: no-cache. I don't think we can change that now, as most implementations that I'm aware of honour that requirement, and more importantly, clients will have a reasonable expectation that this will continue.  A gateway cache can choose to ignore request directives, because it has implicit permission from the origin, of course.

We can have a discussion about CC: no-cache vs. other CC directives, but AFAIK the long-held consensus on that is that max-age (for example) does NOT trump no-store, because no-store is more restrictive than max-age, and implementations need to behave conservatively in the face of conflicting directives. That's one of the reasons that the text in p6 is written like it is.

However, in the case of responses, 2616, defined Pragma: no-cache as

> When the no-cache directive is present in a request message, an application SHOULD forward the request toward the origin server even if it has a cached copy of what is being requested. This pragma directive has the same semantics as the no-cache cache-directive (see section 14.9) and is defined here for backward compatibility with HTTP/1.0. Clients SHOULD include both header fields when a no-cache request is sent to a server not known to be HTTP/1.1 compliant.

and then

>       Note: because the meaning of "Pragma: no-cache as a response
>       header field is not actually specified, it does not provide a
>       reliable replacement for "Cache-Control: no-cache" in a response

So, we already say* that Pragma: no-cache doesn't have any meaning in responses, and should be ignored.

Does that cover the cases you're concerned about, or are you arguing that CC: max-age (etc.) in requests overrides Pragma: no-cache?

Cheers,

* I see we have a bug in p6, in that it says it has the same semantics as the response directive, not the request directive; I'll fix that momentarily.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 20:30:33 +1000

On 04/05/2011, at 8:27 PM, Poul-Henning Kamp wrote:

> In message <[email-masked]>, Mark Nottingham wri
> tes:
> 
>> In requests, Pragma: no-cache is defined to be equivalent to CC: 
>> no-cache. I don't think we can change that now, as most implementations 
>> that I'm aware of honour that requirement, and more importantly, clients 
>> will have a reasonable expectation that this will continue.  A gateway 
>> cache can choose to ignore request directives, because it has implicit 
>> permission from the origin, of course.
> 
> But we can clarify that if there _also_ is a C-C, then Pragma is
> ignored.

Is your use case that you want to be able to send requests that CC-unaware HTTP/1.0 caches will pass through, and yet allow them to be pulled from cache by a CC-aware implementation?

> 
>> Does that cover the cases you're concerned about, or are you arguing 
>> that CC: max-age (etc.) in requests overrides Pragma: no-cache?
> 
> I am arguing that the most expressive and well defined header trumps
> if both are present.
> 
> If there is only Prama, follow it.  If there is only C-C, follow it.
> 
> If both are there, ignore Pragma.
> 
> --

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 04 May 2011 10:32:25 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>Is your use case that you want to be able to send requests that 
>CC-unaware HTTP/1.0 caches will pass through, and yet allow them to be 
>pulled from cache by a CC-aware implementation?

My "use case" is simply that if they are in conflict, we should give
very clear guidance on how that should be resolved, we currently do
not do that.

Further I think it makes sense if this guidance helps eliminate Pragma
in the long run.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 04 May 2011 10:27:25 +0000

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 20:51:05 +1000

> very clear guidance on how that should be resolved, we currently do

OK. As I said, we already have text -- from 2616 -- that explicitly makes Pragma: no-cache equivalent to CC: no-cache in requests. 

Then, p6 2.2 says:

>    For a presented request, a cache MUST NOT return a stored response,
>    unless:
[...]
>    o  the presented request and stored response are free from directives
>       that would prevent its use (see Section 3.2 and Section 3.4)

That's IMO pretty clearly stated (those references are to the sections defining Cache-Control and Pragma, respectively). From here, I can see three possible directions:

1) We could perhaps refine "would prevent its use" here to be more specific; if that's what you're interested in, please say so and we'll pursue that.

2) We could add more text (e.g., in 2.3 Freshness Model) that says that when CC: no-cache and/or Pragma: no-cache are present, the freshness model isn't in use, but we'd be saying the same thing in another way.

On that line, one of the reasons that the original RFC2616 part 13 was so difficult to use was that it said many things twice, and sometimes three times. When I did the initial edit of p6, I was able to remove ten *pages* of such redundant text, and the overwhelming feedback we've had so far is that this has made the specification much clearer, readable and useful. 

So, I'm reluctant to repeat requirements, because we have good reason to believe that it makes the spec worse, not better.

3) OTOH if you're really arguing that we need to change this so that CC's presence signals that Pragma isn't to be followed (your original suggestion, which is why I was asking about use cases), that's a totally different discussion. I've outlined the reasons why I think that won't work; other implementers may have opinions too.

> Further I think it makes sense if this guidance helps eliminate Pragma

Well, I think we're there, as much as we can be; we deprecate the header for new uses. 

The argument behind the current design, AIUI, is that we can't completely get rid of it, because there might be a HTTP/1.0 cache that doesn't understand CC: no-cache in requests. One could say that such caches don't really exist any more (and I'd probably buy it, not sure about how others would feel), but I don't see how making the change you propose helps get rid of the header.

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 04 May 2011 11:03:44 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>So, I'm reluctant to repeat requirements, because we have good reason to =
>believe that it makes the spec worse, not better.

I think simply adding:

	If there is a Cache-Control header, the Pragma header is ignored.

Would clarify it even more.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 22:39:04 +1000

> Would clarify it even more.

That's not clarifying, that's changing.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 4 May 2011 14:57:28 +0200

Hi Poul-Henning,

> Would clarify it even more.

It's different. If we declare that Pragma: no-cache is equivalent to
CC: no-cache, then we simply address conflicts the same way we address
them when multiple CC headers are set.

If you currently know how you deal with two conflicting CC headers, then
you can apply the same method to a CC and a Pragma (eg: probably that the
pragma can win under some circumstances).

Hoping this helps,
Willy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 4 May 2011 16:21:33 -0700

> That's not clarifying, that's changing.

That is what Cache-Control was intended to do.  It replaced Pragma.
If Pragma isn't ignored when CC is present then refinements to its
behavior can't be implemented via CC extensions.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 May 2011 11:21:44 +1000

> behavior can't be implemented via CC extensions.

Right, but the way that you'd do that is by refining CC: no-cache explicitly, rather than relying on a general header override.*

E.g. if I want to define CC: mnot-cacheok in requests to allow enlightened caches to serve a response, while still disallowing unenlightened caches from doing so, I'd define it to override CC: no-cache -- with the side effect of overriding Pragma: no-cache (because they're defined to be equivalent).

Thus, 

GET /foo HTTP/1.1
Host: example.com
Cache-Control: mnot-cacheok
Pragma: no-cache

would have unambiguous semantics to any cache that sees it, and would properly operate.

AFAICT the only reason that you'd way to say that the Cache-Control header's presence means that you can ignore Pragma is if you want to emit requests like this:

GET /foo HTTP/1.1
Host: example.com
Cache-Control: max-age=45
Pragma: no-cache

and have them treated differently by caches that understand CC vs. those that don't. As I've said a few times now, according to the spec, that's currently very clearly equivalent to:

GET /foo HTTP/1.1
Host: example.com
Cache-Control: max-age=45, no-cache

Which I *hope* we can agree should be interpreted conservatively.

If we remove that equivalence and say that Pragma is ignored in the presence of CC, then we'll have situations where *all* CC directives -- whether they intend it or not -- will override the no-cache, and IMO that's a very surprising outcome.

Cheers,

* This makes me realise, however, that the rules in p6 2.1 and p6 2.2 need to allow extensions to override caching semantics in more places. Opening issue <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/291>.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 4 May 2011 18:37:40 -0700

On May 4, 2011, at 6:21 PM, Mark Nottingham wrote:
> and have them treated differently by caches that understand CC vs. those that don't.

Yes, that's exactly what we need to do if there are suspected
HTTP/1.0 intermediaries in the path.  max-age must override Pragma.

> As I've said a few times now, according to the spec, that's currently very clearly equivalent to:
> 
> GET /foo HTTP/1.1
> Host: example.com

No, that's not what the client wants.  The client wants an HTTP/1.1
compliant cache to obey max-age=45, not no-cache, but fears the
presence of non-compliant caches that only understand Pragma.
Hence, both headers are sent.  If the client wanted no-cache
semantics, it would have sent no-cache in *both* fields.

Pragma has the same relation to CC as Expires.  If that isn't defined
in 2616, then it is surely an error in 2616.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 May 2011 12:01:04 +1000

> in 2616, then it is surely an error in 2616.

OK. How do we specify this? Saying "When CC is present, you can ignore Pragma" brings about the case where:

GET /foo HTTP/1.1
Host: example.com
Cache-Control: myweirdextension=foo
Pragma: no-cache

means that a HTTP/1.1 cache that doesn't implement myweirdextension now silently ignores the no-cache.

We do already say

> A client SHOULD include both header fields when a no-cache request is sent to a server not known to be HTTP/1.1 compliant.

... but that's only a SHOULD (hey, the Subject *is* relevant!).

If we go in this direction, AFAICT we'd end up changing 

>    stored copy of what is being requested.  This pragma directive has
>    the same semantics as the no-cache response directive (see Section 3.2.2
>    ) and is defined here for backward compatibility with
>    HTTP/1.0.  A client SHOULD include both header fields when a no-cache
>    request is sent to a server not known to be HTTP/1.1 compliant.  A
>    cache SHOULD treat "Pragma: no-cache" as if the client had sent

to something like

"""
When the no-cache pragma directive is present in a request, a cache MUST treat the request as if it contains the no-cache cache-control directive [ref], unless the Cache-Control header is also present in the request. 

A client SHOULD include both pragma and cache-control directives when sending a no-cache request , unless Cache-Control: no-cache is omitted to target other Cache-Control directives at HTTP/1.1 caches, while precluding HTTP/1.0 implementations from using a cached response.
"""

Is that workable?

Personally, I think that this approach is going to surprise more people than it will help, but I'm not going to lie down in the road about it; highly refined CC request directives are pretty unusual.

Cheers,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 4 May 2011 19:32:27 -0700

> means that a HTTP/1.1 cache that doesn't implement myweirdextension now silently ignores the no-cache.

Which is why it says that extensions can override existing
declarations.  The client would never send the above.  It
would send "Cache-Control: no-cache, myweirdextension=foo" and
myweirdextension would be defined to override the semantics of
no-cache for those caches that support myweirdextension.

> A client SHOULD include both pragma and cache-control directives when sending a no-cache request , unless Cache-Control: no-cache is omitted to target other Cache-Control directives at HTTP/1.1 cac…

Sounds like torture to me.  What is wrong with just ignoring Pragma
if Cache-Control is given and obeyed?

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 May 2011 13:04:53 +1000

> if Cache-Control is given and obeyed?

"Obeyed" really needs to be qualified to those directives that are understood. I've have a go below; if you have suggestions, please provide specific text.

"""
3.4.  Pragma

The "Pragma" header field allows backwards compatibility with HTTP/1.0 caches, so that clients can specify a "no-cache" request that they will understand (as Cache-Control was not defined until HTTP/1.1). When the Cache-Control header is also present and understood in a message, Pragma is ignored. 

In HTTP/1.0, Pragma was defined as an extensible field for implementation-specified directives for recipients. This specification deprecates such extensions to improve interoperability. 

     Pragma           = 1#pragma-directive
     pragma-directive = "no-cache" / extension-pragma
     extension-pragma = token [ "=" ( token / quoted-string ) ]

When the no-cache pragma directive is present in a request, a cache MUST treat the request as if it contains the no-cache cache-control directive [ref], unless the Cache-Control header is also present and the cache obeys those cache-control directives [ref] that it contains (excepting any extensions that are not understood [ref]). 

A client SHOULD include both pragma and cache-control directives when sending a no-cache request, unless Cache-Control: no-cache is purposefully omitted to target other Cache-Control response directives at HTTP/1.1 caches. For example:

GET / HTTP/1.1
Host: www.example.com
Cache-Control: max-age=30
Pragma: no-cache

will constrain HTTP/1.1 caches to serve a response no older than 30 seconds, while precluding implementations that do not understand Cache-Control from serving a cached response.
"""

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 05 May 2011 15:57:14 +1200

having the override be conditional on whether the CC header contained 
understood directives or not seems like not a great idea to me.

I'd have presumed that if the CC header exists, Pragma should be ignored 
- regardless of what directives are contained or whether they are 
understood or enacted.

The reasoning being:

* Pragma was an HTTP/1.0 header
* anyone implementing this RFC is implementing HTTP/1.1
* Pragma was recommended in RFC2616 to cope with HTTP/1.0 caches

therefore an HTTP/1.1 client is only sending Pragma to target HTTP/1.0 
intermediaries, which implementations of this spec are not. The only 
other case where an intermediary would expect to see a Pragma is from an 
HTTP/1.0 client, in which case it wouldn't expect to see C-C.  HTTP/1.1 
clients are still perfectly free to send whatever C-C directives they 
choose.  It's actually more deterministic to ignore Pragma than treat it 
as a repeat C-C header (and therefore combine the headers).  Especially 
since precedence of cache directives isn't explicitly defined (another 
issue).

Regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 May 2011 16:15:33 +1000

It sounds like you're suggesting changing

> When the no-cache pragma directive is present in a request, a cache MUST treat the request as if it contains the no-cache cache-control directive [ref], unless the Cache-Control header is also pres…

to 

"""
When the no-cache pragma directive is present in a request, a cache MUST treat the request as if it contains the no-cache cache-control directive [ref], unless the Cache-Control header is also present.
"""

in the proposal below.

Roy, are you OK with dropping the 'obey' language?

On 05/05/2011, at 1:57 PM, Adrien de Croy wrote:

> 
> having the override be conditional on whether the CC header contained understood directives or not seems like not a great idea to me.
> 
> I'd have presumed that if the CC header exists, Pragma should be ignored - regardless of what directives are contained or whether they are understood or enacted.
> 
> The reasoning being:
> 
> therefore an HTTP/1.1 client is only sending Pragma to target HTTP/1.0 intermediaries, which implementations of this spec are not. The only other case where an intermediary would expect to see a Pragma is from an HTTP/1.0 client, in which case it wouldn't expect to see C-C.  HTTP/1.1 clients are still perfectly free to send whatever C-C directives they choose.  It's actually more deterministic to ignore Pragma than treat it as a repeat C-C header (and therefore combine the headers).  Especially since precedence of cache directives isn't explicitly defined (another issue).
> 
> Regards
> 
> Adrien
> 
> 
> 
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 05 May 2011 06:32:47 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>> When the no-cache pragma directive is present in a request, a cache 
>MUST treat the request as if it contains the no-cache cache-control 
>directive [ref], unless the Cache-Control header is also present and the 
>cache obeys those cache-control directives [ref] that it contains 
>(excepting any extensions that are not understood [ref]).
>
>to 
>
>"""
>When the no-cache pragma directive is present in a request, a cache MUST 
>treat the request as if it contains the no-cache cache-control directive 
>[ref], unless the Cache-Control header is also present.
>"""

Non-english-native-whining:

Text on the general form of "bla and bla or bla and bla, unless
foo" should be avoided, as some non-english natives can be confused
about which "bla" the unless applies to.

It is much better to put the conditional clause first, in
all languages that makes the entire sentence conditional:

In this case it also saves words:

	If Cache-Control is not present, a "Pragma: no-cache" MUST
	have the same effect as if "Cache-Control: no-cache" was
	present.

Poul-Henning

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 May 2011 16:35:20 +1000

> 	have the same effect as if "Cache-Control: no-cache" was

WFM.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 5 May 2011 13:37:46 -0700

>> 	have the same effect as if "Cache-Control: no-cache" was

+1

....Roy

---

From: Dzonatas Sol <[email-masked]>
Date: Fri, 06 May 2011 13:57:43 -0700

> means that a HTTP/1.1 cache that doesn't implement myweirdextension now silently ignores the no-cache.

Is there anything that prevents the Host: field being left out for the 
assumption of volatile (or no-cache) data is then in effect?

If Host: is intentionally left out and if there is any need to know 
where-from then use "best effort" with the assumption of either "guest" 
of known network, or "foreign" or "inherited". If "foreign" or 
"inherited" then assume either the reply-path details are already 
included (by specified protocol).

This could further eliminate the need for User-Agent: in "inherited" and 
"guest" mode. The User-Agent: field doesn't make sense in "guest" mode, 
so I would consider this optional. Could/Would... your opinion, please?

-- 
--- https://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering, Virtual Reality, Consultant

---

From: "William A. Rowe Jr." <[email-masked]>
Date: Fri, 06 May 2011 16:15:24 -0500

On 5/6/2011 3:57 PM, Dzonatas Sol wrote:
> 
> Is there anything that prevents the Host: field being left out for the assumption of
> volatile (or no-cache) data is then in effect?

Huh?  The Host: header is mandatory in HTTP/1.1, without it the only
appropriate response is 400.

---

From: Dzonatas Sol <[email-masked]>
Date: Fri, 06 May 2011 14:35:42 -0700

> appropriate response is 400.

We are aware of this. That 400 response would be an example of something 
that went out of bounds/scope. Thus, volatile.

--

---

From: Dzonatas Sol <[email-masked]>
Date: Sat, 07 May 2011 06:45:44 -0700

On 05/06/2011 01:57 PM, Dzonatas Sol wrote:
>> OK. How do we specify this? Saying "When CC is present, you can 
>> ignore Pragma" brings about the case where:
>>
>> GET /foo HTTP/1.1
>> Host: example.com
> where-from then use "best effort" with the assumption of either 
> "guest" of known network, or "foreign" or "inherited". If "foreign" or 
> This could further eliminate the need for User-Agent: in "inherited" 
> and "guest" mode. The User-Agent: field doesn't make sense in "guest" 
> mode, so I would consider this optional. Could/Would... your opinion, 
> please?
>

Seems like "guest" and "foreign" mode may be to early to introduce to 
HTTP standards for those that have the HTTP/1.0 mindset without any 
virtualization in consideration. With HTTP/1.1 pipelining, however, we 
can still assume "inherited" mode and that the gateway/proxy knows the 
header information if message doesn't specify. There are logical reasons 
that include tokenization, compression, eliminate-redundancy, 
cache/no-cache, and security (with TLS/STLP).

My position is being able to bundle/batch requests based on pivotal 
data, identify where endless transistions need flow stability, and where 
we can use asset caches to immediately replace requests in flow (not 
just respond like Squid or botnets).

For now, we'll consider "guest" and "foreign" mode as less-than-"best 
effort", and "inherited" mode as "best effort" to conform to standard.

--