Subject: WGLC p6 4.2.1

From: "Adrien W. de Croy" <[email-masked]>
Date: Sun, 17 Mar 2013 22:39:21 +0000

Hi all

I see there were some changes made to the 3rd bullet point in 4.2.1 
about selection of representations to update with a 304.

The new text hints that dates other than those received in a previous 
Last-Modified can be used to generate a conditional request with 
If-Modified-Since.

However, there are a number of side-effects with introducing this 
concept.

The previous iterations of the spec requires that If-Modified-Since may 
only contain the content of a previously received Last-Modified.  I 
don't know if it's wise to open this up.  We would need to consider what 
the side-effects of choosing any other date may be.

For instance if the sender (revalidating client) just chose the file 
date on the file in their store, it could bear no resemblance to any 
time on the server, and may appear to be a more recent copy than they 
actually have.  A server could then send a 304 back for content that was 
no longer valid.

So I think the date used in If-Modified-Since needs to have originally 
come from the server.  This only leaves Last-Modified, and Date.  Date 
could have been altered by intermediaries, so actually I feel it's 
unsafe to allow any other value than that contained in Last-Modified.

My original query about this point, which I suspect was a contributing 
factor to this change, was concerning receiving a 304 response to a 
request that related to revalidating a stored entry that had no 
validators.  My point at the time was that a 304 response to a 
non-conditional request is a server bug.  I would argue that 
constructing validators in a request where none were previously received 
is not valid (client can't invent ETag, and shouldn't invent 
Last-Modified), and so therefore my proposal was to strike this 
altogether, or prohibit it.

Regards

Adrien

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 18 Mar 2013 06:58:32 +0000

>Last-Modified can be used to generate a conditional request with 

There are several uses I know of, where IMS is used by clients
without having an older object, as a way to say "Is a recent version
of this object available ?".

One such usage is "Are there any severe weather warnings published
in the last 24 hours ?" which avoids pulling the "no warnings"
boilerplate most of the year.

I will fully agree, that using only values originally received from
the server is a lot more water-tight, and is to be strongly recommended
(at the SHOULD level), but trying to outlaw other values is a waste
of everybodys time, given that such a ban cannot be sensibly enforced
by us.

If the server for some reason insists on not receiving arbitrary
timestamps in IMS, it can use E-tags, which by definition are
impossible to synthesize anywhere else.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 18 Mar 2013 21:17:26 +1300

> impossible to synthesize anywhere else.

+1 on what PHK said.

A client with out-of-band information about the server state should not 
be hindered by HTTP as to the conditionals it makes a request with.
As long as the server response is correct in relation to those 
conditionals it cannot cause problematic side effects elsewhere than the 
client itself, since all intermediaries will be considering that 
request+reply in isolation.

Amos

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Mon, 18 Mar 2013 08:59:02 +0000

there are some interactions with caches.

for instance a cache receiving a request with If-Modified-Since later 
than its own Last-Modified, may presume the client has a later copy, and 
discard its own copy.

I think really if we're to introduce this sort of allowed behaviour, we 
need to do a bit more work on the spec, if only to cover the other parts 
where IMS is covered etc.

And set some rules.

Adrien

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 18 Mar 2013 11:17:38 +0000

>than its own Last-Modified, may presume the client has a later copy, and

Uhm, so you're saying I can clean the entire cache with bogos IMS
requests ?

That doesn't sound optimal or even sensible to me...

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Mon, 18 Mar 2013 11:31:19 +0000

------ Original Message ------
From: "Poul-Henning Kamp" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Amos Jeffries" <[email-masked]>; "[email-masked]" 
<[email-masked]>
Sent: 19/03/2013 12:17:38 a.m.
Subject: Re: WGLC p6 4.2.1
>Uhm, so you're saying I can clean the entire cache with bogos IMS

that's not the point.

There could be any number of ways a cache could be depending on the 
current specification which requires that IMS came from a previous LM.

We're looking to change that between v20 and v21 just before last call.  
In a sub-part of the spec.

I think it's a non-trivial change, and deserves more discussion, and is 
possibly beyond the mandate for httpbis.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 18 Mar 2013 12:05:11 +0000

In message <em94119d23-43b0-4de4-a3e3-8c30e8a40bfc@bombed>, "Adrien W. de Croy" writes:

>>>than its own Last-Modified, may presume the client has a later copy,=20
>>>and
>>>discard its own copy.
>>

It very much is:  A cache would have to be stupid to make that
assumption, and we should not be protecting stupid mistakes with
the specification, we should make things work.

No matter what you write in the specification, you will have IMS
headers with non-server-supplied timestamps, because it is possible
and there are legitimate use-cases.

We can discuss if the text expresses this optimally, but there is
no way the text can put this particular genie back in his bottle.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Mon, 18 Mar 2013 12:19:17 +0000

------ Original Message ------
From: "Poul-Henning Kamp" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Amos Jeffries" <[email-masked]>; "[email-masked]" 
<[email-masked]>
Sent: 19/03/2013 1:05:11 a.m.
Subject: Re: WGLC p6 4.2.1
>the specification, we should make things work.
it was just one example.  There are potentially limitless ones.

fundamentally, we're changing the semantics.

Do we even know what that may break?

I don't recall seeing a discussion about it on list, but maybe it was 
discussed in a meeting.

>and there are legitimate use-cases.
I agree it's possible, I'm not sure about the use-cases. at least not 
the one you mentioned before.

Sending If-Modified-Since currently indicates you have a copy.  So we're 
looking to break that too?

A cache needs to know which version is being re-validated, whether its 
own or the clients.  It's probably not a train wreck, but it's 
convoluted enough trying to figure out what on earth is going on when 
you get a 304 back and you were checking multiple items, some with ETag 
some with IMS.

I never got a straight answer on my query about whether it's valid to 
send a 304 back to a request that contained If-None-Match with a 
different ETag.  Personally I consider that a server bug as well.

>no way the text can put this particular genie back in his bottle.
I think we will need to make further changes, to refer to these changes 
elsewhere in the spec, e.g. where it's discussed that to make a 
conditional request one adds If-Modified-Since using the content of a 
previous Last-Modified response header.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 18 Mar 2013 12:24:12 +0000

>fundamentally, we're changing the semantics.

No, at most we're changing the text.

The HTTP sematics is whatever is in used out there on the big bad net...

---

From: Fabian Keil <[email-masked]>
Date: Mon, 18 Mar 2013 14:17:27 +0100

> Do we even know what that may break?

Privoxy has supported modifying the If-Modified-Since and
Last-Modified dates for years to make using them as cookies
less convenient:
http://www.privoxy.org/user-manual/actions-file.html#HIDE-IF-MODIFIED-SINCE
http://www.privoxy.org/user-manual/actions-file.html#OVERWRITE-LAST-MODIFIED

As far as I know this didn't break anything except for the
user tracking (provided no additional tricks are used).

Modifying the If-Modified-Since date comes with the risk of ending up
with a stale copy, but if the modification range is small it's usually
not a problem.

> looking to break that too?

In my opinion it only indicates that you don't want a copy
if it isn't more recent than the given date.

RFC 2616 14.25 even allows sending an arbitrary date ...

> previous Last-Modified response header.

My interpretation of "4.2. Validation Model" is that it's only
relevant for certain conditional requests and thus doesn't prohibit
sending other dates for other conditional requests.

Fabian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 19 Mar 2013 11:00:24 -0700

On Mar 17, 2013, at 3:39 PM, Adrien W. de Croy wrote:

> Hi all
>  
> I see there were some changes made to the 3rd bullet point in 4.2.1 about selection of representations to update with a 304.
>  
> The new text hints that dates other than those received in a previous Last-Modified can be used to generate a conditional request with If-Modified-Since. 

Yes, because that has always been allowed, including within my
original definition when I invented it in 1993.  IMS is used for
both cache updates and restricted-window traversals (e.g., MOMspider).

> However, there are a number of side-effects with introducing this concept.

It is not being introduced.  p6 was originally extracted to only talk
about the use of IMS in caching, but it still needs to deal with all
valid uses of IMS that were defined in RFC2616, RFC2068, and RFC1945.
The recent changes in p6 just restores the prior definitions.

This dual use of IMS has never been a problem in the past, though
concerns about it was one of the main reasons for introducing etags
as a replacement for validation.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Mar 2013 12:07:49 +1100

> both cache updates and restricted-window traversals (e.g., MOMspider).

Indeed; I've seen a widely-used browser doing it.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 21 Mar 2013 23:57:38 +0000

OK, thanks for clarification.

So this means that maybe it was just my mis-reading of RFC2616 that led 
me to presume IMS could only come from LM.

I guess this means it's possible to make any request conditional, by 
setting an IMS header.

Should there be maybe some SHOULD level requirements around choice of 
value for this?

e.g. if there was a LM sent on the previous response, it SHOULD be used.

Adrien