Subject: Expires header vs Vary header

From: Julian Reschke <[email-masked]>
Date: Mon, 08 Dec 2003 18:48:26 +0100

Hi,

I'm experiencing a somewhat strange behaviour involving "Vary" header, 
"Expires" header and 304 (not modified) responses.

Out server supports content that knows about its Expiry date, has strong 
Etags and indeed (potentielly) varies across a few request header 
dimensions.

I'm testing with Mozilla Firebird (using the excellent Live HTTP headers 
plugin).

After clearing all caches, the trace shows that Mozilla is GETting the 
resource, and that the response is a 200, including "Vary" and "Expires" 
(let's say 60 seconds in the future):

HTTP/1.x 200 OK
Date: Mon, 08 Dec 2003 17:37:13 GMT
Vary: accept-encoding, accept-language, cookie, user-agent, translate
Accept-Ranges: bytes
Etag: "942-1070891632000"
Last-Modified: Mon, 08 Dec 2003 13:53:52 GMT
Expires: Mon, 08 Dec 2003 17:38:13 GMT
Content-Type: image/gif
Content-Length: 942
Server: Apache Coyote/1.0

Continueing browsing, Mozilla indeed does not attempt any additional GET 
on that resource until the advertised expiry date has passed. When this 
happens, Mozilla sends a conditional GET...:

GET .../folder.gif HTTP/1.1
Host: 192.168.1.50:8080
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5) 
Gecko/20031007 Firebird/0.7
Accept: image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://192.168.1.50:8080/wcm/docs/cm/jr/leer/
Cookie: JSESSIONID=AA0E061A3B970E4AFC1BA61B8C973572
If-Modified-Since: Mon, 08 Dec 2003 13:53:52 GMT
If-None-Match: "942-1070891632000"
Authorization: Basic cmVzY2hrZTp3Y20xMjM0

returning...:

HTTP/1.x 304 Not Modified
Date: Mon, 08 Dec 2003 17:38:13 GMT
Vary: accept-encoding, accept-language, cookie, user-agent, translate
Etag: "942-1070891632000"
Expires: Mon, 08 Dec 2003 17:39:13 GMT
Server: Apache Coyote/1.0

Until now all is working as expected, however after receiving a 304 for 
the first time, Mozilla now issues the GET requests *every* time 
(instead of using the cached response for another 60 seconds).

Removing the "Vary" header from the responses "fixes" this behaviour. Am 
I missing an dependency between "Vary" and "Expires" (and/or possible 
strong ETags) I should be aware of?

Feedback appreciated,

Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 8 Dec 2003 12:52:15 -0700 (MST)

On Mon, 8 Dec 2003, Julian Reschke wrote:

> Until now all is working as expected, however after receiving a 304
> for the first time, Mozilla now issues the GET requests *every* time
> Removing the "Vary" header from the responses "fixes" this
> behaviour. Am I missing an dependency between "Vary" and "Expires"
> (and/or possible strong ETags) I should be aware of?

I am not aware of such dependencies. I would have guessed that Mozilla
does not update Expires header when receiving a 304 response, but then
it would probably not have worked without the Vary either.

Alex.

---

From: Jeffrey Mogul <[email-masked]>
Date: Mon, 08 Dec 2003 11:59:31 -0800

From: Alex Rousskov <[email-masked]>
    
    I am not aware of such dependencies. I would have guessed that
    Mozilla does not update Expires header when receiving a 304
    response, but then it would probably not have worked without the
    Vary either.

If Mozilla is not updating the Expires header when it receives
a 304 response with a new value for that header (as in the
example in Julian Reschke's message), then it is violating
this MUST:

   If a cache uses a received 304 response to update a cache entry, the
   cache MUST update the entry to reflect any new field values given in
   the response.

in RFC2616 section 10.3.5 304 (Not Modified).  This is restated
in more detail in section 13.5.3.

-Jeff

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 8 Dec 2003 13:05:50 -0700 (MST)

Yes, of course, but Julian says everything works fine without the
Vary. I find it strange that a Vary header would prevent required
updates.  But it is still possible, of course. Sorry for not being
clear.

Alex.

On Mon, 8 Dec 2003, Jeffrey Mogul wrote:

> If Mozilla is not updating the Expires header when it receives a 304
> response with a new value for that header (as in the example in
> Julian Reschke's message), then it is violating this MUST:
>
>    If a cache uses a received 304 response to update a cache entry,
>    the cache MUST update the entry to reflect any new field values
>    given in the response.
>
> in RFC2616 section 10.3.5 304 (Not Modified).  This is restated in
> more detail in section 13.5.3.
>
> -Jeff
>

---

From: Jeffrey Mogul <[email-masked]>
Date: Mon, 08 Dec 2003 13:42:06 -0800

Sorry, I guess I didn't make my own point clear.

Since RFC 2616 says "the cache MUST update the entry", without
any language about "but this changes if Vary is used", then what
Julian complained about seems to be an implementation bug, if
your interpretation ("Mozilla does not update Expires header when
receiving a 304 response") was correct.

If so, I don't think we really need to discuss it on the HTTP-WG
list.  This should go through Mozilla's bug-reporting process.

On the other hand, Mozilla could simply have decided to disable
caching for any response that carries a Vary header.  This is
perfectly legal, and is a simple but effective way of ensuring
that the Vary specification is observed.

The problem is that we can only infer whether Mozilla is correctly
updating its cache entry, so without some other information
(results of other tests, or reading the source code) we can't
tell whether this is a case of "Vary prevents required updates"
or "Vary disables caching".

I'm sure this has nothing at all to do with Schroedinger's cat.

-Jeff

---

From: Julian Reschke <[email-masked]>
Date: Mon, 08 Dec 2003 22:49:47 +0100

> list.  This should go through Mozilla's bug-reporting process.

Seems so. I just wanted to find out first whether what I am doing is 
supposed to work.

> that the Vary specification is observed.

Yes, but this is not the case. If the Vary header is present, the 
Expires header on the first 200 response works as expected, and 
furthermore the presence of the ETag causes Mozilla to issue a 
conditional GET. This seems to indicate that caching is supposed to work 
in presence of Vary.

> I'm sure this has nothing at all to do with Schroedinger's cat.

:-)

---

From: "Joris Dobbelsteen" <[email-masked]>
Date: Tue, 9 Dec 2003 19:43:24 +0100

Indeed the 200 works fine, but this doesn't automatically mean that 304
works fine too.

Better discuss this with the Mozilla developers first.

-Joris

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Julian Reschke
> Sent: Monday, 8 December 2003 22:50
> To: Jeffrey Mogul
> Cc: Alex Rousskov; [email-masked]
> Subject: Re: Expires header vs Vary header
> 
> 
> Jeffrey Mogul wrote:
> 
> > Since RFC 2616 says "the cache MUST update the entry", without any 
> > language about "but this changes if Vary is used", then what Julian 
> > complained about seems to be an implementation bug, if your 
> > interpretation ("Mozilla does not update Expires header 
> when receiving 
> > a 304 response") was correct.
> > 
> > If so, I don't think we really need to discuss it on the 
> HTTP-WG list.  
> > This should go through Mozilla's bug-reporting process.
> 
> Seems so. I just wanted to find out first whether what I am 
> doing is supposed to work.
> 
> > perfectly legal, and is a simple but effective way of ensuring that 
> > the Vary specification is observed.
> 
> Yes, but this is not the case. If the Vary header is present, 
> the Expires header on the first 200 response works as 
> expected, and furthermore the presence of the ETag causes 
> Mozilla to issue a conditional GET. This seems to indicate 
> that caching is supposed to work in presence of Vary.
> 
> (results 
> > of other tests, or reading the source code) we can't tell 
> whether this 
> > is a case of "Vary prevents required updates"

---

From: "Charumathy Venkatraman" <[email-masked]>
Date: Tue, 16 Dec 2003 15:13:26 +0530

Under what conditions would a HTTP server abort a connection from the
client?

I am testing with a server that intermittently aborts the connection. My
client fails with the error 10053 (WSAECONNABORTED). I am not able to nail
down exactly when this happens, but it seems like when I issue a POST which
the server does not support, it returns error 405 (which I am handling
fine), and the next time I issue a GET or a POST I get the 10053 error.

Regards,
Charu.

---

From: Scott Lawrence <[email-masked]>
Date: Tue, 16 Dec 2003 10:01:42 -0500

> fine), and the next time I issue a GET or a POST I get the 10053 error.

The server is free to close the connection whenever it wants to, and
it's not uncommon to see a close following an error.

You might want to look at a network trace to see exactly when the TCP
connection closes relative to when you are sending the next request.
In any event, the client is responsible for opening a new connection
whenever it needs one.

-- 
Scott Lawrence        
  http://skrb.org/scott/