Subject: Re: i37: Vary and non-existant headers

From: Mark Nottingham <[email-masked]>
Date: Tue, 6 Oct 2009 15:53:23 +1100

Jamie (et al),

Have you had a chance to have a look at the changes in draft 07 of p6?

Specifically,
   http://trac.tools.ietf.org/wg/httpbis/trac/changeset/607
includes the following text:

> The selecting request-headers from two requests are defined to match  
> if and only if the selecting request-headers in the first request  
> can be transformed to the selecting request-headers in the second  
> request by adding or removing linear white space at places where  
> this is allowed by the corresponding ABNF, and/or combining multiple  
> message-header fields with the same field name following the rules  
> about message headers in Section 4.2 of [Part1].
>
> If a header field is absent from a request, it can only match  
> another request if it is also absent there.

Especially noting the last sentence.

Does this address the issue?

Cheers,

On 25/06/2009, at 4:04 PM, Mark Nottingham wrote:

> I haven't seen any comment on this, so my inclination is to  
> incorporate into -07 and see how it stands in context. I'll leave  
> the issue open for now, and close once people have a chance to see  
> that.
>
>
> On 04/06/2009, at 5:06 PM, Mark Nottingham wrote:
>
>> I think the right way to fix this is to move the language about  
>> validation into section 2.4 (validation model) and carefully  
>> updating that and section 2.7 (combining responses) to indicate how  
>> validation affects the response, whether or not there are selecting  
>> headers. This requires some non-trivial changes to the text, but  
>> we've already done that, and I think the result is worth it...
>>
>> Note that 2.4 already says:
>>> When a stored response includes one or more validators, such as  
>>> the field values of an ETag or Last-Modified header field, then a  
>>> validating request SHOULD be made conditional to those field values.
>>
>> Proposal for updated section 2.4:
>>
>> """
>> 2.4. Validation Model
>> Checking with the origin server to see if a stale or otherwise  
>> unusable cached response can be reused is called "validating" or  
>> "revalidating." Doing so potentially avoids the overhead of  
>> retransmitting the response body when a stored response is valid.
>> When a cache has one or more stored responses for a URI, but cannot  
>> serve any of them (see section 2.2), it can use the conditional  
>> request mechanism (part 4) in the forwarded request to select a  
>> valid stored response to be used, and update it.
>>
>> When sending such a conditional request, the cache SHOULD add an If- 
>> Modified-Since header whose value is that of the Last-Modified  
>> header from the selected (see section 2.6) stored response, if  
>> present.
>>
>> Additionally, the cache SHOULD add an If-None-Match header whose  
>> value is that of the ETag header(s) from all responses stored for  
>> that URI (see section 2.6), if present. However, if any of the  
>> stored responses contains only partial content, its entity-tag  
>> SHOULD NOT be included in the If-None-Match header field unless the  
>> request is for a range that would be fully satisfied by that stored  
>> response.
>>
>> A 304 (Not Modified) response status code indicates that a stored  
>> response should be updated and used to satisfy the request; see  
>> Section 2.7.
>>
>> A full response (i.e., one with a response body) indicates that  
>> none of the stored responses nominated in the conditional request  
>> is suitable. Instead, the full response is used both to satisfy the  
>> request and replace the stored response. [[anchor8: Should there be  
>> a requirement here?]]
>> If a cache receives a 5xx response while attempting to validate a  
>> response, it MAY either forward this response to the requesting  
>> client, or act as if the server failed to respond. In the latter  
>> case, it MAY return a stale stored response if allowed (see Section  
>> 2.3.3).
>> If a cache receives a successful response whose Content-Location  
>> field matches that of an existing stored response for the same  
>> Request-URI, whose entity-tag differs from that of the existing  
>> stored response, and whose Date is more recent than that of the  
>> existing response, the existing response SHOULD NOT be returned in  
>> response to future requests and SHOULD be deleted from the cache. 
>> [[anchor13: DISCUSS: Not sure if this is necessary.]]
>> """
>>
>> Proposal for updated section 2.6:
>>
>> """
>> 2.6.  Caching Negotiated Responses
>>
>> When a cache receives a request that can be satisfied by a stored  
>> response with a Vary header field (Section 3.5), it MUST NOT use  
>> that response unless all of the selecting request-headers nominated  
>> by the Vary header match in both the original request (i.e., that  
>> associated with the stored response), and the presented request.
>>
>> The selecting request-headers from two requests are defined to  
>> match if and only if the selecting request-headers in the first  
>> request can be transformed to the selecting request-headers in the  
>> second request by adding or removing linear white space [[anchor11:  
>> [ref]]] at places where this is allowed by the corresponding ABNF,  
>> and/or combining multiple message-header fields with the same field  
>> name following the rules about message headers in Section 4.2 of  
>> [Part1].
>>
>> If a header field is absent from a a request, it can only match  
>> another request if it is also absent there.
>>
>> A Vary header field-value of "*" always fails to match, and  
>> subsequent requests to that resource can only be properly  
>> interpreted by the origin server.
>>
>> If no stored response matches, the cache MAY forward the presented  
>> request to the origin server as a conditional request; see Section  
>> 2.4.
>> """
>>
>>
>> Proposal for updated section 2.7 (there's some duplication with p5- 
>> range section 4 that we need to take care of, btw):
>>
>> """
>> 2.7.  Combining Responses
>>
>> When a cache receives a 304 (Not Modified) response or a 206  
>> (Partial Content) response (collectively, the "new" response), it  
>> needs to update the stored response with the new one, so that the  
>> updated response can be used to satisfy the request.
>>
>> If the new response contains an ETag, it identifies the stored  
>> response to use.  [[ note: may need language about Content-Location  
>> here ]]
>>
>> If the status code is 206 (partial content), both the stored and  
>> new responses MUST have ETags, and those ETags MUST match using the  
>> strong comparison function (see Section 4 of [Part4]). Otherwise,  
>> the responses MUST NOT be combined.
>>
>> The stored response headers are used as those of the updated  
>> response, except that
>>
>>  o Any stored Warning headers with warn-code 1xx (see Section 3.6)  
>> MUST be deleted from the stored response and the updated response.
>>
>>  o Any stored Warning headers with warn-code 2xx MUST be retained  
>> in the stored response and the updated response.
>>
>>  o Any headers provided in the new response MUST replace the  
>> corresponding headers from the stored response.
>>
>> If a header field-name in the new response matches more than one  
>> header in the stored response, all such old headers MUST be replaced.
>>
>> The updated response can [[requirement?]] be used to replace the  
>> stored response in cache. In the case of a 206 response, the  
>> combined entity-body MAY be stored.
>>
>>  [[anchor14: ISSUE: discuss how to handle HEAD updates]]
>> """
>>
>>
>>
>> On 08/05/2009, at 7:29 AM, Julian Reschke wrote:
>>
>>> Roy T. Fielding wrote:
>>>> On May 7, 2009, at 12:43 PM, Julian Reschke wrote:
>>>>> Brian Smith wrote:
>>>>>> Julian Reschke wrote:
>>>>>>> "When a cache receives a request that can be satisfied by a  
>>>>>>> stored
>>>>>>> response that includes a Vary header field (Section 3.5), it  
>>>>>>> MUST NOT
>>>>>>> use that response unless all of the selecting request-headers  
>>>>>>> nominated
>>>>>>> by the *stored* Vary header match in both the original request
>>>>>>> associated with the stored response, and the presented request."
>>>>>> This is wrong, but not for the reason Jamie stated.
>>>>>> Consider this:
>>>>>> 1. Client requests "GET /foo"
>>>>>> 2. Cache forwards the request to the server "GET /foo" with
>>>>>> an If-None-Match: "A", "B", "C"
>>>>>> 3. Server returns "304 Not Modified" with ETag "A".
>>>>>> According to the statement above, the cache could only return  
>>>>>> the cached
>>>>>> response with ETag "A" if the Vary'd headers match; otherwise,  
>>>>>> it couldn't
>>>>>> return any response at all. However, the cache must be able to  
>>>>>> return the
>>>>>> response with ETag "A" regardless of whether the Vary'd headers  
>>>>>> match.
>>>>>
>>>>> Why?
>>>>>
>>>>> What you describe sounds like a bug. If the selecting headers  
>>>>> that resulted in the response with entity tag "A" had a "Accept- 
>>>>> Encoding: gzip", then the response can't be used for a request  
>>>>> without "Accept-Encoding", even if it may be fresh.
>>>> Because the origin server just told the cache to deliver "A" in  
>>>> (3).
>>>> The origin server is authoritative.
>>>> We've had this discussion before.
>>>
>>> Right, sorry.
>>>
>>> So the text is only needs to consider the case where the origin  
>>> server gives its ok.
>>>
>>> ""When a cache receives a request that can be satisfied by a stored
>>> response that includes a Vary header field (Section 3.5), it MUST  
>>> NOT
>>> use that response unless all of the selecting request-headers  
>>> nominated
>>> by the *stored* Vary header match in both the original request  
>>> associated with the stored response, and the presented request,  
>>> *or after validating with the origin server*".
>>>
>>> ?
>>>
>>> BR, Julian
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Mark Nottingham     http://www.mnot.net/
>>
>>
>
>
> --
> Mark Nottingham     http://www.mnot.net/
>
>

--
Mark Nottingham     http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 14 Oct 2009 20:33:33 +1100

Presumed OK, closing. If there are concerns, please raise them on-list.

On 06/10/2009, at 3:53 PM, Mark Nottingham wrote:

>> second request by adding or removing linear white space at places  
>> where this is allowed by the corresponding ABNF, and/or combining  
>> multiple message-header fields with the same field name following  
>> the rules about message headers in Section 4.2 of [Part1].
>>
>>> updating that and section 2.7 (combining responses) to indicate  
>>> how validation affects the response, whether or not there are  
>>> selecting headers. This requires some non-trivial changes to the  
>>> text, but we've already done that, and I think the result is worth  
>>> it...
>>>
>>> When a cache has one or more stored responses for a URI, but  
>>> cannot serve any of them (see section 2.2), it can use the  
>>> conditional request mechanism (part 4) in the forwarded request to  
>>> select a valid stored response to be used, and update it.
>>>
>>> When sending such a conditional request, the cache SHOULD add an  
>>> If-Modified-Since header whose value is that of the Last-Modified  
>>> SHOULD NOT be included in the If-None-Match header field unless  
>>> the request is for a range that would be fully satisfied by that  
>>> stored response.
>>>
>>> is suitable. Instead, the full response is used both to satisfy  
>>> the request and replace the stored response. [[anchor8: Should  
>>> there be a requirement here?]]
>>> case, it MAY return a stale stored response if allowed (see  
>>> Section 2.3.3).
>>> that response unless all of the selecting request-headers  
>>> nominated by the Vary header match in both the original request  
>>> (i.e., that associated with the stored response), and the  
>>> presented request.
>>>
>>> second request by adding or removing linear white space  
>>> [[anchor11: [ref]]] at places where this is allowed by the  
>>> corresponding ABNF, and/or combining multiple message-header  
>>> fields with the same field name following the rules about message  
>>> headers in Section 4.2 of [Part1].
>>>
>>> response to use.  [[ note: may need language about Content- 
>>> Location here ]]
>>>
>>> new responses MUST have ETags, and those ETags MUST match using  
>>> the strong comparison function (see Section 4 of [Part4]).  
>>> Otherwise, the responses MUST NOT be combined.
>>>
>>>>>>> response with ETag "A" regardless of whether the Vary'd  
>>>>>>> headers match.
>>>>>>
>>>>>> Why?
>>>>>>