Subject: #147: header-specific canonicalisation

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Mar 2010 16:59:59 +1100

This has been discussed on-list and in meetings sporadically, but I don't think we've ever focused on it exclusively.

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/147>

Current text at <http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-08#section-2.6>:

>    The selecting request-headers from two requests are defined to match
>    if and only if the selecting request-headers in the first request can
>    be transformed to the selecting request-headers in the second request
>    by adding or removing linear white space [[anchor8: [
> ref
> ]]] at places
>    where this is allowed by the corresponding ABNF, and/or combining
>    multiple message-header fields with the same field name following the
>    rules about header fields in 
> Section 3.2 of [Part1].

Proposed replacement:

  The selecting request-headers from two requests are defined to match
  if and only if those in the first request can be transformed to those in the
  second request by applying any of the following:
    - adding or removing whitespace, where allowed in the headers' ABNF
    - combining multiple message-header fields with the same field name (see
       Section 3.2 of [Part1])
    - canonicalising both headers' values in a way that is known to have identical
       semantics, according to the headers' specification (e.g., re-ordering field values
      when order is not significant; case-normalisation, where values are defined to be
      case-insensitive)

--
Mark Nottingham     http://www.mnot.net/

---

From: Yves Lafon <[email-masked]>
Date: Wed, 3 Mar 2010 04:49:12 -0500 (EST)

>      when order is not significant; case-normalisation, where values are defined to be

Friendly amendment: s/Canonicalising/Normalizing/

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves

---

From: Julian Reschke <[email-masked]>
Date: Wed, 03 Mar 2010 14:05:52 +0100

On 03.03.2010 10:49, Yves Lafon wrote:
>> This has been discussed on-list and in meetings sporadically, but I
>> don't think we've ever focused on it exclusively.
>>
>> Current text at
>> <http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-08#section-2.6>:
>>

This *almost* works for me:

Section 2.6 would become:

-- snip --
    When a cache receives a request that can be satisfied by a stored
    response that has a Vary header field (Section 3.5), it MUST NOT use
    that response unless all of the selecting request-headers nominated
    by the Vary header match in both the original request (i.e., that
    associated with the stored response), and the presented request.

    The selecting request-headers from two requests are defined to match
    if and only if those in the first request can be transformed to those
    in the second request by applying any of the following:

    o  adding or removing whitespace, where allowed in the header's
       syntax

    o  combining multiple message-header fields with the same field name
       (see Section 3.2 of [Part1])

    o  normalizing both header values in a way that is known to have
       identical semantics, according to the header's specification
       (e.g., re-ordering field values when order is not significant;
       case-normalization, where values are defined to be case-
       insensitive)

    If a header field is absent from a request, it can only match another
    request if it is also absent there.

    A Vary header field-value of "*" always fails to match, and
    subsequent requests to that resource can only be properly interpreted
    by the origin server.

    The stored response with matching selecting request-headers is known
    as the selected response.

    If no selected response is available, the cache MAY forward the
    presented request to the origin server in a conditional request; see
    Section 2.4.
-- snip --

Note:

    If a header field is absent from a request, it can only match another
    request if it is also absent there.

Is that still true? I don't think; maybe we can remove the sentence 
completely?

I would also add this to the Changes-from-2616 section (A.2) as:

-- snip --
    Clarify that matching request headers allows header-specific
    normalization.  (Section 2.6)
-- snip --

(proposed draft patch attached to the ticket, see 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/147/i147.diff>)

Best regards, Julian

---

From: Yves Lafon <[email-masked]>
Date: Wed, 3 Mar 2010 15:52:13 -0500 (EST)

On Wed, 3 Mar 2010, Julian Reschke wrote:

>>> in the

I don't remember where we left this one last time we discussed this, and I 
am wondering what caches are doing in this specific case.

         ~~Yves

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 4 Mar 2010 13:57:47 +1100

On 04/03/2010, at 12:05 AM, Julian Reschke wrote:
> Is that still true? I don't think; maybe we can remove the sentence completely?

That means we'd have to re-open <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/37>. 

I think we can address your concern by doing something like

If (after any normalisation that may take place) a header field is absent from a request...

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 4 Mar 2010 00:17:14 -0800

> Is that still true? I don't think; maybe we can remove the sentence completely?

Yes, it is still true.

   Accept:

means accept nothing (give me a 300 instead).  No Accept header means
accept everything.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 04 Mar 2010 09:49:25 +0100

> means accept nothing (give me a 300 instead).  No Accept header means

I do agree for "Accept".

But a header *could* have a normalization behaviour, where the presence 
of an empty header and the absence of the header field have the same 
semantics. (Not a good idea, probably, but that doesn't matter here).

In that case, a header-field specific normalization should allow to 
treat them as matching, no?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 04 Mar 2010 16:48:01 +0100

On 04.03.2010 03:57, Mark Nottingham wrote:
> That means we'd have to re-open<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/37>.
>

Works for me; I have updated 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/147/i147.diff> 
accordingly.

Best regards, Julian

---

From: Brian Smith <[email-masked]>
Date: Thu, 04 Mar 2010 10:51:03 -0600

>        when order is not significant; case-normalisation, where values are defined to be
It would be good to provide recommended canonical forms for 
headers--recommended case, recommended order, recommended whitespace, 
recommended (non-)quoting/escaping, etc. Otherwise, it opens the door 
for non-interoperability in implementors understanding of "identical 
semantics." Plus, a recommended canonical form would allow 
implementations to work better even with caches that don't do any 
header-specific canonicalization. If the working group isn't able to 
agree on canonical forms for headers, then it probably wouldn't be 
possible for everybody to agree on whether two different forms have 
exactly identical semantics either.

Header canonicalization also has benefits for compression of the header 
too. That's (at least one reason) why SPDY requires case normalization.

- Brian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 4 Mar 2010 12:13:10 -0800

On Mar 4, 2010, at 12:49 AM, Julian Reschke wrote:

> But a header *could* have a normalization behaviour, where the presence of an empty header and the absence of the header field have the same semantics. (Not a good idea, probably, but that doesn't matter here).
> 
> In that case, a header-field specific normalization should allow to treat them as matching, no?

No, there are no request header fields with that characteristic
that would appear in Vary, and we are better off letting caches
be ignorant of the meaning of a field while running these tests.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Fri, 05 Mar 2010 11:10:23 +0100

On 04.03.2010 16:48, Julian Reschke wrote:
> ...
> ...

Now applied as <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/771>.

Best regards, Julian