Subject: #231: Considerations for new headers

From: Mark Nottingham <[email-masked]>
Date: Wed, 29 Jun 2011 20:17:56 +1000

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/231>

Proposal:

Move p2 sections 3 and 5 (request and response header fields, respectively, into a new section):

------8<------
3. Header Fields

HTTP header fields are key value pairs that can be used to communicate data about the message, its payload, the target resource, or about the connection itself (i.e., control data).  See [ref to part 1] for a general definition of their syntax.
  
3.1 Considerations for Creating Header Fields

New header fields are registered using the procedures in [RFC3864]. 

Requirements for header field names are defined in [RFC3864], section 4.1. Authors of new fields are advised to keep the name as short as practical, and not to prefix them with "X-" if they are to be registered (either immediately or in the future). Under no conditions can the "Close" field-name be registered, because it is reserved for use as a connection token.

New HTTP header field values typically have their syntax defined using ABNF [ref] (using the extensions defined in [ref to p1] as necessary), and are usually constrained to the range of ASCII characters. Headers needing a greater range of characters can use an encoding such as that defined in [RFC5987]. 

Because commas (",") are used as a generic delimiter between field-values, they need to be treated with care if they are allowed in the field-value's payload. Typically, components that might contain a comma are protected with double-quotes <">. 

For example, a textual date and a URI (either of which might contain a comma) could be safely carried in a field-value like this:

  Example-Field: uri="http://example.com/a.html"; date="Sat, 4 May 1996"

Authors of new header fields are advised to consider documenting:

  * Whether the field is a single value, or whether it can be a list (delimited by commas; see [ref to p1]).
  * Under what conditions the header field can be used; e.g., only in responses or requests, in all messages, only on responses to a particular request method.
  * Whether it's appropriate to list the field-name in the Connection header (i.e., if the header is to be hop-by-hop [ref to p1]).
  * Under what conditions intermediaries are allowed to modify the header field's value, insert or delete it. 
  * Whether the header field is useful or allowable in trailers [ref to p1].
  * Whether the header field should be preserved across redirects.

3.2 Request Header Fields

[ as in the current section 3 ]

3.3 Response Header Fields

[ as in the current section 4 ]

------>8------

--
Mark Nottingham   http://www.mnot.net/

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 30 Jun 2011 00:09:16 +1200

On 29/06/11 22:17, Mark Nottingham wrote:
> Because commas (",") are used as a generic delimiter between field-values, they need to be treated with care if they are allowed in the field-value's payload. Typically, components that might contain a comma are protected with double-quotes<">.
>

Nit:
  should that example not have a comma ',' at the end of the url 
quoted-string?

> [ as in the current section 4 ]

AYJ

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Jun 2011 09:30:00 +0200

On 2011-06-29 14:09, Amos Jeffries wrote:
> ...
>> For example, a textual date and a URI (either of which might contain a
>> comma) could be safely carried in a field-value like this:
>>
> ...

No, the delimiter in this format is indeed ";". (see, for instance, 
"Link", "Content-Type", "Content-Disposition"...)

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 30 Jun 2011 07:37:27 +0000

In message <[email-masked]>, Mark Nottingham wri

>Authors of new header fields are advised to consider documenting:

* How the header field impacts caching.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Jun 2011 09:38:14 +0200

> Move p2 sections 3 and 5 (request and response header fields, respectively, into a new section):

Sounds good overall.

On second though, I can see why Amos was confused by ";" and ",". We 
should clarify the roles of "," and ";", where the latter is frequently 
used in header fields that use the "parameter" ABNF 
(<http://tools.ietf.org/html/rfc2616#section-3.6>).

Indeed, we should encourage authors of new header fields to re-use 
existing ABNF constructs, so that recipients can re-use parser components.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 30 Jun 2011 17:40:17 +1000

New headers can't impact caching on their own; if you mean that authors should document how their header might *interact* with caching, certainly.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Jun 2011 09:40:53 +0200

On 2011-06-30 09:37, Poul-Henning Kamp wrote:
> In message<[email-masked]>, Mark Nottingham wri
> tes:

Can a new header field affect caching?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 30 Jun 2011 17:42:14 +1000

On 30/06/2011, at 5:38 PM, Julian Reschke wrote:
> 
> On second though, I can see why Amos was confused by ";" and ",". We should clarify the roles of "," and ";", where the latter is frequently used in header fields that use the "parameter" ABNF (<http://tools.ietf.org/html/rfc2616#section-3.6>).

Yes, I'm still figuring out how to best illustrate the point I'm trying to make; revised proposal soon.

> Indeed, we should encourage authors of new header fields to re-use existing ABNF constructs, so that recipients can re-use parser components.

Yep; that's already an issue <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/266>.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 30 Jun 2011 07:52:38 +0000

>Can a new header field affect caching?

If not, we should say so.

...not that I think that will have any effect.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 30 Jun 2011 10:48:30 +0200

> ...not that I think that will have any effect.

I agree with PHK here. For instance if some sensitive headers are added
that must not be cached, we should probably add provisions for being able
to do so (just as we have "Cache-control: no-cache=set-cookie").

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 30 Jun 2011 10:58:33 +0200

> Yep; that's already an issue <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/266>.

It's not only a matter of re-using existing parser components, but also to
allow participants to adopt consistent behaviour. For instance, the comma
can be used to fold a list of similar headers which are defined as a list.
New headers might be added that will be defined as lists but unknown from
existing components, which might fold them for consistent processing. This
is problematic with some headers that adopt a date format where the comma
is not a list delimitor.

I'm already facing this issue in haproxy. ACLs make it possible to filter
on a header contents. It can iterate over all of a header's values, whether
they're delimited by commas or all on their own line. A date header will be
seen as two occurrences of partial dates due to the comma.

Also I remember that Apache's mod_proxy folds any multiple headers into a
single list, thus if you send two "Date" headers with both parts, it will
rebuild a properly formatted "Date" header.

I suspect that there are not *that* many different behaviours right now
for existing headers. Maybe we could propose a universal syntax that is
recommended to respect for future headers, and document known exceptions ?
After all, if only 3-4 exceptions are identified, it might be worth hard-
coding them in many products.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jul 2011 13:54:10 +1000

> ...not that I think that will have any effect.

I really don't want to start walking down the route of specifying what can't affect caching; we'll quickly get into phases of the moon, and the people who would benefit most from that won't be reading the spec anyway. 

p6 lays out how HTTP caching works according to a set of rules; if someone wants to override those rules, that's great, but they can't expect people who have never heard of their extension to know about it. Thankfully, reality will teach them pretty quickly.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jul 2011 15:05:27 +1000

Revised proposal:

------8<------
3. Header Fields

For example, a textual date and a URI (either of which might contain a comma) could be safely carried in field-values like these:

  Example-URI-Field: "http://example.com/a.html,foo", "http://without-a-comma.example.com/"
  Example-Date-Field: "Sat, 04 May 1996", "Wed, 14 Sep 2005"

 * How the header might interact with caching [ref to p6].

------>8------