Subject: Re: suggestions for examples and explication wrt ABNF and header fields in draft-ietf-httpbis-p1

From: Julian Reschke <[email-masked]>
Date: Fri, 08 Jan 2010 16:36:16 +0100

=JeffH wrote:
> I have some further suggestions wrt draft-ietf-httpbis-p1 section 
> "1.2.2. Basic Rules".
> 
> HTH,
> 
> =JeffH
> ------
> 
> In Section 1.2.2 Basic Rules...
> 
>  >    Many HTTP/1.1 header field values consist of words separated by
>                                                   ^^^^^
>                                                   tokens

I don't think "token" would be correct here.

>  >    whitespace or special characters.  These special characters MUST be
>  >    in a quoted string to be used within a parameter value (as defined in
>  >    Section 6.2).
> 
> The "special characters" aren't mentioned anywhere in 
> draft-ietf-httpbis-p* spec set other than the above, nor are they 
> defined in ABNF. Inspection of draft-ietf-httpbis-p1 and RFC2616 reveals 

Yes. This happened when we rewrote the ABNF not to use "prose" rules 
anymore. Unfortunately ABNF doesn't have syntax for something like "this 
set of characters, except for that other set".

> that the "special characters" are what RFC2616 defined (using ABNF) as 
> "separators". This is additionally confused in that the below tchar rule 
> directly follows the above paragraph, and it takes a few moments to 
> figure out that the below _are not_ the "special characters" (and it 
> takes even more time to determine that the "special characters" aren't 
> explicitly defined in draft-ietf-httpbis-p*).
> 
> 
>  >      tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
>  >                     / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
>  >                     / DIGIT / ALPHA
>  >
>  >      token          = 1*tchar
> 
> 
> I suggest rewriting the above paragraph as below, also explicitly 
> including the "separators" ABNF from RFC2616 (tho remove the SP and HT 
> from it), and list the "token" rule before the "tchar" rule, such that 
> it reads...
> 
> 
>    Many HTTP/1.1 header field values consist of tokens separated by
>    whitespace (OWS or RWS as appropriate) or separators.  These
>    separators, as well as whitespace, MUST be in a quoted string
>    to be used within a parameter value (as defined in Section 6.2).
> 
> 
>      separators     = "(" | ")" | "<" | ">" | "@"
>                     | "," | ";" | ":" | "\" | <">
>                     | "/" | "[" | "]" | "?" | "="
>                     | "{" | "}"
> 
> 
>      token          = 1*tchar
> 
> 
>      tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
>                     / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
>                     / DIGIT / ALPHA
> 

I have looked at this and decided to leave the prose alone for now. I 
changed the ABNF to:

     token          = 1*tchar

     tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
                    / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
                    / DIGIT / ALPHA
                    ; any VCHAR, except special

     special        = "(" / ")" / "<" / ">" / "@" / ","
                    / ";" / ":" / "\" / DQUOTE / "/" / "["
                    / "]" / "?" / "=" / "{" / "}"

(see <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/744>).

This introduces an unused ABNF production, but I think that's ok for 
clarity.

Feedback appreciated,

Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Jan 2010 18:36:58 +0100

=JeffH wrote:
> ...
> WRT "3.2    Header Fields"..
> 
>  > 3.2.  Header Fields
>  >
>  >    Each HTTP header field consists of a case-insensitive field name
>  >    followed by a colon (":"), optional whitespace, and the field value.
>  >
>  >      header-field   = field-name ":" OWS [ field-value ] OWS
>  >      field-name     = token
>  >      field-value    = *( field-content / OWS )
>  >      field-content  = *( WSP / VCHAR / obs-text )
>  >
>  >    No whitespace is allowed between the header field name and colon.
>  >    For security reasons, any request message received containing such
>  >    whitespace MUST be rejected with a response code of 400 (Bad
>  >    Request).  A proxy MUST remove any such whitespace from a response
>  >    message before forwarding the message downstream.
>  >
>  >    A field value MAY be preceded by optional whitespace (OWS); a single
>  >    SP is preferred.  The field value does not include any leading or
>  >    trailing white space: OWS occurring before the first non-whitespace
>  >    character of the field value or after the last non-whitespace
>  >    character of the field value is ignored and SHOULD be removed without
>  >    changing the meaning of the header field.
> 
> 
> I suggest adding, right here after the above paragraph, some example 
> header fields. At least one should feature comma-separated field-values. 
> e.g.
> 
> 
>      example-header1: foo
> 
>      example-header2:foo
> 
>      example-header3: foo=bar,barfoo;attrib,charlie
> 
>      etc.
> ...

I agree that we currently have too few examples. On the other hand, I'd 
prefer only to add examples if it's clear that they're useful.

I don't think there's any confusion about how header fields look in 
practice, so these totally generic examples don't look too useful to me.

As you mentioned comma-separated values... Did you think about the case 
of list productions, and the way headers using them can be recombined? 
That would probably be a good example to have. Such as:

  example: foo
  example: bar

is equivalent to:

  example: foo, bar

Feedback appreciated, Julian