Subject: Re: allowable characters in token as used in parameter ABNF

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Mar 2010 19:25:35 +0100

Hi,

adding apps-discuss to get more review... -- comments in-line.

On 09.02.2010 14:34, Julian Reschke wrote:
> Mark Nottingham wrote:
>> Speaking personally --
>>
>> I'm torn here. On one hand, I'd very much like to see best practice
>> promoted here, because the wild-west situation of HTTP header parsing
>> is one of the things I really dislike, and suspect causes a lot of
>> problems.
>>
>> OTOH, we don't have any implementers stepping up and saying that
>> they're eager, and in this situation it may be too easy to specify the
>> wrong thing.
>>
>> AIUI the most liberal form of xtoken would be 1*VCHAR without DQUOTE,
>> "," or ";". Correct?
>
> That will make it:
>
> "!" / "#" / "$" / "%" / "&" / "'" / "(" / ")" / "*" / "+" / "-" / "." /
> "/" / DIGIT / ":" / "<" / "=" / ">" / "?" / "@" / ALPHA / "[" / "\" /
> "]" / "^" / "_" / "`" / "{" / "|" / "}" / "~"
>
> which includes the following non-token characters:
>
> "(" / ")" / "/" / ":" / "<" / "=" / ">" / "?" / "@" / "[" / "\" / "]" /
> "{" / "}"
>
> "(" and ")" *might* become a problem in headers that allow comments.
>
> "\" might be confused with the escape character in quoted strings and
> comments.
>
> Other than that, I don't see a problem.

In the meantime, Mark has added this as "ptoken" to the ABNF for the 
HTTP Link Header, see 
<http://greenbytes.de/tech/webdav/draft-nottingham-http-link-header-08.html#rfc.section.5>:

   ptoken            = "!" | "#" | "$" | "%" | "&" | "'" | "("
                     | ")" | "*" | "+" | "-" | "." | "/" | DIGIT
                     | ":" | "<" | "=" | ">" | "?" | "@" | ALPHA
                     | "[" | "\" | "]" | "^" | "_" | "`" | "{"
                     | "|" | "}" | "~"

Are people ok with relaxing this, compared to RFC 2616 token characters?

Assuming this is ok for parameters in the Link header, should I add a 
similar construct to draft-reschke-rfc2231-in-http, to promote re-use in 
new header definitions?

> ...

Best regards, Julian