Subject: [http-state] Netscape Cookie spec allowed chars

From: =JeffH <[email-masked]>
Date: Wed, 23 Feb 2011 15:03:46 -0800

Roy said..
 >
 > Therefore, I would like you to change the ABNF so that it
 > reflects the reality of (Set-)Cookie usage on the Internet,
 > ...
 >  Changing it to
 >
 >  cookie-value      = %x21-2B / %x2D-3A / %x3C-7E / %x80-FF
 >
 > or just the minimum
 >
 >  cookie-value      = %x21-2B / %x2D-3A / %x3C-7E
 >
 > returns the definition to the original Netscape spec (at
 > least in the first case)

Just so we all know for sure what the paraphrased (above) primary source 
actually says, here's the as-is spec being cited and a brief analysis of what 
it means...

[NSCP-COOKIE] "PERSISTENT CLIENT STATE HTTP COOKIES
Preliminary Specification - Use with caution"...
<http://web.archive.org/web/20020803110822/http://wp.netscape.com/newsref/std/cookie_spec.html> 

###
   Syntax of the Set-Cookie HTTP Response Header

     This is the format a CGI script would use to add to the HTTP
     headers a new piece of data which is to be stored by the client
     for later retrieval.

     Set-Cookie: NAME=VALUE; expires=DATE;
     path=PATH; domain=DOMAIN_NAME; secure

     NAME=VALUE
           This string is a sequence of characters excluding semi-colon,
           comma and white space. If there is a need to place such data
           in the name or value, some encoding method such as URL style
           %XX encoding is recommended, though no encoding is defined or
           required.

           This is the only required attribute on the Set-Cookie header.
###

Operative statement from the above quote..

"..characters excluding semi-colon, comma and white space."

(note also that the text above is only explicitly referring to the "NAME=VALUE" 
production; it apparently has been commonly interpreted as referring to the 
entire value string of the "Set-Cookie" HTTP response header; note also that it 
apparently assumes "characters" means "octets", and when it says "whitespace" 
it is meaning the set of chars from the ASCII charset typically referred to as 
that (although that isn't necessarily a rigidly defined set [WS]))

As noted in the table below, the above operative statement has some 
ambiguities. For example, it doesn't explicitly include the high-bit-set chars 
%x80-FF, though apparently many have assumed it does.

----------------------------------------
Original Netscape Cookie spec
Allowed Characters ( [ASCII] (assumed) )
----------------------------------------
Decimal	    Hex       Symbol(s)
----------------------------------------

; %x00-20 -- non-printing control chars (excluded ?)
;            and whitespace ( BS, HT, LF, VT, FF, CR, SP
;                             all excluded ? correct list of
;                             "whitespace" chars as assumed by
;                             [NSCP-COOKIE] authors ? )

;  %x21-2B:

33-43       21-2B       ! " # $ % & ' ( ) * +

; 44          2C          , (excluded)

;  %x2D-3A:

45-47       2D-2F       - . /

48-57       30-39       0 - 9

58          3A          :

; 59          3B          ; (excluded)

;  %x3C-7E:

60-64       3C-40       < = > ? @

65-90       41-5A       A-Z

91-96       5B-60       [ \ ] ^ _ `

97-122      61-7A       a-z

123-126     7B-7E       { | } ~

; 127         7F        DEL  (excluded ?)

;  %x80-FF:

128-256     80-FF       high-bit-set chars  (included ?)

See also:

[ASCII]  ASCII (American Standard Code for Information Interchange) Code
          http://www.december.com/html/spec/asciiall.html

[WS]     Whitespace character
          https://secure.wikimedia.org/wikipedia/en/wiki/Whitespace_character

---
end

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 23 Feb 2011 19:31:12 -0800

On Feb 23, 2011, at 3:03 PM, =JeffH wrote:

> Just so we all know for sure what the paraphrased (above) primary source actually says, here's the as-is spec being cited and a brief analysis of what it means...
> 
> 
> (note also that the text above is only explicitly referring to the "NAME=VALUE" production; it apparently has been commonly interpreted as referring to the entire value string of the "Set-Cookie" HTTP response header; note also that it apparently assumes "characters" means "octets", and when it says "whitespace" it is meaning the set of chars from the ASCII charset typically referred to as that (although that isn't necessarily a rigidly defined set [WS]))
> 
> 
> As noted in the table below, the above operative statement has some ambiguities. For example, it doesn't explicitly include the high-bit-set chars %x80-FF, though apparently many have assumed it does.

Right, but it also defines it as an HTTP Response Header,
and (at the time Lou wrote that spec) all such header fields
forbid CTLs (other than WS) and allowed "Recipients of header
field TEXT containing octets outside the US-ASCII character set
may assume that they represent ISO-8859-1 characters."

Hence, it does say exactly the production that I suggested,
if you know the context, and well-written implementations
generally do exclude %x00-20, ";", ",", and %x7E when
processing what looks to be a Netscape-style cookie.

....Roy