Subject: new issue: mismatch between RFC2616 and RFC4234 CHAR definition

From: Julian Reschke <[email-masked]>
Date: Thu, 24 Jan 2008 14:54:56 +0100

Hi,

I just realized that the "CHAR" core rules in RFC2616 and RFC4234 
differ, in that RFC2616 includes NUL:

     CHAR           = <any US-ASCII character (octets 0 - 127)>

(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.2.2>)

CHAR is only used in two places:

     token          = 1*<any CHAR except CTLs or separators>

and

     quoted-pair    = "\" CHAR

(same section).

Are these productions really intended to allow NUL?

BR, Julian

---

From: [email-masked]
Date: Thu, 24 Jan 2008 15:06:12 +0100

> Are these productions really intended to allow NUL?
The "token" production explicitly disallows NUL ("except CTLs").

As for quoted-string: quoting is meant to enable the usage 
of *any* CHAR within the quotes or am I wrong?

Regards

Ingo Struck

---

From: [email-masked]
Date: Thu, 24 Jan 2008 15:11:00 +0100

> of *any* CHAR within the quotes or am I wrong?
Sorry, I mixed up quoted-pair with quoted-string,
but that does not invalidate the statement:

the quoted-pair-construct should be able to
quote any character, including NUL.

Regards

Ingo Struck

---

From: Julian Reschke <[email-masked]>
Date: Thu, 24 Jan 2008 15:14:24 +0100

> The "token" production explicitly disallows NUL ("except CTLs").

Indeed. Thanks.

> of *any* CHAR within the quotes or am I wrong?

Not convinced. If the intent was to allow NUL in quoted strings (after 
unquoting), why isn't it allowed in qdtext in the first place?

     quoted-string  = ( <"> *(qdtext | quoted-pair ) <"> )
     qdtext         = <any TEXT except <">>
     TEXT           = <any OCTET except CTLs,
                      but including LWS>

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 24 Jan 2008 16:05:39 +0100

> quote any character, including NUL.

FWIW, in mail (RFC 2822) it used to be NO-WS-CTL,
WSP (HTAB or SP), and VCHAR, excluding NUL, CR,
and LF from ordinary <quoted-pair>s.  

The "missing" cases were collected in <obs-qp>
(MUST accept, MUST NOT generate).  In the last
2822upd draft NO-WS-CTL was also moved to "obs".

While RFC 2822 isn't directly relevant for HTTP
Alexey's attempt to update Digest-MD5 for SASL
ran into serious difficulties wrt <quoted-pair>s
in <qstr-val> - in essence the same construct as
in RFC 2617 and 2616, a <quoted-string> without
its quotes:

HTTP has to be clear what it is for Auth Digest.

 Frank

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 24 Jan 2008 12:15:54 -0800

> Are these productions really intended to allow NUL?

No, and no one sends it, so just note the difference and use RFC4234.

....Roy