Subject: rfc2617 BNF quoting issues

From: Jim Luther <[email-masked]>
Date: Mon, 21 Apr 2003 11:46:19 -0700

rfc2617 defines auth-param as:

       auth-param	= token "=" ( token | quoted-string )

and defines challenge and credentials as:

	challenge		= auth-scheme 1*SP 1#auth-param
	credentials	= auth-scheme #auth-param

However, other parts of rfc2617 don't make a couple of things clear:

1 - Can quoted-strings be used where they are not required for the 
value of an auth-param and the BNF for the value of the auth-param 
doesn't mention quoting?

An example of this is the nonce-count auth-param in the digest-response 
(section 3.2.2). Can the nc-value value be quoted (even though it 
doesn't need to be) or does the fact that it's defined as 8LHEX 
override the general auth-param BNF?

I believe that if the general auth-param BNF is overridden by the BNF 
defining a specific auth-param attribute-value pair, then a note in 
section 1.2 saying that should be added; if not, then a note in section 
1.2 saying it cannot be overridden and that any value can be quoted 
should be added.

2 - Are quoted-strings required for some auth-param values?

An example of this is the digest-uri-value auth-param in the 
digest-response (section 3.2.2). Since the digest-uri-value can contain 
comma characters, and because the fields in the digest-response are 
comma delimited, the digest-uri-value must be a quoted-string in order 
to handle all possible digest-uri-value.

I believe that if quoting is required for a specific auth-param value, 
then it should say so in the definition of that auth-param.

In our client implementations of Digest Access authentication, we first 
quoted only those auth-param values which were defined as 
quoted-string, or which included <"> in the BNF, by rfc2617. Following 
the rules that way caused problems (mostly because we weren't quoting 
digest-uri-value), so we changed our implementation to quote all 
auth-param values. Last week, we found a proxy cache which doesn't work 
with the nonce-count quoted. That's why I'm bringing this up.

Thanks,

- Jim

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 21 Apr 2003 13:43:10 -0600 (MDT)

Jim,

	I agree that a (token | quoted-string) rule is not precise
enough alone. It lacks (polished versions of) the following two rules:
	- any quoted string MUST be interpreted only after the
	  quotes are removed
	- any token MAY be quoted

These rules would make quoted things equivalent to unquoted things (as
long as the latter can be correctly parsed) and eliminate some of the
confusion. In other words, it would say "when in doubt, quote". I
suspect this is what the authors implied, but there are no explicit
rules like that, unfortunately. RFC 2616 comes close when it says that
"A string of text is parsed as a single word if it is quoted", but
that phrase is not precise enough and does lead to different
implementations.

	The problem you discuss is not limited to authentication
headers. There are, for example, proxies that quote cache-control
directives confusing many HTTP agents:
	Cache-Control: "private"
versus canonical
	Cache-Control: private

	In general, if there is a more specific BNF that [re]defines a
generic parameter from RFC 2616, that more specific BNF should be
followed. Hopefully, there are no conflicts (requiring a specific part
of an OR statement is not a conflict!) other than those caused by the
lack of the above quotation rules.

	The following RFC 2616 wording may answer your second
question:

   Many HTTP/1.1 header field values consist of words separated by LWS
   or special characters. These special characters MUST be in a quoted
   string to be used within a parameter value

A comma (',') is a special character in a digest-response context and,
hence, digest-uri-value must be quoted if it contains commas. This may
still cause interoperability problems, of course. Many implementations
we work with do not handle quoted strings well enough.

To summarize, here are two compatibility rules of thumb I try to
follow/recommend:
	- if the spec allows you not to quote something,
	  do not quote that something unless quotes are necessary for
	  correct parsing on the other end
	- accept quoted something even if quotes are not necessary;
	  remove quotes before interpreting that something

HTH,

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance