Subject: #195, was: ABNF for Authorization header not quite right

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Jul 2011 19:58:18 +0200

On 2011-04-01 08:17, Manger, James H wrote:
> The ABNF in draft-ietf-httpbis-p7-auth-13 still doesn't match what the BASIC scheme needs (or NTLM or NEGOTIATE or BEARER etc).
>
> Draft 13 says:
>    credentials = auth-scheme ( token / quoted-string / #auth-param )
>
> BASIC, however, effectively uses:
>    credentials = auth-scheme base64
>
> <base64>  includes the characters '/' and '=' that are not in<token>
> [draft-ietf-httpbis-p1-messaging-13#section-1.2.2]

Good catch. Could be fixed by relaxing "token" as well, but I'm not sure 
that we want to go there...

> I suggest changing the ABNF to the following:
>
>    credentials = auth-scheme SP ( b64 / #auth-param )
>
>    b64 = 1*( ALPHA / DIGIT / "-" / "." / "_" / "~" / "+" / "/" ) *"="
>
>    <b64>  includes the 66 unreserved URI characters plus a few others.
>    It can hold a base64, base64url (URL and filename safe alphabet),
>    base32, or base16 (hex) encoding, with or without padding, but
>    excluding whitespace [RFC4648].
>
>
> This accepts authentication schemes that transmit a base64 blob instead of name=value pairs (such as BASIC, NTLM, NEGOTIATE). It also accepts dot-separated base64url blobs, as proposed in new specs such as JSON Web Tokens.
>
> I dropped<quoted-string>  as I don't know where that came from. Perhaps it was added with<token>  as they are often a pair. If there are no existing uses (and I don't know of any) it adds no value.

Indeed.

Maybe we need

   credentials = auth-scheme SP #( b64 / auth-param )

though? As far as I can tell, RFC 4559 uses that.

Also: RFC 4559 seems to need this for the challenge as well...

> I added<SP>  as I understand HTTPbis is making a global change to be explicit with whitespace.

Oops; separate bug. Now #309.

> This change would reopen ticket #195.

It has; sorry for not processing it faster.

Best regards, Julian

---

From: "Manger, James H" <[email-masked]>
Date: Thu, 28 Jul 2011 10:45:49 +1000

>   credentials = auth-scheme SP #( b64 / auth-param )

I doubt we need that. I have never seen a NTLM or Negotiate scheme header with more than 1 base64 blob.

>Also: RFC 4559 seems to need this for the challenge as well...

RFC 4559 "SPEGNO/NTLM/Negotiate" might specify that, but I don't think it can work as it makes parsing ambiguous. For instance, does the following response header include 1 scheme with 4 parameters, or 2 or 3 schemes? Is "tuv" another authentication scheme supported by this server, or a parameter of the "ABC" scheme?
  WWW-Authenticate: ABC xyz, a=1, qrs, tuv

The ambiguity that occurs for <challenge> might not occur for <credentials> if only 1 Authorization header is allowed, but it does show that we cannot match what the ABNF in RFC 4559 theoretically needs. We should match what is actually in use -- which seems to be a single base64 blob.

I prefer
  ( b64 / #auth-param )
instead of
  #( b64 / auth-param )
That is, I think a <b64> blob should only be allowed when it is the first (and only) parameter.

--
James Manger

---

From: Thomas Maslen <[email-masked]>
Date: Thu, 28 Jul 2011 01:06:18 +0000

On Wed 2011-07-27 17:45 +0700  Manger, James H <[email-masked]> wrote:
[...]
> That is, I think a <b64> blob should only be allowed when it is the first (and only) parameter.

I agree, both for challenges and for credentials.

If some auth-scheme needs multiple base64 blobs it should use #auth-param
(i.e. key-value pairs, presumably with quoted-string values) to convey them.

I agree that NTLM, Negotiate, and even Nego2 need only a single base64 blob;
they do not need multiple base64 blobs.

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 28 Jul 2011 13:11:40 +1200

base 64 blobs allow any sort of arbitrary binary structure to be sent.  
Therefore anything can be described by a single one.

On 28/07/2011 1:06 p.m., Thomas Maslen wrote:
> On Wed 2011-07-27 17:45 +0700  Manger, James H<[email-masked]>  wrote:
> [...]
>> That is, I think a<b64>  blob should only be allowed when it is the first (and only) parameter.

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 beta out now - http://www.wingate.com/getlatest/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 28 Jul 2011 17:06:29 +0200

> I doubt we need that. I have never seen a NTLM or Negotiate scheme header with more than 1 base64 blob.

Ack.

>    WWW-Authenticate: ABC xyz, a=1, qrs, tuv

The ABNF says:

    challenge   = auth-scheme 1*SP 1#auth-param

so there needs to be at least one auth-param, separated by one or more SPs.

That being said, we may have to rethink the use of the list production here.

Keep in mind that 1#auth-param allosws

	scheme auth-param

but also

	scheme ,auth-param

because of the specific rules for empty list elements.

*That* would be impossible to parse, and thus we may want not to use the 
# notation over here.

Best regards, Julian

> The ambiguity that occurs for<challenge>  might not occur for<credentials>  if only 1 Authorization header is allowed, but it does show that we cannot match what the ABNF in RFC 4559 theoretically needs. We should match what is actually in use -- which seems to be a single base64 blob.
>
> I prefer
>    ( b64 / #auth-param )
> instead of
>    #( b64 / auth-param )

---

From: Julian Reschke <[email-masked]>
Date: Thu, 28 Jul 2011 18:35:49 +0200

On 2011-07-28 17:06, Julian Reschke wrote:
>>>> This accepts authentication schemes that transmit a base64 blob
>>>> instead of name=value pairs (such as BASIC, NTLM, NEGOTIATE). It
>>>> also accepts dot-separated base64url blobs, as proposed in new specs
>>>> such as JSON Web Tokens.
>>>>
>>>> I dropped<quoted-string> as I don't know where that came from.
>>>> Perhaps it was added with<token> as they are often a pair. If there
>>>> are no existing uses (and I don't know of any) it adds no value.
>>
>>> Indeed.
>>>
>>> Maybe we need
>>>
>> RFC 4559 "SPEGNO/NTLM/Negotiate" might specify that, but I don't think
>> it can work as it makes parsing ambiguous. For instance, does the
>> following response header include 1 scheme with 4 parameters, or 2 or
>> 3 schemes? Is "tuv" another authentication scheme supported by this
>> server, or a parameter of the "ABC" scheme?
> ...

Added test cases at

   http://greenbytes.de/tech/tc/httpauth/#simplebasiccomma

and

   http://greenbytes.de/tech/tc/httpauth/#simplebasiccomma2

Surprisingly, most UAs (exception: Safari) get this right.

That being said, I think we should disallow leading empty list elements 
in prose, or rewrite the ABNF not to use #.

Feedback appreciated,

Julian

---

From: "Manger, James H" <[email-masked]>
Date: Fri, 29 Jul 2011 09:45:41 +1000

> so there needs to be at least one auth-param, separated by one or more SPs.

NTLM doesn't include any params initially.
A recent HTTPbis change made the 1 and only previously-required parameter 'realm' optional.
So the syntax should be adjusted not to require any params. At which point a base64 blob that isn't the first param would be ambiguous.

> *That* would be impossible to parse, and thus we may want not to use the

If you keep the <b64> part out of the list production it should be unambiguous:
 auth-scheme [1*SP (b64 / #auth-param)]

--
James Manger