Subject: Re: [http-auth] a review of draft-ietf-httpauth-scram-auth

From: Alexey Melnikov <[email-masked]>
Date: Sun, 18 May 2014 13:41:03 +0100

Hi Tony,
Sorry for the very slow response on this.
(Also, I discovered that I got unsubscribed from the http-auth mailing 
list, so I reconstructed your email message from the web archive.)

Firstly, thank you very much for your review and comment.
> I just recently finished an implementation of SCRAM support in another
> protocol that is almost identical to HTTP, so my wounds there are rather
> fresh. This is a review of Salted Challenge Response (SCRAM) HTTP
> Authentication Mechanism, draft-ietf-httpauth-scram-auth. (In what
> follows, I'll refer to this version of SCRAM as "HTTP SCRAM" and that in
> RFC 5802 as "SASL SCRAM".) In the following, I raise a large number of
> points about this draft in particular, along with some points related to
> HTTP auth in general.
>
>       Tony Hansen
>
> As I was re-reading this draft, my thoughts kept getting drawn to these
> questions:
>
> 1) what is different in HTTP SCRAM from the original SASL SCRAM
> documented in RFC 5802?
> 2) hash agility
> 3) method agility
>
>
>
> 1. What is different in HTTP SCRAM from the original SASL SCRAM
> documented in RFC 5802?
>
> When a new version of a spec is created by copying such a large portion
> of another protocol document, as this document was from RFC 5802,
> someone who has implemented the earlier protocol or who has a library
> that implements the earlier protocol needs to be concerned about the
> differences between this new incarnation and the previous incarnation.
> If you do a diff between this draft and RFC 5802 (such as using
> tools.ietf.org/rfcdiff), you will see many gratuitous differences (eg
> "left side" vs "left-side", "that" vs "which") whose very quantity could
> hide the actual differences.
I think these are because I started from my pre-SCRAM-as-RFC-publication 
draft versa the final SCRAM RFC...
> First off, get rid of the gratuitous differences! If the copied text is
> going to be kept, make it easy to diff the two.
>
> Now note: actual differences do exist. In particular, I noted several
> specific differences, although I may have missed some:
>
> a) The specification of the gs2 header is different. It has been
> "adjusted" to match the name=value format of www-authenticate's
> 1#challenge. In addition, the use of authzid has been lost. For example,
> in SASL SCRAM the gs2-header uses "p=channel,authzid," and in HTTP
> SCRAM, gs2-header uses "g=p=channel," with no way to specify authzid.
> b) The handling of white space and double quotes.
>
> The consequence of these substantive differences is that existing
> implementations of SASL SCRAM can NOT be used to implement HTTP SCRAM.
> Personally I think that this is a bad consequence.
>
> There are two fixes for going forward:
>
> A) Accept that there are differences. If we do this, I feel that there
> MUST MUST MUST be a serious WARNING section added for implementers to
> let them know that current SASL SCRAM libraries can NOT be used.
One way to handle the differences is to canonicalize SCRAM HTTP 
challenges/responses (strip WSP between parameters, reorder them in the 
order prescribed by SCRAM) before passing canonicalized results to the 
SCRAM library.

And any data generated by SASL SCRAM is pretty much valid HTTP syntax 
(other then the gs2 header and the authorization identity).

However I do find your proposal to just base64 data to be tempting.
> B) Get rid of these differences. See below.
>
> Regarding the handling of white space, one of the hardest parts to get
> right with SCRAM is getting the inputs into the hash algorithms exactly
> right, on BOTH sides. For example, if one side includes a space and the
> other side does not, then the hashes will not match.
Right. See above.
> If you look at the ABNF for "challenge" (HTTP-P7, HTTP-P1) and the ABNF
> for the elements for the name=value pairs in HTTP SCRAM, you will find
> that they are not compatible. The HTTP definition is a superset in that
> it allows for the use of quoted strings and allows for white space to be
> included in various locations, neither of which are allowed by the ABNF
> for the SCRAM-specific elements. I don't think that the ABNF for
> auth-param is compatible at all with gs2-header p= option because
> token68 does not allow an "=".
>
> The examples all use copious white space both between parameters and
> beginnings and ends of multiple lines, as well as the line breaks
> themselves. At a minimum, there need to be a note about this difference,
> as well as a requirement that all white space must be stripped before
> doing any hashing. Or see my note below on eliminating the differences.
>
> As for the quotation marks, there are a couple possibilities. The most
> correct would be to change the ABNF for ALL of the SCRAM elements to be
> compatible with the ABNF for "challenge". This is also required to
> satisfy the gs2-header issue mentioned above. Or see my note below on
> eliminating the differences.
Right.
> The loss of authz is potentially troublesome. I know that HTTP hasn't
> included this concept before, so anyone who has needed this concept has
> had to implement it below HTTP or using a non-standard HTTP header.
> There is discussion of the use of authz elsewhere on the httpauth list.
> One way to correct this is to provide a new "z=" parameter (to pick
> that is allowed as part of gs2-header. NOTE: this further breaks
> compatibility with stock SCRAM implementations. Or see my note below on
> eliminating the differences.
I personally don't object to keeping the authorization identity, but it 
would be good to hear other opinions on this.
> There is no ABNF for the sid value or any constraints on it.
I am hoping that this can be extracted to a separate document, which can 
be shared by all authentication mechanisms.

But either way, I agree that this should be defined.
> 2) Hash agility
>
> HTTP SCRAM is defining a >family< of authentication schemes whose names
> all start with "SCRAM-" and end with a hash value.
>
> a) The ABNF for challenge-scram and credentials-scram are wrong. They
> both specify "SCRAM-SHA-1" as the only string that can be used. Instead
> they should say something like:
>
>       challenge-scram    = scram-name 1*SP 1#auth-param
>       credentials-scram  = scram-name 1*SP 1#auth-param
>
>       scram-name = "SCRAM-SHA-1" / other-scram-name
>           ; SCRAM-SHA-1 is registered by this RFC
>       other-scram-name = "SCRAM-" hash-name
>           ; hash-name is a capitalized form of names from IANA "Hash
> Function Textual Names" registry.
>           ; Additional SCRAM names must be registered in both the IANA
> "SASL mechanisms" registry
>           ; and the IANA "authentication scheme" registry.
>
> The comments about sasl name and hash name registry come from RFC 5802.
>
> It MIGHT be worth mentioning the potential use of SCRAM-SHA-256 in the
> future, but I'm not going to get hung up on this.
Ok. I have no objections to multiple SCRAM mechanisms.
> Actually, the ABNF shown in section 5 and aboe is incorrect as shown
> because auth-param is actually optional when there is no realm. A better
> description might be
>
>       challenge-scram = scram-name (1*SP 1#auth-param)
>       credentials-scram = scram-name (1*SP 1#auth-param)
>
>
>
>
> 3) method agility
>
> Unless I'm totally mistaken, the issues with method agility are
> associated with the protocol exchange example shown in section 5 is
> incorrect with respect to the realm and the description of the original
> 401 Unauthorized challenge along with subsequent descriptions of the realm.
>
> I think it's worth showing a *complete* example 401 challenge exchange,
> filling in the "server might have returned 401 Unauthorized first"
> comment. THIS is where the server might provide one or more realm names
> as part of its challenge. For example, you might see:
>
>       WWW-Authenticate: Basic  Digest realm="realm1"
>           Digest realm="realm2" Digest realm="realm3"
>           SCRAM-SHA-1 realm="realm3" SCRAM-SHA-1 realm="realm4"
>
> In this example, the user may authenticate using Basic with no realm,
> using Digest in realms 1, 2 or 3, and using SCRAM-SHA-1 using realms 3 or 4.
>
> The first client response that includes a WWW-Authenticate header then
> specifies the authentication scheme along with the optional realm that's
> been chosen and any additional parameters needed for the specific
> authentication scheme.
Right.
> The interesting new thing about SCRAM-SHA1 as an authentication scheme
> is that it is the first one that has multiple rounds. The server
> challenge that comes back from an authentication scheme that requires
> such multiple rounds should NOT specify the realm parameter again, nor
> should subsequent client responses include the realm parameter again.
>
>         [...The server might have returned 401 Unauthorized first...]
>         S: WWW-Authenticate: Basic
>               Digest realm="[email-masked]"
>               Digest realm="[email-masked]"
>               Digest realm="[email-masked]"
>               SCRAM-SHA-1 realm="[email-masked]"
>               SCRAM-SHA-1 realm="[email-masked]"
>         S: [...]
>
>         C: GET /resource HTTP/1.1
>         C: Host: server.example.com
>         C: Authorization: SCRAM-SHA-1 realm="[email-masked]",
>                g=n,n=user,r=fyko+d2lbbFgONRv9qkxdawL
>         C: [...]
>
>         S: HTTP/1.1 401 Unauthorized
>         S: WWW-Authenticate: SCRAM-SHA-1
>                sid=AAAABBBBCCCCDDDD,
> r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,
>                i=4096
>         S: [...]
>
>         C: GET /resource HTTP/1.1
>         C: Host: server.example.com
>         C: Authorization: SCRAM-SHA-1 sid=AAAABBBBCCCCDDDD,
>                c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,
>                p=v0X8v3Bz2T0CJGbJQyF0X+HI4Ts=
>         C: [...]
>
>         S: HTTP/1.1 200 Ok
>         S: Authentication-Info: SCRAM-SHA-1
>                sid=AAAABBBBCCCCDDDD,
>                v=rmF9pqV8S7suAoZWja4dJRkFsKQ=
>         S: [...Other header fields and resource body...]
>
> The description of the realm parameter(s) need to be adjusted to reflect
> when the server and client send it/them.
>
> It might be worth introducing SOMEWHERE the terms "challenge #0" as the
> server challenge that lists the acceptable authentication schemes and
> their optional realms, and then the client response is "response #1"
> followed by "challenge #1", and counting up from there. In general
> terms, challenge #0 always lists the available authentication schemes
> and their optional realms (along with any other initial parameters that
> need to be passed along with that authentication scheme). Response #1
> specifies the next round of parameters for the given authentication
> scheme chosen by the client. Challenge #1 either returns the 200 with an
> optional Authorization-Info or another 401 Unauthorized with a
> WWW-Authenticate header. If this is an authentication scheme that allows
> another round to be used for additional information, Challenge #1 would
> also include a sid= parameter (or WWW-Authentication-Sid header as has
> been suggested elsewhere). Subsequent Responses (#2 and beyond) then use
> the same sid=/WWW-Authentication-Sid value to tie the challenges and
> response together.
>
> I'm not sure where SOMEWHERE should be. Is it too late to get this into
> HTTP BIS P7?
>
>
>
>
>
>
>
>
> Getting Rid of the Differences with SASL SCRAM
>
> Now I'm going to present an alternative way of fixing some of the
> problems above. The basic idea is to say that the SCRAM-SHA-1 payloads
> are exactly as specified in RFC 5802, but are base64-encoded for use
> within HTTP. This is exactly the SASL authentication split that is used
> by all other application protocols that use SASL: the SASL mechanism
> specifies what information needs to be included in any given round of
> exchange and the application protocol specifies how that information is
> wrapped for transmission within the application protocol. For example,
> an application protocol could use base64 encoding of that data (as
> suggested here), or could use run-length-encoding if it's binary, or
> could use quoted-printable or hex or whatever. The important thing is
> that is a strict separation of responsibility. The application protocol
> doesn't care what the data is and what is included within it; it just
> has some mechanism for getting that data across the protocol boundary
> safely. The application protocol restores that data back to its octet
> form and passes that data into the authentication mechanism. The
> response from the authentication mechanism is then encoded for the
> application-layer response.
>
> How does this look for HTTP SCRAM? I'm going to use the word "payload"
> to represent the parameter that holds the SCRAM-SHA-1 payload. Each
> challenge and response becomes 1) the scram name, 2) either the realm or
> sid value, and then 3) the payload value. (I don't care what the
> parameter name is called.) In ABNF, this would be something like this:
>
>       challenge-scram   = scram-name WS realm-or-sid BWS "," OWS
> "payload=" payload-value
>       credentials-scram   = scram-name WS realm-or-sid BWS "," OWS
> "payload=" payload-value
>       realm-or-sid = ( "realm=" realm-value / "sid=" sid-value )
>       payload-value = 1*base64 / DQUOTE 1*base64 DQUOTE
>
> If "sid=" were moved out to a different header, then the above would become
>
>       challenge-scram   = scram-name WS [realm BWS ","] OWS "payload="
> payload-value
>       credentials-scram   = scram-name WS [realm BWS ","] OWS "payload="
> payload-value
>       realm-spec = "realm=" realm-value
>       payload-value = 1*base64 / DQUOTE 1*base64 DQUOTE
>       session-id-header = "WWW-Authentication-ID:" *SP session-id
>       session-id = token
>
>
> All of the rest of the document that provides details on what is
> included within that payload would be removed and replaced with a
> reference to RFC 5802 and a small section on how HTTP authentication
> credentials map into the SASL concepts of authentication id,
> authorization id, etc.
I wouldn't mind doing this. One concern I had is that whenever web 
developers see "GSS-API" (e.g. in RFC 5802), they might freak-out. So I 
was trying to avoid references to unusual terminology. But hopefully 
what you suggest would work.
> The document would still only register SCRAM-SHA-1.
>
>
>
>
> Generalizing Just a Bit Further
>
> An interesting consequence of this modification is that it is extremely
> close to what would be needed to generalize HTTP authentication to using
> any given SASL mechanism that is duly registered and supported by the
> server.
>
> The first editing step is to change the ABNF names used above from
> scram-name to sasl-name.
>
> The only other required change is noting somewhere that challenge#0 MAY
> include an optional payload if that particular authentication mechanism
> has a payload value that is to be sent first from the server to the
> client. (Just as Digest has a variety of parameters that are transmitted
> along with challenge#0.)
>
> There might be a constraint placed on the non-use of SASL security
> layers and restricting the set of SASL mechanisms only to those that do
> not require a security layer to be used.
That might be of interest in the future. I used to have a draft on this 
over 10 years ago :-).
> And I think that that's it.
>
>
> I hope this is of use.
It is very much is.