Subject: Re: [Ietf-http-auth] Updating RFC 2617 (HTTP Digest) to use UTF-8

From: "Robert Sayre" <[email-masked]>
Date: Sat, 14 Oct 2006 21:53:18 -0400

On 9/25/06, Lisa Dusseault <[email-masked]> wrote:
> I agree it would be good to update RFC2617.  Is anybody going to take
> a stab at it?
>

As a data point, Mozilla always uses a lossy UTF16-to-ASCII conversion
for Basic [1] and always uses UTF-8 for Digest [2]. Both
implementations date from 2001. We have no bugs filed on Digest
authentication charsets, and one bug on Basic that's been filed
several times.[3]

I don't think a 2617 update is a good use of time, but I will update
our networking code in the event an update is completed. MD5-sess is
essentially unused. An update should drop it.

1.) http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/http/src/nsHttpBasicAuth.cpp#108
2.) http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/http/src/nsHttpDigestAuth.cpp#330
3.) https://bugzilla.mozilla.org/show_bug.cgi?id=41489

-- 

Robert Sayre

---

From: Ingo Struck <[email-masked]>
Date: Sun, 15 Oct 2006 17:39:42 +0000

Hello Robert,

> essentially unused. An update should drop it.
It is "essentially unused" because essentially nobody
created a correct implementation. As for mozilla:
https://bugzilla.mozilla.org/show_bug.cgi?id=270447
(open since 2004, "Nobody's working on this, feel free to take it"
I would if I had some better knowledge of the internal
ns... data storage api)

I currently implemented a server-side implementation 
of rfc2617 supporting MD5-sess like specified.

Bottom-line of the tests performed is, that nearly any
known rfc2617 client-side implementation is more or less
broken (programmers seem to tend to cut'n'paste the example
code found in the rfc and blindly use it for MD5-sess too,
which is plain wrong).

To sum up only the most severe mal-implementations:
- IE only sends the "opaque" value within the first
  subsequent request, thus rendering MD5-sess basically 
  dysfunctional (you have to combine the sessionid and
  the nounce, instead of being able to use opaque for the
  sid like it is intended)
- some netscape descendants tend to use a stale nonce
  after the server sent an updated nonce
- all netscape/mozilla-based browsers recalculate the 
  session-key for each request (see the bug filed above),
  as well as most other browser implementations (e.g.
  opera)
- opera sends MD5-SESS not MD5-sess
- kio-based browsers send qop and algo quoted
- nonce-count calculus is broken for most clients

I would not drop something, only because it is not widely
used as a result of broken implementations -- the better
way would be to clarify the requirements for a conforming
implementation and the advantages of a working MD5-sess
implementation. In fact MD5-sess could be used as a performance
advantage, both for client and for server implementations:
- both of them need to store some "session state" anyway
  if using digest auth
- usage of MD5-sess saves some MD5-operations (which are
  usually not cheap if you use the common implementations)

Kind regards

Ingo Struck

---

From: "Robert Sayre" <[email-masked]>
Date: Sun, 15 Oct 2006 12:56:19 -0400

On 10/15/06, Ingo Struck <[email-masked]> wrote:
> broken

That is the main metric the IETF uses when deciding whether to drop a
capability. Also, there is no reason to include MD5-sess in the same
document.

-- 

Robert Sayre

---

From: Adam Roach <[email-masked]>
Date: Sun, 15 Oct 2006 12:06:13 -0500

>   after the server sent an updated nonce

As I've pointed out many times over the past several years: 2617 
contains _conflicting_ language regarding whether H(A1) should be 
recalculated upon receipt of nextnonce when using MD5-sess. It would 
take one short sentence to resolve this ambiguity one way or the other.

With conflicting language in the spec, it's no wonder that these 
implementations get it "wrong" -- they have to choose between two 
mutually exclusive statements.

/a

---

From: Ingo Struck <[email-masked]>
Date: Sun, 15 Oct 2006 18:30:08 +0000

Adam, Robert,

> take one short sentence to resolve this ambiguity one way or the other.
I agree that a clarification would help, but I do not agree
that the rfc really is ambiguous (see my mail from 2006-08-09
to the wg list).
Anyway, the brokenness of the implementations I pointed out
have nothing to do with issuing a nextnonce.
The clients re-calculate the session-id for *every* request,
even with a pertained nonce (but with a different cnonce), 
which is (unambiguously) a violation of the spec.

The "stale nonce" problem mentioned does not only affect MD5-sess
but simple MD5 too. Some browser implementations seem to use broken
multi-threading in combination with concurrent connections to the
same server such that one thread sends a stale nonce (within MD5 algo!)
after a previous response from the server already marked that nonce
as stale. It seems that access to "session-data" within those
browsers is not synchronised among different i/o-threads.

> capability. Also, there is no reason to include MD5-sess in the same
Using such a metric would mean to drop usage of rfc 2616 / 2617
completely (nearly any implementation out there is "broken" in
the sense of complying to the rfc in some way or other. 
Especially the implementations of the prevalently used browsers
out there are an utter mess -- I guess I do not have to specify
the details here).

I would agree that it could be a good idea, though, to move
the specification of the algorithms used by rfc2617 out to
separate standards tracks.

Kind regards

Ingo Struck

---

From: "Robert Sayre" <[email-masked]>
Date: Mon, 16 Oct 2006 04:51:51 -0400

> out there are an utter mess -- I guess I do not have to specify

No, it's a feature by feature test. Basic works for some people some
of the time, and MD5-sess doesn't. The cause is irrelevant. It could
be that the spec is too difficult to implement, the spec is not worth
implementing, the specified protocol doesn't work at all, or that
browser engineers are dumb. It doesn't really matter--the document is
really old by now, and the workable parts are clear.

-- 

Robert Sayre

---

From: Sylvain Hellegouarch <[email-masked]>
Date: Mon, 16 Oct 2006 10:05:25 +0100

> really old by now, and the workable parts are clear.

The first time I've implemented Digest for the CherryPy server (a Python
HTTP 1.0/1.1 server) I was so surprised RFC2617 was so difficult to
decrypt. I must say to this date I have no idea to say whether or not my
implementation is correct or not and the worse is that there is no way
to test this anyway considering no one can provide a definitive
implementation against which to test or to compare.

I do not pretend having any authority on that matter but as Robert says
the document has grown old now and it really ought to be updated and
clarified.

- Sylvain

BTW, for those interested here is the implementation:
http://www.cherrypy.org/browser/trunk/cherrypy/lib/auth.py
http://www.cherrypy.org/browser/trunk/cherrypy/lib/httpauth.py