Subject: [#177] Realm required on challenges

From: Mark Nottingham <[email-masked]>
Date: Tue, 7 Jul 2009 17:15:17 +1000

[ this was raised anonymously ]

p7 defers to RFC2617 for the definition of challenge.

RFC 2617, section 1.2 says:

challenge = auth-scheme 1*SP 1#auth-param ... The authentication  
parameter realm is defined for all authentication schemes:

realm = "realm" "=" realm-value realm-value = quoted-string

The realm directive (case-insensitive) is required for all  
authentication schemes that issue a challenge.

The interpretation being that challenges (which is what www-  
authenticate is defined as) MUST contain at least one parameter and  
that parameter MUST be a realm.

Is it truly necessary for all authentication schemes to include a  
'realm' paramter? If so, it should be documented (e.g., in the section  
about extension authentication schemes).

--
Mark Nottingham     http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 7 Jul 2009 17:42:07 +1000

Not to argue a particular position WRT #177, but using NTLM is  
probably a bad example, precisely because it does connection  
authentication -- thereby breaking HTTP's assumption of statelessness.

Cheers,

On 07/07/2009, at 5:35 PM, Robert Collins wrote:

> On Tue, 2009-07-07 at 17:15 +1000, Mark Nottingham wrote:
> With you so far.
> Got that too.
> I'd have to check, but I'm fairly sure that NTLM doesn't provide a  
> realm
> in its challenges. I'm also fairly certain, because that scheme does
> connection authentication, not message authentication, that the  
> intended
> use - partitioning a single site - doesn't even make sense for that
> scheme (nor the Negotiate scheme).
>
> -Rob

---

From: Robert Collins <[email-masked]>
Date: Tue, 07 Jul 2009 07:36:43 +0000

With you so far.

Got that too.

> about extension authentication schemes).

I'd have to check, but I'm fairly sure that NTLM doesn't provide a realm
in its challenges. I'm also fairly certain, because that scheme does
connection authentication, not message authentication, that the intended
use - partitioning a single site - doesn't even make sense for that
scheme (nor the Negotiate scheme).

-Rob

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 7 Jul 2009 11:00:59 +0200

On Tue, Jul 7, 2009 at 10:28 AM, Adrien de Croy wrote:
>
> I question the validity of requiring that realm be a parameter of every
> (even new) scheme that has a challenge.
>
> I've never seen a browser use the realm for anything other than a label in a
> dialog box either.

Plain wrong!

See http://ltgt.net/tests/http-auth-realm/

I tested it in IE8, Firefox 3.5, Opera 9.64, Safari 4 and Chrome
3.0.191.3 (Dev channel), all on Windows.
Only Chrome fails the test and do not ask for your credentials when
going from Foo to Bar the first time; all others take the realm into
account when storing the credentials for use in subsequent requests;
realm is not just a label in a dialog box.

-- 
Thomas Broyer

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 07 Jul 2009 09:29:11 +0100

Adrien de Croy wrote:

> I've never seen an NTLM challenge (or Negotiate) with a realm 
> parameter.  Realm isn't referred to in RFC 4559 either.
>
> I question the validity of requiring that realm be a parameter of 
> every (even new) scheme that has a challenge.

I tend to agree that the realm shouldn't be required.

> I've never seen a browser use the realm for anything other than a 
> label in a dialog box either.

It is also used for hash computations (at least for Digest).

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 07 Jul 2009 21:11:01 +1200

> See http://ltgt.net/tests/http-auth-realm/
OK. Glad to be wrong on that count.

One thing, since most of my experience is associated with proxies.  What 
is the realm supposed to be set for when authenticating to a proxy that 
uses the same credentials for all sites.

Regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 07 Jul 2009 20:28:56 +1200

I've never seen a browser use the realm for anything other than a label 
in a dialog box either.

Regards

Adrien

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 7 Jul 2009 11:34:24 +0200

> realm is not just a label in a dialog box.

Oops! it depends on the cache actually!

Re-tested in Chrome 2.0.172.33 and Chrome 3.0.191.3 with a clean cache
(in the mean time, I added a Cache-Control: private on the foo and bar
responses), they behave as expected (ask your credentials when
changing realm).

Tested successfully in IE6 and IE7 too.

-- 
Thomas Broyer

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 7 Jul 2009 14:18:51 +0200

On Tue, Jul 7, 2009 at 9:15 AM, Mark Nottingham wrote:
>
> Is it truly necessary for all authentication schemes to include a 'realm'
> paramter? If so, it should be documented (e.g., in the section about
> extension authentication schemes).

It is totally useless to the Cookie scheme:
http://tools.ietf.org/html/draft-broyer-http-cookie-auth (wow, expires
tomorrow)
At least I cannot find a single case where it could be used.

-- 
Thomas Broyer

---

From: Robert Collins <[email-masked]>
Date: Tue, 07 Jul 2009 10:47:30 +0000

> authentication -- thereby breaking HTTP's assumption of statelessness.

Oh it surely is a pain. You don't want to know how ugly making NTLM work
through squid (to NTLM offering servers on the internet) was/is. Pretty
hard to imagine HTTP/SMTP w/NTLM too.

That said, it exists, and forcing it to add a empty realm would be
pointless IMO - let alone probably fraught and likely to break clients.

-Rob

---

From: Jim Luther <[email-masked]>
Date: Tue, 7 Jul 2009 10:20:33 -0700

> label in a dialog box either.

Apple's network framework code won't use a user's stored credentials  
in a response unless the protection space for the authentication  
challenge and the protection space for the stored credential match.  
Our code considers protection spaces to match if the host, port  
number, server type (http, https, http proxy, https proxy, ftp, etc),  
and authentication scheme (Basic, Digest, etc) all match. If the  
challenge includes a realm, then the stored credential's protection  
space must have a realm and it must match the challenge's realm; if  
there is no realm in the challenge, then the stored credential's  
protection space must not have a realm. So yes, our code checks the  
realm if it is included in a challenge and it require a realm for  
Basic and Digest Access authentication.

> 3.0.191.3 (Dev channel), all on Windows.

And you'll see that Safari 3 and 4 on Mac OS X, and Safari 3 on  
Windows, also behave correctly.

I've had complaints that our matching is too strict, but I don't think  
so. For example, I've been told that other browsers will hand out  
credentials stored for port 80 to a challenge from the same host on  
port 443 if the rest of the challenge is the same -- I consider doing  
that a security hole. Anyone want to write a test to check that?

- Jim

---

From: Jeff Jenkins <[email-masked]>
Date: Tue, 7 Jul 2009 10:56:24 -0700

> realm is not just a label in a dialog box.
I agree!!

Realms do matter!  It would be a security hole to automatically apply  
credentials from realm X to realm Y.
Not all authentication protocols use realms, thus its inclusion should  
be optional.  NTLM/Negotiate authenticate a connection, and do not  
necessarily apply to what can be accessed on the server.

-- jrj

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 08 Jul 2009 10:02:49 +1200

many browsers I've seen don't require there to be a realm as part of the 
challenge.  Some versions of WinGate didn't supply one, and Basic still 
worked in all the major browsers we tested with.  I guess we didn't test 
Safari.

But this is for auth to a proxy.

When we added a realm, we couldn't find anywhere which specified how a 
proxy should specify a realm of the entire universe.  In the end we used 
realm="/"

A lot of docs on the web which "explain" the realm parameter and how to 
set it, tell you to set it to any text that describes the abstract 
concept of the space within which the credentials are usable.  E.g. a 
quick google search shows sites with examples like:

"Secure Area"
"Our Site"
"My website"
"Test Authentication System"
etc

 This is nothing like a sitename, or anything the browser can use to 
judge whether or not the credentials are usable or not for another 
request.  So the browser has to simply assume the actual usability space 
of the credentials are defined by host, port, etc etc.  Surely this 
isn't the intent of realm?

Is one intent of realm to provide information to browsers so that they 
don't automatically re-submit credentials outside the realm (thereby 
potentially leaking sensitive information)?  How should a browser do 
that when it gets "Our Site"?

What happens when more than one site choose the same parameter for realm 
(e.g. read the same example web page).  Will the browser automatically 
submit Basic credentials to the second site based on the first site?

It seems the only safe thing for a browser to do is to not automatically 
resubmit any Basic credentials if any of port, server, protocol change 
regardless of whether they get a challenge with a matching realm to one 
previously seen.

IOW isn't the realm completely redundant?  Or worse?

Adrien

---

From: Jeff Jenkins <[email-masked]>
Date: Tue, 7 Jul 2009 15:23:13 -0700

On Jul 7, 2009, at 3:02 PM, Adrien de Croy wrote:

>
> many browsers I've seen don't require there to be a realm as part of  
> the challenge.  Some versions of WinGate didn't supply one, and  
> Basic still worked in all the major browsers we tested with.  I  
> guess we didn't test Safari.
>
> When we added a realm, we couldn't find anywhere which specified how  
> a proxy should specify a realm of the entire universe.  In the end  
> we used realm="/"
>
> A lot of docs on the web which "explain" the realm parameter and how  
> to set it, tell you to set it to any text that describes the  
> abstract concept of the space within which the credentials are  
> usable.  E.g. a quick google search shows sites with examples like:
>
> "Secure Area"
> "Our Site"
> "My website"
> request.  So the browser has to simply assume the actual usability  
> space of the credentials are defined by host, port, etc etc.  Surely  
> this isn't the intent of realm?
>
> Is one intent of realm to provide information to browsers so that  
> they don't automatically re-submit credentials outside the realm  
> (thereby potentially leaking sensitive information)?  How should a  
> browser do that when it gets "Our Site"?
>
> What happens when more than one site choose the same parameter for  
> realm (e.g. read the same example web page).  Will the browser  
> automatically submit Basic credentials to the second site based on  
> the first site?
You could base your credential "cache" on realms AND server/port  
keys.  If a client is asking for foo.com:8888 with a realm of /bar,  
and then asks for foo.com:8888 /snafu you would not want to  
automatically use the cached credential.  Even a request for foo.com: 
443 with a realm of /bar differs, so no automatic application of  
cached credentials are safe.

>
> It seems the only safe thing for a browser to do is to not  
> automatically resubmit any Basic credentials if any of port, server,  
> protocol change regardless of whether they get a challenge with a  
> matching realm to one previously seen.
>
>> Apple's network framework code won't use a user's stored  
>> credentials in a response unless the protection space for the  
>> authentication challenge and the protection space for the stored  
>> credential match. Our code considers protection spaces to match if  
>> the host, port number, server type (http, https, http proxy, https  
>> proxy, ftp, etc), and authentication scheme (Basic, Digest, etc)  
>> all match. If the challenge includes a realm, then the stored  
>> credential's protection space must have a realm and it must match  
>> the challenge's realm; if there is no realm in the challenge, then  
>> the stored credential's protection space must not have a realm. So  
>> yes, our code checks the realm if it is included in a challenge and  
>> it require a realm for Basic and Digest Access authentication.
>>
>> I've had complaints that our matching is too strict, but I don't  
>> think so. For example, I've been told that other browsers will hand  
>> out credentials stored for port 80 to a challenge from the same  
>> host on port 443 if the rest of the challenge is the same -- I  
>> consider doing that a security hole. Anyone want to write a test to  
>> check that?
>>
>> - Jim
>>
>
> --

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 17 Jul 2009 03:59:27 +0200

> proxy should specify a realm of the entire universe.  In the end we used 

It's an opaque string. There is no structure to realms, just equality.

> isn't the intent of realm?

The realm is in addition to the canonical root of the requested server,
or in case of 407 the Cacnonical-Root is proxy-host:port (or
http://proxy-host:port if you like, doesn't matter).

The protection space is defined by the tuple

 (Canonical-Root, Realm)

See 2617 1.2 Access Authentication Framework

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 17 Jul 2009 04:17:49 +0200

> I've never seen a browser use the realm for anything other than a label 

In addition to presenting it to the user I have also seen it be used to
determine which set of cached credentials to use.

Regarding NTLM/Negotiate, the fact that there is no realm returned in
the challenge is a frequent cause to used confusion as they don't relly
know what they are supposed to login to. And with there being some
servers which do switch NTLM protection space depending on the requested
URI it can become quite messy.. As far as I am concerned the lack of
realm in NTLM/Negotiate is just yet another bug in those authentication
schemes.

Regards
Henrik