Subject: [http-auth] (mutual:P5) Re: possible discussions / call for opinions

From: Yutaka OIWA <[email-masked]>
Date: Tue, 17 Mar 2015 13:49:53 +0900

Dear Ilari,

Thank you very much.

2015-03-16 23:09 GMT+09:00 Ilari Liusvaara <[email-masked]>:
> On Mon, Mar 16, 2015 at 10:20:52PM +0900, Yutaka OIWA wrote:
>> [P5] reason parameter:
>>      Are the listed reasons good and complete?
>
> I'm having trouble understanding why authz-failed isn't
> the same as 403.

In short, 403 is too generic to determine the true reason.

In authorization based on HTTP authentication, there are
three cases of situations where accesses are denied:
  1) authentication failed: user credential (pair of ID and password)
      is unacceptable for the whole realm.
  2) authorization failed: user credential is acceptable for
      the realm, but not for a specific resource within it.
  3) user-based authorization is successful, but for some
      reasons access is denied.

The 403 Forbidden status is returned in so many instances of
case 3), e.g. Authenticated identity is unacceptable, TLS client certificate is
unacceptable, client IP address is unacceptable, "r" permission
is missing for the directory listing, "x" permission in parent directories
are missing, the server administrator is reluctant, whatever else.
Some of those should have statuses 404 or 500, but 403 is widely used.

It is generally difficult to handle case 2) in current HTTP statuses.
Apache has "Require user" directive, which returns 401 status for 2).
In this case, clients cannot distinguish it from 1), so many clients thinks
"password was wrong", and forget the credential for the whole realm.
It's bad.

However, if we use 403 for it, the status does not tell the client that
another credential for the realm may be acceptable.
It's just "forbidden for some reason", period.

the "authz-failed" reason is designed, based on the current
Apache's behavior, to tell client that the case is 2), not 1).

Other solution might be to define a new status code or
a new sub-status information (e.g. Error-reason header) to tell
case 2) as clearly distinguishable from case 3).

-- 
Yutaka OIWA, Ph.D.
           Senior Researcher, Research Institute for Secure Systems (RISEC)
    National Institute of Advanced Industrial Science and Technology (AIST)
                      Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

---

From: Michael Sweet <[email-masked]>
Date: Tue, 24 Mar 2015 07:37:03 -0400

> On Mar 17, 2015, at 12:49 AM, Yutaka OIWA <[email-masked]> wrote:
> ....

This is a classic interop problem with HTTP authentication, and something that varies between implementations.

In the case of CUPS, we decided to return 403 if authentication succeeds (valid credentials) but the user does not have access to the resource.  This potentially exposes information ("I have the right credentials") to an attacker, but we felt that usability in this case trumped that concern (browsers don't retry a 403 response).

I'm not sure whether we need/want to more clearly distinguish why access is forbidden.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 24 Mar 2015 05:53:40 -0700

On Mar 24, 2015, at 4:37 AM, Michael Sweet wrote:

> This is a classic interop problem with HTTP authentication,

Nope.

> and something that varies between implementations.

Yep, because some implementations decide not to reveal that info.

> In the case of CUPS, we decided to return 403 if authentication succeeds (valid credentials) but the user does not have access to the resource.  This potentially exposes information ("I have the ri…

Sending 403 with a WWW-Authenticate header field repeating the challenge
indicates to the client that authenticating as a (different) user might
change the status.

> I'm not sure whether we need/want to more clearly distinguish why access is forbidden.

We don't, in general, but we do provide the mechanism.

....Roy

---

From: Michael Sweet <[email-masked]>
Date: Tue, 24 Mar 2015 10:23:37 -0400

Roy,

> On Mar 24, 2015, at 8:53 AM, Roy T. Fielding <[email-masked]> wrote:
> 

In this case I mean interop for getting a specific user experience from all User Agents when using HTTP Authentication (which is why most web sites do not use it and instead do their own thing with cookies, etc.)

"Authenticated but not authorized" is just one of the issues...

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 25 Mar 2015 00:27:15 +0900

Dear Michael,

> I'm not sure whether we need/want to more clearly distinguish why access is forbidden.

Thank you for the informative reply. It's interesting and valuable
to know the CUPS's decision about the situation.
However, I have some case where 401 is more preferable for the
setting.
(Let's temporarily forget about use of WWW-Authenticate with
 non-401 status.)

Let me assume the following setting:
 - there is two pages authenticated for the same realm
   (i.e. the same set of users),
 - the first one requires general user authentication (as "yutaka"), and
 - the second one requires priviledged user authentication (as "root").
I have credentials for both identities, and prefer to use
the general user identity as much as possible.

I already accessed the first page, an authentication is required,
and provided a credential for "yutaka".
The browser rememberss the credential for the "yutaka" at this moment.

Now, I move to the second page, and find that "yutaka" is
unacceptable.
In this case, there are two conflicting situation for usability.

1) When I do not intend to perform priviledged action,
   I'd like to go back to the first page.
   In this case, returing 403 is preferable.

   If 401 is returned (as Apache does), the browser
   will forget the memorized credential for "yutaka",
   thinking that the provided credential has become invalid.
   I'll need to re-authenticate as "yutaka".

2) If my intention is, oppositely, to perform an action,
   even with providing the priviledged credential (like su or sudo).
   In this case, 401 is more preferable.
   Returning 401 will provide me an opportunity to
   re-authenticate with the "root" credential.

   If 403 is returned (as CUPS does), the browser does not have
   any opportunity to provide me an option to re-authenticate
   with another credential.
   I'd need to close/reopen the browser to let the browser
   forget the authentication status.

That's why I think "401-and-403" status is needed in some way.

I personaly think that the information exposure issue you've mentioned is
not a problem at all in this setting, because an attacker
will know the validity of the credential for "yutaka" anyway,
by simply accessing the first page.

I'll make a reply for the Roy's comment separately.

-- 
Yutaka OIWA, Ph.D.
          Planning Officer, Research Planning office for IT and Electronics
    National Institute of Advanced Industrial Science and Technology (AIST)
                      Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 25 Mar 2015 00:40:50 +0900

Dear Roy,

On 2015/03/24 21:53, Roy T. Fielding wrote:
> ...

Hmm, this makes sense.  Thank you very much.

Actually, at least personally, I understand the clarification in RFC 7235,
Section 4.1 about the use of WWW-Authenticate with a non-401 status as an
extending "changes" from the previous definition.  Anyway, in this case, use of
403 and WWW-Authenticate altogether have really clear meaning which we should
adopt.

If we can have a common consensus about the understanding of this
combination, I might add a note about the "authz-failed" status in
the Mutual draft, saying that "the status-code might be better used together
with the 403 status instead of 401."

(However, adopting this way of understanding gives an interesting
 consequence: 401 and 403 with WWW-Authenticate will have almost
 the same meaning for an non-authenticated request.
 For an authenticated request, 401 and 403 will differentiate whether
 the provided credential is accepted or not.)
## Your parenthesis around the word "different" gave me this insight.

> We don't, in general, but we do provide the mechanism.

I think we do in some cases, as an example provided in the reply to the
Michael's previous mail.

---

From: Michael Sweet <[email-masked]>
Date: Tue, 24 Mar 2015 15:47:47 -0400

Yutaka,

> On Mar 24, 2015, at 11:27 AM, Yutaka OIWA <[email-masked]> wrote:
> ...

Generally speaking, if you return 401 then the User Agent will present an authentication dialog.  If you return 403 it will display the error.

> the general user identity as much as possible.

The issue here is that a) the server does not know that the user has multiple credentials, and b) the client/user agent does not know that the provided credentials were otherwise valid, only that the provided credentials will not authorize access.

>   forget the authentication status.

There *is* that, but in the case of CUPS it is more common to not have multiple accounts or authorization levels - you are a regular user or an admin...  Returning a 401 response is just confusing, and before we started returning 403 I personally got 3-4 emails per week asking what username and password to provide to use the CUPS web interface...

> That's why I think "401-and-403" status is needed in some way.

I like Roy's suggestion of returning WWW-Authenticate information in a 403 response, but that will require standardization and agreement/implementation by user agents.