Subject: WGLC #357: Authentication Exchanges

From: Mark Nottingham <[email-masked]>
Date: Fri, 8 Jun 2012 17:29:00 +1000

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/357>

Alexey, could you say a little more here? The text as it reads doesn't require authentication to happen in one exchange; it only mandates the status codes and headers to use. 

Thanks,

--
Mark Nottingham   http://www.mnot.net/

---

From: Yutaka OIWA <[email-masked]>
Date: Thu, 14 Jun 2012 23:50:02 +0900

Dear Mark,

I'm not Alexey, but (one of) the person(s) proposing an HTTP authentication
not happen in just one exchange.

>> If the origin server does not wish to accept the credentials
>> sent with a request, it SHOULD return a 401 (Unauthorized) response.

My interpretation of this phrase is "if the origin server does not wish to
provide the requested resource with credentials sent within a request"
(slightly rephrased).
Under this interpretation, we can implement multi-exchange authentication
by using the 401 status code as follows:

A non-authenticating request -> 401 Unauthorized (not acceptable)
-> ask user a secret
-> A request with 1st-credential -> 401 Unauthorized (not satisfied yet)
-> A request with 2nd-credential -> 200 Succeed (now satisfied enough)

# Of course, it can be naturally extended for three or more exchanges.

As far as I know, 401/407 are the best choice for this case.
I also think that there were already multi-exchange HTTP authentications
using 401 in this way.

If one thinks the original sentence is bad for this,
his/her understanding of the above flow may be
"the server is accepting the 1st credential, and just requesting more", I guess.

My proposal is either we can just leave the text as is, or rephrase it
like something above.

# Any rephrasing again with better English is welcome.

-- 
Yutaka OIWA, Ph.D.              Leader, Software Reliability Research Group
                             Research Institute for Secure Systems (RISEC)
   National Institute of Advanced Industrial Science and Technology (AIST)
                     Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 15 Jun 2012 09:24:42 +1000

> # Of course, it can be naturally extended for three or more exchanges.

That was my reading too.

> My proposal is either we can just leave the text as is, or rephrase it

"provide the requested resource" isn't correct from an HTTP standpoint, but I see where you're going.

My (personal) inclination is to leave it as-is, but I'm not against rewording if that'll move us forward.

Alexey?

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Jun 2012 12:04:52 +1000

Looking at this, I think this language in the spec isn't very good for other reasons as well:

> If the origin server does not wish to accept the credentials sent with a request, it should return a 401 (Unauthorized) response. The responsemust include a WWW-Authenticate header field containing at least one (possibly new) challenge applicable to the requested resource.
> 
> If a proxy does not accept the credentials sent with a request, it should return a 407 (Proxy Authentication Required) response. The responsemust include a Proxy-Authenticate header field containing a (possibly new) challenge applicable to the proxy for the requested resource.

Because "accept" can be read in so many ways. I think we can fix both problems with something like:

"""
Requests for protected resources that omit credentials, contain invalid credentials (e.g., a bad password), or partial credentials (e.g., when the scheme requires more than one round trip) SHOULD return a 401 (Unauthorized) response. Such responses MUST include a WWW-Authenticate header field containing at least one (possibly new) challenge applicable to the requested resource.

Likewise, requests that require authentication by proxies that omit credentials, or contain invalid or partial credentials SHOULD return a 407 (Proxy Authentication Required) response. The response MUST include a Proxy-Authenticate header field containing a (possibly new) challenge applicable to the proxy.
"""

Thoughts?

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 20 Jun 2012 15:08:18 +1200

On 20.06.2012 14:04, Mark Nottingham wrote:
>> If the origin server does not wish to accept the credentials sent 
>> with a request, it should return a 401 (Unauthorized) response. The 
>> responsemust include a WWW-Authenticate header field containing at 
>> least one (possibly new) challenge applicable to the requested 
>> resource.
>>
>> If a proxy does not accept the credentials sent with a request, it 
>> should return a 407 (Proxy Authentication Required) response. The 
>> responsemust include a Proxy-Authenticate header field containing a 
>> (possibly new) challenge applicable to the proxy for the requested 
>> resource.
>
>
> Requests for protected resources that omit credentials, contain
> invalid credentials (e.g., a bad password), or partial credentials
> (e.g., when the scheme requires more than one round trip) SHOULD
> return a 401 (Unauthorized) response. Such responses MUST include a
> WWW-Authenticate header field containing at least one (possibly new)
> challenge applicable to the requested resource.
>
> Likewise, requests that require authentication by proxies that omit
> credentials, or contain invalid or partial credentials SHOULD return 
> a
> 407 (Proxy Authentication Required) response. The response MUST
> include a Proxy-Authenticate header field containing a (possibly new)
> challenge applicable to the proxy.
> """
>
> Thoughts?
>

second paragraph:  s/The response MUST/ Such responses MUST/ is a bit 
clearer that its the 407 which "MUST", not any other status which may 
happen.

There is also no mention in any of the auth draft about 403 responses 
being used to terminate authentication attempts or cycles regardless of 
credentials validity or presence.

Something informational in this section would be very useful to clarify 
that 403 is one of the authentication-related states, for use in halting 
loops or making N-tries limitations.

AYJ

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Jun 2012 14:11:25 +1000

On 20/06/2012, at 1:08 PM, Amos Jeffries wrote:

>> credentials, or contain invalid or partial credentials SHOULD return a
> second paragraph:  s/The response MUST/ Such responses MUST/ is a bit clearer that its the 407 which "MUST", not any other status which may happen.

OK

> There is also no mention in any of the auth draft about 403 responses being used to terminate authentication attempts or cycles regardless of credentials validity or presence.
> 
> Something informational in this section would be very useful to clarify that 403 is one of the authentication-related states, for use in halting loops or making N-tries limitations.

How about, after those two paragraphs:

"""
A server receiving credentials that are valid, but not adequate to gain access, ought to respond with the 403 (Forbidden) status code.
"""

?

---

From: Julian Reschke <[email-masked]>
Date: Wed, 20 Jun 2012 10:16:15 +0200

On 2012-06-20 06:11, Mark Nottingham wrote:
> ...

Works for me; with that we have:

261,271c261,275
<  If the origin server does not wish to accept the credentials sent
<  with a request, it SHOULD return a 401 (Unauthorized) response.  The
<  response MUST include a WWW-Authenticate header field containing at
<  least one (possibly new) challenge applicable to the requested
<  resource.
<
<  If a proxy does not accept the credentials sent with a request, it
<  SHOULD return a 407 (Proxy Authentication Required) response.  The
<  response MUST include a Proxy-Authenticate header field containing a
<  (possibly new) challenge applicable to the proxy for the requested
<  resource.
---
 >  (e.g., when the authentication scheme requires more than one round
 >  trip) SHOULD return a 401 (Unauthorized) response.  Such responses
 >  MUST include a WWW-Authenticate header field containing at least one
 >  (possibly new) challenge applicable to the requested resource.
 >
 >  a 407 (Proxy Authentication Required) response.  Such responses MUST
 >  A server receiving credentials that are valid, but not adequate to
 >  gain access, ought to respond with the 403 (Forbidden) status code.

(<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/357/357.diff>)

Best regards, Julian

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 20 Jun 2012 17:36:25 +0900

Dear Amos and Mark,

> A server receiving credentials that are valid, but not adequate to gain access, ought to respond with the 403 (Forbidden) status code.

I have a different understanding on the use of 401/403 statuses.
At least on current implementations (e.g. Apache),
auth-succeed and authz-failed status will be represented by
401-status instead of 403.
403 status is used, for example, when the content is
not accessible by underlying filesystem permissions,
or by server configuration for denying directory listing.

I think this (use 401 instead of 403) should be kept for two reasons:

 * Without 401 status, client will not know that changing
    the user name and the password will solve the
    inaccessibility issue.
 * If we change 403 status' behavior, it conflicts with
   the current use of 403 status described above
   (e.g. denied directory listing within auth-protected realm).

# This is part of the reason that I recently (just in httpbis-00 revision)
added "authz-failed" reason code into our HTTP Mutual proposal,
which has OPTIONAL detailed description codes for authentication/
authorization failures.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 20 Jun 2012 11:24:15 +0200

>      the user name and the password will solve the

Sorry?

"The server understood the request, but refuses to authorize it. 
Providing different user authentication credentials might be successful, 
but any credentials that were provided in the request are insufficient. 
The request SHOULD NOT be repeated with the same credentials." - 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-19.html#rfc.section.7.4.3>

Best regards, Julian

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 20 Jun 2012 19:08:26 +0900

Dear Julian,

2012/6/20 Julian Reschke <[email-masked]>:
> "The server understood the request, but refuses to authorize it. Providing
> different user authentication credentials might be successful, but any
> credentials that were provided in the request are insufficient. The request
> SHOULD NOT be repeated with the same credentials." -

I see.  I noticed now that this was changed incompatibly from RFC 2616.
Thank you for telling me.

Now I have a concern with the text for 403 status for two reasons,
 * any recovery from failed authorization (with successful authentication)
   is completely out of the protocol's provision, and
 * the new definition is intentionally breaking existing server implementations.
But it these are under clear consensus and intention,
it's OK for me (but sure?).  Mark, can I ask how do you think?

# With the older semantics to use 401 for authz-failure,
# failed authorization was recoverable
# (although in somewhat uncomfortable way).

Accepting the above consequences, the Mark's text is making sense.
However, we will not able to define 403 to be "one of the
authentication-related states" as an original proposal, because
403 already has several other use cases which is directly conflicting.
# More specifically, clients could not assume that 403 has
# any meaning for authentication-related status.

Don't we need (do we have a clear way, or do we need to define how)
to distinguish failed authz from other generic access privilege failures?

# For example, is "403 response with WWW-Authenticate: header,
# as a response for request with Authorization: header"
# restrictive enough for defining such a status?

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Jun 2012 20:24:30 +1000

>   is completely out of the protocol's provision, and

The client is still able to submit a new request with different credentials (or no credentials, if it needs a new challenge).

> But it these are under clear consensus and intention,

It doesn't make them non-conformant; they can still choose to use those status codes in the manner they are.

> it's OK for me (but sure?).  Mark, can I ask how do you think?

See:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/294

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Wed, 20 Jun 2012 12:29:53 +0200

> Thank you for telling me.

Context: <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/294>

>   * the new definition is intentionally breaking existing server implementations.

How so? Example?

Best regards, Julian

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 20 Jun 2012 19:37:43 +0900

Thank you Mark,

> The client is still able to submit a new request with different credentials (or no credentials, if it needs a new challenge).

My point was "the client will not have a way to know about
that possibility on the protocol level, as compared with 401-based solution".
I agree that if the client knows it in some way, they can do as you mentioned.

> It doesn't make them non-conformant; they can still choose to use those status codes in the manner they are.

The current implementations including Apache Basic/Digest auth
are not compatible with the proposed text

> A server receiving credentials that are valid, but not adequate to gain access, ought to respond with the 403 (Forbidden) status code.

as they currently use 401 status for that meaning.
Although it is not "MUST" nor "SHOULD",
they are still somewhat "against" the new specifications.
If we say they are compliant,
is there any good way to describe both possibilities and
the preference for future implementations?

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 20 Jun 2012 19:48:18 +0900

Thank you,

> Context: <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/294>

Thanks. I've been late for entering this discussion
as I did not noticed that the title was related to authentication issues.

# I still *personally* prefer the RFC 2616 design :-)

>>  * the new definition is intentionally breaking existing server

The above URL have a link to the page saying IIS use 401 status
for that case, and so is Apache.
We may say the 403 text alone is not breaking existing implementations
(in the way that they say nothing about 401), but the amendment text
currently proposed is conflicting with existing implementations.

My current feeling after this thread of discussions is to honor both 401 and 403
as desired responses with authz-failure, possibly with preference for 403 for
future implementations (if we want).

# I would be updating my drafts if such preference is clearly defined.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Jun 2012 21:00:11 +1000

> future implementations (if we want).

That's effectively where we are; note that there aren't any RFC2119 conformance requirements placed around this.

Cheers,

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 20 Jun 2012 20:31:01 +0900

Dear Mark,

> That's effectively where we are; note that there aren't any RFC2119 conformance requirements placed around this.

I prefer to be explicit that use of 403 is just a preference
and is not RFC2119 nor other "requirements".
"Ought-to" sounds to be louder than RECOMMENDED, as a natural language.
# correct me if I have an English problem.

P.S. ("Intended Status: Informational")

I'd like to append that, (although I prefer 401 approach as default,)
allowing 403 for authz-failure *is* a good thing for deployability.

HTTP Authentication and authorization are implemented in
various software layers, depending on each software's own design.
If both of them are implemented in the same software layer
(both within Web servers, or both within applications),
there are no problem.   However, if auth is in the Web server,
and if authz in the application logic, it is hard to send
a valid 401 response to the client when
the used authentication scheme is more complex than Basic.
Allowing 403 (or empty 401, alternatively) will solve this problem.

In reality, we have experienced implementation difficulty
exactly in this case for proposed Mutual authentication.
It is because Apache web server has separate sub-layers for
auth and authz, and the same problem arised.
Until version 2.2, Basic and Digest
were hard-coded for error handling in the authz layer.
In the current 2.4 series of Apache, it is fixed by adding
an additional API in the auth layer.

---

From: David Morris <[email-masked]>
Date: Wed, 20 Jun 2012 11:01:11 -0700 (PDT)

> > A server receiving credentials that are valid, but not adequate to gain
> > access, ought to respond with the 403 (Forbidden) status code.
> > """

Seems to me that 'ought to' should either be 'SHOULD' or 'MAY'?

---

From: Julian Reschke <[email-masked]>
Date: Wed, 20 Jun 2012 20:15:11 +0200

> Seems to me that 'ought to' should either be 'SHOULD' or 'MAY'?

"MAY" isn't helping at all. "SHOULD" would make those who do not do this 
non-compliant. Do we want to do that? (note it would conflict with the 
"If the server does not wish to make this information available to the 
client, the status code 404 (Not Found) MAY be used instead." that we 
have in the definition of 403).

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 21 Jun 2012 09:57:37 +1000

> # correct me if I have an English problem.

We've consistently used "ought to" to give advice and encourage certain behaviours without making it a conformance requirement (thereby breaking existing implementations). The normative requirements are expressed (and always have been) in RFC2119 language.

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 21 Jun 2012 14:55:07 +1200

On 21.06.2012 06:01, David Morris wrote:
>> gain

If it were, that would make the existing implementations 
non-conformant.

My intention with bringing this up at all was to make it clear there 
was an preferred alternative to infinite 401 loops, should the server 
need to use it.

401/407 being the try-again response, indicates new credentials are 
REQUIRED. It *remains* the preferred status when credentials are valid 
but different ones would gain access in a followup request.

403 being the preferred halt status for authn and authz indicating that 
repeating or retrying is not going to be useful.

We need to do this to prevent future auth schemes or implementations 
using 400 or 404 or such status. We have had several proposals just 
since I joined the list this year for creating a whole new status code 
duplicating 403 semantics because people did not understand that on 
first reading of the Drafts.

Simply put 403 says two things:
   1) I checked your credentials, they were fine but you are not allowed 
access here. Go away.
   2) You are not allowed access here. I didn't even bother to check 
your credentials, they won't help. Go away.

  case (1) is when for example Corporate users machine is trying to 
access CEOs private pages. authn passes, authz failed. Only the CEO 
machine allowed to access the CEOs content.

  case (2) is when for example External IP attacker is trying to access 
internal-only resources. IP or something non-authn is the primary authz 
check and 403 prevents credentials leakage to the attacker.

  Either way authz failed AND the server policy desires the user 'never' 
to re-try.

Important note: this above is an explicit server policy, not a protocol 
mandatory response.

AYJ

---

From: Julian Reschke <[email-masked]>
Date: Thu, 21 Jun 2012 10:18:54 +0200

> We've consistently used "ought to" to give advice and encourage certain behaviours without making it a conformance requirement (thereby breaking existing implementations). The normative requirement…

Indeed.

Change applied with 
<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1681>.

Best regards, Julian

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 29 Jun 2012 16:52:58 +0100

Hi Mark,

> Likewise, requests that require authentication by proxies that omit credentials, or contain invalid or partial credentials SHOULD return a 407 (Proxy Authentication Required) response. The response…

This reads better, yes.