Subject: #78: Relationship between 401, Authorization and WWW-Authenticate

From: Mark Nottingham <[email-masked]>
Date: Sun, 24 Jul 2011 14:06:17 -0400

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/78>

Proposal:

1) Clarify that WWW-Authenticate can appear on any response, and that when it appears on any other than a 401, it means that the client can optionally present the request again with a credential.

and,

2) Clarify that an Authentication scheme that uses WWW-Authenticate and/or 401 MUST use the Authorization header in the request, because of its implications for caching. Schemes MAY specify additional headers to be used alongside it.

--
Mark Nottingham   http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 24 Jul 2011 20:11:38 +0200

> 1) Clarify that WWW-Authenticate can appear on any response, and that when it appears on any other than a 401, it means that the client can optionally present the request again with a credential.

Does this mean it's only for other 4xx or for any status ? It might have
implications with non-idempotent requests if a client can repost a request
that led to a 200 for instance.

> 2) Clarify that an Authentication scheme that uses WWW-Authenticate and/or 401 MUST use the Authorization header in the request, because of its implications for caching. Schemes MAY specify additio…

This looks important indeed !

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 24 Jul 2011 14:31:02 -0400

> that led to a 200 for instance.

Any status. Good point about non-idempotent requests; we'll need to make clear it's not about automatically retrying requests, but instead that sending the same request with credentials might have a different affect.

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 13:29:30 +1200

> Any status. Good point about non-idempotent requests; we'll need to make clear it's not about automatically retrying requests, but instead that sending the same request with credentials might have…

isn't this redundant?

I see requests with credentials all the time, when no previous 
WWW-Authorize had been sent in any response.

So clients are already taking any liberties they like to send 
credentials when they please.  I don't know that it adds anything to 
HTTP to explicitly tell them they may do this in protocol.  They are 
doing it anyway.

Otherwise are we going to prohibit the sending of creds when no 
WWW-Authorize had been sent?

Adrien
-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 beta out now - http://www.wingate.com/getlatest/

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 25 Jul 2011 15:12:35 +1200

On 25/07/11 13:29, Adrien de Croy wrote:
>>>> 1) Clarify that WWW-Authenticate can appear on any response, and
>>>> that when it appears on any other than a 401, it means that the
>>>> client can optionally present the request again with a credential.
>> Any status. Good point about non-idempotent requests; we'll need to
>> make clear it's not about automatically retrying requests, but instead
>> that sending the same request with credentials might have a different
>> affect.
>
> isn't this redundant?
>

I disagree on the redundant. This clarifies and documents the existing 
behaviour which must be handled by implementers to improve 
compatibility. Not all of us see these same requests you do.

> credentials when they please. I don't know that it adds anything to HTTP
> to explicitly tell them they may do this in protocol. They are doing it
> anyway.
>

Prohibiting that would be a bad idea. It is possible for the client 
agent to use non-HTTP means to verify the connection security 
requirements before sending credentials over it.

Likewise the mere presence of a WWW-Authorize challenge means nothing 
about the security of the channel. Despite NTLM assumptions to the contrary.

A mention in the security considerations should be enough there.

AYJ

---

From: "Manger, James H" <[email-masked]>
Date: Mon, 25 Jul 2011 13:19:37 +1000

> 1) Clarify that WWW-Authenticate can appear on any response, and that when it appears on any other than a 401, it means that the client can optionally present the request again with a credential.

Great.

> 2) Clarify that an Authentication scheme that uses WWW-Authenticate and/or 401 MUST use the Authorization header in the request, because of its implications for caching. Schemes MAY specify additio…

Not so great.

If an authentication mechanism uses the Authorization header then it benefits from some default caching rules. Good.
Plenty of other authentication mechanisms don't use that header, primarily because they operate at higher or lower layers of the protocol stack (eg forms, cookies, TLS...). Even in these cases a WWW-Authenticate response header can be a useful signal about the authentication options available. They may need to handle caching explicitly, but they can do that.

If anything needs to be said, perhaps something like the following could be appended to section 4.1 "Authorization":

  A server may need to explicitly indicate the cachability of responses
  if a request uses an authentication mechanism that does not involve
  the Authorization header.

--
James Manger

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 25 Jul 2011 06:40:46 +0200

On Mon, Jul 25, 2011 at 01:29:30PM +1200, Adrien de Croy wrote:
> >>>1) Clarify that WWW-Authenticate can appear on any response, and that 
> >>>when it appears on any other than a 401, it means that the client can 
> >>>optionally present the request again with a credential.
> >Any status. Good point about non-idempotent requests; we'll need to make 
> >clear it's not about automatically retrying requests, but instead that 
> >sending the same request with credentials might have a different affect.
> 
> isn't this redundant?
> 

Normally they're doing this when they have seen once a 401 with
WWW-Authenticate. Here the idea is that the server might present
the header on something different from the 401 (eg: here is your
data, but if you authenticate I could probably serve something
more specific to you).

Regards,
Willy

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 19:37:03 +1200

> data, but if you authenticate I could probably serve something

actually we mostly see it when the client agent is a script, downstream 
proxy, tunneling (non-http) client or other non-browser.

They send the header from request #1.

Of course this ignores the issue about whether the method chosen is 
supported / advertised.  Normally it's Basic as well.

Adrien

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Jul 2011 23:54:07 +0200

On 2011-07-25 05:19, Manger, James H wrote:
> ...
> ...

Good catch. I knew we were missing something.

Maybe...:

Use of the Authorization header to transfer credentials implies 
"Cache-Control: private" [ref] and thus affects cacheability of 
responses. Thus, definitions of new authentication schemes that do not 
use "Authorization" will need to ensure that response messages do not 
leak in an unintended way, for instance by specifying "Cache-Control" or 
"Vary: *" [ref] explicitly.

Feedback appreciated,

Julian

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 26 Jul 2011 00:44:02 +0200

Hi Julian,

> "Vary: *" [ref] explicitly.

I can read the first sentence in two ways :
  - if a server or intermediary receives an Authorization header, it must
    assume that "Cache-Control: private" is implied
  - if a client wants to emit an Authorization header, it must also add
    a "Cache-Control: private" header

I think the former was meant given the second sentence, though I'm not
100% certain. If so, maybe we should focus on the recipient of the message
and replace "Use of" with "Presence of" (or anything equivalent).

The second part is clear enough however.

Regards,
Willy

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 26 Jul 2011 11:14:06 +1200

On Mon, 25 Jul 2011 23:54:07 +0200, Julian Reschke wrote:
>>> 2) Clarify that an Authentication scheme that uses WWW-Authenticate 
>>> and/or 401 MUST use the Authorization header in the request, because 
>>> of its implications for caching. Schemes MAY specify additional 
>>> headers to be used alongside it.
>>
>> Not so great.
>>
>> If an authentication mechanism uses the Authorization header then it 
>> benefits from some default caching rules. Good.
>> Plenty of other authentication mechanisms don't use that header, 
>> primarily because they operate at higher or lower layers of the 
>> protocol stack (eg forms, cookies, TLS...). Even in these cases a 
>> WWW-Authenticate response header can be a useful signal about the 
>> authentication options available. They may need to handle caching 
>> explicitly, but they can do that.
>>
>> If anything needs to be said, perhaps something like the following 
>> could be appended to section 4.1 "Authorization":
>>
> leak in an unintended way, for instance by specifying "Cache-Control"
> or "Vary: *" [ref] explicitly.
>
> Feedback appreciated,
>
> Julian

 Was about to suggest exactly that. Sound great.

 AYJ

---

From: "Manger, James H" <[email-masked]>
Date: Tue, 26 Jul 2011 10:13:11 +1000

>The second part is clear enough however.

The first sentence should be read a 3rd way:
  - if an Authorization header is present in a request, the corresponding
    response MUST be treated as though it includes "Cache-Control: private",
    unless it explicitly includes a Cache-Control header

draft-ietf-httpbis-p7-auth-15#section-4.1 already contains 20 lines of text (1 paragraph plus 3 dot points) about caching when a request includes an Authorization header. This shouldn't be paraphrased immediately after that text with the first sentence above "... implies Cache-Control: private...". I am not sure that the 20 lines are totally consistent with this first sentence.

Perhaps the existing 20 lines were going to be removed, to be replaced with a single sentence about implying "Cache-Control: private" by default? That sounds ok to me, as long as the first sentence makes it clear that "Cache-Control: private" is implied for the corresponding response.

Alternatively, if the existing 20 lines are kept, then just add the 2nd sentence of the Julian's text as a new paragraph at the end of section 4.1 [draft-ietf-httpbis-p7-auth-15#section-4.1]:

  Use of authentication schemes that do not 
  use "Authorization" will need to ensure that response messages do not 
  leak in an unintended way, for instance by specifying "Cache-Control" or 
  "Vary: *" [ref] explicitly.

--
James Manger

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 26 Jul 2011 02:38:37 +0200

>"Vary: *" [ref] explicitly.

This should refer to disclosure or something like that rather than leak-
age (you wouldn't design a protocol that intentionally leaks something),
and `Vary: *` strikes me as odd in this context (why, then, doesn't the
use of Authorization imply just `Vary: Authorization`, for instance).

I would rather say something along the lines that use of "Authorization"
implies that the message is confidential with respect to the credentials
provided in that header, meaning messages should be treated as if they
had `Cache-Control: private`, and that new schemes must take explicit
measures to ensure the confidentiality of messages, like using that same
header, because deployed servers are otherwise unaware of the semantics.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 26 Jul 2011 13:35:46 +1200

>     unless it explicitly includes a Cache-Control header

 That wording in turn implies "Cache-Control: something-new" reverts the 
 object to public access. Not a good idea IMHO to premise it on the 
 existing of the header field.
  "unless it explicitly includes Cache-Control: public" would be better 
 wording.

 AYJ

>
> draft-ietf-httpbis-p7-auth-15#section-4.1 already contains 20 lines
> of text (1 paragraph plus 3 dot points) about caching when a request
> includes an Authorization header. This shouldn't be paraphrased
> immediately after that text with the first sentence above "... 
> implies
> Cache-Control: private...". I am not sure that the 20 lines are
> totally consistent with this first sentence.
>
> Perhaps the existing 20 lines were going to be removed, to be
> replaced with a single sentence about implying "Cache-Control:
> private" by default? That sounds ok to me, as long as the first
> sentence makes it clear that "Cache-Control: private" is implied for
> the corresponding response.
>
> Alternatively, if the existing 20 lines are kept, then just add the
> 2nd sentence of the Julian's text as a new paragraph at the end of
> section 4.1 [draft-ietf-httpbis-p7-auth-15#section-4.1]:
>

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 26 Jul 2011 13:44:37 +1200

> use of Authorization imply just `Vary: Authorization`, for instance).

 I've been thinking along these lines recently too. It seems 
 Authorization, ETag, and maybe others should be implicit Vary: headers. 
 But that text seems to belong better in the definition of Vary: and 
 seems rather out of place in areas like auth.
  For instance, non-caching agents do not need to bother with Vary:, but 
 may want to do authentication. So there should be no need to make them 
 care about Vary: by mentioning a dependence on its syntax by 
 Authoritzation:.

 AYJ

---

From: Yutaka OIWA <[email-masked]>
Date: Tue, 26 Jul 2011 22:28:14 +0900

2011/7/25 Mark Nottingham <[email-masked]>:
> 1) Clarify that WWW-Authenticate can appear on any response, and that when it
> appears on any other than a 401, it means that the client can optionally
> present the request again with a credential.

Just for confirmation:
I remember we had some discussion about this years ago.
This change will break SPNEGO (see RFC 4559, Sec. 5 example)
and other other authentication schemes which uses
WWW-Authenticate on 200 as a carrier for authentication
exchanges, instead of Authentication-Info.
Is this incompatible change OK?
(I prefer this direction, though.)

And if this change text intends to introduce any opportunity
for optional authentication to HTTP at this time,
I think we need more detailed restrictions to make it really work.
If the intention is just to clarify header meanings and
leave the rest for future work, it is OK for me.

> 2) Clarify that an Authentication scheme that uses WWW-Authenticate and/or
> 401 MUST use the Authorization header in the request, because of its
> implications for caching. Schemes MAY specify additional headers to be used
> alongside it.

+1. Good way.

-- 
Yutaka OIWA, Ph.D.                                       Research Scientist
                            Research Center for Information Security (RCIS)
    National Institute of Advanced Industrial Science and Technology (AIST)
                      Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

---

From: Yutaka OIWA <[email-masked]>
Date: Tue, 26 Jul 2011 22:47:11 +0900

> leave the rest for future work, it is OK for me.

just FYI, the following is the list of required additional rules
to make optional auth work.

(1) The response for successful authentication MUST NOT contain
    any WWW-Authenticate: header.

(2) The response for failed authentication is RECOMMENDED to be
    401 status, even if a request for the same URL and method without
    Authorization: header will result in 200 status with WWW-Authenticate:
    header.

At least one of the above condition must be met, otherwise
clients cannot determine whether the authentication is successful or not.
Of course, the clause (1) will break some existing authentication scheme.

If interested, please also refer my Mutual authentication proposal
which also contains detailed rules (including those two above)
for realizing optional HTTP authentication.

If the #78 change intends to realize optional auth at this time,
I propose two above clauses to be included.
Otherwise, It's OK and I'll work on this later in future http-auth activity.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Jul 2011 20:29:11 +0200

> header, because deployed servers are otherwise unaware of the semantics.

Björn, thanks. To the point as always...

So:

"Use of the Authorization header to transfer credentials implies that 
the message is confidential with respect to the credentials provided in 
that header field, meaning response messages ought to be treated as if 
they had "Cache-Control: private", and that new authentication schemes 
will have to take explicit measure to ensure the confidentiality of 
messages, such as by using that very header, because deployed recipients 
are otherwise unaware of the semantics."

?

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 26 Jul 2011 21:04:30 +0200

> are otherwise unaware of the semantics."

Looks a lot better to me :-)

Best regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Jul 2011 21:29:28 +0200

> are otherwise unaware of the semantics."

Or even....:

"The credentials carried in an Authorization header field are specific 
to the User Agent, and therefore have the same effect on HTTP caches as 
the "private" Cache-Control response directive, within the scope of the
request they appear in.

Therefore, new authentication schemes which choose not to carry 
credentials in the Authorization header (e.g., using a newly defined
header) will need to explicitly disallow caching, by mandating the use 
of either Cache-Control request directives (e.g., "no-store") or 
response directives (e.g., "private")."

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jul 2011 15:43:05 -0400

On 26/07/2011, at 9:15 AM, Yutaka OIWA wrote:

> This change will break SPNEGO (see RFC 4559, Sec. 5)

Well, RFC4559 is already broken, because it makes assumptions about the relationship between messages in a connection. 

Regardless, I think we can word it in such a way that Negotiate isn't any more broken; people already know that they need to handle it differently.

> And if this change text intends to introduce opportunity
> I think we need more details and restrictions to make it work.

I think it's the latter.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jul 2011 15:45:15 -0400

Nice text; +1.

On 26/07/2011, at 3:29 PM, Julian Reschke wrote:
> Or even....:
> 
> "The credentials carried in an Authorization header field are specific to the User Agent, and therefore have the same effect on HTTP caches as the "private" Cache-Control response directive, within the scope of the
> request they appear in.
> 
> Therefore, new authentication schemes which choose not to carry credentials in the Authorization header (e.g., using a newly defined
> header) will need to explicitly disallow caching, by mandating the use of either Cache-Control request directives (e.g., "no-store") or response directives (e.g., "private")."

---

From: Yutaka OIWA <[email-masked]>
Date: Wed, 27 Jul 2011 04:53:36 +0900

Thanks,

> Regardless, I think we can word it in such a way that Negotiate isn't any more broken; people already know that they need to handle it differently.

I see, then I agree on your proposal.

Does anyone have a list of HTTP authentication schemes
(either RFC-defined of de-fact deployed) so that we can check
the whole list of to-be-differently-handled schemes?
If there is such a list, I (we) can work on making such checklist.
(unless it has 50 or 100 entries :-))

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Jul 2011 21:55:23 +0200

>      any WWW-Authenticate: header.

Not sure about that.

If we allow WWW-A on a non-authenticated 200 response, why not also on 
an authenticated one?

>      Authorization: header will result in 200 status with WWW-Authenticate:

I agree with this one, but, as Mark said, let's leave that to future work.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Jul 2011 22:05:00 +0200

> 2) Clarify that an Authentication scheme that uses WWW-Authenticate and/or 401 MUST use the Authorization header in the request, because of its implications for caching. Schemes MAY specify additio…

OK, proposed patch: 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/78/78.diff>

This adds the following point to the Considerations for new schemes:

    o  The credentials carried in an Authorization header field are
       specific to the User Agent, and therefore have the same effect on
       HTTP caches as the "private" Cache-Control response directive,
       within the scope of the request they appear in.

       Therefore, new authentication schemes which choose not to carry
       credentials in the Authorization header (e.g., using a newly
       defined header) will need to explicitly disallow caching, by
       mandating the use of either Cache-Control request directives
       (e.g., "no-store") or response directives (e.g., "private").

An updates the description of WWW-A to:

4.4.  WWW-Authenticate

    The "WWW-Authenticate" header field consists of at least one
    challenge that indicates the authentication scheme(s) and parameters
    applicable to the effective request URI (Section 4.3 of [Part1]).

    It MUST be included in 401 (Unauthorized) response messages and MAY
    be included in other response messages to indicate that supplying
    credentials (or different credentials) might affect the response.

      WWW-Authenticate = 1#challenge

    User agents are advised to take special care in parsing the WWW-
    Authenticate field value as it might contain more than one challenge,
    or if more than one WWW-Authenticate header field is provided, the
    contents of a challenge itself can contain a comma-separated list of
    authentication parameters.

Best regards, Julian

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 27 Jul 2011 08:11:02 +1200

apologies, but I'm still not convinced overloading a new function onto 
WWW-Authenticate is the best way to advertise the availability of 
optional authentication.

It creates an immediate dilemma for any UA that receives such a message.

What are the options for the UA, and how will they affect user experience?

If the UA always elects to proceed to auth, then it's the same as 
sending back a 401
if the UA tries to give the choice to the user, that's (IMO) asking for pain
otherwise the UA can ignore it, and it's just more bloat.

Also I just see it breaking a whole heap of agents who switch behaviour 
on the presence of that header (rather than the status).

Finally, we see UAs starting auth without this header in the first 
place.  So does this really need advertising anyway?

If this is to be new behaviour, shouldn't we use a new header or status? 
That way we can keep it out of the way.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jul 2011 16:18:04 -0400

On 26/07/2011, at 4:11 PM, Adrien de Croy wrote:

> apologies, but I'm still not convinced overloading a new function onto WWW-Authenticate is the best way to advertise the availability of optional authentication.
> 
> If the UA always elects to proceed to auth, then it's the same as sending back a 401
> Also I just see it breaking a whole heap of agents who switch behaviour on the presence of that header (rather than the status).
> 
> Finally, we see UAs starting auth without this header in the first place.  So does this really need advertising anyway?
> 
> If this is to be new behaviour, shouldn't we use a new header or status? That way we can keep it out of the way.

All we're doing is leaving the door open for the possibility in the future, explicitly; we're not requiring anything, and a future effort can figure out what the best thing to do is.

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 27 Jul 2011 11:56:51 +1200

On Tue, 26 Jul 2011 16:18:04 -0400, Mark Nottingham wrote:
>> apologies, but I'm still not convinced overloading a new function 
>> onto WWW-Authenticate is the best way to advertise the availability of 
>> optional authentication.
>>

 The server also does not know which of those two behaviours the client 
 UA will choose.

 This opens the door for the form-base auth currently being proposed in 
 other WG. Websites placing login forms on, for example, their front 
 page. User agents with credentials available already can login silently. 
 Those without can choose to happily display the non-authed page without 
 annoying the user overly much. If the user does fill out the form 
 credentials become available and the page view can change without 
 caching problems.

 With some extra UA support this is all possible now. Just requires a 
 few things outside the HTTP spec to happen. Which means when this is 
 implemented compatibility will be a problem.

>>
>> Also I just see it breaking a whole heap of agents who switch 
>> behaviour on the presence of that header (rather than the status).
>>
>> If this is to be new behaviour, shouldn't we use a new header or 
>> status? That way we can keep it out of the way.
>
>
> All we're doing is leaving the door open for the possibility in the
> future, explicitly; we're not requiring anything, and a future effort
> can figure out what the best thing to do is.
>

 +1 to foresight.

 AYJ

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Jul 2011 15:23:54 +0200

> <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/78/78.diff>

Applied as <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1360>.