Subject: #177: Realm required on challenges

From: Mark Nottingham <[email-masked]>
Date: Sun, 24 Jul 2011 14:20:48 -0400

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/177>

Proposal: 

1) put the text on realms into a separate subsection (editorial suggestion), and

2) in p7, change:

>   The authentication parameter realm is defined for all authentication schemes:

to:

"""
The authentication parameter 'realm' is reserved for use by authentications schemes that wish to indicate the scope of protection:
""" 

3) drop this sentence:

> The realm directive (case-insensitive) is required for all authentication schemes that issue a challenge.

--
Mark Nottingham   http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Sun, 24 Jul 2011 21:12:03 +0200

>> The realm directive (case-insensitive) is required for all authentication schemes that issue a challenge.

+1

Proposed patch: 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/177/177.diff>

New text of Section 2 (after putting in more structure):

2.  Access Authentication Framework

2.1.  Challenge and Response

    HTTP provides a simple challenge-response authentication mechanism
    that can be used by a server to challenge a client request and by a
    client to provide authentication information.  It uses an extensible,
    case-insensitive token to identify the authentication scheme,
    followed by a comma-separated list of attribute-value pairs which
    carry the parameters necessary for achieving authentication via that
    scheme.

      auth-scheme    = token
      auth-param     = token "=" ( token / quoted-string )

    The 401 (Unauthorized) response message is used by an origin server
    to challenge the authorization of a user agent.  This response MUST
    include a WWW-Authenticate header field containing at least one
    challenge applicable to the requested resource.

    The 407 (Proxy Authentication Required) response message is used by a
    proxy to challenge the authorization of a client and MUST include a
    Proxy-Authenticate header field containing at least one challenge
    applicable to the proxy for the requested resource.

      challenge   = auth-scheme 1*SP 1#auth-param

       Note: User agents will need to take special care in parsing the
       WWW-Authenticate and Proxy-Authenticate header field values
       because they can contain more than one challenge, or if more than
       one of each is provided, since the contents of a challenge can
       itself contain a comma-separated list of authentication
       parameters.

       Note: Many browsers fail to parse challenges containing unknown
       schemes.  A workaround for this problem is to list well-supported
       schemes (such as "basic") first.

    A user agent that wishes to authenticate itself with an origin server
    -- usually, but not necessarily, after receiving a 401 (Unauthorized)
    -- MAY do so by including an Authorization header field with the
    request.

    A client that wishes to authenticate itself with a proxy -- usually,
    but not necessarily, after receiving a 407 (Proxy Authentication
    Required) -- MAY do so by including a Proxy-Authorization header
    field with the request.

    Both the Authorization field value and the Proxy-Authorization field
    value consist of credentials containing the authentication
    information of the client for the realm of the resource being
    requested.  The user agent MUST choose to use one of the challenges
    with the strongest auth-scheme it understands and request credentials
    from the user based upon that challenge.

      credentials = auth-scheme ( token
                                / quoted-string
                                / #auth-param )

    If the origin server does not wish to accept the credentials sent
    with a request, it SHOULD return a 401 (Unauthorized) response.  The
    response MUST include a WWW-Authenticate header field containing at
    least one (possibly new) challenge applicable to the requested
    resource.

    If a proxy does not accept the credentials sent with a request, it
    SHOULD return a 407 (Proxy Authentication Required).  The response
    MUST include a Proxy-Authenticate header field containing a (possibly
    new) challenge applicable to the proxy for the requested resource.

    The HTTP protocol does not restrict applications to this simple
    challenge-response mechanism for access authentication.  Additional
    mechanisms MAY be used, such as encryption at the transport level or
    via message encapsulation, and with additional header fields
    specifying authentication information.  However, such additional
    mechanisms are not defined by this specification.

    Proxies MUST forward the WWW-Authenticate and Authorization headers
    unmodified and follow the rules found in Section 4.1.

2.2.  Protection Space (Realm)

    The authentication parameter realm is reserved for use by
    authentication schemes that wish to indicate the scope of protection:

      realm       = "realm" "=" realm-value
      realm-value = quoted-string

    The realm value (case-sensitive), in combination with the canonical
    root URI (the scheme and authority components of the effective
    request URI; see Section 4.3 of [Part1]) of the server being
    accessed, defines the protection space.  These realms allow the
    protected resources on a server to be partitioned into a set of
    protection spaces, each with its own authentication scheme and/or
    authorization database.  The realm value is a string, generally
    assigned by the origin server, which can have additional semantics
    specific to the authentication scheme.  Note that there can be
    multiple challenges with the same auth-scheme but different realms.

    The protection space determines the domain over which credentials can
    be automatically applied.  If a prior request has been authorized,
    the same credentials MAY be reused for all other requests within that
    protection space for a period of time determined by the
    authentication scheme, parameters, and/or user preference.  Unless
    otherwise defined by the authentication scheme, a single protection
    space cannot extend outside the scope of its server.

2.3.  Authentication Scheme Registry

    The HTTP Authentication Scheme Registry defines the name space for
    the authentication schemes in challenges and credentials.

    Registrations MUST include the following fields:

    o  Authentication Scheme Name

    o  Pointer to specification text

    Values to be added to this name space are subject to IETF review
    ([RFC5226], Section 4.1).

    The registry itself is maintained at
    <http://www.iana.org/assignments/http-authschemes>.

Feedback appreciated, Julian

---

From: "Manger, James H" <[email-masked]>
Date: Mon, 25 Jul 2011 10:47:01 +1000

Julian,

The concept of a "protection space" is quite important (eg for automatically applying credentials), regardless of whether or not a 'realm' parameter is present. Unfortunately, the proposed patch to make 'realm' optional also effectively makes a protection space optional. How about changing the 1st sentence of the 2nd paragraph of 2.2 "Protection Space (Realm)" to the following:

  A protection space is defined by the canonical root URI (...)
  of the server being accessed, in combination with the realm
  value if present.

The ABNF for <credentials> is still wrong, but I guess that is being tracked by a separate ticket [http://trac.tools.ietf.org/wg/httpbis/trac/ticket/195].

The ABNF for <challenge> should probably be changed in a similar way to <credentials>: accept a base64 blob as an alternative to comma-separated attribute-value pairs. That would match widespread existing behaviour with the NTLM scheme.

--
James Manger

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Julian Reschke
Sent: Monday, 25 July 2011 5:12 AM

Proposed patch: 

      auth-scheme    = token

...

...

...

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 13:39:00 +1200

Hi

firstly, thanks for the expanded text around realm, it makes it a lot 
clearer what it is for.

It does raise an immediate issue for proxy auth though.  Realm in the 
context of the authority + Scheme doesn't make sense for proxy auth.

If a proxy must provide a realm with any challenge, (BTW NTLM ignores 
this), then what is the base effective URI?  How can a proxy indicate 
that proxy creds can be re-used for access to the entire internet?

We may need to propose different restrictions.  For instance on the net 
it would be very dangerous to be able to specify a wild-card or 
universal realm (e.g which encompasses more than one site) due to 
credential leakage issues, but for a proxy that's exactly what you want.

Or, the realm for proxy auth must only apply to the proxy itself (which 
isn't stated anywhere).

In any case, the client will need to treat the realm differently for 
proxy auth vs auth to an O-S.

Yet another reason why intercepting connections is a bad idea.

Adrien

On 25/07/2011 7:12 a.m., Julian Reschke wrote:
>> The authentication parameter 'realm' is reserved for use by 
>> authentications schemes that wish to indicate the scope of protection:
>> """
>>
>> 3) drop this sentence:
>>
>>> The realm directive (case-insensitive) is required for all 
>>> authentication schemes that issue a challenge.
>
> +1
>
> Proposed patch: 

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 beta out now - http://www.wingate.com/getlatest/

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 25 Jul 2011 15:00:53 +1200

> credential leakage issues, but for a proxy that's exactly what you want.

AIUI, WWW-Auth and Proxy-Auth are defined explicitly with distinct 
end-to-end and hop-by-hop requirements to prevent exactly that leakage. 
There is no leak problem unless the implementation is non-compliant or 
misconfigured.

There is the edge case we hit occasionally. Where two chained proxies 
require unique Proxy-Auth credentials from the client. Admin obsession 
with end-to-end single-signon appears to be avoiding this problem for 
now on most networks. But these per-user chaining configurations are 
still being asked about in our user base occasionally. It is reasonable 
to assume they are being implemented ... somehow.

> Or, the realm for proxy auth must only apply to the proxy itself (which

Section 4 paragraph 2, sentence 1 seems to be that.

"
    Unlike Authorization, the Proxy-Authorization header field applies
    only to the next outbound proxy that demanded authentication using
    the Proxy-Authenticate field.
"

That and the wording about consuming the header. Implies any realm 
stated on it is proxy/hop specific. No leaks.

> Yet another reason why intercepting connections is a bad idea.

What has that got to do with this topic? any halfway sane interceptor 
won't touch auth at all. The insane ones break things regardless of what 
gets specified.

AYJ

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 15:30:35 +1200

On 25/07/2011 3:00 p.m., Amos Jeffries wrote:
> end-to-end and hop-by-hop requirements to prevent exactly that 
> leakage. There is no leak problem unless the implementation is 
> non-compliant or misconfigured.
>
> still being asked about in our user base occasionally. It is 
> reasonable to assume they are being implemented ... somehow.
>

How does a proxy state (using Realm)

"use those creds for any site you access through me"

if the base URL must be combined with the realm.  Unless you can say 
realm ="../../*" or something.

The problem is

a) the proxy MUST provide a realm
b) the realm must be combined with the base URI

> won't touch auth at all. The insane ones break things regardless of 
> what gets specified.

Therefore there are a lot of insane network admins out there who insist 
on the ability to

a) intercept connections (so they don't need to configure a proxy in the 
client); AND
b) authenticate those connections at the intercepting proxy

sure it sucks, but when you turn on automatic logon in IE, it actually 
kinda works.  Sure it's less work to just configure the browser to use 
the proxy and you get a better experience.

Problem is there are a bunch of proxy vendors (ourselves included) who 
support intercepted connections, because that's what users think they 
want.  We no longer recommend it, but others do, and on the face of it, 
the benefits seem worth-while, and therefore often form part of a 
purchasing decision.

---

From: "Yngve N. Pettersen" <[email-masked]>
Date: Mon, 25 Jul 2011 05:53:52 +0200

> b) the realm must be combined with the base URI

The solution is IMO simple: The "base URI" for the proxy is its hostname  
and port. That is how it is implemented in Opera.

Alternatively, the WWW and Proxy auth headers define different types of  
scopes, where the WWW variant is realm + base URI, and the proxy one is  
realm+hostname+port; the effect is the same.

The WWW and proxy auth systems have, by design, different scopes; which is  
why they are using different headers. The definitions for how they are  
used need to take that into account.

-- 
Sincerely,
Yngve N. Pettersen

********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 25 Jul 2011 18:21:01 +1200

> "use those creds for any site you access through me"

Your argument appears to assume the credentials are going to a "site" 
itself. Which is only true for origins. Proxy-auth are stripped by the 
proxy.

The proxy relevant version of your statement is:
  "use those creds for any access through me"

   Proxy-Authenticate: Foo realm="e"

The 'm' part of "me" is built into the browsers knowledge of what proxy 
IP:port or FQDN:port it is contacting.

To form two-part combo distinguishing "m"+"e" from "m"+"y landlords" and 
"m"+"r joe blogs" proxies. All chained and accessed through the proxy 
gateway "m".

> realm ="../../*" or something.

You are thinking the base URL has to be inside the realm string? I don't 
read the RFC that way. It makes no statement at all of _how_ things have 
to be combined. Merely that they need to be. (Though it does not say MUST).

As Yvnge mentioned, when talking to a proxy the service site and scope 
is proxy-IP:port. Adding the particular scheme realm= value to that 
generates a scope for the credentials.

ie three-factor authentication (TCP data, realm, credentials). With two 
factors beyond the browsers ability to forge.

> b) authenticate those connections at the intercepting proxy

Yes there are. Fortunately they can be educated. All it takes is a clear 
explanation of the security leak introduced by handing your private 
credentials to unknown third-parties. I've ended this email with the 
effective example I usually use.

> kinda works.

Really? what browsers respond to Proxy-Auth challenges when they 
explicitly contacted the origin directly?

Note that NTLM does not authenticate HTTP. But only authenticates the 
client end of the TCP link, sometimes the machine rather than user. So 
when the DC trusts the proxy the actual source of the credentials become 
irrelevant. NTLM and Kerberos credentials can be (and often are) 
injected by the middleware without the browser actually sending any 
credentials at all.

> Sure it's less work to just configure the browser to use

I share that hat. Yet my view here apparently varies.

"intercepted connections" or "intercepted connections _by_ an 
authenticating proxy"?
  Lets be clear on that before I ask you for your credit card pin 
number. Would you give it to me on demand? or would I have to try asking 
your bank?
  The same security mechanisms are in play here. With the same 
categories of possible checks and loopholes.

  I could intercept your phone line and hear you telling it to someone 
you trusted. Or you could talk about ponies all day and I'd know squat. 
Or you could pass the phone to a friend halfway and I get theirs by 
mistake.
  Even so, Would you answer my voice if I interrupted and asked halfway 
through your conversation with someone else?

So I restate; no halfway sane interceptor touches auth. The half-sane 
ones might try and log it, might even background verify it. But not 
challenge or depend on it.

AYJ

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 19:34:39 +1200

On 25/07/2011 6:21 p.m., Amos Jeffries wrote:
> The 'm' part of "me" is built into the browsers knowledge of what 
> proxy IP:port or FQDN:port it is contacting.
>
> To form two-part combo distinguishing "m"+"e" from "m"+"y landlords" 
> and "m"+"r joe blogs" proxies. All chained and accessed through the 
> proxy gateway "m".
>
sorry, what I meant was

"use these creds to auth to me (your proxy for today) for any site you 
access through me".

The creds I'm talking about aren't the site creds, but the proxy creds.

Nowhere in RFC2616 or the proposed amendmend does it state that in the 
case of authenticating TO a proxy the client should do something 
different in terms of deriving values for the Authority

> You are thinking the base URL has to be inside the realm string? I 
> don't read the RFC that way. It makes no statement at all of _how_ 
> things have to be combined. Merely that they need to be. (Though it 
> does not say MUST).

nope, it's appended to the base URI.

> generates a scope for the credentials.

that's not in the spec

>
> ie three-factor authentication (TCP data, realm, credentials). With 
> two factors beyond the browsers ability to forge.
>
>>
>> The problem is
>>
> Yes there are. Fortunately they can be educated. All it takes is a 
> clear explanation of the security leak introduced by handing your 
> private credentials to unknown third-parties. I've ended this email 
> with the effective example I usually use.

most countries spend a significant portion of GDP on educating their 
populace.  It's not cheap.  So, saying we can simply educate clients 
about things isn't correct, it's not simple or cheap.

> explicitly contacted the origin directly?

all of them do.  They don't know they are being intercepted by a proxy.  
They just think the site challenged them.

There's a setting in IE that allows it to try authing to all and sundry 
with current windows credentials.  This then bypasses the logon dialog 
(which otherwise clients pop many times for a page - 1 for each 
different site)

> client end of the TCP link, 

???  It's used to associate credentials with the connection between 
client and the other end of the NTLM handshake, whether that's a proxy 
or O-S.

> sometimes the machine rather than user. So when the DC trusts the 
> proxy the actual source of the credentials become irrelevant. 

It's still quite useful to be able to identify who made a request.

The source of the credentials is the human who typed in their password 
when they logged into windows / their domain.

> NTLM and Kerberos credentials can be (and often are) injected by the 
> middleware without the browser actually sending any credentials at all.

sure.

> "intercepted connections" or "intercepted connections _by_ an 

the latter.

> number. Would you give it to me on demand? or would I have to try 
> asking your bank?
> you trusted. Or you could talk about ponies all day and I'd know 
> squat. Or you could pass the phone to a friend halfway and I get 
> theirs by mistake.

who told you about the ponies!!?!!?

> challenge or depend on it.

judge not lest ye be judged.  There are plenty of people intercepting to 
an authenticating proxy.

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 25 Jul 2011 19:42:43 +1200

On 25/07/2011 7:34 p.m., Adrien de Croy wrote:
> all of them do.  They don't know they are being intercepted by a 
> proxy.  They just think the site challenged them.

sorry - to clarify, we don't challenge an intercepted connection with 
407 + Proxy-Authorize.  That would be pointless since the client has no 
knowledge of the proxy.

We of course respond with 401 + WWW-Authorize.

Adrien

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 26 Jul 2011 00:21:35 +1200

> We of course respond with 401 + WWW-Authorize.

I wondered. Having seen plenty of proxy-auth challenges get silently 
rebuffed by the client agents.

So, in this case you are not an authenticating proxy. But a proxy 
claiming to _be_ the origin. And are thus bound by the origin permitted 
behaviour. www-auth being *this* domain, should be naturally bound to 
the domain by one end or the other, hopefully both.

(I take it you know the usual result when this is tried? a constant 
barrage of auth popups by the browser.)

That also explains your (apparent) mistake in "use those creds for any 
site you access through me".  Your text implied that it was a proxy 
question. Not an origin question being posed by a lying proxy. And you 
are right. Its intentionally not easy to pose that question if it was 
bound firmly to domain+realm.
  Sounds like good security intentionally getting in the way of a nasty 
practice. I like it.

AYJ

---

From: Julian Reschke <[email-masked]>
Date: Mon, 25 Jul 2011 15:48:10 +0200

>    of the server being accessed, in combination with the realm

OK; new proposed patch: 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/177/177.diff>. 
The subsection would then read:

    A protection space is defined by the canonical root URI (the scheme
    and authority components of the effective request URI; see Section
    4.3 of [Part1]) of the server being accessed, in combination with the
    realm value if present.  These realms allow the protected resources
    on a server to be partitioned into a set of protection spaces, each
    with its own authentication scheme and/or authorization database.
    The realm value is a string, generally assigned by the origin server,
    which can have additional semantics specific to the authentication
    scheme.  Note that there can be multiple challenges with the same
    auth-scheme but different realms.

Best regards, Julian

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 26 Jul 2011 12:16:18 +1200

Actually this whole thread is a diversion.  I'm not concerned about the 
spec in relation to intercepting.

My concern is how to specify the base URI when it's proxy auth (not 
intercepted).

The definitions of that refer to the site being requested, not the proxy 
itself.

On 26/07/2011 12:21 a.m., Amos Jeffries wrote:
> claiming to _be_ the origin. And are thus bound by the origin 
> permitted behaviour. www-auth being *this* domain, should be naturally 
> bound to the domain by one end or the other, hopefully both.
>

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Jul 2011 17:47:48 +0200

On 2011-07-25 15:48, Julian Reschke wrote:
>> The concept of a "protection space" is quite important (eg for
>> automatically applying credentials), regardless of whether or not a
>> 'realm' parameter is present. Unfortunately, the proposed patch to
>> make 'realm' optional also effectively makes a protection space
>> optional. How about changing the 1st sentence of the 2nd paragraph of
>> 2.2 "Protection Space (Realm)" to the following:
>>

...applied with 

Best regards, Julian