Subject: Secure (https) proxy authentification

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 16 Feb 2012 15:36:47 +0100

Hi,

Now that browsers have started refusing redirection of https sessions, there
is no clean way for a proxy to point browsers to an https authentication
portal when they need to be authenticated or re-authenticated.

The 407 error must be extended to indicate the https proxy authentication
portal location to handle the cases where it is not desirable to have proxy
auth transmitted in clear, and clients are too dumb to support anything more
complex than basic auth over http or https.

(the other “solution” is DPI, but that's not really appealing except to proxy
aplicance manufacturers)

Best regards,

-- 
Nicolas Mailhot

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 16 Feb 2012 18:44:00 +0100

On Thu, Feb 16, 2012 at 03:36:47PM +0100, Nicolas Mailhot wrote:
> (the other ???solution??? is DPI, but that's not really appealing except to proxy
> aplicance manufacturers)

Well, this is one more reason for urging all browser vendors to support
proxying over https. This will put an end to this redirection madness
which prevents most HTTP agents from working in such environments (eg:
firefox cannot even update itself at a customer's due to such proxies,
so everyone uses outdated versions until they decide to download the full
image again).

Regards,
Willy

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Fri, 17 Feb 2012 21:20:09 +0100

> so everyone uses outdated versions until they decide to download the full

Thank you for your support
I believe fixing this hole in the spec is also needed to get working captive
portals (instead of the current half-working piles of bandaids that never
quite work correctly)

-- 
Nicolas Mailhot

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 19 Feb 2012 06:17:55 +1300

>>> complex than basic auth over http or https.

The counter side of that is whether such clients likely to be smart 
enough to do TLS to the proxy anyway?

You mention some recent change in browser behviour on redirect to 
CONNECT. Can you point me at some reference material for that change?

> portals (instead of the current half-working piles of bandaids that never

IME you will find all of us proxy folk in support of the idea. The 
browser folk seem strangely quite whenever it comes up.

AYJ

---

From: Henry Story <[email-masked]>
Date: Sat, 18 Feb 2012 18:29:53 +0100

> portal when they need to be authenticated or re-authenticated.

Hi Nicolas. I am working on WebID - an https protocol ( http://webid.info/spec ) -
so this sounds like it could be important to us. Do you have a pointer to 
explain the situation here in more detail? I am not sure what kind of redirects
get refused, for what reason, etc....

> (the other “solution” is DPI, but that's not really appealing except to proxy

Social Web Architect
http://bblfish.net/

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Sat, 18 Feb 2012 19:56:14 +0100

> get refused, for what reason, etc....

Hi Henry,

The problem as I understand it is pretty simple.

When you're managing an hotspot or corporate proxy you want to make sure only
authorized persons pass through. It's pretty easy to set up the network so
browsers have to pass through some control equipment to reach the internet.
However it's much more difficult to convince them to display some
authorization page so users can actually provide their credentials.

In both cases you need to handle generic browsers without any special
configuration, otherwise you can not support visitors (and they can use a wide
range of devices, from desktops to laptops, netbooks, tablets, the range
always increases).

So the usual workflow would be:

1. user with some browser-capable equipment arrives on the network (usually
you can not control physical access well enough to be sure every connecting
user is authorized to)
2. he types some address in his browser
3. network routes the request through the control equipment
4. the control equipment needs to know who the user is. In case of a visitor
it also probably needs to display some information to tell the user where to
get the required authorization. To get widest possible web client support,
it's usually done with http basic auth on an https portal page
5. the user fills in a valid user id and password
6. browser continues on the actual URL the user typed

In some cases it's even more complex since the authorization is not eternal or
the connection load is shared by several equipments that may be rebooted. In
that case setp 2. becomes:

2. the user is already engaged in some workflow on a web site (may have
started filling forms, have some http session state, etc)
3. network routes the request through the control equipment but the user auth
expired/was forgotten/the user never provided auth to this specific equipment
4. the control equipment needs to know/confirm who the user is (without
breaking the user workflow)
5. the user fills in a valid user id and password
6. browser continues whatever request triggered step 3 (forgetting forms users
may have spent quite a lot of time filling is not nice)

At step 2., the user is not typing the captive portal address (he might not
know it). To point the browser the right way, such equipments used to issue a
redirect. However that was never ideal (it's basically a man in the middle
attack) and browsers will not honour such redirects when the user is accessing
an https site since 2010 IIRC (I don't remember the CVE numbers, but every
major browser stopped honoring those redirects at about the same time) . So
now users get an empty page or a 'proxy does not like you' error page in the
middle of their https browsing

See also: https://en.wikipedia.org/wiki/Captive_portal

But the control equipments do not want to attack the user session. All they
want is to get the browser to display the captive portal auth page. All the
hacks detailed on the wikipedia page would never have been necessary if the
http protocol actually provided a standard way to get the browser to display
this page.

I'd really like the working group to define such a standard method. It
wouldn't be complex or difficult to implement in browsers, and it would solve
many actual problems now.

-- 
Nicolas Mailhot

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 19 Feb 2012 15:46:16 +1300

> 2. he types some address in his browser
<snip>
> major browser stopped honoring those redirects at about the same time) . So

Ah, that restricted to 302, 300 and unknown 3xx redirects AFAIK, which 
are unsafe or difficult to allow safe auto-redirect for.

> http protocol actually provided a standard way to get the browser to display

Most of the hacks on the wikipedia page are involved with getting the 
packets to the portals proxy software without making the browser aware 
that it exists. Simple proxy auto-configuration avoids all of these 
hacks. They all happen long before HTTP gets a byte in edgewise.

How do you propose HTTP spec updates to solve "users who first use an 
email client or other will find the connection not working without 
explanation"? This being the problem which covers intercepted port 443 
packets for HTTPS portal.

> many actual problems now.

Do you mean a mechanism like the status 303 (See Other), 305 (Use Proxy) 
and 511 (Network Authentication Required)?

Notice how 302 is not in that list despite Wikipedia noting that most 
portals use 302. That would seem to be one part of the underlying 
problem. The other part being browsers defaulting to disable WPAD 
support (needed for 303 to work) or treating 303 as 302 including those 
security protections.

The "correct" workflow for a captive portal depends on whether the 
packets are intercepted:
  511 with login "splash" page doing some form of side-band 
authentication, or
  305 whereon the browser uses the portal proxy and hopefully sends 
credentials

... or not intercepted:
  303 redirecting to a resource which emits 407 then on success 303 
redirects back to the original URI, whereon the browse sends credentials.

Note that only the 511 NAT intercept case involves changes to the spec, 
and that is being done as 
http://tools.ietf.org/id/draft-nottingham-http-new-status-04.txt.

AYJ

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Sun, 19 Feb 2012 11:22:52 +0100

Amos Jeffries writes:
> > (I don't remember the CVE numbers, but every

Yes, I totally understand the CVE logic, however this change broke most
authenticated proxy setups (and now that enterprises are finally retiring or
replacing ie6 they are stuck)

> hacks. They all happen long before HTTP gets a byte in edgewise.

This part does not require browser configuration, and proxy autoconfiguration
can be challenging on a huge corporate network. The main problem right now is
getting browser cooperation to display the authentication form.

> packets for HTTPS portal.

I don't have any bright idea there. Though if people continue migrating to
webmail, that will eventually be academic.

> and 511 (Network Authentication Required)?

511 is exactly what I need. I was not aware of it. Is it simplemented in any
browser yet? Where should I point the browser writers to get it implemented?

http://tools.ietf.org/id/draft-nottingham-http-new-status-04.txt ?

(except for the part where 511 generates a certificate error)

> portals use 302.

Most portals use 302, Bluecoat uses 307, and everyone is getting blocked by
browsers.

305 is useless for my needs as long as the RFC states that:

 Note: RFC 2068 was not clear that 305 was intended to redirect a
      single request, and to be generated by origin servers only.  Not
      observing these limitations has significant security consequences.

Anyway, 511 is much cleaner

> That would seem to be one part of the underlying

The RFC does state:

Note: Many pre-HTTP/1.1 user agents do not understand the 303
      status. When interoperability with such clients is a concern, the
      302 status code may be used instead

Thank you for your help!

BTW : it would be nice if 7.3.6. 7.3.7. 7.4.8. referenced the error 511 (and
if error 511 description included the proxy keyword) or people won't find it.

-- 
Nicolas Mailhot

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Tue, 21 Feb 2012 09:58:16 +0100

> http://tools.ietf.org/id/draft-nottingham-http-new-status-04.txt ?

I take that back. 511 is almost exactly what we need. However, when I pointed
the authors of some of the tools that pass through our proxy to it (curl, git)
they told me they could not parse html code in their tools, so they really
need a location (or similar) field containing the address of the
authentication portal to communicate it to the user. Without this field, they
can only stop with 'Network authentication is needed' instead of 'Please open
<url> in your browser to proceed'.

http://article.gmane.org/gmane.comp.version-control.git/191085
http://article.gmane.org/gmane.comp.version-control.git/191087
http://article.gmane.org/gmane.comp.version-control.git/191086

(the nearest thing there is in the spec is the url in meta, but it's only in
the example, not mandatory, and no one will write code for something they can
not be sure will exist)

We'd like to support those tools properly as their users' previous clumsy
attempts to navigate our current non-standard redirection method resulted in
internal security investigations.

It is a problem in our setup as we only block some URLs (others are allowed
transparently without auth), and we use several proxy farms in different
physical sites (to avoid spofs). So just opening any url in a browser won't
trigger an authentication request (the url may not be blocked, or the browser
may pass through a gateway where the user IP is already authorized, while
git/etc tried to access through another one).

Could you please revise the error 511 definition to add such a field ?

Regards,

-- 
Nicolas Mailhot

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Feb 2012 10:28:33 +0100

> <url>  in your browser to proceed'.

Yes. The definition of status code 511 did not attempt to solve more 
problems than that.

> Could you please revise the error 511 definition to add such a field ?

The specification has already been approved, so it's too late to make 
more than editorial changes.

Best regards, Julian

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Tue, 21 Feb 2012 10:44:45 +0100

> more than editorial changes.

I know it has been approved, and we'll go with it as is for now on our network.
However, could it be extended to add this location field later please? (taking
into account the tool authors' feedback)

Best regards,

-- 
Nicolas Mailhot