Subject: HTTPS, proxy environment variables and non-CONNECT access

From: Robert Collins <[email-masked]>
Date: Tue, 16 Jul 2013 18:08:31 +1200

So [fairly recently] squid and other proxies can retrieve resources
over HTTPS. However user agents generally don't take advantage of
this, instead using CONNECT, to do end to end encryption.

One of the ramifications of saying we'd like to be able to do explicit
recommendations for a proxy [which I know is a longer term thing] is
that clients need to be /able/ to do HTTPS via their proxy - e.g.
client <- https -> proxy <- https -> origin, and/or client <- http ->
proxy <- https -> origin.

Today, for unix UAs, https_proxy when set results in a connection to
the named proxy and then a CONNECT verb.

I'd like to start getting UAs to stop doing the CONNECT verb. but
redefining the behaviour of https_proxy seems super risky. So -
looking for input (and here is as good a place as any, so please
excuse the technically off-topic thread) on how to signal that.

We could add a new variable to control the behaviour of https_proxy :
e.g. https_proxy_mode=[connect|native] with unrecognised or absent
values treated as 'connect'.

Or we could add a new variable e.g. https_trusted_proxy=<URI> and have
that take priority.

Does anyone have:
 - a better place to raise this
 - or thoughts on this?

I'm sure that implementing this will start to raise issues like 'how
do we signal client certificates indirectly' and so on, which *will*
be HTTP protocol issues, but one step at a time.

-Rob

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Tue, 16 Jul 2013 10:35:11 +0200

> this, instead using CONNECT, to do end to end encryption.

Is there a spec somewhere on how it's supposed to work ?

> be HTTP protocol issues, but one step at a time.

Some more questions:
1. How do you protect the client <-> proxy link?
2. how do you send auth from the client to the proxy in a secure way
without it leaking them outside? (some http_proxy users just add proxy
auth headers everywhere even when the proxy didn't ask for them, in basic
auth, so they are leaking secrets to the outside like sieves)
3. more generally how are the client and proxy supposed to distinguish
between client <-> proxy signaling and client <-> web site signaling ?
4. Is proxy chaining possible? (I've seen proxy used both to authorize
connexions to the outside, and as gateway for connexions inside. So how
can a poor user that needs access to a resource protected by and
Internet-to-inside proxy traverse his own inside-to-Internet gateway to
reach it?)

Regards,

-- 
Nicolas Mailhot

---

From: Robert Collins <[email-masked]>
Date: Tue, 16 Jul 2013 21:52:36 +1200

> Is there a spec somewhere on how it's supposed to work ?

Just HTTP/1.1 - http://www.apps.ietf.org/rfc/rfc2616.html#sec-5.1.2
-  The absoluteURI form is REQUIRED when the request is being made to
a proxy. The proxy is requested to forward the request or service it
from a valid cache, and return the response. Note that the proxy MAY
forward the request on to another proxy or directly to the server

ask for an HTTPS URI and the proxy should get the HTTPS URI for you;
early HTTP proxies couldn't make requests to HTTPS sites, nor could
they listen on HTTPS, both issues have been fixed some time ago.

> 1. How do you protect the client <-> proxy link?

It's up to the client I think. Some obvious options:
 - use an HTTPS connection to the proxy.
 - use mandatory IPSEC within your network
 - have the proxy be on your local machine

> without it leaking them outside?

I think you mean 'If the origin is an HTTPS origin which uses
replayable (e.g. basic) auth, how do you prevent that leaking [vs e.g.
how do you authenticate to the proxy itself]. So for that you'd want
to have the client switch to CONNECT if the client -> proxy link is
subject to observation and any of the following are true:
 - there are HTTPS only cookies
 - it wants to try a replayable auth mechanism

But the response might deliver an HTTPS only cookie, which suggests
that we should mandate only doing this when either a) the client
doesn't care about having some traffic observed or b) the client ->
proxy link is secured (either at transport level of IPSEC etc).

> (some http_proxy users just add proxy

Basic auth is terrible in so many ways :).

> between client <-> proxy signaling and client <-> web site signaling ?

Thats covered by HTTP already.

> Internet-to-inside proxy traverse his own inside-to-Internet gateway to

There's no facility for tunnelling *auth* from a client through
multiple proxies in HTTP today. It might be interesting to contemplate
but I think it's orthogonal.

Remember that I proposing an optional thing folk can opt into, when it
makes sense for their environment.

For instance - I have a proxy on my laptop, traffic to it is entire
unobservable by other people, and it would make an excellent shared
cache for the growing number of sites (e.g. pypi) that are https-only.

-Rob

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Tue, 16 Jul 2013 12:07:44 +0200

Le Mar 16 juillet 2013 11:52, Robert Collins a écrit :

> how do you authenticate to the proxy itself].

No, I really meant "how do you prevent web site auth leaking proxy-side,
and proxy auth leaking web site-side, without assuming one of those auths
is worthless and can be shared or exposed non-encrypted in the name of
cutting corners". And that in a world where the only auth most web clients
will use reliably is basic auth.

-- 
Nicolas Mailhot

---

From: Robert Collins <[email-masked]>
Date: Tue, 16 Jul 2013 22:15:21 +1200

> will use reliably is basic auth.

If a proxy forwards Proxy-Auth headers on it is either a deliberate
strategy for working in a proxy hierarchy, or buggy as hell :). So I'm
not worried about proxy auth leaking web site-side, it's a non-problem
(or a problem already present in the use of proxies, so unrelated to
the use of proxies to obtain https entities.

The web site auth leaking proxy-side aspect I answered in my prior email.

As for dealing with broken web clients - sure, that is a real concern,
but I have no ideas beyond good specs, and filing bugs/not using bad
clients.

-Rob