Subject: why not WPAD?

From: Peter Lepeska <[email-masked]>
Date: Wed, 15 Jan 2014 14:09:14 -0500

Salvatore's recent draft on trusted proxies
(http://www.ietf.org/internet-drafts/draft-loreto-httpbis-trusted-proxy20-00.txt)
presents one approach for browsers to learn about the presence of
proxies, even when the browser is first using HTTPS to talk to the
Internet.

But WPAD already exists for this purpose and all of the browsers
support it in one form or another -- chrome recently added support for
WPAD over DHCP as I understand it. I know there are implementation
problems with WPAD and proxy autoconfig but fundamentally what is
wrong with the approach of leveraging DHCP and DNS to discover proxies
and then relying on a simple javascript-based script to determine when
the proxy should be used?

Is there something fatally flawed about the WPAD/PAC model for dynamic
proxy detection? If this topic is covered in another thread, please
send me a link to it.

Thanks,

Peter

---

From: Martin Thomson <[email-masked]>
Date: Wed, 15 Jan 2014 11:42:55 -0800

On 15 January 2014 11:09, Peter Lepeska <[email-masked]> wrote:
> javascript-based

I think that's one problem.  Not every HTTP client comes with a
JavaScript interpreter.

---

From: Eliot Lear <[email-masked]>
Date: Wed, 15 Jan 2014 22:24:23 +0100

Peter,

Without addressing your question specifically, who do you trust?  If the
information comes off DHCP do you trust the local network
administrator?  What if your device is mobile?  What if it's in
Starbucks?  If we're talking about DNS-based WPAD, perhaps a configured
domain that one trusts is more interesting, especially if you can play
proximity games...

Eliot

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 15 Jan 2014 21:39:42 +0000

Hi Peter

in general, WPAD involves up to 4 different systems.

DHCP
DNS
http server for WPAD.dat URL
client (must be configured to use auto proxy detect)

then there's the Proxy

this is 4 places for failure in the WPAD setup.

We find in practise deploying WPAD to be very problematic for customers. 
  If however they could divert ports via the proxy, there's 1 system to 
enforce and advertise the requirements for connection, and it's the 
proxy.  Therefore the proxy vendor has complete ability to develop and 
deploy all the necessary bits.  Neither is it dependent on client 
config.

Since clients may start their browsing with https, therefore there needs 
to be a way within TLS to advertise this.  So actually I think the 
approach is a very good one, and stands to make life a great deal easier 
for all my customers in any case.

Adrien

---

From: Peter Lepeska <[email-masked]>
Date: Wed, 15 Jan 2014 17:38:18 -0500

Eliot,

Maybe the problem is that today we not only use WPAD to detect a
proxy, but we also use it to determine whether or not to use the
proxy. Or, as you say, to trust it to look at our traffic. Maybe those
two actions -- discovery and decision to trust -- should be separate.
In the context of the trusted eproxy discussion, I think we're all
assuming that there will be an additional trust decision made by the
user, or by an administrator in the enterprise case. So I'm asking the
question of whether WPAD is an okay mechanism, perhaps with some
enhancements, to give us the proxy discovery piece.

In that case, do you still see a problem with WPAD if it is only
responsible for discovery?

Peter

---

From: Peter Lepeska <[email-masked]>
Date: Wed, 15 Jan 2014 18:39:12 -0500

Adrien,

That makes sense to me -- it's overly complex to deploy and therefore
prone to failure. As far as needing the client to be configured to use
proxy auto detect, that might also be the case for the TLS-based proxy
detection scheme. Also, the TLS-based scheme requires an in-path
proxy, which sounds like its not an issue for your customers. And it's
also not an issue for mine, though I think the ability to deploy out
of path opens up some interesting possibilities. I wonder if you'd
feel the same way about WPAD if implementors had fully committed to it
-- if browsers implemented the spec consistently with auto proxy
detect enabled most of the time. Inconsistent implementation due to
lack of commitment might also turn out to be a problem with the
TLS-based scheme.

I'm still not sure that WPAD is fundamentally worse. And plus it already exists.

Peter

On Wed, Jan 15, 2014 at 4:39 PM, Adrien de Croy <[email-masked]> wrote:
> We find in practise deploying WPAD to be very problematic for customers.  If
> however they could divert ports via the proxy, there's 1 system to enforce
> and advertise the requirements for connection, and it's the proxy.
> Therefore the proxy vendor has complete ability to develop and deploy all
> the necessary bits.  Neither is it dependent on client config.
>
> Since clients may start their browsing with https, therefore there needs to
> be a way within TLS to advertise this.  So actually I think the approach is
> a very good one, and stands to make life a great deal easier for all my
> customers in any case.
>
> Adrien
>
>
>
>
>

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 15 Jan 2014 23:59:53 +0000

Hi Peter

actually IME the client config is the smallest problem.  Usually it's 
the DHCP, DNS or WPAD.dat http server.

DHCP points to the WPAD URL.  So, client needs to be using DHCP for 
starters.  Name in the WPAD URL needs to be resolvable to the correct 
target http server.  Target of the URL needs to be running (and on the 
correct port), and have the WPAD.dat file to serve.  WPAD.dat file needs 
to be correct (no Javascript errors) and point to the correct proxies / 
ports etc.  Initial setup and change-management is burdensome and 
error-prone.

So I'm not sure complete buy-in by client vendors really would help much 
with this.  To get a properly integrated system to do all this 
automatically would require WPAD-smart webservers that can talk to DNS 
servers and DHCP servers.  DHCP (apart from AD) doesn't have an external 
mechanism to drive it.

Another issue  with WPAD compared to a per-connection 
intercept/enforcement scheme is that it's one-off.  The client performs 
WPAD discovery at the start, and has a static config.  After that, it 
won't alter behaviour.

If OTOH you are doing something per-connection, you have the ability to 
alter/tailor behaviour. For example don't bug the client for certain 
destination IPs, or source IPs, time of day etc. etc.  And this has the 
opportunity to greatly reduce the number of systems involved and 
opportunity for errors.

Adrien

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 16 Jan 2014 14:04:26 +1300

On 2014-01-16 12:59, Adrien de Croy wrote:
> correct port), and have the WPAD.dat file to serve.  WPAD.dat file
> needs to be correct (no Javascript errors) and point to the correct
> proxies / ports etc.  Initial setup and change-management is
> burdensome and error-prone.
> 
> So I'm not sure complete buy-in by client vendors really would help
> much with this.  To get a properly integrated system to do all this
> servers and DHCP servers.  DHCP (apart from AD) doesn't have an
> external mechanism to drive it.

DHCP is effectively optional. If all clients supported the DNS 
mechanism(s) then DHCP could be dropped and proxies simply use a web 
server API to update the WPAD.dat files served by the relevant domain 
server(s).

NP: with an appropriate server script the Squid proxy send_announce 
mechanism can already to this type of auto-registration.

> intercept/enforcement scheme is that it's one-off.  The client
> performs WPAD discovery at the start, and has a static config.  After
> that, it won't alter behaviour.
> 

This is a bigger problem than DHCP. Even the proxy auto-registration 
cannot avoid this one.

On the other hand something as simple as clients properly obeying any 
Cache-Control/Expires headers on the GET response and re-fetching 
wpad.dat when it expires would go a long way towards fixing this issue.

> If OTOH you are doing something per-connection, you have the ability
> to alter/tailor behaviour. For example don't bug the client for
> certain destination IPs, or source IPs, time of day etc. etc.  And
> this has the opportunity to greatly reduce the number of systems
> involved and opportunity for errors.

Biggest problem of all IME is the lack of clients supporting WPAD in any 
form. It is competing with MITM and loosing solely due to this lack of 
implementations. At the end of the day a network admin correctly 
implementing all possible WPAD mechanisms still only captures a small 
amount of the client traffic. Many are deciding the effort is not worth 
the gain and pick simple MITM despite its flaws.

This one will only have a chance at being resolved by fixing the other 
issues and standardizing WPAD.

Any other known problems with WPAD?

Amos

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 16 Jan 2014 01:20:20 +0000

------ Original Message ------
From: "Amos Jeffries" <[email-masked]>

>server API to update the WPAD.dat files served by the relevant domain 
the issue with DNS, is that the URI for the WPAD.dat file becomes fixed 
to http://wpad/wpad.dat

This may or may not work very well for people.

The benefit of DHCP I guess is it allows an admin to specify different 
WPAD URIs to different clients, and also more easily host the WPAD.dat 
file, since the admin can specify the full URI to it.

For many years, WinGate resolved any DNS lookup for WPAD to it's own IP, 
and auto-generated the WPAD.dat file with internal data, and served it 
from the www proxy.

Even then, clients still have problems.

>This is a bigger problem than DHCP. Even the proxy auto-registration 
possibly, unless we made it able to cope with this, e.g by allowing a 
proxy to assert that a client must re-configure itself or re-discover 
the proxy or whathaveyou.

Also, it is interesting considering the opportunity for a different 
entity than the proxy (e.g. border router) doing the interception and 
challenging.  This would then allow for features like load balancing 
etc.

>wpad.dat when it expires would go a long way towards fixing this issue.
Doesn't cope well with outages.  Until the WPAD.dat file expires, the 
client will keep trying the same proxy.

I guess we could all agree that if the proxy was uncontactable that the 
client should re-do proxy auto-detection.

>Biggest problem of all IME is the lack of clients supporting WPAD in 
>any form. It is competing with MITM and loosing solely due to this lack 
>of implementations. At the end of the day a network admin correctly 

MITM (and in this case we're talking about plain old TCP interception 
for plaintext http) is the easiest way for an admin to really enforce 
compliance even if it means some things just don't work.  It's one stop, 
you do it in one place and it's done.  By the time it's in play, all the 
other hurdles (e.g. DNS resolution, routing etc) have been cleared.

Adrien

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 16 Jan 2014 10:43:15 +0100

> proxy detection? If this topic is covered in another thread, please

As far as I know there are several problems:

1. javascript: much too flexible, badly supported by web clients,
difficult to get right (needs a more restrictive subset that can do less
but sanely). I know I will kill cisco voip client should I reorder our
pacfile (they choke on some js directives and cisco refused to fix the
clients)

2. lack of trusted proxy link and browser support : many security teams
will disable by default wpad as there is no technical way to identify
trusted proxies and browsers apply wpad silently without giving any
feedback to the user about it (making hijacking trivial)

3. wrong layer : it's not in the http exchanges, it's not in the firewall
that blocks not-proxified accesses, it's in yet another equipment making
deployment hell (and Firefox still refused to do anything with dhcp wpad).
The notification really needs to happen in the equipment that refuses not
proxified accesses (or redirects to the proxy), in the protocol the web
client uses to reach this equipment

4. quite unspecified web client & site behaviour in the presence of
several potential proxy paths in the pacfile. Balancing is hell.

Regards,

-- 
Nicolas Mailhot

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 16 Jan 2014 10:54:22 +0100

> domain that one trusts is more interesting, especially if you can play

The fact is, you should not have to trust "here is the proxy" hint or not.
A correct security model is "do you trust the proxy or not", regardless of
how you discovered it

Regards,

-- 
Nicolas Mailhot

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 16 Jan 2014 11:09:22 +0100

> the DHCP, DNS or WPAD.dat http server.

The DNS and DHCP parts are big problems. The http one is not. Right now an
embedded http server in the proxy is a de facto requirement to communicate
with web clients, since there is no web client <-> proxy signalling layer
in http/1

> won't alter behaviour

That's a huge problem you can't notify clients a proxy went down before
they restart. Some users like to keep browsers open ad vitam eternam. Some
browsers keep stuck on the dns resolution as it happened during the
pacfile read, even if the dns of some web sites has changed since.

> opportunity to greatly reduce the number of systems involved and

That does not work so well if only part of the traffic is proxified. With
pacfile you can tell web client "here are internal websites, you can reach
them directly, don't bother with me" and in big corporations internal
accesses are the bulk of the traffic, forcing it to a per-connexion proxy
node will degrade availability and scalability.

(and sometimes a different proxy gateway sits before a private
non-internet link with a partner)

Regards,

-- 
Nicolas Mailhot

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 16 Jan 2014 11:31:31 +0000

------ Original Message ------
From: "Nicolas Mailhot" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "Peter Lepeska" <[email-masked]>; "[email-masked]" 
<[email-masked]>
Sent: 16/01/2014 23:09:22
Subject: Re: why not WPAD?

>>performs
>>to

What I meant was really the opportunity to do something each connection 
rather than necessarily doing it.

After the client knows it must use a proxy, it will not be directly 
connecting via the router that was issuing this challenge.  Normally 
also intranet traffic need not go via this either so you avoid proxying 
intranet.

So I guess you have the same failover issues as WPAD.  But you wouldn't 
want to add another few RTs plus proxy estabilshment for each 
connection.

Adrien

---

From: Peter Lepeska <[email-masked]>
Date: Thu, 16 Jan 2014 10:19:37 -0500

Taking a step back from the details of this discussion a little bit,
the fundamental difference between WPAD and the TLS-based scheme
proposed in Salvatore's draft (as well as standard MITM) is
out-of-path versus in-path. The pros and cons of each flow from this
difference, I think.

And it sounds like the prevailing opinion (based on the experience of
the few who have posted to this thread) is that in-path is better for
reasons given above. At the same time we're trying to design an
improved "explicit proxy" for HTTP 2. But that term usually implies
configuring the address and port of a proxy explicitly in the browser,
as opposed to using network interception to discover the proxy. It
seems like we're moving towards a new model of an explicit,
intercepting proxy that makes the user aware of its presence but is
deployed in-path.

Peter

On Thu, Jan 16, 2014 at 6:31 AM, Adrien de Croy <[email-masked]> wrote:
> connecting via the router that was issuing this challenge.  Normally also
> intranet traffic need not go via this either so you avoid proxying intranet.
>
> So I guess you have the same failover issues as WPAD.  But you wouldn't want
> to add another few RTs plus proxy estabilshment for each connection.
>
> Adrien
>
>
>>