Subject: absolute URI in request to origin servers which are also proxies.

From: Adrien de Croy <[email-masked]>
Date: Tue, 02 Jun 2009 14:15:32 +1200

There is an ambiguity brought about by the requirements of RFC2616 to 
require origin servers to support absolute URI in requests (RFC2616 s 
19.6.1.1).

If you consider the case of a proxy which is also an origin server, then 
you get a problem when the proxy receives a request for a URI for which 
the proxy is an origin server, and that request contains an absolute 
URI, and the proxy requires the client to be authenticated.

In such cases, it's impossible to tell whether to send a 
Proxy-Authenticate, or a WWW-Authenticate header in the response, since 
you can't tell from the syntax of the request whether the client 
believes it is talking to a proxy or an origin server.

I've never seen a client send an absoluteURI in a request to an origin 
server, so the obvious solution is to ignore RFC 19.6.1.1, and base the 
decision on the semantics of the request.

However that doesn't seem like a great option if clients are one day 
going to start send requests to origin servers with absolute URIs.

Should this requirement be deprecated?  Otherwise how should this case 
be handled?  A proxy that can also be an origin server needs to know 
whether the client believes it's talking to a proxy or not for auth and 
other reasons (e.g. default handling of origin-server requests for a 
proxy vs proxy requests).

Regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: David Morris <[email-masked]>
Date: Mon, 1 Jun 2009 19:21:42 -0700 (PDT)

On Tue, 2 Jun 2009, Adrien de Croy wrote:

> There is an ambiguity brought about by the requirements of RFC2616 to require 
> origin servers to support absolute URI in requests (RFC2616 s 19.6.1.1).
>
> If you consider the case of a proxy which is also an origin server, then you 
> get a problem when the proxy receives a request for a URI for which the proxy 
> is an origin server, and that request contains an absolute URI, and the proxy 
> requires the client to be authenticated.
>
> In such cases, it's impossible to tell whether to send a Proxy-Authenticate, 
> or a WWW-Authenticate header in the response, since you can't tell from the 
> syntax of the request whether the client believes it is talking to a proxy or 
> an origin server.

Huh ... a proxy which can't differentiate based on local configuration 
whether content is local or proxied is fundamentally flawed and I suspect 
has the potential for security issues. The shape of the URL should not 
influence how the server differentiates handling.

> decision on the semantics of the request.

NO ... the decision must be made based on the configuration of the proxy 
software.  Having the client able to influence the choice makes no sense.

> However that doesn't seem like a great option if clients are one day going to 
> start send requests to origin servers with absolute URIs.
>
> Should this requirement be deprecated?  Otherwise how should this case be 
> handled?  A proxy that can also be an origin server needs to know

It doesn't need to know what the client believes. I frankly think it 
foolish to combine the functions in a single software installation, But
the combined server needs external configuration to know what it is 
willing to do for the client.

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 02 Jun 2009 14:40:28 +1200

I think you'll find there are a lot of proxy products which also act as 
origin servers for whatever reason.  Admin sites, or even reverse 
proxying are common examples of this.

There were semantic differences in HTTP/1.0 between "proxy" and "server" 
requests - e.g. those which contained an absoluteURI or not.

This distinction was removed in 1.1, by requiring origin servers to 
accept absoluteURI in a request.

however the distinction is still required in cases such as auth.  
Configuration doesn't make any difference for this.  the proxy / server 
can no longer tell what the appropriate response should be.

Take a simple example.  Proxy at A is also a server for www.example.com, 
but requires authentication for access to that site and also requires 
proxy users to auth.  Proxy A receives a request from a client on the 
internal network for

GET http://www.example.com HTTP/1.1
Host: www.example.com
etc

Should the proxy send back  a 407 or a 401?  should it send 
Proxy-Authenticate or WWW-Authenticate?

How should it tell what to do?

This isn't a configuration issue.

Adrien

David Morris wrote:
>> If you consider the case of a proxy which is also an origin server, 
>> then you get a problem when the proxy receives a request for a URI 
>> for which the proxy is an origin server, and that request contains an 
>> absolute URI, and the proxy requires the client to be authenticated.
>>
>> Proxy-Authenticate, or a WWW-Authenticate header in the response, 
>> since you can't tell from the syntax of the request whether the 
>> client believes it is talking to a proxy or an origin server.
>
> whether content is local or proxied is fundamentally flawed and I 
> suspect has the potential for security issues. The shape of the URL 
> should not influence how the server differentiates handling.
>
>> I've never seen a client send an absoluteURI in a request to an 
>> origin server, so the obvious solution is to ignore RFC 19.6.1.1, and 
>> base the decision on the semantics of the request.
>
> NO ... the decision must be made based on the configuration of the 
> proxy software.  Having the client able to influence the choice makes 
> no sense.
>
>> Should this requirement be deprecated?  Otherwise how should this 
>> case be handled?  A proxy that can also be an origin server needs to 
>> know
>

---

From: David Morris <[email-masked]>
Date: Mon, 1 Jun 2009 19:55:50 -0700 (PDT)

On Tue, 2 Jun 2009, Adrien de Croy wrote:

> origin servers for whatever reason.  Admin sites, or even reverse proxying 
> are common examples of this.
>
> This distinction was removed in 1.1, by requiring origin servers to accept 
> absoluteURI in a request.
>
> Configuration doesn't make any difference for this.  the proxy / server can 
> no longer tell what the appropriate response should be.
>
> Take a simple example.  Proxy at A is also a server for www.example.com, but 
> requires authentication for access to that site and also requires proxy users 
> to auth.  Proxy A receives a request from a client on the internal network 
> for
>

Yes it is ... in this example, it would see the request as local content 
since it was recognized as such and perform WWW-Authenticate, nothing else
would be sensible.

It must be able to recognize local content and it's own host name.

>>> the proxy is an origin server, and that request contains an absolute URI, 
>>> and the proxy requires the client to be authenticated.
>>> 
>>> you can't tell from the syntax of the request whether the client believes 
>>> it is talking to a proxy or an origin server.
>> 
>>> However that doesn't seem like a great option if clients are one day going 
>>> to start send requests to origin servers with absolute URIs.
>>> 
>> the combined server needs external configuration to know what it is willing 
>> to do for the client.
>> 
>
> --

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 02 Jun 2009 15:15:13 +1200

David Morris wrote:
>> I think you'll find there are a lot of proxy products which also act 
>> as origin servers for whatever reason.  Admin sites, or even reverse 
>> There were semantic differences in HTTP/1.0 between "proxy" and 
>> "server" requests - e.g. those which contained an absoluteURI or not.
>>
>> Configuration doesn't make any difference for this.  the proxy / 
>> server can no longer tell what the appropriate response should be.
>>
>> Take a simple example.  Proxy at A is also a server for 
>> www.example.com, but requires authentication for access to that site 
>> and also requires proxy users to auth.  Proxy A receives a request 
>> from a client on the internal network for
>>
> Yes it is ... in this example, it would see the request as local 
> content since it was recognized as such and perform WWW-Authenticate, 
> nothing else
> would be sensible.
OK, so we send back a 401, with WWW-Authenticate

But then what does the client do if it thought it was talking to a 
proxy?  I guess there's no problem, since auth will work through a proxy 
already...

So IOW the handling of the request is based on the proxy / servers 
knowledge of which sites it's an O-S for.

> It must be able to recognize local content and it's own host name.
understood.

What about the case where a proxy offers authenticated proxy services 
over the internet but is also an origin server.  If a request to an 
origin server can contain an absolute URI then the proxy can no longer 
tell whether it's being asked to be a proxy for that domain, or be a 
server for that domain.   If it does host the domain, then it can assume 
it's being an origin server without problem, but what if it doesn't host 
it?  Should it then presume it's being asked to be a proxy?

if it sends back a 407 (which could be reasonable), then a UA may choke 
on that if it thought it was making a request to an origin server.

I guess this isn't a big issue... but if relative URI requests die out, 
then this could become bigger.  Which makes me wonder, why after over 10 
years of RFC2616 doesn't any UA send absoluteURI in requests except when 
configured to talk to a proxy?

>>>> There is an ambiguity brought about by the requirements of RFC2616 
>>>> to require origin servers to support absolute URI in requests 
>>>> (RFC2616 s 19.6.1.1).
>>>>
>>>> for which the proxy is an origin server, and that request contains 
>>>> an absolute URI, and the proxy requires the client to be 
>>>> authenticated.
>>>>
>>> Huh ... a proxy which can't differentiate based on local 
>>> configuration whether content is local or proxied is fundamentally 
>>> flawed and I suspect has the potential for security issues. The 
>>> shape of the URL should not influence how the server differentiates 
>>> handling.
>>>
>>>> origin server, so the obvious solution is to ignore RFC 19.6.1.1, 
>>>> and base the decision on the semantics of the request.
>>>
>>> proxy software.  Having the client able to influence the choice 
>>> makes no sense.
>>>
>>>> However that doesn't seem like a great option if clients are one 
>>>> day going to start send requests to origin servers with absolute URIs.
>>>>
>>>> case be handled?  A proxy that can also be an origin server needs 
>>>> to know
>>>

---

From: David Morris <[email-masked]>
Date: Mon, 1 Jun 2009 21:37:43 -0700 (PDT)

On Tue, 2 Jun 2009, Adrien de Croy wrote:

> But then what does the client do if it thought it was talking to a proxy?  I 
> guess there's no problem, since auth will work through a proxy already...
>
> So IOW the handling of the request is based on the proxy / servers knowledge 
> of which sites it's an O-S for.
>
>> 
> What about the case where a proxy offers authenticated proxy services over 
> the internet but is also an origin server.  If a request to an origin server 
> can contain an absolute URI then the proxy can no longer tell whether it's 
> being asked to be a proxy for that domain, or be a server for that domain. 
> If it does host the domain, then it can assume it's being an origin server 
> without problem, but what if it doesn't host it?  Should it then presume it's 
> being asked to be a proxy?

Obviously.

>
> if it sends back a 407 (which could be reasonable), then a UA may choke on 
> that if it thought it was making a request to an origin server.

Doesn't much matter because something is seriously wrong if the client 
thinks it is talking to an origin server and the server it connects 
directly to as the origin for x.mydomain.com, presumably because the DNS 
the client is configure to resolve names with pointed at this server. 
Clients are already a bit weak in general in the decision process related 
to whether a host name is local and shouldn't proxy or is remote and 
should proxy so in this case we have a poorly configured DNS and/or client 
proxy selection script, etc. And we either get a 407 from the proxy which 
is an early clue to whomever the user gets tech support from, or they get
some other 4xx or 5xx error related to the server not existing.

>
> I guess this isn't a big issue... but if relative URI requests die out, then 
> this could become bigger.  Which makes me wonder, why after over 10 years of 
> RFC2616 doesn't any UA send absoluteURI in requests except when configured to 
> talk to a proxy?

My guess would be that this condition exists because the host: header / 
non-proxy URI form is known to work.