Subject: p1: Via and gateways

From: Mark Nottingham <[email-masked]>
Date: Sat, 20 Apr 2013 14:07:11 +1000

p1 Section 2.3 says:

> However, an HTTP-to-HTTP gateway that wishes to interoperate with third-party HTTP servers must conform to HTTP user agent requirements on the gateway's inbound connection and must implement the Connection (Section 6.1) and Via (Section 5.7.1) header fields for both connections.

This means that accelerators and CDNs MUST generate a Via header on the outbound connection. This isn't widely practiced, and I'm not sure it's necessary. Comments?

--
Mark Nottingham   http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 20 Apr 2013 08:23:39 +0200

> This means that accelerators and CDNs MUST generate a Via header on the outbound connection. This isn't widely practiced, and I'm not sure it's necessary. Comments?

I know no load-balancer which does it anyway. Especially in hosted
environments where it is desired that the infrastructure is as much
transparent to the hosted servers as possible.

I must say I never understood the rationale behind Via because for
incoming traffic we don't care and for outgoing traffic we don't
want to disclose to the world our inside details.

Another example of a MUST which makes people think that MUSTs are at user
option I think.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 20 Apr 2013 16:44:30 +1000

> want to disclose to the world our inside details.

Yes. It makes sense for proxies, so that the endpoints can discover the capabilities of the whole path. I'm not convinced that applies to gateways, because they're taking the responsibility of the origin server.

> Another example of a MUST which makes people think that MUSTs are at user

Indeed.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 19 Apr 2013 23:47:45 -0700

Via makes sense for proxies that don't act with the authority of either
end-point, and are likely speaking to public-internet IPs on both sides.
There are relatively few of these that would bother to add Via given that
they're likely transparent proxies and increasing packet size is likely a
pain.

A loadbalancer (reverse proxy) wouldn't do Via, because it IS the website,
for all intents and purposes, and what happens behind it (even if it is
just HTTP proxying) is simply part of generating the response as far as the
rest of the world is concerned.
It is unlikely that one would wish to expose the inner workings of one's
private network by exposing this information in a Via (that goes for
proxies that are part of the client's network too...)

So, ++ to what Willy said.
-=R

On Fri, Apr 19, 2013 at 11:23 PM, Willy Tarreau <[email-masked]> wrote:

> > > However, an HTTP-to-HTTP gateway that wishes to interoperate with
> third-party HTTP servers must conform to HTTP user agent requirements on
> the gateway's inbound connection and must implement the Connection (Section
> 6.1) and Via (Section 5.7.1) header fields for both connections.
> >
> > This means that accelerators and CDNs MUST generate a Via header on the
> outbound connection. This isn't widely practiced, and I'm not sure it's
> necessary. Comments?
>

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 20 Apr 2013 08:51:05 +0200

On Sat, Apr 20, 2013 at 04:44:30PM +1000, Mark Nottingham wrote:
> Yes. It makes sense for proxies, so that the endpoints can discover the
> capabilities of the whole path. I'm not convinced that applies to gateways,
> because they're taking the responsibility of the origin server.

But are client and servers really making use of the Via header to discover
this ?

My impression is that nowadays proxies tend to build messages looking the
closest to what they receive and will either mimmick the HTTP version of
the proxied request, or arrange for the required conversions (eg: chunked-
encoding to content-length or close).

I think Amos might have some experience on this.

Willy

---

From: David Morris <[email-masked]>
Date: Sat, 20 Apr 2013 00:21:52 -0700 (PDT)

On Sat, 20 Apr 2013, Mark Nottingham wrote:

> > third-party HTTP servers must conform to HTTP user agent requirements
> > on the gateway's inbound connection and must implement the Connection
> > (Section 6.1) and Via (Section 5.7.1) header fields for both
> > connections.
> 

I don't care about MUST, but I think the Via header can be useful for
problem determination. A smart content server could also adjust for
a detected accelerator and/or transcoder ... perhaps by avoiding
optimizations dependant on a direct connection and byte/byte transfer
between the client and the server.

So I'm very much in favor of keeping the Via: header.

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 20 Apr 2013 09:25:09 +0200

> So I'm very much in favor of keeping the Via: header.

OK but Mark is not discussing about removing it but keeping the MUST
since a number of existing products already don't add it and will not
for some of the reasons mentionned in this thread.

So I think that based on your feedback this MUST should be turned into
a SHOULD.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 20 Apr 2013 17:55:59 +1000

> So I'm very much in favor of keeping the Via: header.

Definitely not talking about getting rid of it. The (only, specific) point here is whether a gateway that doesn't add Via to responses should be called non-conformant.

Personally, I think it should be a MUST for proxies, in both directions. However, for a gateway, it either shouldn't be a requirement at all (for responses), or it should be a SHOULD with a get-out clause for reasons of security (along with a note that they'll need to accept responsibility for any issues caused by omitting Via). Still should probable be a MUST for requests from gateways.

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 20 Apr 2013 10:09:19 +0200

On Sat, Apr 20, 2013 at 05:55:59PM +1000, Mark Nottingham wrote:
> Definitely not talking about getting rid of it. The (only, specific) point
> here is whether a gateway that doesn't add Via to responses should be called
> non-conformant.
> 
> Personally, I think it should be a MUST for proxies, in both directions.
> However, for a gateway, it either shouldn't be a requirement at all (for
> responses), or it should be a SHOULD with a get-out clause for reasons of
> security (along with a note that they'll need to accept responsibility for
> any issues caused by omitting Via). Still should probable be a MUST for
> requests from gateways.

But then do we want to declare all gateways non-conformant ? The only gateway
I've seen use the Via header was abusing it to put the client's IP address
into it, and none of the hosted servers behind it were ever confused by this
despite being a few tens of various products!

The only use I see with Via is to convey the original message's HTTP version,
but most (all?) gateways do not change this version because it's already
painful to be a gateway, you generally don't want to add more burden by
changing the protocol version between the two sides!

Willy

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 20 Apr 2013 23:46:01 +1200

> I think Amos might have some experience on this.

I do indeed.

Not so much capability detection, but path properties detection. The 
important _capabilities_ tend to be the per-hop ones as mentioned and 
I've yet to see anything using it to handle Expect properly.

Property detection is another story entirely...

Squid currently uses Via as the primary tool for detecting forwarding 
loops which can (and still frequently do) ocur at any time, for any 
number for reasons outside of HTTP itself (NAT misconfiguration, absence 
of split-DNS pointing the proxy at itself or a parent as origin, 
misinformed load balancer, intercepting proxy, etc)
In all these cases the request header is the important one and sending 
it from an intermediary to the server regardless of whether the server 
is believed to be another intermediary or an origin is the key part of 
Via usage. I will vote for keeping at least the request Via as a MUST 
send condition and any HTTP intermediary not sending it should be kicked 
for violation and putting its host machine at risk of that nasty DoS thingy.

I've not seen the reply path Via: having much usage to date. But IMHO 
with the growing usage of interception proxies for port 443 both 
directions will grow in importance as a method of signalling that the 
protocol is end-to-end secure (or not). Note that a lot of 
vendor-specific hacks such as Frontend-HTTPS, Forwarded-For, X-Protocol, 
X-HTTPS etc are simply duplicating information which is supposed to be 
relayed in Via protocol-name field.

FYI: we will shortly be building some extra functionality into Squid to 
make use of these signals for added security in HTTPS and long-term I 
intend to ensure that FTP ICY, gopher etc are properly labeled in Via by 
Squid to showcase and demonstrate the proper usage. No idea at this 
point if that latter part will be of any use to clients but you never 
know - some fancy validation mechanism may want to use it.

Amos

---

From: David Morris <[email-masked]>
Date: Sat, 20 Apr 2013 08:17:31 -0700 (PDT)

On Sat, 20 Apr 2013, Mark Nottingham wrote:

> Definitely not talking about getting rid of it. The (only, specific)
> point here is whether a gateway that doesn't add Via to responses should
> be called non-conformant.

I didn't think you were making that suggestion, but I see comments about
lack of use turning into suggestions for removal.

> responses), or it should be a SHOULD with a get-out clause for reasons
> of security (along with a note that they'll need to accept
> responsibility for any issues caused by omitting Via). Still should
> probable be a MUST for requests from gateways.

I'm not sure how to express it, but the requirement probably should be
based on whether the traffic leaves the control of the organization that
owns the gateway (or even proxy). I think Willy Tarreau noted a
performance concern for some types of gateways if the requirement is
added to modify the message.

Dave Morris

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 21 Apr 2013 13:10:12 +1200

> changing the protocol version between the two sides!

IMO, that is behaviour compliant with the Via semantics in the last 
paragraph(s) of section 5.7.1 of the latest drafts text. A series of 
labels can be compacted down to one label if they are all conveying the 
same version information. A load balancer, SOCKS gateways, magic devices 
only have to ensure they are accurately preserving the traffic behaviour 
used by their sources at each end in order to be compliant under that 
loophole.

The ones I've seen doing HTTP<->HTTPS translation have largely been 
adding some form of "X-HTTPS: on" header anyways, so asking them to use 
the Via as standard mechanism instead would seem not to be a burden.

Section 5.7.1 are a little obtuse and hide this case a bit. So that 
section could perhapse be clarified to explain that hops preserving the 
protocol semantics are not obliged to *add* to Via (although they are 
still required to relay any existing one).

However, all said and done I'm for retaining the sentence in p1 section 
2.3 as-is.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Apr 2013 16:08:14 +1000

Now:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/460

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 23 Apr 2013 08:43:25 +0200

Hi Amos,

> still required to relay any existing one).

I think that could make sense, especially with the loop detection you
explained in your other mail. It's true that proxies are clearly at
risk of looping due to external misconfigurations and I understand
why it makes sense to use it in your case. So I believe that what makes
a different use case of it is the sensibility of the component to its
environment. For example, a proxy such as Squid will rely on the
correspondance between the Host header field and the DNS configuration
so it's totally at risk of looping. A reverse-proxy such as a load
balancer will just use its own static configuration and is not exposed
to this case.

So when someone deploys haproxy in front of squid, it's OK if haproxy
does not add Via as long as it does not remove it.

I know that Roberto has a good point of view about reverse proxies, he
regularly says that they can be seen as part of the server (in that they
are totally useless alone). I think it probably is the correct way of
describing it, as in the example above, the haproxy+squid example can
be seen as a larger proxy and the Via header field is still added by
this "component".

> However, all said and done I'm for retaining the sentence in p1 section 

That said, Mark's initial question is still valid. Considering that many
existing components don't add it anyway, do we really need to declare them
non-compliant ?

I'm now thinking that maybe the goal of this Via header field needs to be
described prior to saying what must be done with it, so the two first
sentences should be swapped :

5.7.1 Via
   The "Via" header field is used in HTTP for tracking message forwards,
   avoiding request loops, and identifying the protocol capabilities of
   all senders along the request/response chain. The "Via" header field
   MUST be sent by a proxy or gateway in forwarded messages to indicate
   the intermediate protocols and recipients between the user agent and
   the server on requests, and between the origin server and the client
   on responses.

Then I would propose this addition :

   Multiple Via field values represent each proxy or gateway that has
   forwarded the message.  Each recipient MUST append its information
   such that the end result is ordered according to the sequence of
-  forwarding applications.
+  forwarding applications. A gateway MAY simply relay any existing Via
+  header field if it does not change the HTTP version, but it MUST NOT
+  remove it.

What do you think ?

Willy

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 24 Apr 2013 01:08:14 +1200

> does not add Via as long as it does not remove it.

And so long as the HTTP version is the same yes.

> existing components don't add it anyway, do we really need to declare them

IMO, for most no, for some which are changing the version and thus 
expected semantics yes and it would be a Good Thing to do so in those cases.

> +  header field if it does not change the HTTP version, but it MUST NOT

Sounds good to me.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 24 Apr 2013 02:09:31 +1200

On 20/04/2013 11:46 p.m., Amos Jeffries wrote:
>>>>>> third-party HTTP servers must conform to HTTP user agent 
>>>>>> requirements on the gateway's inbound connection and must 
>>>>>> implement the Connection (Section 6.1) and Via (Section 5.7.1) 
>>>>>> header fields for both connections.
>>>>> This means that accelerators and CDNs MUST generate a Via header 
>>>>> on the outbound connection. This isn't widely practiced, and I'm 
>>>>> not sure it's necessary. Comments?

> vendor-specific hacks such as Frontend-HTTPS, Forwarded-For, 
> X-Protocol, X-HTTPS etc are simply duplicating information which is 
> supposed to be relayed in Via protocol-name field.

The interesting corollary to come out of all the above is that HTTPS/1.1 
protocol-name is never defined or registered anywhere that I can find. 
So its use will be somewhat non-standard.

The use-case for its need is a reverse-proxy receiving port 443 traffic 
for an unsecured backend or one contacted via means other that TLS over TCP.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 24 Apr 2013 13:27:45 +1000

This is already covered further down:

> A proxy or gateway may combine an ordered subsequence of Via header field entries into a single such entry if the entries have identical received-protocol values.

The question I was raising was specific: can we relax the requirement for a gateway to add Via to responses if there isn't already one present?

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 24 Apr 2013 08:38:10 +0200

On Wed, Apr 24, 2013 at 01:27:45PM +1000, Mark Nottingham wrote:
> > A proxy or gateway may combine an ordered subsequence of Via header field
> > entries into a single such entry if the entries have identical
> > received-protocol values.

I don't read it exactly the same way, but probably it achieves the same in
the end.

> The question I was raising was specific: can we relax the requirement for a
> gateway to add Via to responses if there isn't already one present?

I think yes, we can relax it, as if there was none, then it means the
gateway has not changed the connection's behaviour and the next hop just
assumes the connection comes from the client. Again, in my understanding,
Via is used to understand what transformation was operated on the path
and what capabilities are offered. If the gateway does not change anything,
Via offers no value except detecting loops. And since most gateways simply
forward to the configured next hop, the risk of loop is not caused by
external environment (eg: DNS) but by the configuration so it's not a
problem.

But if we do so, we must specify that a gateway MUST NOT remove the
Via header field in any case, otherwise it will break the loop detection
mechanism of some other components.

Regards,
Willy