Subject: I-D draft-petersson-forwarded-for-00.txt

From: Andreas Petersson <[email-masked]>
Date: Thu, 7 Apr 2011 11:32:17 +0200

Hi,

First, maybe a short introduction of who I am. I work for Opera
Software with Opera Mini in the mini server team in Linköping, Sweden.
Now, to business.

I recently submitted a draft for standardizing a replacement for the
X-Forwarded-For header field. 

Please read the draft and come with feedback.

Regards, 
 Andreas Petersson

---

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title           : Forwarded-For HTTP Extension
	Author(s)       : A. Petersson, M. Nilsson
	Filename        : draft-petersson-forwarded-for-00.txt
	Pages           : 8
	Date            : 2011-04-05

This document standardizes an HTTP extension header field that allows
proxy components to disclose the originating IP number of a request.
Given a trusted path of proxying components, each subsequent
component will have access to all IP numbers used in the chain of
proxied HTTP requests.

This document also standardizes ways for a proxy possessor to
anonymize the origin of a request.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-petersson-forwarded-for-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
<ftp://ftp.ietf.org/internet-drafts/draft-petersson-forwarded-for-00.txt>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 07 Apr 2011 09:43:33 +0000

>Please read the draft and come with feedback.

One: Can we please settle on a single IPv6 representation, and make the 
square brackets mandatory in all cases, not just when there is
a port number present ?

Two: shouldn't this be folded into httpbis part 6

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 07 Apr 2011 12:14:23 +0200

On 07.04.2011 11:43, Poul-Henning Kamp wrote:
> In message<20110407113217.5662559a@hetzer>, Andreas Petersson writes:
>

Without having looked: how about just sticking to the notation in RFC 
3986 (<http://greenbytes.de/tech/webdav/rfc3986.html#rfc.section.3.2.2>)?

> Two: shouldn't this be folded into httpbis part 6

No, for procedural reasons (we're just revising, not adding anything new).

Best regards, Julian

---

From: Andreas Petersson <[email-masked]>
Date: Thu, 7 Apr 2011 13:13:43 +0200

On Thu, 07 Apr 2011 09:43:33 +0000

> square brackets mandatory in all cases, not just when there is

Yes, that is probably wise, the objection one might have is that it is
not the way IPv6-addresses is written in X-Forwarded-For.

 /Andreas Petersson

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 07 Apr 2011 11:20:42 +0000

>not the way IPv6-addresses is written in X-Forwarded-For.

I would be surprised of nobody does...

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 7 Apr 2011 13:59:39 +0200

Hi Poul-Henning,

> I would be surprised of nobody does...

Well, I'd say I have never observed the square brackets in an x-f-f header
yet. I know for sure haproxy does not set them, I don't know how squid,
nginx nor apache proceed. I deduce from your question that varnish does
use them.

I don't see any reason to make them mandatory in place where no ports are
indicated, because there is no ambiguity. However, supporting both formats
at least for compatibility with existing products seems wise.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 07 Apr 2011 12:07:04 +0000

In message <[email-masked]>, Willy Tarreau writes:

>indicated, because there is no ambiguity.

One bery good reason for making it mandatory is that having a big fat '['
makes it much easier and faster to parse the header.

---

From: Karl Dubost <[email-masked]>
Date: Thu, 7 Apr 2011 08:08:49 -0400

> I would be surprised of nobody does...

Do we have a survey of who does what?
X-Forwarded-For is used by Opera Mini servers. What about others?

That would help to establish frequencies, etc.

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 07 Apr 2011 12:24:02 +0000

>X-Forwarded-For is used by Opera Mini servers. What about others?

I thought the objective here was to define a replacement for X-Forwarded-For ?

I know this is not a big deal to most of you, but in high performance
applications, the difference between always having [...] around IPv6
or having that be optional is a relevant cost factor.

The matter of IPv6 address notation has been a total nightmare from day
one, addressed by at least 11 RFC which do not agree on very much, leaving
implementors with a nightmarish morass of code[1]

It would be nice if we didn't add further to the confusion, and tried
to show a bit of sanity for the rest of the community, so can we
please make the '[...]' mandatory even if there is no port number ?

Thanks,

Poul-Henning

[1]  From my own brief survey:
	RFC1884 -> 2373         1080::8:800:200C:417A
	RFC1884                 ::13.1.68.3
	RFC1884                 ::FFFF:129.144.52.38
	RFC2133 -> 2133
	RFC2292 -> 3542
	RFC2373 -> 3513
	RFC2428 EPRT |2|1080::8:800:200C:417A|5282|
	RFC2553 -> RFC3493
	RFC2732 -> 3986 http://[3ffe:2a00:100:7031::1]:8080/
	RFC3493 "numeric format"
	RFC3513 -> 4291
	RFC3986 BNF form
	RFC3986 http://[3ffe:2a00:100:7031::1]:8080/
	RFC3986 http://[v%x.????]/
	RFC5952 [2001:db8::1]:80

And it's really unfair to leave this one out:
	RFC1924 4)+k&C#VzJ4br>0wv%Yp
Despite the fact that it is published on april 1st, it does not stick
significantly out from the confusion of the rest.

---

From: Karl Dubost <[email-masked]>
Date: Thu, 7 Apr 2011 08:34:10 -0400

> I thought the objective here was to define a replacement for X-Forwarded-For ?

Yup. Sorry for not being clear. 
Are there other products/companies using X-Forwarded-For?
What are the products emitting the header?
What are the products parsing the header? (libraries, etc)
What are the usual mistakes, errors, etc we might have to face when parsing this header?

---

From: Matt Domsch <[email-masked]>
Date: Thu, 7 Apr 2011 09:43:19 -0500

> What are the products emitting the header?

Apache mod_proxy emits the header, at least when used as a reverse proxy (front end for web applications).

> What are the usual mistakes, errors, etc we might have to face when parsing this header?

My MirrorManager app parses this header to retrieve the originating
client's IP address, to perform GeoIP lookups thereof.  The python
code is really simple though:

    return xforwardedfor.split(',')[-1].strip()

The resultant string is fed to the python-IPy library.  This library
does not (currently) handle brackets in IPv6 addresses.

Thanks,
Matt

-- 
Matt Domsch
Technology Strategist, Dell Office of the CTO

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 07 Apr 2011 15:05:21 +0000

>The python

<Facepalm/>

Will the horror never cease...

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 7 Apr 2011 18:08:23 +0200

> Are there other products/companies using X-Forwarded-For?

I know quite a number of companies using it for logging purposes.

> What are the products emitting the header?

At least apache , haproxy and squid immediately come to mind.
Haproxy already emits it for IPv4 and IPv6 addresses and uses
no brackets, it emits it as returned by inet_ntop().

> What are the products parsing the header? (libraries, etc)

haproxy is able to use it for various purposes (IP-based ACL
filtering, transparent binding, etc...). Right now it only
parses it for IPv4 addresses, so adding support for both
brackets and non-brackets forms is not a problem.

> What are the usual mistakes, errors, etc we might have to face when parsing this header?

The most common one in my experience is that many people don't understand
that it is a list, and that having multiple header occurrences is equivalent
to having their values delimited by commas. Many of those users tend to log
only the first occurrence then complain they find stupid values there.

Regards,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 8 Apr 2011 06:31:44 +0800

> X-Forwarded-For is used by Opera Mini servers. What about others?

Some form of standardization of this would be useful for us.  We plan to use this for more than just logging.  Of course, we'll cope without a formal definition. 

I'll note that adding a new header is nice, but it actually means more work - we still have to process X-Forwarded-For to support all the existing stuff.  Is the "X-" so offensive?  Is there a reason not to simply specify what everyone does.

--Martin

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 08 Apr 2011 10:28:52 +0900

Hello Poul-Henning, others,

On making '[...]' mandatory, I agree. Moving from an X- header to a 
header without X- is often a very good place to clean up the syntax.

> 	RFC3986 http://[v%x.????]/

As the coauthor of RFC 3987, which copies all the relevant syntax from 
RFC 3986, I'm very familiar with the RFC 3986 syntax, but the last line 
look very suspicious.

There is indeed a syntax that uses a 'v' inside a '[', but it's 
currently not used. The intent is to use it for future versions of the 
IP protocol (just in case :-) and potentially for variants beyond the 
currently allowed syntax. The letter immediately following the 'v' is a 
simple hex digit, no '%' involved.

Here's the relevant text from the spec:

       IP-literal = "[" ( IPv6address / IPvFuture  ) "]"

       IPvFuture  = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )

    The version flag does not indicate the IP version; rather, it
    indicates future versions of the literal format.  As such,
    implementations must not provide the version flag for the existing
    IPv4 and IPv6 literal address forms described below.  If a URI
    containing an IP-literal that starts with "v" (case-insensitive),
    indicating that the version flag is present, is dereferenced by an
    application that does not know the meaning of that version flag, then
    the application should return an appropriate error for "address
    mechanism not supported".

At one time, there has been a proposal to use one of the versions for an 
extended form of IPv6 addresses to include scope zone identifiers
http://tools.ietf.org/html/draft-fenner-literal-zone-02
(don't ask me what scope zone identifiers are, I was just working on the 
URI syntax side of the problem).

> significantly out from the confusion of the rest.

Yes. And having an April fools' RFC on a topic may also indicate that 
there was a strong feeling of "too many, so why not one more".

Regards,    Martin.

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 8 Apr 2011 10:05:12 +0800

> > 	RFC3986 http://[v%x.????]/
[...]
> currently allowed syntax. The letter immediately following the 'v' is 
> a simple hex digit, no '%' involved.

I see that you didn't recognize the printf syntax there.  %x = hex integer

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 08:24:36 +0000

In message <[email-masked]>, =?ISO-8859-1?Q?=22Martin_J=2E_D=

>The letter immediately following the 'v' is a 

That's what '%x' signifies to an old 'C' programmer like me.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 08 Apr 2011 17:34:30 +0900

On 2011/04/08 17:24, Poul-Henning Kamp wrote:
> In message<[email-masked]>, =?ISO-8859-1?Q?=22Martin_J=2E_D=
> FCrst=22?= writes:
>

I'm an "old 'C' programmer", too, and I'm also teaching C, so I would 
have understood that easily *in a C context*.

Regards,   Martin.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 8 Apr 2011 18:49:32 +1000

> I'll note that adding a new header is nice, but it actually means more work - we still have to process X-Forwarded-For to support all the existing stuff.  Is the "X-" so offensive?  Is there a reas…

It's not so much that; it's that there's no interop on the contents of X-Forwarded-For for IPv6 headers (AFAIK), so we should take the opportunity to define a new header that specifies it well.

While we're at it, I'd like to see extensibility points added; there are a bunch of bits of per-hop metadata that would be nice to allow in here, instead of defining separate headers. For example, in some use cases it'd be good to identify the receiving address, as well as the sending one.

E.g.,

Forwarded-For: 1.2.3.4:5678; by=4.3.2.1:3128, 5.6.7.8:9012; by=3.2.1.0:80

--
Mark Nottingham   http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Apr 2011 11:00:15 +0200

> Forwarded-For: 1.2.3.4:5678; by=4.3.2.1:3128, 5.6.7.8:9012; by=3.2.1.0:80

Agreed. I'm used to see that info almost as commonly as the x-f-f header,
because people who want one generally want the other one too. In haproxy
it's implemented in the x-original-to header. The format above is much
better, but I suspect that some users will already complain that it's
harder to parse for them...

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 8 Apr 2011 19:04:59 +1000

> harder to parse for them...

Perhaps, but as long as they make a minimal effort and ignore the parameters, it'll be OK. If the draft goes in this direction, it would be good to give some examples so parser authors can make sure they do the right thing.

Cheers,

---

From: Andreas Petersson <[email-masked]>
Date: Fri, 8 Apr 2011 15:13:35 +0200

On Thu, 07 Apr 2011 12:24:02 +0000
> 	RFC3986 http://[v%x.????]/

Which RFC do you suggest we refer to in this document?

 /Andreas

---

From: Andreas Petersson <[email-masked]>
Date: Fri, 8 Apr 2011 15:36:03 +0200

On Fri, 8 Apr 2011 18:49:32 +1000

> Forwarded-For: 1.2.3.4:5678; by=4.3.2.1:3128, 5.6.7.8:9012; by=3.2.1.0:80

It's probably better to get a real specification than trying to specify
something that's been used in the wild for a long time without being
specified. Especially it is important to clarify how to handle IPv6.

Mark, I'm a bit skeptic about a parameter list. Doing this header field
more complicated makes the threshold for implementing support for it
higher. There is also more likely that people will handle it in wrong
and obscure ways.

What do the list think?
What parameters would be relevant? ("by"? "port"? "proto"? "host"?
other..?)

Also, I'd like to know if the list have opinions on whether this should
be published as an informational rfc or if we should try making it a
standard tracks rfc.

rgds,
 Andreas Petersson

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 13:37:51 +0000

>Which RFC do you suggest we refer to in this document?

I think RFC5952 is an attempt to sanitize this area so I don't see
there being any reason but to follow it.  It generally makes
sense and uses "SHOULD" about [...] where port numbers may be
resent.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 14:08:09 +0000

>What parameters would be relevant? ("by"? "port"? "proto"? "host"?

I've pondered this some more, and I fear we are headed the wrong way.

This header, like X-Forwarded-For will become part of the toolbox
for answering criminal investigations log-requests and similar and
we must think about it in that light.

Spoofing X-F-F headers have been used in the past, both to circumvent
badly thought out security-barriers and generally confuse people
reading logfiles.

The IP#'s you receive in such headers from the outside, are only
badly hashed cryptographic nonces:  You don't need to sit in the
White House to send an (X-)F-F with IP# 198.137.241.43.

Nothing is actually won by mandating that they look like IP# in the
first place, but it is very convenient for well-behaved-net-entities
that they are IP#, and this probably covers 99.99% of the cases
where somebody has to deal with them.

So my considered proposal would be:

The standard should make this header as a comma separated list of
identifiers defined like:

	1*<any CHAR except LWS and ",">

The standard should make it very clear that they are nonces which
can only be correctly interpreted in the context where they were
created.  (In other words:  A lawyer cannot sue based on the
identifier, until whoever created it translates its meaning for him,
even if it looks like an IP number).

And then we SHOULD strongly encourage that they follow this form:

	src-IP ':' src-port [ '/' dst-IP ':' dst-port ]

To make trouble-shooting and abuse@ processing as simple as possible
for all law-abiding net-citizens.

This allows $BIGCORP to cloak their identifiers to avoid leaking
internal network topology which can be used by criminals, and it
encourages all 3rd parties (ISP's etc) to provide IP#s for speedy
resolution of issues.

Poul-Henning

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Apr 2011 17:36:31 +0200

> 	src-IP ':' src-port [ '/' dst-IP ':' dst-port ]

While I agree with the principle, I would render the port optional.
It's almost always wrong anyway because you have the equipments in
the following order :

   client
   firewall
   load balancer
   reverse-proxy
   ...
   server

The load balancer almost always translates the source port (unless it's
doing DSR, which is progressively disappearing), and nobody car correlate
this source port seen by the reverse-proxy to anything logged anywhere.
So while there are *some* situations where the port can be exploited, i
practice where admins do care about logs, it's meaningless.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 15:58:45 +0000

>practice where admins do care about logs, it's meaningless.

You overlooked the bit about it only having meaning in the context
where it was generated.  To derive that meaning, you will need to
investigate logs from firewalls, load balancers and everything else.

But IP+port is what identifies the far end, and is the only handle
you can give the police for the remote end.  What the remote end
or the police can do with it, is their problem

So we should log the port number, always.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Apr 2011 18:12:32 +0200

> investigate logs from firewalls, load balancers and everything else.

Except that you will hardly find a product which logs a source port which
is randomly choosen by either the system or the lower layers for an outgoing
connection.

I completely agree with you concerning the context where the information
was generated. It's just that contexts where the port is meaningful are
very rare, and as such, making it mandatory is counter-productive and can
be misleading.

> or the police can do with it, is their problem

For the remote end I agree. I was talking about the information which is
already corrupted on your own side before the correct one was logged.

> So we should log the port number, always.

I simply disagree here with "always".

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 19:42:54 +0000

>is randomly choosen by either the system or the lower layers for an outgoing

... unless the owners happen to know that they are under a legal
obligation to be able to produce logging records for the next two
years detailing such communications.

>I simply disagree here with "always".

Please remember that we are talking about a SHOULD item, and that
the definition allows you to write "FOOBAR" if you want to.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Apr 2011 21:56:27 +0200

> years detailing such communications.

I am not talking about configurations but equipment capabilities.
Most people would not even know that their LB connects from the
original IP with another port, and blindly logging that info without
being aware of this could result in pointing the finger at the wrong
session on a remote end. I'm not saying the info is useless, I'm saying
it's almost always wrong, which is worse. Logging wrong information is
worse than not logging it by default, because the people who will want
to log it will be concerned about its contents.

> the definition allows you to write "FOOBAR" if you want to.

The SHOULD was for us to encourage pushing the format. I'm discussing
the proposed format. I'd rather see :

   src-IP [ ':' src-port ] [ '/' dst-IP ':' dst-port ]

Which also happens to be compatible with current uses.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 20:12:38 +0000

>Most people would not even know [...]

That should not cause us to write standards that make life impossible
for the people who do know and who do have the need.

>Which also happens to be compatible with current uses.

There is no current uses of "Forwarded-For", backwards compatibility
therefore does not prevent us from doing it right.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Apr 2011 22:41:10 +0200

> for the people who do know and who do have the need.

Sorry, but I fail to see what makes your life impossible when
parsing this an IP address followed by an optional port. I would
add that it's much easier than deciding what to write as a fake
port value when you know the port is wrong and don't want to fool
your logs.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Apr 2011 20:43:44 +0000

>port value when you know the port is wrong and don't want to fool

You seem to confuse "fake" and "useless" here.  In a Danish
court-room the difference is about 4 years of jail.

---

From: Brian Pane <[email-masked]>
Date: Fri, 8 Apr 2011 15:49:47 -0700

> therefore does not prevent us from doing it right.

I look at it a bit differently: there are many current uses of
Forwarded-For.  Every web proxy I've seen in the past decade uses it,
every web server has an option to log it, and every nontrivial web
analytics system parses it.  They all do expect the "X-" in front,
though.  The rollout of a naming change will be difficult; I
anticipate that proxy servers will end up sending both X-Forwarded-For
and Forwarded-For for many years.  That might be inevitable, but it
brings to mind two concerns:
- People who plan to log Forwarded-For data for forensic purposes will
need to keep logging X-Forwarded-For as well.
- X-F-F values can get somewhat large, and sending the value twice
(once as X-F-F and once as F-F) will help push more requests past 1MSS
in size, resulting in an extra round-trip delay in cases where the
proxy does not already have an established connection to the
destination.

I don't think either of those points is a show-stopper for
Forwarded-For, but they're nontrivial implementation challenges.

With regard to src-port, I think it should be optional.  Here's my rationale:

Pragmatically speaking, some non-empty subset of the people who run
proxy servers are going to have valid reasons why they can't share
clients' source ports with a third party destination server.  Here are
a couple of examples that come to mind:
- You operate an online business in a jurisdiction where IP addresses
are considered PII.  Your published privacy policy tells users that
you reveal their IP address to various 3rd parties as a necessary part
of whatever service you provide; on the back end, your application
makes HTTP calls to various service providers and sends the client's
address in the X-F-F.  As you read the release notes for the latest
version of your proxy software, you notice that it now can send source
port along with source address in the F-F header.  You ask your
organization's legal counsel for guidance, and their response is,
"that's a grey area, so we'd want to update the privacy policy in
advance of implementing the change," and so you back off on sending
src-port.
- You run multiple tiers of caching servers as part of an ISP or CDN.
You're willing to expose your internal proxies' IP addresses in the
X-F-F headers you send to 3rd-party web servers, because you've found
that the troubleshooting benefits outweigh the risks of that
transparency.  You decide not to also expose source ports, though,
because you don't want 3rd-parties attempting to deduce request rates
or concurrent connection counts within your internal infrastructure by
studying source port numbers for the hops between your proxy tiers on
successive requests.

For cases like these, it seems preferable to let implementations send
src-IP without src-port, rather than to making them choose between
"src-IP ':' src-port" or "no F-F header whatsoever."

-Brian

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 9 Apr 2011 12:02:55 +1000

> encourages all 3rd parties (ISP's etc) to provide IP#s for speedy

-1 on this approach; it's a spec (not even technical) solution to a purported legal issue. 

If a proxy doesn't want to leak internal network addresses in requests, IME they'll either a) not add them, or b) strip them at the border. Those that want to track internal requests through a series of devices can either take the (b) approach (which is IME more common, because it's easier to administer) or will identify each node with a pseudonym. This is easy to accommodate in the spec, just as it is for the Via header. 

E.g.,

forwarded-for = 1#( hop *[ ";" parameter ] )
hop = address | pseudonum
pseudonum = token
address = ...

I.e., you don't have to make the header syntactic mush just to make it clear that it doesn't always have semantic integrity.

One of the reasons for allowing extension parameters, btw, is to allow someone to sign the hops if they want to.

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 9 Apr 2011 13:10:31 +1000

On 09/04/2011, at 8:49 AM, Brian Pane wrote:

> The rollout of a naming change will be difficult; I

Why so? If they control the software that's consuming it, they can change it in a reasonably bounded amount of time. If they don't, why will they care? XFF is a completely non-standard header that is more than often NOT sent by proxies, so removing it shouldn't break anything.

> need to keep logging X-Forwarded-For as well.

If they're that interested, they're likely to log everything anyway...

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 9 Apr 2011 07:45:32 +0200

> One of the reasons for allowing extension parameters, btw, is to allow someone to sign the hops if they want to.

That's a good point. We could also imagine that some people might want
to specify an incoming interface name, an AS number or whatever.

Regards
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 09 Apr 2011 07:54:34 +0000

In message <[email-masked]>, Mark Nottingham wri

>hop = address | pseudonum

You do realize that this is superset of my proposal, which complicates
the parsing, without adding any value ?

>One of the reasons for allowing extension parameters, btw, is to allow 
>someone to sign the hops if they want to.

You can do the same i both my and your own proposal by hexdumping the
signature and send it as a pseudonym.

Do you at least agree, that we have to state clearly that a received
F-F header's content can only be correctly translated in the context
it was generated ?

---

From: Andreas Petersson <[email-masked]>
Date: Mon, 18 Apr 2011 16:22:43 +0200

On Fri, 8 Apr 2011 11:00:15 +0200

> harder to parse for them...

What happens if someone wants to send the forwarded-by-information, but
not the forwarded-for-information?
Does that never occur? 
There seems to be an X-Forwarded-By in use. Maybe it would be better
to have a header:

Forwarded: for=5.6.7.8:3456, for=8.9.1.2;by=4.5.6.7
Forwarded: for=1.2.3.4:5678;by=4.3.2.1:3128;proto=https

and thereby let all parameters being equal?

 /Andreas Petersson

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Apr 2011 09:01:35 +1000

I like it. Producing it is easy, and there are less special cases for parsing than the format I suggested.

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 19 Apr 2011 07:51:27 +0200

> I like it. Producing it is easy, and there are less special cases for parsing than the format I suggested.

Then let's specify all the possible fields. Quite a number of people
like to also specify the original destination, and some like to set
the original Host header when their reverse-proxies rewrite it before
passing on the request.

So we could have something like (non exhaustive) :
  - from
  - to
  - proto
  - host

Or maybe even better, let's have all the information available in
separate fields for easier parsing :

  - family (v4/v6)
  - src4
  - src6
  - sport
  - dst4
  - dst6
  - dport
  - host
  - proto

Cheers,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 19 Apr 2011 07:53:33 +0000

In message <[email-masked]>, Mark Nottingham wri

>> Forwarded: for=1.2.3.4:5678;by=4.3.2.1:3128;proto=https

This format needs a strict definition to be unambigious.

For instance, if the first proxy adds only "for" and the next adds
only "by", there is no way to tell if one or two proxies were
involved.

I still think it is a better idea that each proxy adds exactly one
element, and that the single element contains whatever information
the proxy is willing to disclose.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 19 Apr 2011 10:47:48 +0200

> only "by", there is no way to tell if one or two proxies were

Yes it's easy, because each subfield is delimited by a semi-colon.
So if you have :

  Forwarded: for=1.2.3.4; by=5.6.7.8

then those are the same proxy, but if you have :

  Forwarded: for=1.2.3.4, by=5.6.7.8

then those are two distinct proxies since there are two header values.

> the proxy is willing to disclose.

This is exactly the case : each proxy adds exactly one element here
(one header value) and it indicates the fields it wants to disclose.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 19 Apr 2011 08:52:06 +0000

>Yes it's easy, because each subfield is delimited by a semi-colon.

Sorry, I overlooked that detail.

To the list of proposed subfields you should add

	"nonce=" TOKEN

For sites which do not want to disclose details.

And In order to not give judges, lawers and law-enforcement the
wrong idea, I still think it is very important that the text makes
it absolutely clear, that even if the header contains what looks
like IP numbers etc, only the originator of this header can correctly
interpret the contents.

---

From: Andreas Petersson <[email-masked]>
Date: Tue, 19 Apr 2011 11:09:17 +0200

On Tue, 19 Apr 2011 07:53:33 +0000

> only "by", there is no way to tell if one or two proxies were

One proxy should add one element, which is a list of key=values
separated by semicolon.

Two proxies:
Forwarded: for=1.2.3.4, by=4.5.6.7

One proxy:
Forwarded: for=1.2.3.4; by=4.5.6.7

Or do I miss something?

> the proxy is willing to disclose.

Can you give an example of how such an element would look like?
Only sending the IP-address of the client you are forwarding for would
maybe be the cleanest, but there seems to be a need for disclosing
other information too when proxying. That information would need to be
connected to the forwarded-for somehow. 

E.g:
Forwarded-For: 1.2.3.4, 5.6.7.8
X-Forwarded-By: 3.4.5.6

would be ambiguous.

I am afraid that one will sooner or later see the need for
standardizing also X-Fowarded-(By|Proto|Host ...) and doing that and
keeping the format of Forwarded-For would be hard. 
I don't think it'd be a good situation if we had yet another way of
disclosing such information, therefore maybe it's better to support all
of them here? 

/Andreas Petersson

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Apr 2011 21:34:05 +1000

I have no problem at all with text like that.

Cheers,

---

From: Andreas Petersson <[email-masked]>
Date: Thu, 28 Apr 2011 11:48:36 +0200

On Tue, 19 Apr 2011 07:51:27 +0200

> separate fields for easier parsing :

I don't think it would be that much harder to parse out the address
family. By specifying family at one place seems strange if you have 
incoming on IPv4 and outgoing on IPv6. src/from and dst/to should be
unique within one proxy step. I also think it is not so much gain by
having separate port-parameters. 
By specifying only "from" and maybe "to" as a token that should look
like an IP-address you can also fairly clear say in the standard that
this only have a meaning at the proxy that appended the value (Hi PHK!).

/Andreas Petersson

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 28 Apr 2011 12:12:03 +0200

On Thu, Apr 28, 2011 at 11:48:36AM +0200, Andreas Petersson wrote:
> incoming on IPv4 and outgoing on IPv6.

I don't see why. The family is necessarily unique at one step. In fact,
if a proxy puts you an IPv4 source and an IPv6 destination, how are you
supposed to parse those addresses ? With an explicit family, at least
you know which one is wrong.

> src/from and dst/to should be unique within one proxy step.

I'm not sure what you mean here.

> I also think it is not so much gain by

It simplifies parsing as you don't need delimiters. We already know all
the issues we can encounter with multi-level delimiters. For instance,
if we make brackets mandatory, an attacker could decide to emit this
header with a missing closing bracket. The proxy won't care about the
syntax and will only have to complete the header by adding a new value.
You'd then have something like this :

attacker sends :
  Forwarded: from=[2001::1:2:3

proxy adds :
  Forwarded: from=[2001::1:2:3
  Forwarded: from=[2002::5:6:7]:12345; to=[2002::8:9:A]:80

which results in :
  Forwarded: from=[2001::1:2:3, from=[2002::5:6:7]:12345; to=[2002::8:9:A]:80

It's expected that many server-side parser will parse it that way :
  from=[2001::1:2:3, from=[2002::5:6:7]:12345
  to=[2002::8:9:A]:80

Then they'd strip the address around the first opening bracket and the
last closing one (strchr/strrchr), and pass their own inet_pton() over
the result :
  my_inet_pton("2001::1:2:3, from=[2002::5:6:7")

They's get 2001::1:2:3, happily ignore the error pointing to the comma
and result in using the attacker-fed address as the source, with the
proxy-fed port as the port.

Last point is that right now apps don't care about the port as it's the
proxy's role to log it. Apps want the IP address for statistics, tracking,
gelocation, etc... I predict that if we make it optional, almost nobody
will enable it because they don't care a dime. Thus, let's not force
everyone to complexify their parsers because we added information they're
not interested in.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 28 Apr 2011 10:39:35 +0000

>> separate fields for easier parsing :

Don't forget:

	- opaque

For people who don't want to expose the information, but who have
to offer logging for law-enforcement.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 28 Apr 2011 13:59:16 +0200

> to offer logging for law-enforcement.

Good point, thanks for reminding.

Willy

---

From: Andreas Petersson <[email-masked]>
Date: Thu, 28 Apr 2011 17:04:03 +0200

On Thu, 28 Apr 2011 12:12:03 +0200

> you know which one is wrong.

Yes, sorry, that wasn't really thought through. I still think it might
be redundant information.

> I'm not sure what you mean here.

There should not be both src6 and src4 present.

> proxy-fed port as the port.

Is this a problem in all other places where you have brackets?
I have hard to believe we can specify the standard in a way that
totally eliminates the need for input sanitation. 

> everyone to complexify their parsers because we added information they're

I can agree to some extent, I see your point. Do others have an opinion
here?

 /Andreas Petersson

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 28 Apr 2011 20:19:56 +0200

> There should not be both src6 and src4 present.

Inside a single step, I agree, but I did not propose to post the info
twice, only once per step. Ah I think I understand. You mean that
instead of having "src4" or "src6" we could simply have "src". Yes
of course. I'd even say that either we have "family= ; src=;" or we
have "src6=" and can get rid of the family.

(...)
> Is this a problem in all other places where you have brackets?

I don't precisely know, I'm sure people are more careful when it comes
to something that is planned at the beginning of their work. The use of
the proxied IP address generally comes after deployment and is done as
a quick and dirty hack. What I experience a lot is the following scenario :

- a guy contacts me and tells me "hey, I have installed your haproxy and now
  my server only sees its IP address instead of my client's, how do I have
  to configure it ?"

- I explain that he has to enable logging of the "X-Forwarded-For" header
  in his application or on his server (depending on his needs) ;

- then the same guy comes the next day later saying "your haproxy has another
  bug, it randomly sends me corrupted headers which look like this :
     X-Forwarded-For: 192.168.1.2, 10.0.0.1
  and this makes my server do [insert your favorite horror story here]"

- then I point to the spec and explain how he must parse it, and insist
  that his server should never have performed the horror described above.

- only after that I get "OK it works thanks".

So basically, this is the type of thing that is often developped in a hurry
because they discover they need it when it's already too late. Given the
horrors I've read (application crash, session loss, ...) and the references
they're systematically pointing as "the spec" without knowledge of header
folding (http://en.wikipedia.org/wiki/X-Forwarded-For), I'm sure quite
certain there is no reason for the situation to change whatever we describe.

> totally eliminates the need for input sanitation. 

I perfectly agree. However, some error patterns are more common than other
ones, and when we know that some practices tend to trigger them quite often 
while other practices don't, I tend to stay on the side of not triggering
them.

Regards,
Willy