Subject: The HTTP Origin Header (draft-abarth-origin)

From: "Larry Masinter" <[email-masked]>
Date: Thu, 22 Jan 2009 09:33:13 -0800

proposes a new HTTP header and rules for its use as a way of addressing
Cross-Site Request Forgery (CSRF) attacks. This was part of the
HTML5 work in WhatWG and W3C HTML working group.

Is there's a better venue for discussion of this draft
than [email-masked]?

Larry
-- 
http://larry.masinter.net

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 15:07:07 -0800

> The document  http://tools.ietf.org/html/draft-abarth-origin

I have no interest in implementing or standardizing this header.

1) CSRF is not a security issue for the Web.  A well-designed Web
service should be capable of receiving requests directed by any host,
by design, with appropriate authentication where needed.  If browsers
create a security issue because they allow scripts to automatically
direct requests with stored security credentials onto third-party
sites, without any user intervention/configuration, then the obvious
fix is within the browser.

2) It doesn't prevent CSRF attacks because it doesn't prevent the
Origin header field from being spoofed.  It therefore only improves
a very small situation: a new browser that has implemented this new
header *and* allows cross-site scripted requests *and* prevents any
manipulation of the request header fields *and* is still stupid
enough to include stored credentials in the cross-site request.
It does nothing for existing browsers and actively interferes with
existing non-browser user agents.

3) It requires servers to maintain a table of "compliant" user
agents in order to distinguish spoofed or dropped origin fields,
which is known to be harmful to the introduction of new clients
and results in all clients using copycat user-agents fields,
thereby defeating the purpose of the table.

4) Even if such a feature becomes necessary, it can be far
easier accomplished by changing the operational behavior of
browsers such that they always send Referer and simply reduce
the value of that field (similar to that specified for Origin)
in those cases where it is currently not set at all.  No change
would then be needed to HTTP and existing agents that already
send Referer for these cases would already comply.

5) The author seems to be unaware that URL is a deprecated term
defined in RFC 3986. The field values would be better defined
in terms on the standard URI grammar and terminology.

Cheers,

Roy T. Fielding                            <http://roy.gbiv.com/>
Chief Scientist, Day Software              <http://www.day.com/>

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 22 Jan 2009 17:28:23 -0600

> in terms on the standard URI grammar and terminology.

When does RFC 3986 become binding on HTTP/1.1?  I was presuming that
doesn't happen until the next RFC, although the author should take
note of the discrepancy.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Jan 2009 10:53:47 +1100

As I responded to Ian, I think here is at least as appropriate as  
elsewhere.

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 15:54:44 -0800

> Entirely agreeing with your points, I'm confused on one point...
>
> Roy T. Fielding wrote:
>>

It is already binding (to the extent RFCs are binding on anyone).
STD 66/RFC 3986 defines how software making use of the Internet
correctly send/receive/parse URIs.  It automatically applies to
any protocols that make use of URIs as a protocol element, even
if the protocol spec was written long before 3986.

In any case, the term URL has been deprecated since RFC 2396
because it is effectively a synonym for URI and leads to all
sorts of useless arguments about the perceived limitations of
locators.  The HTML5 editor believes this allows him to redefine
URL within the exclusive context of HTML5, hence the continued
misuse of this term here.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Jan 2009 11:20:08 +1100

> fix is within the browser.

Personally, I tend to agree; the number of problems caused by  
scripting are voluminous, and I think we'd do well to question its  
boundaries.

However, AIUI Origin isn't about exposing credentials, directly. The  
Referer header would be just as adequate for this purpose (assuring  
that the originating content is from an allowed host), except that the  
authors feel that Referer is stripped by proxies and user-agents in an  
attempt to improve privacy / limit exposure of sensitive data in URIs.

I don't think this argument is valid. The number of proxies that do  
this is very small. Those that exist will come under pressure to  
change their behaviour when their users notice that Web applications  
that rely upon this feature break behind those proxies.

In other words, just as the authors are satisfied having a staged  
deployment where some browsers on the Web support cross-site requests  
using their mechanism, while some still do not, I see no reason why it  
isn't just as acceptable to accept that some proxies will support it,  
and some will not; over time, more will.

Of course, *some* users have more direct control over what browser  
they use than whether a proxy is interposed, but not all do.

> existing non-browser user agents.

I think they're relying on existing browsers not allowing cross-site  
requests at all programmatically, and not having the ability to  
manipulate headers using other means (e.g.,   script src, etc.).

You make a good point about non-browser user agents. Origin  
effectively creates a new (and very primitive, as you note)  
authentication protocol for requests to what some would call  
"services," unless the server decides to also allow requests that omit  
the Origin header altogether. If they do that, however, it seems like  
a non-programmatic, non-Origin-creating browser would leave a hole...

> thereby defeating the purpose of the table.

I understand the drop scenario (see above), but I don't see how it can  
be spoofed by a browser cross-site, provided that Origin is  
implemented with appropriate restrictions in new browsers (which may  
be a big ask, given the history of browser implementation...).

> send Referer for these cases would already comply.

I don't agree. Unless it's very well-specified and implemented, this  
will have the effect of dumbing down Referer, reducing its utility for  
other purposes.

Cheers,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 16:41:37 -0800

> On 23/01/2009, at 10:07 AM, Roy T. Fielding wrote:
>>
> I don't agree. Unless it's very well-specified and implemented,  
> this will have the effect of dumbing down Referer, reducing its  
> utility for other purposes.

I don't understand -- the only case that would be affected
is the one wherein no Referer is sent today.  It is easy
to distinguish that case from other Referer values because it
excludes anything after the URI authority (normal "http" Referer
values always have a path portion of at least "/").  Hence,
the change is both HTTP-compliant and detectable by origin
servers (if they cared, which I don't expect they would).

....Roy

---

From: Adrien de Croy <[email-masked]>
Date: Fri, 23 Jan 2009 13:51:06 +1300

google analytics for a compelling reason why.

It allows a site to determine if it was hit by way of linking from 
another site, or presume that the human typed directly the URI into the 
browser.

I don't see why servers can't protect themselves without changing 
Referer though.

Adrien

Roy T. Fielding wrote:
>
> On Jan 22, 2009, at 4:20 PM, Mark Nottingham wrote:
>> will have the effect of dumbing down Referer, reducing its utility 
>> for other purposes.
>

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Jan 2009 11:55:33 +1100

Why would even that change be necessary? AIUI browsers sent no value  
when the request wasn't sourced from a particular HTTP URI; that's  
information that's valuable to the server (as Adrien points out).

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 17:16:21 -0800

> Ah, I missed the clause "where it is currently not set at all."
>
> Why would even that change be necessary? AIUI browsers sent no  
> value when the request wasn't sourced from a particular HTTP URI;  
> that's information that's valuable to the server (as Adrien points  
> out).

Er, right, I should have limited it to the https case where
nothing is sent for (what I've always considered bogus)
privacy reasons.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 17:35:28 -0800

> by design, with appropriate authentication where needed.

Many Web sites contains CSRF vulnerabilities and find it difficult to
engineer CSRF defenses.  The goal of the Origin header is to make it
easier for these sites to defend themselves against CSRF attacks.  For
example, a site can use the header to defend itself against CSRF using
a simple Web application firewall.

> If browsers

What obvious fix do you have in mind?  A browser vendor who removes
cookies from cross-site requests would quickly find that their users
choose to use another browser.

> Origin header field from being spoofed.

Do you have an example of how the Origin header can be spoofed in a
cross-site request in a browser with more than 1% market share?  You
can "spoof" the Origin header when sending a same-site request, but I
don't know of any techniques for spoofing the header in a cross-site
request.

> It therefore only improves

An browser-side CSRF defense will have this property.  As more
browsers adopt the header, this situation will grow to encompass a
large percentage of the browser market share.

> agents in order to distinguish spoofed or dropped origin fields,

I experimentally measured how often the Origin header is dropped in
the real world, an it is not dropped greater than 99.9% of the time.
The details of the experiment are described in:

http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf

I don't believe such a table is necessary.  The presence of the Origin
header itself indicates that the user agent supports the header.

> browsers such that they always send Referer

The Referer header is suppressed in the network for HTTP requests
approximately 3% of the time.  As a Web site operator, I cannot use
strict Referer validation to defend against CSRF because I'll miss out
on 3% of my customers.  (Again, these numbers come from the same
measurement study.)

> and simply reduce
> in those cases where it is currently not set at all.

The Referer header is stripped in the network layer regardless of the
value it contains.  Truncating its contents does not help defend
against CSRF.

> in terms on the standard URI grammar and terminology.

I'll fix this in the next version.

Thanks for your comments,
Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 17:45:23 -0800

> Personally, I tend to agree; the number of problems caused by scripting are
> voluminous, and I think we'd do well to question its boundaries.

An attacker can mount a CSRF attack without scripting simply by using
an HTML <form> element.

> However, AIUI Origin isn't about exposing credentials, directly. The Referer
> header would be just as adequate for this purpose (assuring that the
> originating content is from an allowed host), except that the authors feel
> that Referer is stripped by proxies and user-agents in an attempt to improve
> privacy / limit exposure of sensitive data in URIs.

Indeed.  The Referer header is stripped by proxies frequently enough
to make it unusable as a CSRF defense over HTTP.

> I don't think this argument is valid. The number of proxies that do this is
> very small.

In fact, this occurs for about 3% of users, which is a significant number.

> Those that exist will come under pressure to change their
> behaviour when their users notice that Web applications that rely upon this
> feature break behind those proxies.

Unfortunately, these proxies prevent Web sites from relying on the
Referer header, and so the operators of these proxies never come under
pressure to stop suppressing the header.

> In other words, just as the authors are satisfied having a staged deployment
> where some browsers on the Web support cross-site requests using their
> mechanism, while some still do not, I see no reason why it isn't just as
> acceptable to accept that some proxies will support it, and some will not;
> over time, more will.

The difference is that there is a deployment plan self-interested
agents to bring about the usefulness of the Origin header for CSRF
defense, but there is no such plan for the Referer header.

Origin header deployment plan:

1) Each user agent implements the Origin header and advertise that
they support more secure than those that do not implement the header.

2) Sites add rules to their Web application firewall rules to protect
those of their users that use a supporting browser.

3) Users choose supporting user agents because they are more secure.

4) Supporting user agents come to dominate the market.

> requests at all programmatically, and not having the ability to manipulate
> headers using other means (e.g.,   script src, etc.).

I am not aware of a popular browser that allows sites to spoof the
Origin header in cross-site requests.

> unless the server decides to
> also allow requests that omit the Origin header altogether.

In fact, the ID requires participating sites to do this.

> If they do that,
> however, it seems like a non-programmatic, non-Origin-creating browser would
> leave a hole...

Can you explain this hole?

Thanks for your feedback,
Adam

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 22 Jan 2009 19:46:44 -0600

> On Thu, Jan 22, 2009 at 3:07 PM, Roy T. Fielding <[email-masked]> wrote:

Does a protocol which provides for inherent spoofing methods actually add
value to the design of the application, or simply provide another security
check box which authors can apply to simply be routed about in the very
next request.

If you really wanted to solve this programmaticly, you would add a specific
hash or noonce to identify the origin to itself...

oh wait, nevermind, that's digest authentication.

Bill

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 17:47:59 -0800

> is the one wherein no Referer is sent today.

The problematic case is when the Referer header is suppressed by the
network (e.g., proxies).  In this case, the Referer header is
suppressed regardless of its value.  Choosing a different value will
not help Web sites defend themselves against CSRF.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 17:52:57 -0800

> I don't see why servers can't protect themselves without changing Referer
> though.

A Web site that wishes to use the Referer header to defend itself
against CSRF has two choices:

Strict Referer validation:
1) If the Referer header is present, ensure that it contains a "trusted" value.
2) If the Referer header is absent, *reject* the request.

Lenient Referer validation:
1) If the Referer header is present, ensure that it contains a "trusted" value.
2) If the Referer header is absent, *accept* the request.

Web sites cannot use strict Referer validation because the Referer
header is legitimately absent for 3% of users, causing the site to
lose out on a significant amount of business.

Web sites cannot use lenient Referer validation because the attacker
can maliciously force the browser to omit the Referer header, causing
the site to be vulnerable to CSRF.  There are several techniques for
doing this, the simplest is to issue the request from an FTP URL.

You can find a more detailed explanation here:
http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf

Adam

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 22 Jan 2009 19:58:04 -0600

> On Thu, Jan 22, 2009 at 4:41 PM, Roy T. Fielding <[email-masked]> wrote:

Ok - hold up... a 'strict' proxy which is stripping all but trusted
headers is going to pass the Origin header, why?

If you can't fix the bug in the proxies, adding another header for
them to ignore is not a solution.  Agreed this is a problem, but not
one to be resolved by adding more datum to be eliminated.  They only
have RFC2616 to strip hop-by-hop headers, so I would study this
problem set from the scope of non-compliant proxies and explain
to that group (through a best practices RFC or direct bug report)
why this is harmful.

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 18:16:37 -0800

> hash or noonce to identify the origin to itself...

This design rules out common use cases such as berkeley.facebook.com
POSTing a request to www.facebook.com.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 18:21:48 -0800

<[email-masked]> wrote:
> Adam Barth wrote:

According to my experiments, this is not how proxies function.
Instead, proxies strip headers on a blacklist of headers.  A user
behind a hypothetical "strict" proxy you describe would be unable to
use many existing Web sites because those sites require custom
headers, such as X-JSON, to function properly.

> them to ignore is not a solution.

In fact, the vast majority of proxies leave the Origin header
unmolested and so do not do not require modification.

> They only

I suspect the operators of these proxies will ignore such efforts
because they have business reasons for stripping the Referer header.
For example, it is common in enterprise configurations to strip the
Referer header to prevent intranet URLs from leaking to Internet Web
sites.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 18:29:03 -0800

> not help Web sites defend themselves against CSRF.

The feature of "defend themselves against CSRF by identifying
the referral page" is satisfied by "don't allow requests that
lack an appropriate Referer".  Your estimate that it would also
block some 3% of false negatives does not lessen its defense.
The 3% would get an error message in response.

Your claims are based on the assumption that those very same
3% proxies will forward the Origin header unchanged.  Your
assumption is wrong.  The proxies that remove request headers
today are the ones that remove all request headers and rewrite
each request on their own terms -- the main ones being the
content-translating proxies used by some wireless networks and
the firewall-translating proxies used by a few large corporations.
Both of those cases are just as likely to adapt their feature set
to unreachable services "protected" by Referer as they are to
unreachable services "protected" by Origin.

Furthermore, unlike Origin, Referer is in use today, would not
add extra bits on the wire, and doesn't require further standards
work (outside the browser implementation standards) to define.
A 3% initial failure rate is just noise compared to deploying an
entirely new header field.

....Roy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 23 Jan 2009 03:41:45 +0100

>2) If the Referer header is absent, *accept* the request.

This is a false dichotomy; servers also have the option to request more
information before making their final determination whenever deemed ne-
cessary as long as human interaction is possible. For example, having a
user re-enter his credentials is a common technique.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adrien de Croy <[email-masked]>
Date: Fri, 23 Jan 2009 16:14:53 +1300

requests?  e.g were any investigated to see if there was a common theme 
regards which model of proxy they may have been going through?

I would have thought that if a system admin decided to strip referer for 
privacy reasons, then they may also decide (whether in ignorance or not) 
to strip the Origin header as well.

Also I presume this scenario is coupled with a requirement for browsers 
to make it impossible for a script to touch the Origin header?

Adrien

Adam Barth wrote:
> On Thu, Jan 22, 2009 at 5:58 PM, William A. Rowe, Jr.

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 19:51:52 -0800

> The 3% would get an error message in response.

These 3% of potential users would be unable to use the Web site.  In
talking with folks who run large Web sites, I've been told that
excluding 3% of your potential customers is not acceptable.

> 3% proxies will forward the Origin header unchanged.

This is not an assumption.  In April 2008, measured how often various
headers were suppressed for 283,945 browsers who viewed an
advertisement we placed with a minor ad network.  We observed that the
Referer header was suppressed for approximately 3% of requests whereas
the Origin header was only suppressed 0.029-0.047% of requests (95%
confidence).  For more detailed results and a description of the
methedology, please see Section 4.2.1 of
http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf

> Your assumption is wrong.

What evidence do you have to back up this claim?

> The proxies that remove request headers
> each request on their own terms

These proxies do not appear to be nearly as common as proxies that
strip the Referer header.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 19:54:37 -0800

> * Adam Barth wrote:
[...]

> user re-enter his credentials is a common technique.

To fully defend themselves against CSRF attacks, Web sites must
protect every request that modifies state.  It is impractical to ask
users to re-enter their credentials for every side effecting
operation.  Also, this technique cannot be used to defend against CSRF
attacks on a site's login form.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jan 2009 20:03:36 -0800

> was there any trend you were able to recognise from the 3% of stripped

Due to ethical restrictions, we were unable to collect
user-identifiable information (such as IP addresses) during this
study.  We did observe a number of times that the Referer header was
replaced with an advertisement for a privacy firewall (one of whose
advertised features was stripping the Referer header).  I also have
worked for companies (and hear about similar deployments) whose
corporate firewall/proxy strips the Referer header to protect the
confidentiality of intranet URLs.

> privacy reasons, then they may also decide (whether in ignorance or not) to
> strip the Origin header as well.

Unlike the Referer header, the Origin header is not set for GET or
HEAD requests.  In practice, this immensely reduces how often the
header is sent.  For example, imagine a corporate wiki page at
http://wiki/Main/PlansToBuildAnIPhoneKiller has a hyperlink to Apple's
iPhone marketing materials.  An employee who follows this link will
leak sensitive information to Apple in the Referer header, but the
employee will not leak any information in the Origin header because it
is not sent for GET requests.

> Also I presume this scenario is coupled with a requirement for browsers to
> make it impossible for a script to touch the Origin header?

This is already the case for cross-site requests in browsers today.
The XMLHttpRequest specification in the W3C already forbids overriding
the Origin header for same-site requests.  This change has already
been implemented by several major browser vendors.

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 23 Jan 2009 09:36:02 +0100 (CET)

> headers were suppressed for 283,945 browsers who viewed an advertisement we 
> placed with a minor ad network.  We observed that the Referer header was 
> suppressed for approximately 3% of requests whereas the Origin header was 
> only suppressed 0.029-0.047% of requests (95% confidence).

Surely this isn't really surprising. Referer is a standardized and established 
header that has been in use for a long time and proxy 
admins/products/companies have adapted and reacted.

Origin is a newly suggested header that certainly none of the 
admins/products/companies have bothered about since it isn't standardized nor 
in actual use and thus they don't block it - yet.

Further, the argument:

> the employee will not leak any information in the Origin header because it 

... will thus break when that same intranet has a 'search the with loogle' 
field that sends a POST to the external site?

-- 

  / daniel.haxx.se

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 Jan 2009 09:30:49 +0100

looking at <http://tools.ietf.org/html/draft-abarth-origin-00#section-5>:

    Whenever a user agent issues an HTTP request whose method is neither
    "GET" nor "HEAD", the user agent MUST include exactly one HTTP header
    named "Origin".

What about other safe methods, such as PROPFIND, REPORT or SEARCH? 
Shouldn't the spec just say:

    Whenever a user agent issues an HTTP request whose method is not
    known to be safe (see ...), the user agent MUST include exactly
    one HTTP header named "Origin".

?

BR, Julian (always nervous when definitions refer to a certain set of 
named methods)

---

From: Ian Hickson <[email-masked]>
Date: Fri, 23 Jan 2009 10:26:37 +0000 (UTC)

> 
> Further, the argument:
> 
> > the employee will not leak any information in the Origin header 
> > because it is not sent for GET requests.
> 
> ... will thus break when that same intranet has a 'search the with 
> loogle' field that sends a POST to the external site?

Search is usually done with GET, but even if it was, leaking a hostname 
isn't a big deal -- it's unlikely that confidential information will be in 
a hostname. (This is one reason why the Origin header doesn't include the 
path information.)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Adam Barth <[email-masked]>
Date: Fri, 23 Jan 2009 09:14:54 -0800

> What about other safe methods, such as PROPFIND, REPORT or SEARCH? Shouldn't
> the spec just say:
>

Good point.  What should I cite as the authoritative list of safe methods?

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 Jan 2009 18:35:20 +0100

> On Fri, Jan 23, 2009 at 12:30 AM, Julian Reschke <[email-masked]> wrote:

Just say "safe", reference RFC 2616, Section 9.1.1 for now. HTTPbis will 
introduce an IANA registry for HTTP methods, which contains the flag 
(see 
<http://tools.ietf.org/html/draft-ietf-httpbis-method-registrations-01>).

BR, Julian

---

From: Adam Barth <[email-masked]>
Date: Fri, 23 Jan 2009 09:50:02 -0800

You can find the up-to-the-minute version of the draft at:

http://webblaze.cs.berkeley.edu/2009/origin/origin.txt

I'll upload new versions to IETF as appropriate.

Adam

On Fri, Jan 23, 2009 at 9:35 AM, Julian Reschke <[email-masked]> wrote:
> introduce an IANA registry for HTTP methods, which contains the flag (see

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 23 Jan 2009 17:17:18 -0800

On Jan 22, 2009, at 7:51 PM, Adam Barth wrote:
> On Thu, Jan 22, 2009 at 6:29 PM, Roy T. Fielding  
> <[email-masked]> wrote:

No, those 3% of users would have to use TLS/SSL, which the site could
redirect to automatically (and should have been using anyway for a
state-changing authenticated request.  As your study shows, https
exchanges are not modified by those proxies.

Obviously, you would have had a different response if you had asked
them a more accurate question, like "would you prefer a 97% success
rate today or a 0% success rate today."

Referer was introduced in 1993, deployed by spiders in 1994, grudgingly
accepted by browsers in 1995, and only reached the 90% range sometime
around the 1999-2000 timeframe.  The notion that we can introduce Origin
now and reach anywhere near 97% correct deployment is absurd.

Meanwhile, those same sites will implement this CSRF protection
using Referer, if they need it, since 97% protection is obviously better
than no success if they want to protect against CSRF. That percentage
can already be improved to 99% if non-Referer state-changing requests  
are
(re)directed to an https site.  So, what you are actually asking is
that we standardize an entirely new header field in the vague hope
that it will eventually be deployed correctly on better than 97% of
browsers in the wild so that some 2.x% of those browsers that happen
to still be behind old non-CSRF-protecting proxies at that future
date might be able to use a separate codepath on the server that
duplicates all of the security checks for Referer instead of just
using an https connection, assuming that none of the browser
developers introduce a bug that makes Origin less secure than Referer.

Sorry, the answer from this server developer is NO.  Spend the
effort on improving the definition of Referer instead so that it
will be blocked by fewer proxies, and then help educate those
proxy vendors as to why they shouldn't make their own users more
vulnerable to CSRF attacks.  I suspect that you will have little
problem convincing sites to upgrade their old proxy if you can
show them a CSRF attack on salesforce.com, Google domains, or AWS
that is enabled by the proxy vendor's bad code or config.

> http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf

I read the paper.  I am not sure if automated ad serving technology
would change the sample size to be self-selecting (and therefore not
representative of deployed browsers), but I doubt that it would change
the analysis much.  Regardless, Origin is not deployed and there is
no reason to believe that it would ever reach the 97% deployment
threshold.  Heck, HTTP parsing isn't implemented correctly in far
more than 3% of the browsers in the wild.

> What evidence do you have to back up this claim?

16 years of Web protocol implementations, 14 years being the editor
of the HTTP specs, and 13 years in which my contact information has
been supplied for people to send "why doesn't HTTP work?" messages
for both standards and Apache httpd.  For example, there was a period
when some MS folks thought that HTTP/1.1 chunked encoding didn't work,
until I pointed out that all of their tests were being made though
their own corporate firewall (eating characters).  I get that stuff
all the time.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Fri, 23 Jan 2009 19:04:14 -0800

The key point that you misunderstand is that the Origin header is
incrementally useful as a CSRF defense whereas the Referer header is
not.  This is a subtle, but important, point in the design of the
Origin header.

On Fri, Jan 23, 2009 at 5:17 PM, Roy T. Fielding <[email-masked]> wrote:

[Re-ordered]

> The notion that we can introduce Origin

Suppose a browser with 10% market share implements the Origin header.
Then a Web site can protect 10% of their users from CSRF by adding
three rules to their Web application firewall (WAF).

> Regardless, Origin is not deployed and there is
> threshold.

There is no deployment threshold for the Origin header.

> So, what you are actually asking is
> browsers

The Origin header need not be deployed to 97% of browsers to be
useful.  The Origin header design achieves this by having three
states:

1) Present and containing a scheme/host/port serialization
2) Present and containing the string "null"
3) Absent

States (2) and (3) exist to distinguish supporting and non-supporting
user agents.  Sites can protect users of supporting user agents
without affecting those that use non-supporting user agents.

The Referer header has only two states:

A) Present and containing a URL
B) Absent

Because the Referer header collapses states (2) and (3) into (B), the
sites might lock out non-supporting user agents (or those behind
proxies that make them appear to be non-supporting) in order for users
of supporting user agents to reap the benefits of CSRF protection.
For this reason, the Referer header is not incrementally useful as a
CSRF defense.

> state-changing authenticated request.

Sadly, deploying HTTPS is not as easy as adding three rules to a WAF.
Consider a site like Digg.  Digg requires CSRF protection to prevent
stories from digging themselves and causing other mischief.  However,
implementing HTTPS requires Digg to pay its CDN more money for HTTPS
service, limits Digg to advertising networks that support HTTPS, and
prevents Digg from showing YouTube videos on its front page.

> than no success if they want to protect against CSRF.

The 3% lock-out rate for using the Referer header causes sites to use
other CSRF defenses that are expensive to engineer and difficult to
retrofit onto existing sites.  This prevents the bootstrapping effect
you envision for convincing proxies to change their behavior.

> Spend the
> vulnerable to CSRF attacks.

The timeline for this approach is unbounded.  The Origin header can
become useful within one or two browser update cycles.

> Sorry, the answer from this server developer is NO.

Then you are free to operate a non-participating server and ignore the
header.  Other server developers have expressed their interest in
operating a participating server.

> Heck, HTTP parsing isn't implemented correctly in far

Yet somehow we muddle along.

> for both standards and Apache httpd.

This is not evidence.  This is an argument from authority.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 24 Jan 2009 07:59:03 +0100

* Adam Barth wrote:
>operation.

I am unsure what point you are trying to make. You gave the impression
that there are only two options, and neither of them is ever acceptable.
That is not the case, there are more options, and some of them lead to
acceptable results for some applications. There may be others, but that
is no reason to claim a greater problem than there really is.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Fri, 23 Jan 2009 23:45:45 -0800

On Fri, Jan 23, 2009 at 10:59 PM, Bjoern Hoehrmann <[email-masked]> wrote:
> I am unsure what point you are trying to make.

I was trying to make the point that Web sites cannot rely on the
Referer header to build a CSRF defense.

> You gave the impression

Most of the time, sites that use the Referer header to defend
themselves against CSRF use lenient Referer validation (meaning they
accept requests that lack a Referer header).  These sites are easy for
attackers to exploit because the attacker can suppress the Referer
header in a number of ways.

In the past, when I've brought these attacks to the attention of these
sites, they explain that they can't use strict Referer validation due
to the 3% of users this locks out.  After the dust settles, these
sites either remain vulnerable or implement a more complex CSRF
defense based on secret tokens.

> is no reason to claim a greater problem than there really is.

Yes, there are techniques that sites can use to defend themselves
against CSRF, but those techniques are (a) expense/complex to engineer
and (b) difficult to retrofit onto existing web sites.  The goal of
the Origin header is to make it easier for most sites to defend
themselves against CSRF.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 24 Jan 2009 17:44:50 +0100

>Referer header to build a CSRF defense.

I believe that point is somewhere between misleading and incorrect, but
for the sake of argument let me make this point instead: Web sites can-
not rely on the Origin header to build a CSRF defense. Now I'd like to
know how your point can reasonably believed to be correct, but my point
reasonably believed to be incorrect, at some point within the next seven
years. Or, if you agree with my point, why you raise this point here.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sat, 24 Jan 2009 10:20:20 -0800

On Sat, Jan 24, 2009 at 8:44 AM, Bjoern Hoehrmann <[email-masked]> wrote:
> not rely on the Origin header to build a CSRF defense.

Why is that?  I missed the argument that supports this claim.  It
seems that sites can use the Origin header to defend themselves
against CSRF by implementing the following three mod_security rules:

SecRule REQUEST_HEADERS:Host !^www\.example\.com(:\d+)?$ deny,status:403
SecRule REQUEST_METHOD ^POST$ chain,deny,status:403
SecRule REQUEST_HEADERS:Origin !^(https?://www\.example\.com(:\d+)?)?$

A user who visits such a site with a supporting user agent (which, at
the moment, includes the Chrome dev channel and the WebKit nightlies)
will be protected from CSRF attacks.  A site can implement these rules
without impacting users of non-supporting user agents.

> Now I'd like to
> years.

Which mod_security rules should a Web site implement to use the
Referer header as a CSRF defense without locking out some of its
users?

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 24 Jan 2009 22:03:07 +0100

>Referer header as a CSRF defense without locking out some of its

Adapting the header name and regular expression should do. Of course the
difference between the two would be that, and I am assuming proposal and
implementations are perfect, the Origin header would protect a few users
against all CSRF attacks, while the Referer header would protect nearly
all users against some CSRF attacks. Either way the web site would leave
some users vulnerable to some CSRF attacks.

The attraction of the Origin header is inversly proproptional to a site
owner's willingness to leave a substantial, but over time diminishing,
portion of his user base vulnerable to attack. If that is okay, you have
a great solution. If it is not okay, you will implement some fallback
defense, in which case you might just have used the Referer header to
decide when to trigger the fallback, if it does not work always anyways.

In my book, the latter would be relying on the Referer header to build
a CSRF defense, while the the former, relying on the Origin header, is
not really a CSRF defense until perfect implementations are universally
deployed. But arguing about that is not very useful, my point was and is
that you are oversimplifying and thereby overstating your case.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sat, 24 Jan 2009 15:16:12 -0800

The essential point which you are misunderstanding is this:

1) The attacker can force a user agent to suppress the Referer header,
mimicking a user behind a Referer-stripping proxy.

2) The attacker cannot force a supporting user agent to suppress the
Origin header and therefore cannot mimic a non-supporting user agent.

Fact (1) renders the Referer header useless as a CSRF defense, whereas
fact (2) allows the Origin header to be incrementally useful as a CSRF
defense.

On Sat, Jan 24, 2009 at 1:03 PM, Bjoern Hoehrmann <[email-masked]> wrote:
> Adapting the header name and regular expression should do.

That would not be an effective CSRF defense for any users.  The rules
you suggest implement lenient Referer validation, which does not
prevent an attacker from mounting a CSRF attack (as explained
previously in this thread).

> Of course the
> all users against some CSRF attacks.

An attacker has no trouble crafting an exploit against lenient Referer
validation.  It's pointless to consider defenses that can be defeated
by an adaptive adversary.

> Either way the web site would leave

The rules you suggest leave all users vulnerable.

> portion of his user base vulnerable to attack.

This is the case for any new browser feature designed to help with any
web security vulnerability.  We should not simply give up.  In fact,
the cost for a site to deploy an Origin-header based CSRF defense is
so low, I would expect many sites to deploy the defense, at first, to
supplement their existing CSRF defenses.

> If that is okay, you have

You haven't stated precisely how you envision this working, but
suppose you rely on your fall back defense when the Referer header is
absent.  If you have a vulnerability in your fall back, then, with the
Referer approach, 100% of your users are vulnerable CSRF.  With
Origin, users with supporting user agents are protected from CSRF.

> deployed.

In fact, the situation is precisely the opposite.  The Referer header
is useless as a CSRF defense until it reaches 100% deployment
(including not being removed by proxies), whereas the Origin header is
incrementally useful as a CSRF defense.

> But arguing about that is not very useful, my point was and is

I do not believe I am oversimplifying or overstating my case.  The
Referer-based CSRF defense you suggest as an alternative doesn't
actually block CSRF attacks.  By contrast, the Origin-based CSRF
defense I suggest does actually block CSRF attacks.

Adam

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 25 Jan 2009 15:30:47 +1100

On 23/01/2009, at 12:45 PM, Adam Barth wrote:
>> this is

I'd like to dig into that. You believe that most of the suppression of  
the Referer header is done in proxies, due to the differences seen in  
HTTP and HTTPS. However, there are also considerable differences  
between the block rates for same-domain vs. cross-domain requests; are  
you implying that these proxies are parsing the Referer and only  
blocking those that are cross-domain? If so, this seems an odd  
rationale; a person or company blocking referers for purpose of  
privacy would presumably be doing so for all values, not just cross- 
domain referers. Likewise, someone doing it to hide intranet URLs  
would be more likely to only hide those, rather than to stop cross- 
domain referers.

Additionally, discriminating requests as cross-domain is more  
expensive to implement in an intermediary, and these implementers are  
famously sensitive to performance issues. All of the products that I'm  
aware of would easily allow wholesale blocking of a header, but would  
require a relatively expensive (and thereby less likely) callout  
(e.g., with ICAP) to selectively block them based upon request state.

On the other hand, I do notice that Firefox has the ability to  
selectively configure how Referer headers are blocked, both in terms  
of same-site vs. cross-site and HTTP vs. HTTPS;
   http://kb.mozillazine.org/Network.http.sendRefererHeader
   http://kb.mozillazine.org/Network.http.sendSecureXSiteReferrer
Couldn't that account for at least a portion of the discrepancies you  
saw?

BTW, did you look for vanilla wafers <http://www.junkbusters.com/ijbfaq.html#wafers 
 > to see how much of this stripping could be attributed to JunkBuster?

Also, did you find any rationale for the difference between rates seen  
on network A vs. network B? It's a pretty wide range...

The numbers that I found especially interesting were for stripping of  
same-site XmlHttpRequest-generated Referer headers, which came in at  
(eyeballing Figure 3) about 0.6% on HTTP and 0.2% on HTTPS  
(discounting the Firefox 1.x bug, which isn't relevant to this  
discussion, since we're talking about updated browsers as a pre- 
condition). Aren't these numbers closer to what one would expect? In  
particular, they're much closer to the numbers for "custom" headers  
that you measure, which means we are looking at implementations that  
white-list as a significant factor (as well as statistical error, of  
course)...

Lastly -- Figure 3 says that its unit is requests; wouldn't IP  
addresses be a more useful number here? Or, better yet, unique clients  
(tracked with a cookie)? Otherwise it seems that the results could be  
skewed by, for example, a single very active proxy. Likewise, did you  
record the geographical distribution of clients? It would be nice to  
have assurances that this sample represents a global audience, and not  
just a selective (read: US) one.

> pressure to stop suppressing the header.

Certainly they do. If Cool New Cross-Site Web Apps are broken, and  
they explain to the user why it is broken, both ISPs and companies  
will come under pressure.

>> deployment

IMO 99% of the driving factor for deployment here is going to be new  
features -- supporting cross-site XmlHttpRequest with authentication,  
etc.

> Can you explain this hole?

Well, we'd be in the same situation as today; a current (non-Origin)  
browser would be able to make cross-site requests (using IMG,  
form.submit, etc.).

Cheers,

---

From: Adam Barth <[email-masked]>
Date: Sun, 25 Jan 2009 11:31:12 -0800

On Sat, Jan 24, 2009 at 8:30 PM, Mark Nottingham <[email-masked]> wrote:
> I'd like to dig into that. You believe that most of the suppression of the
> Referer header is done in proxies, due to the differences seen in HTTP and
> HTTPS.

That and the low suppression rates of document.referrer (see Figure 3
of http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf).

> However, there are also considerable differences between the block
> rates for same-domain vs. cross-domain requests; are you implying that these
> proxies are parsing the Referer and only blocking those that are
> cross-domain?

That's what appears to be going on.

> If so, this seems an odd rationale; a person or company
> blocking referers for purpose of privacy would presumably be doing so for
> all values, not just cross-domain referers.

Same-domain Referer headers do raise a privacy concern because the
server already knows which URL the user requested previously.
Cross-domain Referer headers can inform an entirely different entity
of what page you were just viewing.

> Likewise, someone doing it to
> hide intranet URLs would be more likely to only hide those, rather than to
> stop cross-domain referers.

We were unable to measure this in our experiment because we ran our
experiment on the Internet.

> Additionally, discriminating requests as cross-domain is more expensive to
> implement in an intermediary, and these implementers are famously sensitive
> to performance issues.

This can be done with a simple regular expression by matching the Host
header with the Referer header.

> All of the products that I'm aware of would easily
> allow wholesale blocking of a header, but would require a relatively
> expensive (and thereby less likely) callout (e.g., with ICAP) to selectively
> block them based upon request state.

Do you have an alternate explanation for the data we observe?

> On the other hand, I do notice that Firefox has the ability to selectively
> configure how Referer headers are blocked, both in terms of same-site vs.
> cross-site and HTTP vs. HTTPS;

This preference blocks both the Referer header and and the
document.referrer property (see documentation at
<http://kb.mozillazine.org/Network.http.sendRefererHeader>).  Our data
indicates that a vanishing number of users enable this preference.

>  http://kb.mozillazine.org/Network.http.sendSecureXSiteReferrer

This blocks cross-domain Referer headers when both domains are using
HTTPS.  Our data indicates that virtually no one enables this
preference.

> Couldn't that account for at least a portion of the discrepancies you saw?

No, for the reasons stated above.

> BTW, did you look for vanilla wafers
> <http://www.junkbusters.com/ijbfaq.html#wafers> to see how much of this
> stripping could be attributed to JunkBuster?

Unfortunately, we did not.  If we'd known about them at the time, then
we would have.  As far as I can tell from the JunkBuster documentation
at <http://www.junkbusters.com/ijbman.html>, they block the Referer
header for both same-domain and cross-domain requests.

> Also, did you find any rationale for the difference between rates seen on
> network A vs. network B? It's a pretty wide range...

This is a bit of a puzzle.  I suspect these network are targeting
different demographics and the truth lies somewhere in the middle.

> (eyeballing Figure 3) about 0.6% on HTTP and 0.2% on HTTPS (discounting the
> Firefox 1.x bug, which isn't relevant to this discussion, since we're
> talking about updated browsers as a pre-condition). Aren't these numbers
> closer to what one would expect?

I'm not surprised by a 3% suppression rate given the feedback from Web
sites that try strict Referer validation.  Note that the same-domain
suppression rate of the Referer header is much larger, around 6%, on
network B.

> In particular, they're much closer to the
> numbers for "custom" headers that you measure, which means we are looking at
> implementations that white-list as a significant factor (as well as
> statistical error, of course)...

The "suppression" of custom headers appears to be mostly due to
oddball user agents that don't have fully implemented XMLHttpRequest
objects.  Looking at events that occur less than 0.2% of the time puts
you WAY out in the tail of user agents.

Statistical error is not much of a factor, given the enormous sample
size.  Sampling bias is a concern, however, which is why we tried two
sampling techniques.

> Lastly -- Figure 3 says that its unit is requests; wouldn't IP addresses be
> a more useful number here? Or, better yet, unique clients (tracked with a
> cookie)?  Otherwise it seems that the results could be skewed by, for
> example, a single very active proxy.

We ran the numbers all three ways and they're similar.  Tracking
unique users with cookies is a bit unreliable due to third-party
cookie blocking in some user agents.

> Likewise, did you record the
> geographical distribution of clients? It would be nice to have assurances
> that this sample represents a global audience, and not just a selective
> (read: US) one.

Our add campaigns targeted the US.  We were unable to record IP
addresses due to ethical concerns.  We did record a keyed hash of the
IP addresses to re-identify requests from the same IP address, but we
deleted the key after the experiment to prevent further
re-identification.

> Certainly they do. If Cool New Cross-Site Web Apps are broken, and they
> explain to the user why it is broken, both ISPs and companies will come
> under pressure.

Unfortunately, folks developing Cool New Cross-Site Web Apps know that
the Referer is often suppressed and therefore do not rely on the
header in their designs.  Thus, step 1 never occurs.

> features -- supporting cross-site XmlHttpRequest with authentication, etc.

Great.  A number of browser vendors are interested in implementing the
header, giving their users yet more reasons to upgrade.

> Well, we'd be in the same situation as today; a current (non-Origin) browser
> would be able to make cross-site requests (using IMG, form.submit, etc.).

The Origin header is incrementally useful as a CSRF defense.  Users
with supporting user agents will benefit.  Users without supporting
user agents will be no worse off than they are today.  This is
different than the situation we are in today because sites must
engineer complex CSRF defense to help any of their users.  The Origin
header lets sits protect some of their users with minimal effort.

Adam

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 26 Jan 2009 12:51:31 +1300

> header lets sits protect some of their users with minimal effort.

My question then is should we be pouring effort into a solution that is 
only incrementally useful.

A a site operator myself, I'm not particularly interested in very easily 
being able to protect a very small number of users.  We need to protect 
ALL our users.

If this means we have to go to some secure token-based approach, then 
why bother with anything else as well?

As long as there is an appreciable proportion of our user-base using a 
browser that doesn't support Origin, we will need to cater to them.  By 
your argument, a show-stopper proportion is at least 3% or less.

It just seems to make the Origin header a bit redundant.

Also, without any sort of decent crypto involved, any reliance on 
client-supplied data for real security seems destined to fail. 

Even if you could get the major browsers to support it, getting servers 
to support it would be several orders of magnitude more difficult.  For 
many sites it would require patching scripts.. lots of them.  Why would 
I go to all the effort to patch all our scripts to check Origin to only 
protect a vanishingly-small-maybe-growing-but-never-enough proportion of 
my users when I could get them all with a decent system?

Regards

Adrien

---

From: Adam Barth <[email-masked]>
Date: Sun, 25 Jan 2009 16:14:15 -0800

On Sun, Jan 25, 2009 at 3:51 PM, Adrien de Croy <[email-masked]> wrote:
> My question then is should we be pouring effort into a solution that is only
> incrementally useful.

Several browser vendors have expressed interest in implementing a
feature to help sites defend themselves against CSRF.  By definition,
such as feature is only incrementally useful.

> being able to protect a very small number of users.  We need to protect ALL
> our users.

This number will grow over time.  Most sites have a list of browsers
which they support (i.e., expend resources to ensure their site works
properly).  After a few browser update cycles, this list may well
contain entirely supporting user agents.  If we don't start now, we'll
never improve the status quo.

In a sense, the same argument could be advanced about any browser
feature (CSS, <canvas>, etc).  This shouldn't stop us from innovating.

> If this means we have to go to some secure token-based approach, then why
> bother with anything else as well?

In the near term, you might as well use both secret tokens and the
Origin header.  At some point, your secret token defense will have a
vulnerability (just due to its complexity), and users of supporting
browsers will be protected by the Origin header.

> browser that doesn't support Origin, we will need to cater to them.  By your
> argument, a show-stopper proportion is at least 3% or less.

The difference is, with a Referer-based CSRF defense, the site is
unusable by 3% of its users.  With the Origin header, the site is
usable, but contains vulnerabilities, just like your Web site contains
XSS vulnerabilities in IE5 and Firefox 1.0 that you don't patch.

> It just seems to make the Origin header a bit redundant.

In reality, things will likely play out as follows in the near term:

1) OMG, my site has CSRF!
2) Add Web application firewall rules that blocks CSRF for Origin
header supporting browsers.
3) Spend the engineering resources to implement a secret token CSRF defense.

How far a site operator gets down this list of steps depends on their
engineering resources, their risk tolerance, and the deployment of the
Origin header.  Adding step (2) to the equation improves the situation
dramatically, especially as user agents that support the Origin header
gain market share.

> client-supplied data for real security seems destined to fail.

Unfortunately, you must rely on client-supplied data in order to build
a Web application.

> Even if you could get the major browsers to support it, getting servers to
> support it would be several orders of magnitude more difficult.  For many
> sites it would require patching scripts.. lots of them.

You should be able use the Origin header by adding a few rules to your
Web application firewall (see the three mod_security rules earlier in
this thread).  If you're not using a Web application firewall, you can
start using one by adding one line to your apache config file.

> Why would I go to
> all the effort to patch all our scripts to check Origin to only protect a
> vanishingly-small-maybe-growing-but-never-enough proportion of my users when
> I could get them all with a decent system?

One of the design goals of the Origin header is making is very easy to
use as a CSRF defense.  Compared the cost of implementing a secret
token based defense, implementing an Origin-based CSRF defense as well
is insignificant.

Adam

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 27 Jan 2009 13:40:28 +1300

> feature (CSS, <canvas>, etc).  This shouldn't stop us from innovating.

this is security we're talking about though, not additional nice-to-have 
features.  Again, we still need to secure all users, and can't wait 
until deployment of some new header.

> browsers will be protected by the Origin header.

I think the converse is more likely.  But even if the token-based 
approach is found to be insecure, the solution would be to simply fix that.

> dramatically, especially as user agents that support the Origin header
this presumes

a) there is a WAF.
b) people will even think about / know about Origin.  The set of people 
who are site developers is completely different to the set of people who 
are browser developers.  Even if you got 100% coverage of all browsers 
out of the box, it would still take a very long time to get the message 
out to every site operator.

I also think, that anyone who even does know about the Origin header 
will still look at the deployment level when deciding whether a more 
secure approach is necessary.

> Unfortunately, you must rely on client-supplied data in order to build
for security you don't need to rely on it.  You can use methods to 
greatly increase your confidence in submitted data.

> start using one by adding one line to your apache config file.
what about people who aren't running apache?

> token based defense, implementing an Origin-based CSRF defense as well
sure, the additional cost might be very small, but unless it can be 
completely relied on, it can't replace any other scheme, therefore 
doesn't save any expense.

Adrien

---

From: Adam Barth <[email-masked]>
Date: Mon, 26 Jan 2009 17:07:45 -0800

On Mon, Jan 26, 2009 at 4:40 PM, Adrien de Croy <[email-masked]> wrote:
> features.  Again, we still need to secure all users, and can't wait until
> deployment of some new header.

It is impossible to secure all the users who visit your Web site.  You
cannot secure users with IE5 or Firefox 1.0, for example.  Moreover,
the header provides incremental value while it is being deployed.

> I think the converse is more likely.  But even if the token-based approach
> is found to be insecure, the solution would be to simply fix that.

Regardless of which has a vulnerability, some fraction of your users
will be secure as long as BOTH don't have a vulnerability
simultaneously.

> this presumes
>
> a) there is a WAF.
> b) people will even think about / know about Origin.  The set of people who
> are site developers is completely different to the set of people who are
> browser developers.  Even if you got 100% coverage of all browsers out of
> the box, it would still take a very long time to get the message out to
> every site operator.

One common way I expect sites to make use of the header is via
security consultants.  Consider the case of WhiteHat Security.  If you
run a large Web site, you can hire WhiteHat to help you secure your
site.  Their preferred mod of operation (as I understand it from
Jeremiah's blog posts) is to install a Web application firewall and
apply "virtual patches" (i.e., WAF rules) to block vulnerabilities
until the site's developers can fix the underlying hole (assuming the
developers actually get around to fixing the holes).  The Origin
header is a powerful tool to add to the WAF arsenal against CSRF.

> I also think, that anyone who even does know about the Origin header will
> still look at the deployment level when deciding whether a more secure
> approach is necessary.

It's helpful to think about this in cost/benefit terms.  The cost of
adding the rules to your WAF is small and the benefit scales with the
header's deployment.

> for security you don't need to rely on it.  You can use methods to greatly
> increase your confidence in submitted data.

You simply cannot secure your Web site in IE5 or Firefox 1.0.  It is
simply impossible because the client-supplied data from these browser
lacks confidentiality or integrity.  Ultimately, the secret token
defense also relies on the browser for security.

> what about people who aren't running apache?

There is an entire industry of companies whose business plan involves
making it easy to deploy a WAF.

> completely relied on, it can't replace any other scheme, therefore doesn't
> save any expense.

It provides value:

1) Between the time a CSRF vulnerability is discovered and when its fixed.

2) For those sites that don't have the will / engineering time / money
to implement a complex secret token-based defense.

Of course, once all the browsers your site supports implement the
Origin header, then you don't need to implement a secret token based
defense at all.

Adam

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 27 Jan 2009 14:34:27 +1300

> the header provides incremental value while it is being deployed.
Do you have any more information on this you could refer me to?  I find 
it hard to believe that there can be no security scheme which would be 
browser-independent.

Regards

Adrien

---

From: "Larry Masinter" <[email-masked]>
Date: Mon, 26 Jan 2009 17:41:49 -0800

(cutting down distribution list)

Treating Referer like some kind of urban blight which we just leave to
molder seems odd to me. Is Referer deprecated? Expected to be sent as well,
but then frequently stripped? If the main difference between "Referer" and
"Origin" is that Referer contains too much information or is sent at the
wrong time, why not just fix "Referer"? It shouldn't be any harder to change
an existing header (compatibly) than it is to invent a new one.

For example, you could normalize Referer paths, use a different URI scheme,
replace the path (or even host name) with a different string (e.g., the hash
of the original), without introducing a new header. 

I think the discussion confuses the cost/benefits for different
constituencies. There are perhaps six categories:  implementers and
users/operators of browsers, firewall/gateway/proxies, web sites. 

* Browser makers:   Yay!
* Browser users: ? sounds confusing ? Is this like cookies? Don't they carry
viruses?
* Firewall makers:  Origin blocking! New feature! Oh boy!
* Firewall admins:  Low benefit (not on enough sites). Not blocking it might
cost me! (reveal host names)
* Server makers:   Roy sees no benefit, not worth the time.
* Server operators: doesn't solve problem until implementing browser share
is substantial.

You can't benefit browser users until a significant number of server
operators deploy this. But server operators will be reluctant to deploy
without browser implementation. Meanwhile, if firewalls start blocking the
header, then the benefit curve will go down, not up, and it will be another
set of useless bytes sent over the wire only to be parsed and blocked later
(i.e., just more cost).

Larry

---

From: Adam Barth <[email-masked]>
Date: Mon, 26 Jan 2009 18:37:34 -0800

On Mon, Jan 26, 2009 at 5:34 PM, Adrien de Croy <[email-masked]> wrote:
> Do you have any more information on this you could refer me to?  I find it
> hard to believe that there can be no security scheme which would be
> browser-independent.

These browsers are no longer maintained by their vendors.  Whenever
you see a vulnerability patched for IE7 or Firefox 3, there is a good
chance that vulnerability also exists in IE5 or Firefox 1.0.  In the
context of this discussion, that means the "secret" tokens you rely
upon for CSRF protection are not secret, and the attacker is free to
mount a CSRF attack against your site.

Adam

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 27 Jan 2009 15:46:51 +1300

> mount a CSRF attack against your site.

I was referring to a secure system that does not rely on secrecy, since 
we all know that secrecy is not security.

E.g. some sort of random token + hashing, where you pass a token to the 
browser, it does something to it (e.g. in script), and passes the result 
back.

one that can't be forged, and can't be replayed

Regards

Adrien

---

From: Adam Barth <[email-masked]>
Date: Mon, 26 Jan 2009 18:47:25 -0800

On Mon, Jan 26, 2009 at 6:46 PM, Adrien de Croy <[email-masked]> wrote:
>>>> cannot secure users with IE5 or Firefox 1.0, for example.

[snip]

> I was referring to a secure system that does not rely on secrecy, since we
> all know that secrecy is not security.
>

These browsers are unable to distinguish "your site" from "the
attacker's site."  Anything that your site can do, the attacker can do
on behalf of your site.  It's as if your site has an XSS vulnerability
that you cannot patch.

Adam

---

From: Adam Barth <[email-masked]>
Date: Mon, 26 Jan 2009 18:55:21 -0800

On Mon, Jan 26, 2009 at 5:41 PM, Larry Masinter <[email-masked]> wrote:
> but then frequently stripped?

Although the Referer header is not useful as a CSRF defense, the
Referer header is useful for a number of other purposes.  For example:

1) The Referer header tells me which search terms visitors to
http://www.adambarth.com/ used to find my home page.

2) The Referer header helps me track down pages that are still linking
to my old home page (at adambarth.org).

3) Many bloggers use the Referer header to link to pages that link to
their posts (trackback).

> If the main difference between "Referer" and

That would break the above use cases.

> constituencies.

I didn't understand the point you were trying to make with this
analysis.  Instead of listing the costs and benefits of various
actions to various constituencies, you just described what you thought
each party would do without explaining why.

> operators deploy this.

A user of a supporting browser will benefit as soon as one Web site
they use adopts the defense.

> But server operators will be reluctant to deploy
> without browser implementation.

The cost to deploy the defense is small and the benefit scales with
the deployment of supporting browsers.  At some point, the benefits of
deploying the defense will eclipse its cost.  My guess is this turning
point will come long before the Origin header reaches 90% deployment.

> Meanwhile, if firewalls start blocking the

We have yet to see even anecdotal evidence that firewall customers
will benefit from demanding that firewall vendors to block the header.

Adam

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 30 Jan 2009 19:23:59 +1100

> mimicking a user behind a Referer-stripping proxy.

Can you walk us through this attack, please? Or give a reference...

> Origin header and therefore cannot mimic a non-supporting user agent.

This supposes perfect implementation...

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 30 Jan 2009 14:19:36 +0100

http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf
Page 6: "Case Study: Facebook" contains an example; though I suspect
other ways of suppressing the Referer.

-- 
Thomas Broyer

---

From: Adam Barth <[email-masked]>
Date: Fri, 30 Jan 2009 09:56:12 -0800

> Can you walk us through this attack, please? Or give a reference...

As Thomas says, there are lots of ways to do this, mostly by design.
Here is a list of the top of my head:

1) The attacker hosts his HTML page over HTTPS and mounts a CSRF
attack against an HTTP URL.
2) The attacker hosts his HTML page over FTP.  Browsers don't send FTP
URLs in the Referer header.
3) The attacker hosts his HTML page over GOPHER.  Browsers don't send
GOPHER URLs in the Referer header
4) The attacker hosts his HTML in a DATA URL.  Browser don't send DATA
URLs in the Referer header.
5) The attacker hosts his HTML in a frame whose URL is the empty string.

There are many more ways, but none of these are bugs.

> This supposes perfect implementation...

Bugs happen, but they get patched.  If users aren't applying patches,
then they will very quickly have a user agent that is vulnerable to
much more severe attacks (e.g., universal cross-site scripting or
arbitrary code execution).

That said, it's actually pretty easy to enforce this property.  You
just add a branch in the code near where the request hits the wire
that says "if this is a non-safe request and I don't have an Origin
header yet, add "Origin: null".

Adam

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 31 Jan 2009 09:30:41 +1100

> There are many more ways, but none of these are bugs.

OK, so can't we get incremental improvement by specifying what Referer  
should be in these situations, and having browsers implement that?

---

From: Adam Barth <[email-masked]>
Date: Fri, 30 Jan 2009 14:36:07 -0800

[...]

> should be in these situations, and having browsers implement that?

Yes.  That's an interesting idea.  We could let user agents send the
value "null" in the Referer header and then require user agents to
always send a Referer header (possibly with the value "null").  This
would let servers distinguish between a header suppressed by the
attacker (value is null) and suppressed by the network (header is
gone) in the same way the Origin header proposes.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 30 Jan 2009 15:16:23 -0800

> gone) in the same way the Origin header proposes.

I was thinking something like

    Referer: data:hidden
    Referer: about:bookmarks
    Referer: https:

and others where appropriate.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Fri, 30 Jan 2009 15:19:56 -0800

> and others where appropriate.

There is some value in having a catch-all "OMG, I can't figure it out"
value.  Keep in mind that you want to have a branch somewhere deep in
the bowels of the HTTP stack that enforces this requirement, and that
code might not have enough context to figure out that this was the
user clicking on a bookmark.

Adam

---

From: Amit Klein <[email-masked]>
Date: Sat, 31 Jan 2009 20:04:50 +0200

Historically it was possible to set some/all HTTP request headers via
client side scripting (this was demonstrated with Flash several times,
e.g. http://www.securityfocus.com/archive/1/441014). Referer was thus
spoofed, rendering Referer-based defense methods useless. Obviously
this was/is an implementation bug, but perhaps one that could be
avoided had the HTTP standard mandated an explicit list of
disallowed-to-set-from-client-side headers. Would it thus be possible
to address such issue with the current proposal?