Subject: [http-state] Ticket 3: Public Suffixes

From: Adam Barth <[email-masked]>
Date: Sat, 16 Jan 2010 00:01:00 -0800

I'd like to start discussing the issues in the tracker.  Hopefully
we'll be able to resolve them and make progress on the draft.

The first issue I'd like to discuss is what to do with public suffixes:
http://trac.tools.ietf.org/wg/httpstate/trac/ticket/3

I'm not setting any particular deadline to responding to this
proposal, but hopefully we can sort something out relatively soon and
then move on to the next issue.

== Overview ==

When a server sets a cookie, the server tells the user agent which
domains the cookie is for.  The protocol restricts the server to
specifying a suffix of its own host name.  For example, if the server
is foo.example.com, the server can set a cookie for foo.example.com,
example.com, or com.

This behavior turns out to be a security problem because when
attacker.com sets a cookie for com, that cookie is sent to example.com
and can confusing the example.com server.  This attack technique has
had a number of names of the years, but I'm going to call it "cookie
forcing" in this discussion.

Cookie forcing arises because of two design flaws in the cookie
protocol: (1) servers are allowed to set cookies for superdomains and
(2) when receiving a cookie, a server cannot determine the scope of
the cookie.  Unfortunately, fixing these underlying issues is out of
scope for our present work (although cookie2 has some interesting
ideas in this regard).

What user agents actually do is slightly data-intensive.  Essentially,
each user agent has a list of so-called public suffixes (or "registry
controlled" domains).  The user agent then refuses to accept cookies
whose scope is a public suffix.  For example, user agents will not
accept cookies for "com" or for "co.uk".  See, for example,
<http://publicsuffix.org/>, which is the list used by Mozilla and some
other open-source browsers.  Microsoft has it's own list for IE, which
I don't believe is publicly available.  These lists are somewhat long
and dynamic because there are a lot of domain registries in the world
and the rules are different in every province of Japan.

== Proposal ==

I propose that the spec contain a description of the vulnerability and
recommend at the SHOULD level that user agents drop cookies set for
public suffixes.

== Alternatives ==

One alternative is to require that all user agents use a specific
well-known public suffix list.  The problem with this approach is that
the list evolves with time and tracking the list is some amount of
effort.  Not all user agents will want to update when the list
updates.  These user agents will need to make a risk management
decision about how to approximate the list based on their security and
compatibility needs.

Another alternative is to recommend a heuristic that works in many
cases and then further recommend that user agents use the full list.
The problem with this approach is that I don't know of any simple
heuristics that provide reasonable behavior.  In the past, some user
agents have used heuristics based on the length of the top-level
domain (i.e., two characters => ccTLD => foo.cc is a public suffix).
Unfortunately, this heuristic has undesirable consequences for some
small countries that let folks register domains directly in the ccTLD.

A third alternative is to ignore the issue entirely, but this approach
would be a disservice to folks who need to write secure user agents.

Comments?
Adam

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Sat, 16 Jan 2010 12:39:54 +0100

Comments below:

> would be a disservice to folks who need to write secure user agents.

Alternative 4: Use a DNS based heuristic by only allowing cookies set to  
domains that have an IP address defined. If there is no IP address, remove  
the domain attribute

    See my draft describing Opera's implementation:  
http://www.ietf.org/internet-drafts/draft-pettersen-dns-cookie-validate-05.txt

Alternative 5: (which require an extensive change of the cookie  
specification) Reverse the domain semantics. Do not allow a site to set  
cookies for a parent domain, only the server itself or its subdomain any  
host with the server's name as the suffix (that is, example.com can set  
cookies either for the server example.com or the entire domain  
example.com; www.example.com cannot set a cookie for the domain  
example.com)

See the following for my suggestion  (the draft is currently expired)

    http://files.myopera.com/yngve/blog/draft-pettersen-cookie-v2-03.txt

Regarding public suffix, see my proposal

    http://www.ietf.org/internet-drafts/draft-pettersen-subtld-structure-05.txt

and Opera's upcoming system public suffix system (which is already live):

    http://my.opera.com/yngve/blog/2009/06/17/refreshed-subtld-public-suffix-drafts
    http://my.opera.com/rootstore/blog/2009/06/17/swisssign-ev-enabled-and-a-public-suffix-list

And my older articles about the topic

    http://my.opera.com/yngve/blog/show.dml/267415
    http://my.opera.com/yngve/blog/show.dml/388840

-- 
Sincerely,
Yngve N. Pettersen
 
********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Daniel Stenberg <[email-masked]>
Date: Sat, 16 Jan 2010 16:23:49 +0100 (CET)

> http://www.ietf.org/internet-drafts/draft-pettersen-dns-cookie-validate-05.txt

That's a 404, but I found it here:

   http://tools.ietf.org/html/draft-pettersen-dns-cookie-validate-05

... so Opera is actually using this now?

I know for a fact that there is at least one domain in .se that is actually 
using one of the documented suffixlist domains for a company (apparently it 
was baught and handed out more or less by mistake but...). This actually makes 
it possible to resolve the name to an IP  even though browsers using the 
suffixlist will treat it as invalid. This would speak FOR this concept. But 
likewise I can imagine there are sites that own example.com but without IP and 
they use two subdomain sites like foo.example.com and bar.example.com that 
both set cookies for example.com...

Unfortunately I don't have any particularly good idea on how to include all 
this in our spec. I think perhaps the only thing to do is to include a 
discussion about the problems and mention exactly this: a few ways that 
current implmentations use.

> specification)

Right, but if changing the spec would even be considered we could easily come 
up with many more alternatives... .-)

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Sat, 16 Jan 2010 10:18:48 -0800

On Sat, Jan 16, 2010 at 3:39 AM, Yngve N. Pettersen (Developer Opera
> domains that have an IP address defined. If there is no IP address, remove

One problem with this approach is that a number of major ISPs,
including COMCAST, return IP addresses for all non-existent domains.
The ISPs use these non-existent domains to show advertisements and
increase revenue.  Personally, I'd rather that they abstain from this
practice, but it demonstrates that this heuristic is somewhat fragile
for the security property we need.

> Alternative 5: (which require an extensive change of the cookie

As Daniel Stenberg says, extensive changes to the spec are off the
table in phase one.  However, we should revisit this approach in phase
two.

Adam

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Sat, 16 Jan 2010 20:01:56 +0100

On Sat, 16 Jan 2010 16:23:49 +0100, Daniel Stenberg <[email-masked]> wrote:

>> Alternative 4: Use a DNS based heuristic by only allowing cookies set  
>> to domains that have an IP address defined. If there is no IP address,  
>> remove the domain attribute
>>

Sorry, thought it was still valid. Will renew shortly. Archive copies are  
also available from my homepage.

> ... so Opera is actually using this now?

We have been using this method since v3.6, about 9-10 years.

> I know for a fact that there is at least one domain in .se that is  
> actually using one of the documented suffixlist domains for a company  
> (apparently it was baught and handed out more or less by mistake  
> but...). This actually makes it possible to resolve the name to an IP   
> even though browsers using the suffixlist will treat it as invalid. This  
> would speak FOR this concept. But likewise I can imagine there are sites  
> that own example.com but without IP and they use two subdomain sites  
> like foo.example.com and bar.example.com that both set cookies for  
> example.com...

These kind of sites is what is causing occasional problems. In Opera they  
can be worked around by setting an allow filter for the domain.

IMO this problem is alleviated by the common procedure by sites to use  
example.com as an alias for www.example.com.

> Unfortunately I don't have any particularly good idea on how to include  
> all this in our spec. I think perhaps the only thing to do is to include  
> a discussion about the problems and mention exactly this: a few ways  
> that current implmentations use.

I might note that the DNSop community have strong opinions against  
assuming anything about different levels of the DNS hierarchy. (My  
summary: They don't like the current method of domain specification in  
cookies, and they don't like public suffix)

> Right, but if changing the spec would even be considered we could easily  
> come up with many more alternatives... .-)

Some have suggested that clients probe servers for permission before  
sending cookies.

I think that long term domain owners should have a way define policies  
(such as cookies, scripting etc.) for their domain, possibly using a file  
format similar to what I have proposed for the public suffix system.

One way to proceed, what might be called alternative 6 in combination with  
other requirements, is that each cookie is associated with its origin  
hostname, for example in the form of a "$origin=server.example.com" value.  
This would allow the server to distinguish between valid and invalid  
cookies. This does however require an extension of the Cookie header  
syntax and semantics. It would IMO be easy to implement, and servers  
should ignore the values if they do not understand them.

Another way to proceed, which requires more changes to the spec, would be  
to deprecate "domain" for parent domain, and encourage adoption of  
domain=.hostname and/or a "subdomain=true" attribute, in combination with  
the $origin attribute

-- 
Sincerely,
Yngve N. Pettersen

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Sat, 16 Jan 2010 20:08:51 +0100

> including COMCAST, return IP addresses for all non-existent domains.

I know, and it could become a signifcant problem with the system,  
depending on how those systems work.

The question is though: Do they do this for domain.com if www.domain.com  
exists?

I know ISPs and others do this for unknown domains, but do they do it for  
registered domains, or the public suffixes that actually have domains  
under them?

The situation in which the heuristics will operate is that  
server.domain1.domain2.tld tries to set a cookie for domain2.tld. Would a  
ISP highjack the name domain2.tld to display ads, even if domain2.tld is a  
public suffix?

> table in phase one.  However, we should revisit this approach in phase

Could elements of the cookie-v2 spec be used? For example to initiate a  
move towards a better domain designation method?

-- 
Sincerely,
Yngve N. Pettersen

---

From: "corvid" <[email-masked]>
Date: Sat, 16 Jan 2010 19:47:16 +0000

> small countries that let folks register domains directly in the ccTLD.

This seems good to me to both
- let implementors know that they can use the publicsuffix list
- try to provide the best heuristic we know of for user agents who might
  not have the luxury of using publicsuffix for whatever reason (or can't
  depend on it)

---

From: Adam Barth <[email-masked]>
Date: Sat, 16 Jan 2010 14:16:32 -0800

On Sat, Jan 16, 2010 at 11:08 AM, Yngve N. Pettersen (Developer Opera
Software ASA) <[email-masked]> wrote:
> I know, and it could become a signifcant problem with the system, depending
> on how those systems work.
>
> registered domains, or the public suffixes that actually have domains under
> them?
>

I don't know, and I expect it varies from ISP to ISP.  In general,
though, whether or not a domain resolves is a fragile heuristic.
These folks have strong business incentives for creating these
records.  Even if their current behavior is ok, we don't have any
guarantees that it will remain so.

> Could elements of the cookie-v2 spec be used? For example to initiate a move
> towards a better domain designation method?

I don't think it's a good idea to defocus phase 1 by mixing in ideas
for changing the protocol.  The charter specifically forbids us from
introducing new syntax or semantics.

Adam

---

From: Adam Barth <[email-masked]>
Date: Sat, 16 Jan 2010 14:24:29 -0800

On Sat, Jan 16, 2010 at 11:01 AM, Yngve N. Pettersen (Developer Opera
Software ASA) <[email-masked]> wrote:
> I think that long term domain owners should have a way define policies (such
> as cookies, scripting etc.) for their domain, possibly using a file format
> similar to what I have proposed for the public suffix system.

That sounds like a reasonable long-term solution.

> cookies. This does however require an extension of the Cookie header syntax
> and semantics. It would IMO be easy to implement, and servers should ignore
> the values if they do not understand them.

Unfortunately, this requires introducing new syntax and is therefore
off the table in phase 1.

> Another way to proceed, which requires more changes to the spec, would be to
> deprecate "domain" for parent domain, and encourage adoption of

Even if we recommended that servers not use domain cookies (for
superdomains), we'd still have to instruct user agent how to handle
them securely, which is what this ticket is about.

Adam

---

From: Adam Barth <[email-masked]>
Date: Sat, 16 Jan 2010 14:39:35 -0800

>  not have the luxury of using publicsuffix for whatever reason (or can't

Here's the best heuristic I know.  The algorithm can probably be
simplified and explained more clearly.

[[
Roughly, getDomain(strFQDN) amounts to:

1> If the final label is empty, drop it for the purposes of this
1> algorithm
// Otherwise "www.example.com." would have four labels "www",
"example", "com", "".  Instead, we drop the final label.

2> Name the labels Ln,...,L3,L2,L1; decreasing from start
(Leftmost=Ln) to finish (Rightmost=L1).
// If at any point in this algorithm the result demands >n labels,
getDomain returns "".

3> Check n > 1.  If not, there's no domain, just a plain hostname.
Return ""; exit.
// Dotless FQDNs consist of a host only, there is no domain.

4> Check L1 == "tv".  If so, getDomain returns L2.L1; exit.
// "tv" is a special-case "completely flat" ccTLD for historical reasons.

5> Check Len(L1) > 2.  If so, getDomain returns L2.L1; exit.
// Len(L1)>2 suggests L1 is a gTLD rather than a ccTLD.
// If Len(L1)<=2 we assume L1 is a part of a ccTLD.

6> Check if L2 in gTLD list "com,edu,net,org,gov,mil,int".  If so,
getDomain returns L3.L2.L1; exit.
// gTLDs, when they appear immediately left of a ccTLD (modulo
exception in step 4), are considered a part of the TLD.

7> If L1 is in the list "GR,PL" AND L2 is NOT in the gTLD list,
getDomain returns L2.L1; exit.
// GR and PL are considered "flat" ccTLDs EXCEPT when a gTLD appears in L2.
// getDomain("a.pl") returns "a.pl"
// getDomain("a.uk") returns ""

8> If Len(L2) < 3 getDomain returns L3.L2.L1; exit.
// getDomain("aa.bb.cc") returns "aa.bb.cc"

9> Otherwise, getDomain returns L2.L1
// getDomain("aa.bbb.cc") returns "bbb.cc"
]]

The heuristic is sufficiently ugly and wrong that I'd prefer to
recommend that user agent that care about security use the public
suffix list.  For example, it breaks the cookie protocol for domains
in the "to" ccTLD.  If a user agent doesn't care about security, then
it can skip the public suffix check and the protocol will still
function fine.

Adam

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Sun, 17 Jan 2010 00:10:26 +0100

> it can skip the public suffix check and the protocol will still

Just to confirm the "wrong" part:

This algorithm would at least classify two norwegian public suffixes,  
vgs.no (highschools of Norway) and kommune.no (municipalities/counties of  
Norway), as ordinary domains, as well as 400+ others in dot-no namespace.

It would also classify the domain of Norway's largest print and online  
newspaper, vg.no, as a public suffix, and Norways largest daily economic  
newspaper's domain, dn.no, as well.

IMO looking up an online public suffix repository, like Opera will be  
doing shortly, is probably the best option, as it does not require  
hardcoding a list of domains into the executable, and eliminates the need  
to update the execuatable when the list changes.

-- 
Sincerely,
Yngve N. Pettersen

---

From: David Morris <[email-masked]>
Date: Sat, 16 Jan 2010 20:58:04 -0800 (PST)

Is a WG meeting planned for Anaheim? If the HTTPBIS group is meeting
hopefully coordinated to not conflict and be the same day?

Dave

---

From: Bil Corry <[email-masked]>
Date: Sun, 17 Jan 2010 12:43:04 -0800

> hopefully coordinated to not conflict and be the same day?

We're having a meeting in Anaheim; Jeff is taking care of the necessary actions required to do so.  Jeff, can you request that our meeting doesn't conflict with HTTPBIS?

- Bil