Subject: [http-state] two or three dots in domain name

From: Daniel Stenberg <[email-masked]>
Date: Thu, 6 Aug 2009 09:35:19 +0200 (CEST)

Hey

An interesting phrasing from the netscape spec on 'domain' is this:

   Only hosts within the specified domain can set a cookie for a domain and
   domains must have at least two (2) or three (3) periods in them to prevent
   domains of the form: ".com", ".edu", and "va.us". Any domain that fails
   within one of the seven special top level domains listed below only require
   two periods. Any other domain requires at least three. The seven special top
   level domains are: "COM", "EDU", "NET", "ORG", "GOV", "MIL", and "INT".

To me, this looks like artifacts of how things perhaps once used to work and 
it makes funny conclusions that I believe makes no sense. Does any current 
parser special cases these mentioned TLDs in any way? I know for sure libcurl 
doesn't.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Thu, 6 Aug 2009 00:49:31 -0700

To do correct cookie processing, you need to know about the public suffix list:

http://publicsuffix.org/

This isn't really static information, and it gets really complex in
Japan.  It's unclear to me how to handle this issue in the document.
We could either:

1) Require use of the public suffix list.

2) Give some reasonable approximation for user agents without access
to the public suffix list and recommend using the public suffix list
if available.

Adam

On Thu, Aug 6, 2009 at 12:35 AM, Daniel Stenberg<[email-masked]> wrote:
> parser special cases these mentioned TLDs in any way? I know for sure
> libcurl doesn't.
>
> --
>
>  / daniel.haxx.se
> _______________________________________________
> http-state mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/http-state
>

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 6 Aug 2009 09:51:53 +0200 (CEST)

> To do correct cookie processing, you need to know about the public suffix 

I question wether most existing implementations really use that. We need to 
separate "correct cookie processing" from "how cookie parsing is done in the 
real world today".

I know libcurl certainly does not use any type of public suffix list or even 
concept.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Thu, 6 Aug 2009 01:07:03 -0700

> separate "correct cookie processing" from "how cookie parsing is done in the

It's certainly done by Firefox and Google Chrome.  We should write a
test and see how other major implementations behave.  If you don't use
this list, a lot of people in Japan can't use the web securely.  If
you do what RFC 2109 says, then folks in the UK (and a lot of other
countries) can't use the web securely either.

We're going to very quickly become blocked behind the lack of a test
harness.  I'm hopeful someone will volunteer to help out in this area.

> I know libcurl certainly does not use any type of public suffix list or even

That is a useful data point.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 6 Aug 2009 10:59:02 +0200 (CEST)

On Thu, 6 Aug 2009, Adam Barth wrote:

>>> http://publicsuffix.org/
>
> It's certainly done by Firefox and Google Chrome.  We should write a test 
> and see how other major implementations behave.  If you don't use this list, 
> a lot of people in Japan can't use the web securely.

Yeah, IE for example still has a pretty fair percentage of the users. Does any 
version of IE use that/a suffix list?

Also, regarding the list handling. It's a pretty big list and I'm sure 
mistakes sneak in, both as in missing suffixes or in suffixes no longer being 
suffixes. How do you recommend the browsers to deal with the dynamic nature of 
it? Does it update every N days, or days it check for changes frequently or... 
?

And I figure that list should be received using HTTPS to reduce the risk of 
someone forging it.

> If you do what RFC 2109 says, then folks in the UK (and a lot of other 

Does anyone really consider RFC2109 as anything else than a curious episode 
we've since forgotten?

> harness.  I'm hopeful someone will volunteer to help out in this area.

Right.

It struck me that to test cookies against a test harness we also need to fool 
the clients name-resolving wise so that they think they're using "foo.bar.tld" 
when accessing the test server so that we can check things such as the suffix 
list usage.

So possibly the test suite should be made so that the tested clients are told 
to always go over a proxy, and the entire magic is done there.

We actually have a test suite in curl that is made up like this, with each 
test case in a single file using a XML-ish format, that could be a starting 
point.

-- 

  / daniel.haxx.se

---

From: "Yngve N. Pettersen" <[email-masked]>
Date: Thu, 06 Aug 2009 16:25:30 +0200

On Thu, 06 Aug 2009 09:35:19 +0200, Daniel Stenberg <[email-masked]> wrote:

> To me, this looks like artifacts of how things perhaps once used to work  
> and it makes funny conclusions that I believe makes no sense. Does any  
> current parser special cases these mentioned TLDs in any way? I know for  
> sure libcurl doesn't.

This is known as the "Cookie Monster Bug", and have been publicly known  
for more than 10 years.

Please see

   http://my.opera.com/yngve/blog/show.dml/267415
   http://my.opera.com/yngve/blog/show.dml/388840

for my coverage of the background, and my drafts

   http://www.ietf.org/internet-drafts/draft-pettersen-dns-cookie-validate-05.txt
   http://www.ietf.org/internet-drafts/draft-pettersen-subtld-structure-05.txt
   http://www.ietf.org/id/draft-pettersen-cookie-v2-04.txt

for my suggestions on the topic, as well as the already mentioned

   http://publicsuffix.org

for information on a database of such domains, and

   http://my.opera.com/yngve/blog/2009/06/17/refreshed-subtld-public-suffix-drafts

for information about Opera's variant of that database.

In my opinion the best way to solve the problem is to use the approach in  
my cookie-v2 draft, where the domain is either the server itself, or all  
hostnames having the stting server's name as a dot-suffix. That requires a  
significant update of the cookie specification, though.

-- 
Sincerely,
Yngve N. Pettersen

********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 6 Aug 2009 16:35:44 +0200 (CEST)

On Thu, 6 Aug 2009, Yngve N. Pettersen wrote:

> In my opinion the best way to solve the problem is to use the approach in my 
> cookie-v2 draft, where the domain is either the server itself, or all 

And we're now here to document existing practises, which is why I asked about 
if anyone was following that advice I referred to.

There are multiple ways this could be fixed if the spec was to be rewritten. 
But it isn't!

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Thu, 6 Aug 2009 10:17:28 -0700

On Thu, Aug 6, 2009 at 1:59 AM, Daniel Stenberg<[email-masked]> wrote:
> Yeah, IE for example still has a pretty fair percentage of the users. Does
> any version of IE use that/a suffix list?

I agree that our primary criterion for modifying the spec should be
widespread implementation.  My understanding is that IE does use a
suffix list, but we shouldn't take my word for it.  We should test.
:)

> mistakes sneak in, both as in missing suffixes or in suffixes no longer
> being suffixes. How do you recommend the browsers to deal with the dynamic
> nature of it? Does it update every N days, or days it check for changes
> frequently or... ?

Yngve may know more about this, but don't think the list changes often
enough for this to be a big problem in practice.  My understanding is
that browser vendors update the list when they ship new versions (be
they point releases or major releases).  Keep in mind that browsers
have a number of lists like this (e.g., the root CAs), so this is a
problem they need to solve anyway.

> And I figure that list should be received using HTTPS to reduce the risk of

I don't think we should get into specifying how to update the list.
That's doesn't seem to affect interoperability.

> Does anyone really consider RFC2109 as anything else than a curious episode

I've seen it quoted as authoritative on topics where it makes no
sense, leading me to believe that some number of people are confused
into thinking it describes reality.

> It struck me that to test cookies against a test harness we also need to
> fool the clients name-resolving wise so that they think they're using
> "foo.bar.tld" when accessing the test server so that we can check things
> such as the suffix list usage.
>
> So possibly the test suite should be made so that the tested clients are
> told to always go over a proxy, and the entire magic is done there.
>

Thanks for sending the mail about CURL's test suite.  I'll try to
investigate that later today.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 6 Aug 2009 10:25:27 -0700

Hi Yngve,

I'm glad you're participating in this group so we can all benefit from
your extensive knowledge of cookies.

> significant update of the cookie specification, though.

I think that cookie-v2 is important for the future of cookies, but
it's important to understand that we have different goals for the
current draft.  Our goal is to produce a spec that implementors can
use to interoperate with existing user agents and servers.  I agree
with Daniel that we should consider major changes, such as reversing
the trust hierarchy, out of scope for this work.

It seems likely that we'll generate lots of good ideas for how to
improve cookies in the course of our work here.  Hopefully you'll be
willing to incorporate the best of those ideas into cookie-v2.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 6 Aug 2009 23:55:18 +0200 (CEST)

On Thu, 6 Aug 2009, Adam Barth wrote:

> My understanding is that browser vendors update the list when they ship new 
> versions (be they point releases or major releases).  Keep in mind that 
> browsers have a number of lists like this (e.g., the root CAs), so this is a 

... and thus users can add (and remove) ca-certs to all major browsers.

> That's doesn't seem to affect interoperability.

Actually, I wasn't asking for it to be added to the spec, I was just curious 
on how browsers deal with this list. I can imagine how usage of browsers using 
different versions of that list built-in can make for some afternoon fun.

BTW, I could rather quickly find a domain in that list that looks just like a 
regular company site with no particular suffix "status" as far as I could see. 
I bet they wonder why they can't play cookies like other sites can (with some 
browsers)...

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Thu, 6 Aug 2009 15:02:09 -0700

On Thu, Aug 6, 2009 at 2:55 PM, Daniel Stenberg<[email-masked]> wrote:
> BTW, I could rather quickly find a domain in that list that looks just like
> a regular company site with no particular suffix "status" as far as I could
> see. I bet they wonder why they can't play cookies like other sites can
> (with some browsers)...

It might be entertaining to go through their amendment process to
revise this error (if indeed it is an error):

http://publicsuffix.org/submit/

If you're interested in how often the list changes, you can see the log here:

http://hg.mozilla.org/mozilla-central/log/91356aa326c2/netwerk/dns/src/effective_tld_names.dat

Note that not all of those changes are substantive.  Some are
licensing, reverting, etc.

Adam