Subject: [http-state] Cookie storage limits (was Re: Updated draft)

From: Adam Barth <[email-masked]>
Date: Mon, 17 Aug 2009 09:48:25 -0700

On Mon, Aug 17, 2009 at 9:34 AM, Bil Corry<[email-masked]> wrote:
> A bit dated, Nicholas Zakas of Yahoo! talks about the maximum number of cookies[1]:
>
> ---8<---
>    * Microsoft indicated that Internet Explorer 8 increased the cookie limit per domain to 50 cookies but I’ve found that IE7 also allows 50 cookies per domain. Granted, this may have been increased with a system patch rather than having the browser’s first version ship like this, but it’s still more than the 20 that was commonly understood to be the limit.

I discussed this issue with Eric Lawrence.  Apparently, IE raised the
cookie limit from 20 to 50 in a patch to improve compatibility with a
major site that required 24 cookies to function properly.  This patch
rendered a major Japanese bank's web site completely inoperable
because it was relying on the browser to evict the 21rst cookie and
fell over and died when it got too many cookies.  There was strong
pressure to reverse the patch, but they did not.

>    * Firefox has a per-domain cookie limit of 50 cookies.
>    * Opera has a per-domain cookie limit of 30 cookies.
>    * Safari/WebKit is the most interesting of all as it appears to have no perceivable limit through Safari 3.1. I tested setting up to 10,000 cookies and all of them were set and sent along in the Cookie header. The problem is that the header size exceeded the limit that the server could process, so an error occurred.
> --->8---

Yeah, so if I were writing a cookie implementation today, I would
definitely use 50 cookies per domain.  However, putting that as a hard
limit into the spec doesn't seem very future proof.  I think
suggesting a limit of 50 is appropriate because that's the limit used
by >90% of the market.

> He also mentions differences in cookie eviction:
>
> ---8<---
>   1. The least recently used (LRU) approach automatically kicks out the oldest cookie when the cookie limit has been reached in order to allow the newest cookie some space. Internet Explorer and Opera use this approach.
>   2. Firefox does something strange: it seems to randomly decide which cookies to keep although the last cookie set is always kept. There doesn’t seem to be any scheme it’s following at all. The takeaway? Don’t go above the cookie limit in Firefox.
> --->8---

The draft current specs LRU for cookie eviction, but I haven't figured
out a good way to test this yet.  It might be worth reading Firefox's
code to understand what algorithm they're using.

> And differences in cookie size support (although this conflicts with results Daniel Stenberg posted earlier[2]):
>
> ---8<---
>    * Firefox and Safari allow cookies with up to 4097 characters, that’s 4096 for the name and value and one for the equals sign.
>    * Opera allows cookies with up to 4096 characters, which is for the name, value, and equals sign.
>    * Internet Explorer allows cookies with up to 4095 characters, which is for the name, value and, equals sign.
> --->8---

Interesting.  I have two size limit tests, but I haven't figured out
anything conclusive here.  Looks like the limit is something on the
order of 4096.  We should definitely try to figure out some broadly
interoperable, least common denominator limits to recommend in the
server conformance section.

Adam

---

From: Bil Corry <[email-masked]>
Date: Mon, 17 Aug 2009 12:19:12 -0500

> suggesting a limit of 50 is appropriate because that's the limit used

I agree, we should suggest 50 but leave it to the implementor to decide.

> code to understand what algorithm they're using.

Having an understanding and recommendation for total cookie jar size and how it relates to cookie eviction may be worthwhile.  And not mentioned yet (at least I don't think it has) is how cookie eviction works with "secure" cookies -- do UAs evict non-secure cookies first?  Do they evict secure cookies from HTTP?  Do they evict HTTPOnly cookies from JavaScript?

Many of these issues are mentioned in the Browser Security Handbook, we should work through them at some point:

	http://code.google.com/p/browsersec/wiki/Part2#Same-origin_policy_for_cookies

> server conformance section.

Another unknown in all of this is if JavaScript cookies are handled differently than non-JS cookies.

- Bil

---

From: Adam Barth <[email-masked]>
Date: Mon, 17 Aug 2009 11:32:33 -0700

> Having an understanding and recommendation for total cookie jar size and how it relates to cookie eviction may be worthwhile.  And not mentioned yet (at least I don't think it has) is how cookie ev…

Great questions.  We need to improve our test harness so we can
experiment with these behaviors.

Adam

---

From: Jim Manico <[email-masked]>
Date: Mon, 17 Aug 2009 14:38:09 -0400

If you are going to make a recommendation about the maximum number of  
stored cookies per domain, but really just leave it up to the  
implementor (which I am not fond of), can you at least reference or  
describe the risk of straying from the 50 cookie per domain browser  
"standard" in this most excellent document?

Jim Manico

On Aug 17, 2009, at 12:48 PM, Adam Barth <[email-masked]> wrote:

>>    * Microsoft indicated that Internet Explorer 8 increased the  
>> cookie limit per domain to 50 cookies but I’ve found that IE7 also 
>>  allows 50 cookies per domain. Granted, this may have been increas 
>> ed with a system patch rather than having the browser’s first vers 
>> ion ship like this, but it’s still more than the 20 that was commo 
>> nly understood to be the limit.
>
>>    * Safari/WebKit is the most interesting of all as it appears to  
>> have no perceivable limit through Safari 3.1. I tested setting up  
>> to 10,000 cookies and all of them were set and sent along in the  
>> Cookie header. The problem is that the header size exceeded the  
>> limit that the server could process, so an error occurred.
>> --->8---
>
>>   1. The least recently used (LRU) approach automatically kicks out  
>> the oldest cookie when the cookie limit has been reached in order  
>> to allow the newest cookie some space. Internet Explorer and Opera  
>> use this approach.
>>   2. Firefox does something strange: it seems to randomly decide  
>> which cookies to keep although the last cookie set is always kept.  
>> There doesn’t seem to be any scheme it’s following at all. The  
>> takeaway? Don’t go above the cookie limit in Firefox.
>> --->8---
>
>> And differences in cookie size support (although this conflicts  
>> with results Daniel Stenberg posted earlier[2]):
>>
>> ---8<---
>>    * Firefox and Safari allow cookies with up to 4097 characters,  
>> that’s 4096 for the name and value and one for the equals sign.
>>    * Internet Explorer allows cookies with up to 4095 characters,  
>> which is for the name, value and, equals sign.
>> --->8---
>
> _______________________________________________
> http-state mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/http-state

---

From: Adam Barth <[email-masked]>
Date: Mon, 17 Aug 2009 11:58:47 -0700

On Mon, Aug 17, 2009 at 11:38 AM, Jim Manico<[email-masked]> wrote:
> If you are going to make a recommendation about the maximum number of stored
> cookies per domain, but really just leave it up to the implementor (which I
> am not fond of), can you at least reference or describe the risk of straying
> from the 50 cookie per domain browser "standard" in this most excellent
> document?

Sure.  I've added TODOs to the draft.  We might want to collect
various considerations like this in an appendix of some sort.

Adam