Subject: [http-state] Combined headers

From: Thomas Broyer <[email-masked]>
Date: Wed, 25 Nov 2009 17:08:08 +0100

Hi all,

HTTP (both RFC1945 [1], RFC2616 [2] and HTTPbis [3]) state (this is
taken from HTTPbis, RFC2616 is slightly different in wording, but not
in meaning):
   Multiple header fields with the same field name MUST NOT be sent in a
   message unless the entire field value for that header field is
   defined as a comma-separated list [i.e., #(values)].  Multiple header
   fields with the same field name can be combined into one "field-name:
   field-value" pair, without changing the semantics of the message, by
   appending each subsequent field value to the combined field value in
   order, separated by a comma.

Currently, draft-abarth-cookie (checked the -05 draft) is in violation
of this rule [4]. This wasn't the case however of neither RFC 2109
[5], RFC 2965 [6] or cookie-v2 [7]. Netscape's cookie_spec (which can
hardly be called a "spec") didn't formally defined the syntax of the
Set-Cookie header so it didn't explicitly violated HTTP, though it is
implicit that each cookie would be sent in its own Set-Cookie header
(without taking into account that those headers could be combined into
a single one).

I think the new Cookies RFC we're preparing shouldn't violate this
rule from HTTP. If it is somehow required for compatibility ("don't
break the web") that each cookie be sent in its own Set-Cookie header
and those headers shouldn't be combined; then instead of violating
HTTP, we could just add a note (a SHOULD?) in a "compatibility"
section.

Does anyone know how current browsers handle such combined headers?
are there known intermediaries (proxies) that combine Set-Cookie
headers?

Apache HTTPD at least special-cases Set-Cookie headers and doesn't
combine them [8]:
    /* The HTTP specification says that it is legal to merge duplicate
     * headers into one.  Some browsers that support Cookies don't like
     * merged headers and prefer that each Set-Cookie header is sent
     * separately.  Lets humour those browsers by not merging.
     * Oh what a pain it is.
     */
There are unfortunately no mention of which browsers "some browsers" refers to.

Here are some cookie-related Apache HTTPD bugs, for reference:
 - https://issues.apache.org/bugzilla/show_bug.cgi?id=18230
 - https://issues.apache.org/bugzilla/show_bug.cgi?id=19934
 - https://issues.apache.org/bugzilla/show_bug.cgi?id=2609
 - https://issues.apache.org/bugzilla/show_bug.cgi?id=7195

AFAICT, the Cookie header isn't expected to be sent multiple times in
the same request, so it doesn't have this issue; but maybe we should
allow commas as separators, and multiple headers that could be
therefore merged (as proposed by RFC 2109)

[1] http://tools.ietf.org/html/rfc1945#section-4.2
[2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
[3] http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-08#section-3.2
[4] http://tools.ietf.org/html/draft-abarth-cookie-05#section-4.1.1
[5] http://tools.ietf.org/html/rfc2109#section-4.2.2
[6] http://tools.ietf.org/html/rfc2965#page-5
[7] http://tools.ietf.org/html/draft-pettersen-cookie-v2-04#section-3.2.2
[8] http://svn.apache.org/viewvc/httpd/httpd/trunk/server/util_script.c?view=markup#l414

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Nov 2009 17:13:16 +0100

Thomas Broyer wrote:
> ...

But note 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-latest.html#rfc.section.3.2>:

"Note: the "Set-Cookie" header as implemented in practice (as opposed to 
how it is specified in [RFC2109]) can occur multiple times, but does not 
use the list syntax, and thus cannot be combined into a single line. 
(See Appendix A.2.3 of [Kri2001] for details.) Also note that the 
Set-Cookie2 header specified in [RFC2965] does not share this problem."

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Wed, 25 Nov 2009 08:18:00 -0800

> HTTP, we could just add a note (a SHOULD?) in a "compatibility"

My understanding is that we are constrained by web compatibility here
because folks use "," characters in the Expires attribute.

I agree that it would be aesthetically pleasing for the Set-Cookie
header to be combinable, but, unfortunately, that's not how the cookie
protocol works in practice.

> Does anyone know how current browsers handle such combined headers?

I'll add a test to the test suite.  My believe is that they parse the
header as described in Section 5.1, but we should test to see.

> are there known intermediaries (proxies) that combine Set-Cookie

I don't know.

> There are unfortunately no mention of which browsers "some browsers" refers to.

We should test, but I'd bet at least IE and Firefox.

>  - https://issues.apache.org/bugzilla/show_bug.cgi?id=7195

Thanks!

Adam

---

From: Adam Barth <[email-masked]>
Date: Wed, 25 Nov 2009 08:26:50 -0800

> header as described in Section 5.1, but we should test to see.

This is now covered by 0017 and 0018:

0017:

Set-Cookie: z=y, a=b

results in

Cookie: z=y, a=b

which means this is being treated as a single cookie with a comma in it's value.

0018:

Set-Cookie: z=y; foo=bar, a=b

results in

Cookie: z=y

which means the comma is being sucked into the attribute value.

These tests pass in IE8, Firefox 3.5, and Google Chrome.

Adam

---

From: David Morris <[email-masked]>
Date: Wed, 25 Nov 2009 08:28:30 -0800 (PST)

>> Does anyone know how current browsers handle such combined headers?

PLease keep in mind that browsers aren't the only User-Agents which 
utilize cookies. There are many special purpose clients which would
break if you try to change this constraint and a server or proxy is
foolish enough to fold multiple set cookies into a single line.

Leave bad enough alone.

Dave Morris

---

From: Thomas Broyer <[email-masked]>
Date: Wed, 25 Nov 2009 18:05:23 +0100

On Wed, Nov 25, 2009 at 5:13 PM, Julian Reschke wrote:
> "Note: the "Set-Cookie" header as implemented in practice (as opposed to how
> it is specified in [RFC2109]) can occur multiple times, but does not use the
> list syntax, and thus cannot be combined into a single line. (See Appendix
> A.2.3 of [Kri2001] for details.) Also note that the Set-Cookie2 header
> specified in [RFC2965] does not share this problem."

Thanks for pointing it out Julian; it was just a few lines below the
passage I quoted, but I didn't scroll down enough and completely
missed it!

On Wed, Nov 25, 2009 at 5:18 PM, Adam Barth wrote:
> On Wed, Nov 25, 2009 at 8:08 AM, Thomas Broyer wrote:

My understanding of the HTTP rule is that it doesn't mandate that a
header has to be splittable on commas; i.e. this is not a reversible
operation, not a bijective function. Still from HTTPBis-p1:
   The order in which header fields with
   the same field name are received is therefore significant to the
   interpretation of the combined field value

On Wed, Nov 25, 2009 at 5:26 PM, Adam Barth wrote:
> On Wed, Nov 25, 2009 at 8:18 AM, Adam Barth wrote:

This however is a more-than-compelling argument...

...for the addition of a "compatibility" section that'd urge
implementors to just not combine Set-Cookie headers, without
necessarily ruling it out as invalid (and a "willful violation" of
HTTP) ;-)
(Well, actually, test 0017 is quite convincing though...)

I mean, nobody combines Set-Cookie headers (or quickly find out it
doesn't work as expected with most browsers, including the 2 most used
worldwide: IE and Firefox), so there's little compatibility risk with
allowing such merging *but* telling implementors not to do so (so
everyone is compliant with HTTP; except browsers, until they update to
draft-abarth-cookie or whatever RFC it'll become).

It would, however, complicate parsing, as you just couldn't just split
on semi-colons and then split on equal signs.
But more tests are needed first: how browsers (I know there are other
"classes" or UAs, but cookies are mainly directed towards browsers)
handle this for example (to be repeated for other attributes; test
0018 uses an unknown attribute):
   Set-Cookie: a=b;max-age=3600, c=d;path="/"
And more research too: are there web sites today relying on test 0017?
are there popular frameworks and their implementations (ASP.NET,
servlets, etc.) that would emit such a Set-Cookie header when passed a
{ "x": "y, a=b" } tuple?)

Now the question is: is it worth it? and what's the httpbis WG
position? (the "note" in httpbis-p1, as pointed out by Julian above,
talks about a compatibility concern, not some kind of "willful
violation" as draft-abarth-cookie is making; so would the httpbis WG
be OK with such a "willful violation"?)

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Adam Barth <[email-masked]>
Date: Wed, 25 Nov 2009 10:11:47 -0800

> (Well, actually, test 0017 is quite convincing though...)

I don't understand what you're proposing here.  The point of the
protocol specification is to help folks write interoperable
implementations.  The server conformance section requires that server
not combine Set-Cookie header field (or else they'd break existing
clients).  The client conformance section requires a detailed parsing
algorithm reverse engineered from existing clients.

Which conformance class are you suggesting we modify?

> draft-abarth-cookie or whatever RFC it'll become).

The interoperability risk is that server that combine Set-Cookie
headers will fail to interoperate with existing user agents.

> on semi-colons and then split on equal signs.

Are you suggesting we change the parsing algorithm required of
conforming user agents?  If so, do you have tests that demonstrate
that the change you're suggesting is already in wide use?

>   Set-Cookie: a=b;max-age=3600, c=d;path="/"

I've added this as 0019, with a slight tweak to make it work in our
testing framework:

Set-Cookie: foo=b;max-age=3600, c=d;path=/

results in

Cookie: foo=b

This test passes in IE8, Firefox 3.5, Google Chrome.  Note that I've
removed the quotes from the path attribute as those are
non-conforming.  We can test that behavior too, but it seems like a
separate issue from parsing commas.

> And more research too: are there web sites today relying on test 0017?

I don't have evidence for this belief, but I suspect the answer is yes.

> { "x": "y, a=b" } tuple?)

I don't understand why that matters.  If these frameworks exist, they
would fail to interoperate today.  Perhaps they'll read this spec and
interoperate better in the future.  Or, worse, they would break if we
started recording these cookies.

> Now the question is: is it worth it?

Is what worth what?

> and what's the httpbis WG position?

They seem fine with the behavior given that they wrote it into their spec.

> (the "note" in httpbis-p1, as pointed out by Julian above,

I don't understand the phrase "willful violation."  In the end, the
cookie protocol works the way it does, with or without anyone's will.

Adam

---

From: Adam Barth <[email-masked]>
Date: Wed, 25 Nov 2009 10:13:50 -0800

> started recording these cookies.

Apologies, I misread your question.  I don't understand the relevance
of framework-level APIs on the wire-level protocol.

Adam