Subject: [http-state] Conformance classes

From: Adam Barth <[email-masked]>
Date: Mon, 14 Dec 2009 15:41:58 -0800

On Mon, Dec 14, 2009 at 3:33 PM, Daniel Stenberg <[email-masked]> wrote:
> On Fri, 11 Dec 2009, Adam Barth wrote:
>> Is this text clearer?
>>
>> NOTE: If the server supplies Set-Cookie headers that conform to the
>> grammar in this section, then the user agent will return a Cookie header
>> that conforms to the grammar above.  However, if the user agent receives a
>> Set-Cookie header that deviates from the grammar in this section, the
>> requirements in the next section might cause the user agent to return a
>> Cookie header that deviates from the preceding grammar.
>
> They way I read this is still like this:
>
> If a server violates the spec and sends something that isn't a valid cookie,
> the server should expect that the client *too* may violate the spec and send
> something non-conformant back.

I think it's helpful to think of things in terms of conformance
classes.  The spec has two conformance classes: server and user agent.

1) The server is constrained to sending a Set-Cookie header in a
particular grammar.  Headers in that grammar have some understandable
semantics defined by the spec.

2) The user agent conformance class is more intricate and is required
to behave in a certain way upon receiving a sequence of bytes from the
server.

Neither is violating the spec.  As Postel tell us, "Be conservative in
what you do; be liberal in what you accept from others."  These
conformance classes merely codify precisely how the server and the
user agent ought to do this.

> Invalid headers are wrong, they should be treated as protocol errors and the
> world will be a much better place if clients IGNORE such cookies instead of
> trying hard to obey.

While it might be nice to live in that world, if we write the spec in
that way, we will fail to achieve our goals because the spec will be
insufficiently precise to be helpful in constructing a user agent that
interoperates with existing servers.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 14 Dec 2009 17:13:09 -0800

> user agent ought to do this.

It is "be conservative in what you send ..."

> interoperates with existing servers.

That is not our goal.  IETF specs do not live in the bizarre
world of implementation spec religion that is HTML5.

The requirements are on what is sent in the message and
how it should be interpreted by the recipient, not on
how a specific implementation must be coded to do everything
in the same order and with all the same bugs as some legacy PoS.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Mon, 14 Dec 2009 19:24:56 -0800

Roy, I would ask you to keep a civil tone on this mailing list.
People wonder why user agent implementors are reluctant to engage with
the IETF.  Likening their work product to excrement is certainly not
helping the situation.

On Mon, Dec 14, 2009 at 5:13 PM, Roy T. Fielding <[email-masked]> wrote:
>> As Postel tell us, "Be conservative in
>> what you do; be liberal in what you accept from others."
>

That's what I though too, but I was quoting from Wikipedia:

http://en.wikipedia.org/wiki/Robustness_principle

> world of implementation spec religion that is HTML5.

Perhaps I spoke to broadly.  My goal is to write a spec that helps
folks implementing the cookie protocol in user agents that wish to
interoperate with existing and future servers.

> in the same order and with all the same bugs as some legacy PoS.

Right, we're concerned only with what messages to send on the wire.
In this case, if a user agent receives these messages:

Set-Cookie: foo
Set-Cookie:

what message should it send in the Cookie header?  Should it send
"foo" or should it omit the header?

We can phrase the question semantically, if you like.  What does an
empty Set-Cookie header field mean?  Does it mean "ignore this
message" or does it mean "the value of the nameless cookie is the
empty string"?

In this case, the question is largely academic.  What matters is what
dwitte decides to do in Firefox.  If he decides to do nothing, then
we'll likely spec that either behavior is acceptable.  If he decides
to match IE, then we'll likely spec that an empty Set-Cookie header
field means the latter.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 14 Dec 2009 20:01:54 -0800

>> On Dec 14, 2009, at 3:41 PM, Adam Barth wrote:

[snip]

> interoperate with existing and future servers.

Indeed, it's definitely necessary to process some bogus cookie-related  
headers in a specific way to successfully interoperate with many  
existing servers. For people who want an idealized cookie feature and  
don't care about working with arbitrary existing Web servers, there is  
always RFC2965.

> message" or does it mean "the value of the nameless cookie is the

I assume you mean "ignore this header" rather than "ignore this  
message" (ignoring the whole HTTP response message would be harsh  
indeed).

> to match IE, then we'll likely spec that an empty Set-Cookie header

Why is Firefox the decisive factor? From other messages it sounds like  
there's a split, and the IE behavior is not IE-only. Is there a reason  
in principle to prefer one or the other?

Regards,
Maciej

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 14 Dec 2009 20:32:28 -0800

> Roy, I would ask you to keep a civil tone on this mailing list.

That was civil.  Please don't sideline discussions with irrelevant
points of order.

> the IETF.  Likening their work product to excrement is certainly not

It isn't their work product, then.

> http://en.wikipedia.org/wiki/Robustness_principle

Ah, that's from RFC 761.  The one most people use is the original
in RFC 760:

  The implementation of a protocol must be robust.  Each implementation
  must expect to interoperate with others created by different
  individuals.  While the goal of this specification is to be explicit
  about the protocol there is the possibility of differing
  interpretations.  In general, an implementation should be conservative
  in its sending behavior, and liberal in its receiving behavior.  That
  is, it should be careful to send well-formed datagrams, but should
  accept any datagram that it can interpret (e.g., not object to
  technical errors where the meaning is still clear).

That's the one I am familiar with from the IETF hallway conversations.

> interoperate with existing and future servers.

Right, and the correct interoperation in this case is to not
send an invalid Cookie header after receiving an invalid Set-Cookie.
It is to ignore the invalid Set-Cookie or translate it into
something valid that is known to be an acceptable replacement.

> message" or does it mean "the value of the nameless cookie is the

It means nothing, so ignore the field (not the message).

> to match IE, then we'll likely spec that an empty Set-Cookie header

What matters is rough consensus *and* running code.  Neither one
requires unanimity, but it does require an agreement on what we
are willing to send.  The interoperable behavior is for the
server to fix its output.

Quite frankly, it doesn't matter what IE does in this case, nor
Firefox, so their particular behavior should not result in a change
to the protocol either way.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Mon, 14 Dec 2009 23:28:42 -0800

On Mon, Dec 14, 2009 at 8:01 PM, Maciej Stachowiak <[email-masked]> wrote:
> I assume you mean "ignore this header" rather than "ignore this message"
> (ignoring the whole HTTP response message would be harsh indeed).

Yes.  :)

> there's a split, and the IE behavior is not IE-only. Is there a reason in
> principle to prefer one or the other?

The other option is for IE and Safari to change to match Firefox and
Chrome.  We should be very cautious about speccing something that IE
won't implement.  The easiest way to achieve that is to spec the IE
behavior.  In some cases, there are good reasons not to spec the IE
behavior, but this case doesn't seem that compelling.

Another way to think about this case is that if Firefox changes to
match IE and Safari, it's a slam dunk for me to convince Chrome to
match as well.

This case is fairly obscure because (1) the server needs to be using
nameless cookies, which aren't very common, and (2) the server needs
to send an empty Set-Cookie header.  We could measure to see how often
this actually occurs, but I suspect it's less than 0.001% of the time,
which is below my compatibility target of 99.999%.

Adam

---

From: Adam Barth <[email-masked]>
Date: Mon, 14 Dec 2009 23:39:35 -0800

> something valid that is known to be an acceptable replacement.

We could define a valid Cookie header in such a way to make this
possible.  However, if we did this, the grammar for Cookie would be
fairly loose.  I'd have to think about it carefully, but I suspect you
can cause user agents to generate most sequences of bytes in a Cookie
header.

I think it's helpful to provide a grammar for "common" Cookie headers.
 Meaning, if the server doesn't send something bizarre, the user agent
will return a Cookie header from some simple grammar.  That's what the
grammar in that section is meant to convey.  I can make the section
non-normative if that would be helpful.

> It means nothing, so ignore the field (not the message).

What's your basis for this conclusion?  Some user agents treat it as
meaning "the value of the nameless cookie is the empty string".  I
don't see that we have a clear basis for preferring "means nothing"
over that semantics.

> server to fix its output.

Although fixing the server output might be desirable, user agents in
the real world have to interoperate with existing servers that won't
fix their output.

> to the protocol either way.

It matters a great deal how existing user agents implement the cookie
protocol.  If we ignore them, we'll end up repeating the mistakes of
RFCs 2109 and 2965.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 14 Dec 2009 23:41:04 -0800

> behavior, but this case doesn't seem that compelling.

Hmm, I realize now that I hastily misread the original message and the  
other option was "spec that either behavior is acceptable". In general  
that's not a very appealing option but you make a decent argument that  
this particular case is obscure.

> which is below my compatibility target of 99.999%.

Potentially relevant questions:

- How common is use of nameless cookies in general?
- If not with an empty header than how can you clear them?
- Dealing with nameless cookies via script was cited - would that have  
the same behavior?

It seems to me like it would be a good idea to make the spec something  
that HTML5 could reference to define document.cookie in more detail.  
As things stand, the HTML5 spec for document.cookie is alarmingly  
vague. <http://dev.w3.org/html5/spec/Overview.html#dom-document-cookie>

Regards,
Maciej

---

From: Adam Barth <[email-masked]>
Date: Mon, 14 Dec 2009 23:50:07 -0800

On Mon, Dec 14, 2009 at 11:41 PM, Maciej Stachowiak <[email-masked]> wrote:
> Hmm, I realize now that I hastily misread the original message and the other
> option was "spec that either behavior is acceptable". In general that's not
> a very appealing option but you make a decent argument that this particular
> case is obscure.

Yes.  I agree that we should prefer to produce a precise spec.

> - How common is use of nameless cookies in general?

I believe they're required at the 99% or 99.9% compatibility levels,
but that's an estimate based on old information.  We could measure if
we wanted to know exactly.

> - If not with an empty header than how can you clear them?

You can clear them with either of these headers:

Set-Cookie: =
Set-Cookie: foo; Expires=<date in the past>

> - Dealing with nameless cookies via script was cited - would that have the
> same behavior?

We already need to distinguish between setting a cookie via an HTTP
header and setting a cookie via script to properly deal with HttpOnly
cookies.  I don't have a good test harness for script yet, but it
sounds like it's important that we allow script to clear nameless
cookies by setting document.cookie to the empty string.

If I were dwitte, I would change Firefox to match IE, but that's his
call.  Having different behavior for Set-Cookie and document.cookie
seems like needless complexity.

> It seems to me like it would be a good idea to make the spec something that
> HTML5 could reference to define document.cookie in more detail. As things
> stand, the HTML5 spec for document.cookie is alarmingly vague.
> <http://dev.w3.org/html5/spec/Overview.html#dom-document-cookie>

HTML5 already references draft-abarth-cookie:

http://dev.w3.org/html5/spec/Overview.html#refsCOOKIES

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Dec 2009 09:37:50 +0100

Maciej Stachowiak wrote:
> ...
> ...

I probably sound like a broken record, but anyway...

If there's no interoperability for this today then that is a signal that 
this does not *need* to be specified.

It's ok to point out the issue and to mention different strategies, but 
there's no reason to make a normative requirement here.

The cookie spec is for UA implementers, but not only for them. 
Overspecification leads to huge specs that nobody except UA implementers 
is going to *really* read. If you feel that UA implementers need spec 
text oon top of that, then please consider moving that into a separate 
implementer's guide.

Best regards, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 15 Dec 2009 01:02:37 -0800

> don't see that we have a clear basis for preferring "means nothing"

The original Netscape Set-Cookie spec requires a NAME=
There is no such thing as a nameless cookie.  At least one
implementation I know of depends on that fact, according
to MSDN's description of cookie handling in WinHTTP

<http://msdn.microsoft.com/en-us/library/aa383261%28VS.85%29.aspx#removing_cookie_headers>

It is using an empty Cookie field as a special-case trigger
to remove all cookies from a request.

Apache httpd will toss empty Set-Cookie headers, AFAICT.

> protocol.  If we ignore them, we'll end up repeating the mistakes of

It matters a great deal how ALL implementations process the Cookie
and Set-Cookie header fields.  More importantly, the spec needs to
define what most of those implementations can meaningfully handle.

A cookie without a name field is very clearly not going to be
handled in an interoperable manner, even if all of the browsers
did the same thing (which has already been shown to be false).
That's why it isn't allowed in the first place.

Please don't create problems that do not exist.  There is no need
to invent nameless cookies and no need to change the protocol to
pretend that such an invalid field value would be handled in
anything other than random fashion.

....Roy

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 15 Dec 2009 01:14:43 -0800

On Dec 15, 2009, at 12:37 AM, Julian Reschke wrote:

> Maciej Stachowiak wrote:
>> ...
>> Why is Firefox the decisive factor? From other messages it sounds  
>> like there's a split, and the IE behavior is not IE-only. Is there  
>> a reason in principle to prefer one or the other?
>> ...
>
> If there's no interoperability for this today then that is a signal  
> that this does not *need* to be specified.

It may be that for this case, there isn't one specific behavior that  
is needed. But I wouldn't assume that just from the fact that there  
are differences in behavior. To give an obvious counter-example: IE's  
event model has many differences from DOM Events Level 2, and nearly  
all sites work with IE. But most people would not conclude that we  
should therefore delete nearly all normative requirements in DOM2EV. I  
have also encountered cases where sites rely on something that has  
unspecified behavior, and only one of exactly two behaviors will work,  
even if it seems like the intersection semantics are thin or  
nonexistent. Thus, you can't conclude much from the bare fact of  
implementation differences.

There are many issues to consider when deciding whether to mandate a  
specific behavior or leave it unspecified. One is the benefit of a  
single consistent behavior. Another is the benefit of having an extra  
degree of implementation freedom.

> It's ok to point out the issue and to mention different strategies,  
> but there's no reason to make a normative requirement here.
>
> Overspecification leads to huge specs that nobody except UA  
> implementers is going to *really* read.

I fail to see how giving multiple suggested behaviors will result in a  
more compact spec than specifying one. A lot of people fear this  
bogeyman of overspecification, but in my experience working with  
implementations of many different specifications, I have yet to find  
one that was too precise and comprehensive to be usable. Usually the  
opposite problem occurs.

> If you feel that UA implementers need spec text oon top of that,  
> then please consider moving that into a separate implementer's guide.

Why should anyone have to read an additional document to find out what  
"Set-Cookie:" with no value does? If it needs a specific behavior it  
should be in the spec. If it doesn't, then not. Shuffling it off to a  
separate document to satisfy some people's sense of design taste is  
not a rational solution.

Regards,
Maciej

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 01:21:08 -0800

On Tue, Dec 15, 2009 at 1:02 AM, Roy T. Fielding <[email-masked]> wrote:
> There is no such thing as a nameless cookie.

Actually, nameless cookies pre-date named cookies.  The original
cookie protocol used only nameless cookies.  By your reasoning, we
should forbid named cookies.

> At least one

I don't see the relevance of Win32 APIs on wire-level protocols.

> Apache httpd will toss empty Set-Cookie headers, AFAICT.

Great!  The spec forbids servers from sending these headers on the wire.

> define what most of those implementations can meaningfully handle.

I agree that this is a reasonable criterion for the server conformance
class.  Servers ought to use the cookie protocol in a way that
interoperates with most user agents.  Similarly, user agents ought to
use the cookie protocol in a way that interoperates with most servers.

> That's why it isn't allowed in the first place.

Why?  The behavior of nameless cookies is actually largely
interoperable.  The interoperability failures are only at the extreme
edge case of an empty Set-Cookie header.  I suspect this is because
browsers that don't implement nameless cookies are ruthlessly beat
into conformance with the consensus cookie protocol by servers that
rely on them.  Maciej has mentioned his implementation experience of
this exact effect.

> to invent nameless cookies

I didn't invent nameless cookies.  Implementing the cookie protocol in
a user agent is a real pain point and a large barrier to new entrants
in the browser market.

> and no need to change the protocol to

When you talk about changing the protocol, what baseline are you
referring to?  The netscape cookie spec?  RFC 2109?  RFC 2965?  The
real cookie protocol that is in wide use has never actually been
specified.  That's why this working group exists.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 15 Dec 2009 01:23:53 -0800

On Dec 15, 2009, at 1:02 AM, Roy T. Fielding wrote:

> At least one implementation I know of depends on that fact, according
> <http://msdn.microsoft.com/en-us/library/aa383261%28VS.85%29.aspx#removing_cookie_headers 
> >
>

How does this rely on nonexistence of nameless cookies? I don't  
believe the existence of nameless cookies ever requires a client to  
send an actual empty Cookie header.

> Apache httpd will toss empty Set-Cookie headers, AFAICT.

It seems like reasonable behavior for a server to avoid sending  
invalid response headers. This doesn't seem to rely on whether or not  
other servers do send invalid headers.

Regards,
Maciej

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Dec 2009 10:41:49 +0100

Maciej Stachowiak wrote:
> implementations of many different specifications, I have yet to find one 
> that was too precise and comprehensive to be usable. Usually the 
> opposite problem occurs.
> ...

The difference is the amount and readability of normative parts of the 
spec.

>> If you feel that UA implementers need spec text oon top of that, then 
>> please consider moving that into a separate implementer's guide.
> 
> separate document to satisfy some people's sense of design taste is not 
> a rational solution.

I agree with that. We just need to come to WG consensus (not UA vendor 
consensus) whether somethings needs to be in. If we decide it doesn't 
need to, implementers are still free to write an implementer's guide 
that has more details, potentially even as a WG product (or even an 
appendix to the cookie spec).

BR, Julian

---

From: David Morris <[email-masked]>
Date: Tue, 15 Dec 2009 01:59:53 -0800 (PST)

> a user agent is a real pain point and a large barrier to new entrants

Huh ... I've been a software engineer for 40+ years and based on my
experience, I really doubt that handling cookie headers is more than 1% 
of the effort in creating a new browser. And as a correlary, I suspect
cookie issues don't even surface in the high-level planning process.

Lets not exagerate too much.

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 15 Dec 2009 02:11:22 -0800

On Dec 15, 2009, at 1:59 AM, David Morris wrote:

> experience, I really doubt that handling cookie headers is more than  
> 1% of the effort in creating a new browser. And as a correlary, I  
> suspect

I've implemented the cookie code in a browser (Safari), and it was a  
surprisingly large amount of work, considering how simple the feature  
seems. And compatibility bugs were still being found years after the  
initial implementation. Note that your "1% of effort" estimate is  
actually a ridiculously high amount of effort for such a trivial  
feature, relative to the overall complexity of a browser.

Regards,
Maciej

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 15 Dec 2009 11:43:32 -0800

> should forbid named cookies.

I don't know what you are talking about. The original proposal was

http://1997.webhistory.org/www.lists/www-talk.1995q2/0134.html

and it was defined the same way in Lou's patent, in the Netscape
docs, and in Netscape Navigator's implementation.  NAME has always
been required.

> I don't see the relevance of Win32 APIs on wire-level protocols.

Because a Win32 client trying to send a nameless cookie would
trigger that API.

I don't have time to investigate the thousand or so implementations
of HTTP to find out which ones will explode on a nameless cookie.
I don't want to add junk to the specification that has never
existed before and for which existing implementations are not
prepared to handle.

> rely on them.  Maciej has mentioned his implementation experience of

I don't believe it.  I have seen no evidence to suggest that any
server on the planet is dishing out Set-Cookie headers with no name
and expecting to receive those same nameless cookies back from
the user agent.  I might believe that some joker is using it to
detect user agent versions, but not for some application that
expects it to be interoperable.

> specified.  That's why this working group exists.

As near as I can tell, you are making statements solely on the
basis of what appears inside the javascript-accessible DOM after
a protocol exchange.  The cookie protocol is what goes over the wire
and works consistently across all implementations.

....Roy

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 15 Dec 2009 12:02:43 -0800

> Because a Win32 client trying to send a nameless cookie would

That is not supported by the documentation you linked:

a) The special case is only triggered by setting the Cookie header to  
an empty value - however, nothing about nameless cookies ever requires  
sending an empty "Cookie" header. The special case we are concerned  
with at the moment is what to do when receiving an empty "Set-Cookie"  
header.

b) The special case is only triggered when the replace flag is used to  
set the header. Thus, even if a client for some reason added an extra  
empty Cookie header in addition to any others, they would not trigger  
the behavior of removing all cookies from the request.

Regards,
Maciej

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 12:13:23 -0800

> docs, and in Netscape Navigator's implementation.  NAME has always

Hum...  I don't remember off-hand what my source is for the history of
nameless cookies.  In any case, it's not relevant how it is that we
arrived in our current situation.  The fact remains that nameless
cookies are widely implemented and occasionally used.  User agents are
forced to implement them to interoperate with existing servers.

> existed before and for which existing implementations are not

You keep asserting that nameless cookies do not exist.  The fact is
that they *do* exist.  If there are clients that explode upon
encountering a nameless cookie, they're in trouble already.  By
defining how a user agent ought to handle them, these implementations
be fixed to not explode.

> expects it to be interoperable.

Well, enough user agent implementors have seen servers do this that
IE, Firefox, Safari, and Chrome all implement nameless cookies the
same way.  We've seen bugs cited on this mailing list where the
Firefox developers have had to fix their nameless cookie
implementation to more closely match IE to get web sites to work.  If
you a haven't seen this evidence, you haven't been paying attention.

> and works consistently across all implementations.

We're talking about what value the Cookie header should take when
returned to the server.  There are also considerations about how to
process the set-cookie-string in non-HTTP scenarios, but let's ignore
those for now.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 15 Dec 2009 22:33:33 +0100 (CET)

On Tue, 15 Dec 2009, Adam Barth wrote:

> The fact remains that nameless cookies are widely implemented and 
> occasionally used.  User agents are forced to implement them to interoperate 
> with existing servers.

This is yet another statement I simply cannot agree with. I've played with 
client side cookies for many years with many users and nameless cookies have 
*NEVER* been an issue with existing servers. I've never even seen a server in 
the wild actually send one.

Sure, it might be just because I'm ignorant and so, but still.

> If there are clients that explode upon encountering a nameless cookie, 
> they're in trouble already.

I can guarantee that we can find MANY such implementations if we just wanted 
to. I'm now again speaking from the angle of non-browser (focused) 
implementations.

> By defining how a user agent ought to handle them, these implementations be 
> fixed to not explode.

If we would accept a very (and I mean VERY) small fraction of bad behavior and 
call that "what implementations use in the wild" we would yes. But I'm in the 
camp that says we should mark out stupid things as: stupid. For all we know 
those three cases could just as well be a bug in an implementation and I quite 
expect that there are LOTS of bugs in many implmentations and we don't go 
adding those to the spec.

> We've seen bugs cited on this mailing list where the Firefox developers have 
> had to fix their nameless cookie implementation to more closely match IE to 
> get web sites to work.

Really? I certainly missed that. Can you give me/us a link?

BTW, when I opposed to the phrasing in the spec I didn't mean just nameless 
cookies. It way that paragraph is put it does not limit it to just that, but 
would included lots of other syntax mistakes as well.

If nameless cookies are to be considered "kosher" we should make sure that the 
spec clearly documents that. We should not put it into a "if the server 
violates the spec so may the client" section.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 13:58:11 -0800

On Tue, Dec 15, 2009 at 1:33 PM, Daniel Stenberg <[email-masked]> wrote:
> *NEVER* been an issue with existing servers. I've never even seen a server
> in the wild actually send one.
>

Well, I'm happy to measure how often user agents receive nameless
cookies.  If we see that nameless cookies are used by existing
servers, will that convince you?  If so, what compatibility threshold
shall we use?

> If we would accept a very (and I mean VERY) small fraction of bad behavior
> and call that "what implementations use in the wild" we would yes. But I'm
> in the camp that says we should mark out stupid things as: stupid.

What criterion shall we use for identifying behaviors as "bad" or
"stupid"?  These are very judgmental words.

>> We've seen bugs cited on this mailing list where the Firefox developers
>> have had to fix their nameless cookie implementation to more closely match
>> IE to get web sites to work.
>

Here's an example of Bank of America using nameless cookies (sent to
this list recently by dwitte):

https://bugzilla.mozilla.org/show_bug.cgi?id=169091

> would included lots of other syntax mistakes as well.

I've removed this paragraph because it seems to be more confusing than
it's worth.

> If nameless cookies are to be considered "kosher" we should make sure that
> the spec clearly documents that. We should not put it into a "if the server

I feel like I'm repeating myself, but servers are forbidden from
sending Set-Cookie headers that contain nameless cookies.  User agents
have to deal with Set-Cookie headers generated by non-conformant
servers.  Pretending otherwise is fantasy.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 15 Dec 2009 23:18:30 +0100 (CET)

On Tue, 15 Dec 2009, Adam Barth wrote:

> Well, I'm happy to measure how often user agents receive nameless cookies. 
> If we see that nameless cookies are used by existing servers, will that 
> convince you?  If so, what compatibility threshold shall we use?

If nothing else it would indeed interest me, as I was completely honest when I 
said I've never seen any.

Compatibility thresholds are of course very tricky. I even suspect that if the 
company in the firefox bug report hadn't been a very large american one the 
verdict might've been different.

> What criterion shall we use for identifying behaviors as "bad" or "stupid"? 
> These are very judgmental words.

Yeah, those are my own personal judgements of weirdnesses. I don't think we as 
a group should use those terms at all. I think nameless cookies are stupid, 
even if you'd show that some percentage of sites actually use such.

> I feel like I'm repeating myself, but servers are forbidden from sending 
> Set-Cookie headers that contain nameless cookies.  User agents have to deal 
> with Set-Cookie headers generated by non-conformant servers.  Pretending 
> otherwise is fantasy.

Ok then, so let me repeat myself too: if we want a client to act in some 
particular defined way on this illegal header, then we shouldn't make the 
header illegal in the first place.

Set-Cookie: headers that are using illegal syntaxes should be ignored. You 
seem to suggest that if the illegal syntax concerns the name and it is a zero 
length name, then the header shouldn't be ignore anyway and instead the client 
should act in some specific way. In my book, the received header is then not 
illegal anymore but is in fact a header that the client needs to deal with and 
thus legal.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 14:27:49 -0800

> header illegal in the first place.

One solution is to make the server conformance section non-normative.
Instead of giving rules for how a server ought to behave, we can
recommend a profile of the cookie protocol that has cleaner semantics
than the full, legacy-encumbered protocol.

> seem to suggest that if the illegal syntax concerns the name and it is a
> zero length name, then the header shouldn't be ignore anyway and instead the
> client should act in some specific way. In my book, the received header is
> then not illegal anymore but is in fact a header that the client needs to
> deal with and thus legal.

There seems to be some value in requiring servers to use a subset of
the full semantics understood by the client.  For example, one could
hope that in the future these conformant servers would sufficiently
over-populate the non-conformant servers that a future version of the
cookie protocol (e.g., cookie2 or cookie3) could drop support for some
of the less aesthetic parts of the current cookie protocol.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 15 Dec 2009 15:13:19 -0800

> https://bugzilla.mozilla.org/show_bug.cgi?id=169091

That issue lists one site that used javascript to set and test an
invalid implementation assumption about cookies in the DOM.
Said site was later fixed by the site owner.  The cookie
protocol had nothing to do with it.

> servers.  Pretending otherwise is fantasy.

They can deal with nameless cookies set via javascript in whatever
way you want to specify in HTML5/DOM5.  The only issue we have is whether
the user agent is supposed to send a nameless cookie to the server
in a Cookie header field of HTTP.  I think the answer to that is "no",
regardless of how the cookie was set.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 15:20:04 -0800

> protocol had nothing to do with it.

I've offered several times to measure how often this part of the
cookie protocol is used on the wire.  You seem uninterested in the
facts on the ground.  Would you consider such a measurement
dispositive?

> regardless of how the cookie was set.

You have yet to provide a basis for this belief.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 15 Dec 2009 15:49:50 -0800

> facts on the ground.  Would you consider such a measurement

I would consider the measure for what it measured, yes.  I just
looked through all of my persistent cookies (back about five years
worth of stored copies) and none of them had empty names.  I don't
know whether that is because none were received/set or whether the
storage interface simply refuses or is unable to store cookies
without names.

> You have yet to provide a basis for this belief.

Yes, I did.  All of the specifications of Cookies state that NAME
is required.  Server implementors actually read specifications and,
occasionally, enforce them.  In order to introduce invalid syntax
to the protocol at this time, all of the implementations would have
to agree.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 16:07:59 -0800

On Tue, Dec 15, 2009 at 3:49 PM, Roy T. Fielding <[email-masked]> wrote:
> I would consider the measure for what it measured, yes.

That's a clever way of not answering the question.  If I measured that
user agents send nameless cookies to servers more than X% of the time
they send Cookie headers, would you agree that the spec should explain
how to handle these cookies (for some value of X)?

> is required.

The previous specifications of cookies also say that a $Version
attribute is required.  If those specifications said sensible things,
we wouldn't need this working group.

> Server implementors actually read specifications and,
> occasionally, enforce them.

Even if 99% of server implementations do not use nameless cookies,
user agents still need to interoperate with the 100th most-popular web
site.

> In order to introduce invalid syntax

The most widely used user agents to agree: nameless cookies exist.  Do
you have test cases that demonstrate otherwise?

Nameless cookies can only be "invalid" with respect to some normative
requirements.  Citing the dead letter of the Netscape cookie spec, RFC
2109, or RFC 2965 is pointless.

Adam

---

From: David Morris <[email-masked]>
Date: Tue, 15 Dec 2009 18:04:14 -0800 (PST)

> facts on the ground.  Would you consider such a measurement

Well, you haven't described your methodology so it would be impossible to 
answer that question. Since I don't the effort required or the statistical 
validity of the sample I can only suggest that you conduct your 
'measurement' and describe the methodolgy ... unless the effort is too 
great or the description would be proprietary. Right now we have nothing
more that speculative.

> You have yet to provide a basis for this belief.

With nothing but anecdotal evidence to the contrary and citations that
demonstrate this was always forbidden, good engineering principles suggest
that garbage should be eliminated.

If your measurements identify a significant volume of such cookies or 
set-cookies, the wording might need to vary, but the case should still
be that the usage is forbidden. But if the sample shows a pattern of
a few major sites, I'd propose a bit of jawboning warning them that the
new spec will be forbidding the contstruct and that future browser
versions will be following the new specification. Q.E.D, it is 1999 and
you'd best get your y2k date issues fixed.

There are various strategies for getting the situation fixed depending on
the usage pattern and number of discrete origin servers. For example,
someone could create an Apache module to notice this situation and create
a special log, email it once a day to the webmaster, etc.

There is however a class of potentially broken servers, which are unlikely 
to be in your measurement ... the embedded HTTP servers included in small 
network appliances ... I'd wager that most of them receive infrequent
maintenance and I also suspect that some of the vendors don't provide
updates for older versions...

David Morris

---

From: Adam Barth <[email-masked]>
Date: Tue, 15 Dec 2009 18:10:32 -0800

On Tue, Dec 15, 2009 at 6:04 PM, David Morris <[email-masked]> wrote:
> answer that question.

The methodology would be analogous to that described in
<http://www.adambarth.com/papers/2009/barth-caballero-song.pdf>.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 15 Dec 2009 19:35:09 -0800

> how to handle these cookies (for some value of X)?

No, but I would if the servers can be shown to be expecting those
nameless cookies (i.e., this isn't just a parsing bug on the client
that is wasting network bits).

> we wouldn't need this working group.

One of them does, I guess -- not the ones I referenced.

> user agents still need to interoperate with the 100th most-popular web

No, they don't, but if you can find one of those that actually
depends on receiving a nameless Cookie header field via HTTP,
then we should change the grammar accordingly (even if I think
that slapping the server owner with a trout would be a more
productive use of our time).

> you have test cases that demonstrate otherwise?

No.  I HAVE NEVER SEEN ONE IN MY ENTIRE LIFETIME.  That's 16 years
of looking at HTTP traces and I had never even heard of such a thing
until this thread began.  Why would I test for it?  How am I supposed
to test all of the other HTTP implementations for interoperability
on a feature that doesn't even exist?

....Roy

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 15 Dec 2009 21:36:36 -0800

On Dec 15, 2009, at 2:18 PM, Daniel Stenberg wrote:

> Ok then, so let me repeat myself too: if we want a client to act in  
> some particular defined way on this illegal header, then we  
> shouldn't make the header illegal in the first place.
>
> Set-Cookie: headers that are using illegal syntaxes should be  
> ignored. You seem to suggest that if the illegal syntax concerns the  
> name and it is a zero length name, then the header shouldn't be  
> ignore anyway and instead the client should act in some specific  
> way. In my book, the received header is then not illegal anymore but  
> is in fact a header that the client needs to deal with and thus legal.

Why is "ignore" the only acceptable behavior when processing a  
nonconforming header field? Or to turn it around, why must everything  
with a processing requirement for the recipient other than "ignore" be  
considered legal to send? Surely it can be rational to define error  
handling other than "ignore" in at least some cases. For example,  
HTML4.01 requires clients to render the contents of unknown elements,  
even though such elements are a conformance error for the producer of  
the document.

We have here a situation where there are both legacy servers and  
legacy clients. Sending certain unusual constructs may break some  
legacy clients. Thus, it is prudent that new servers not do so. That  
seems to justify an RFC2119 keyword of some sort. On the other hand,  
some legacy servers may use these unusual constructs regardless. It  
seems prudent for new clients to handle these constructs as apparently  
expected, to work with those servers. This also seems to justify an  
RFC2119 keyword of some sort. The fact that the required syntax to  
accept is broader than the legal syntax to send is unfortunate, but  
not fundamentally illogical.

Regards,
Maciej

---

From: Daniel Stenberg <[email-masked]>
Date: Wed, 16 Dec 2009 14:51:58 +0100 (CET)

On Tue, 15 Dec 2009, Maciej Stachowiak wrote:

>> Set-Cookie: headers that are using illegal syntaxes should be ignored.
>
> Why is "ignore" the only acceptable behavior when processing a nonconforming 
> header field?

Because that's the sane way to treat protocol errors. If we deem another way 
to be the correct way, then the protocol dictates this and then it isn't 
nonconforming anymore.

> Or to turn it around, why must everything with a processing requirement for 
> the recipient other than "ignore" be considered legal to send?

I don't necessarily think that incoming must match outgoing, but if a client 
is supposed to accept incoming Set-Cookie: headers given a certain format, 
then that format is the legal one. If then some cases of accepted headers lead 
to a different action and a different output in subsequent requests, then the 
spec should detail this.

> Surely it can be rational to define error handling other than "ignore" in at 
> least some cases.

Like how? In this case it was not even suggested to be treated as "error 
handling" but simply to pretend that the misformed incoming header is valid 
and then generate an invalid header back and thus produce another 
non-conformant header. That's not my idea of error handling.

> For example, HTML4.01 requires clients to render the contents of unknown 
> elements, even though such elements are a conformance error for the producer 
> of the document.

1) This is not HTML and I don't think HTML rules or thoughts apply very good 
on httpstate and 2) I'm not sure I even agree that those are good ideas for 
HTML.

> We have here a situation where there are both legacy servers and legacy 
> clients. Sending certain unusual constructs may break some legacy clients.

I think you're turning this into too many shades of grey. While it may be 
hard, I truly believe we will be much better off by clearly specifying what is 
conforming to the spec and what is not. Incoming headers that aren't legal can 
and should be ignored.

If certain bugs (yes I view them as bugs) are significant enough to be 
supported by general clients and servers to make them interoperable to a 
satisfying level, then we make them part of this spec. And then we stop 
viewing them as bugs.

If the bugs are minor enough to not cause a noticable difference, we make them 
remain bugs and they do not make it to the spec. In this category I consider 
the nameless cookies and the requirement on sort order to be.

The greyness here is only where we draw the line and we can argue back and 
forth of what to consider actual practise and what's a bug and so on, but 
inside the spec there should (ideally) be no doubt.

> Thus, it is prudent that new servers not do so.

Isn't that why we use SHOULD and MUST NOT etc in specs?

> The fact that the required syntax to accept is broader than the legal syntax 
> to send is unfortunate, but not fundamentally illogical.

It could indeed be wider, it should just clearly state what the accecept 
syntax is. As well as for the sending one.

-- 

  / daniel.haxx.se

---

From: Julian Reschke <[email-masked]>
Date: Wed, 16 Dec 2009 15:30:34 +0100

> even though such elements are a conformance error for the producer of 

Does it?

<http://www.w3.org/TR/html4/appendix/notes.html#h-B.1>:

"This specification does not define how conforming user agents handle 
general error conditions, including how user agents behave when they 
encounter elements, attributes, attribute values, or entities not 
specified in this document.

However, to facilitate experimentation and interoperability between 
implementations of various versions of HTML, we recommend the following 
behavior:

     * If a user agent encounters an element it does not recognize, it 
should try to render the element's content...."

Note it says "recommend".

Or are you referring to something else?

(Note that this is orthogonal to the question whether this 
recommendation is a good one)

BR, Julian

---

From: Adam Barth <[email-masked]>
Date: Wed, 16 Dec 2009 10:08:59 -0800

Daniel,

My understanding of your position is that we should make the entire
cookie protocol (meaning everything the user agent is required to
understand) conforming for servers.  We can then recommend (at the
SHOULD level?) that servers stick to a subset of the protocol that's
easier to understand / has cleaner semantics.

Is that correct?  If so, that seems reasonable to me.

Adam

On Wed, Dec 16, 2009 at 5:51 AM, Daniel Stenberg <[email-masked]> wrote:
>> nonconforming header field?
>
>> Or to turn it around, why must everything with a processing requirement
>> for the recipient other than "ignore" be considered legal to send?
>
> then that format is the legal one. If then some cases of accepted headers
> lead to a different action and a different output in subsequent requests,
> then the spec should detail this.
>
>> Surely it can be rational to define error handling other than "ignore" in
>> at least some cases.
>
> hard, I truly believe we will be much better off by clearly specifying what
> is conforming to the spec and what is not. Incoming headers that aren't
> legal can and should be ignored.
>
> If the bugs are minor enough to not cause a noticable difference, we make
> them remain bugs and they do not make it to the spec. In this category I
> consider the nameless cookies and the requirement on sort order to be.
>
>> The fact that the required syntax to accept is broader than the legal
>> syntax to send is unfortunate, but not fundamentally illogical.
>

---

From: Daniel Stenberg <[email-masked]>
Date: Wed, 16 Dec 2009 19:16:04 +0100 (CET)

On Wed, 16 Dec 2009, Adam Barth wrote:

> My understanding of your position is that we should make the entire cookie 
> protocol (meaning everything the user agent is required to understand) 
> conforming for servers.  We can then recommend (at the SHOULD level?) that 
> servers stick to a subset of the protocol that's easier to understand / has 
> cleaner semantics.
>

Yes, I believe that's a pretty good way to describe my position!

-- 

  / daniel.haxx.se

---

From: Dan Winship <[email-masked]>
Date: Wed, 16 Dec 2009 21:56:38 +0100

On 12/16/2009 02:51 PM, Daniel Stenberg wrote:
> I don't necessarily think that incoming must match outgoing, but if a
> client is supposed to accept incoming Set-Cookie: headers given a
> certain format, then that format is the legal one.

The asymmetry in incoming vs outgoing is to deal with the fact that we
know there are lots of bad servers out there, and there will almost
certainly continue to be lots of bad servers out there after we publish
the new spec, and so we want to explain to good clients how to deal with
bad servers, but we don't want to encourage servers to be bad.

That is, (as I see it) we're trying to define (a) a nice, clean grammar
and semantics, and (b) a gross, nasty client-side processing model, such
that:

    1. If a server sends out Set-Cookie headers that conform to the
       nice grammar, and makes only the assumptions that are supported
       by the nice semantics, then we believe that that server will be
       compatible with essentially all cookie-handling HTTP clients,
       *even if those clients don't use the gross, nasty processing
       model*.

    2. If a client uses the gross, nasty processing model, then we
       believe that that client will be compatible with essentially all
       cookie-sending HTTP servers, *even if those servers don't use
       the nice, clean grammar and semantics*.

Another reason to want a clean grammar is that people might want to cite
our new cookie RFC in the future (eg, if draft-broyer-http-cookie-auth
or something like it gets started up again), and we want them to be
citing something sane, not "cookies have names except when they don't,
and here are the 17,000 date formats you should know how to parse". (We
can even say explicitly that protocols that build on top of our spec
MUST conform to the nice grammar and MUST NOT assume that clients will
follow the nasty processing model.)

As far as the rest of the spec goes, I think I'm fine now with having
the nasty processing model be free of MUSTs and SHOULDs. It's not
necessarily *the* cookie-processing model, it's just *a*
cookie-processing model--one that we recommend because it's very good at
dealing with lots of different broken servers, but if you don't care
about all of those broken servers then you don't need to use all of the
rules.

-- Dan