Subject: [http-state] Comments on draft-ietf-httpstate-cookie-08.txt (4.1.2.2 - 5.2.6)

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 28 May 2010 21:16:59 +0200

Hi,

  Some additional comments on draft-ietf-httpstate-cookie-08.txt:

In section 4.1.2.2, "The user agent is not required to retain the cookie
until the specified date has passed." `Max-Age` does not specify a date,
it specifies a duration.

Later in the section: "WARNING: Not all user agents support the Max-Age
attribute. User agents that do not support the Max-Age attribute will
ignore the attribute." I see no reason why this needs to be a "WARNING"
rather than a note, and the text needs to clarify that support for the
attribute is not optional but rather required.

In section 4.1.2.3, 'For example, if the Domain attribute contains the
value "example.com", the user agent will include the cookie in the ...';
"Contains" is the wrong word here, the value would have to be equivalent
to it, not just contain it as substring. And "will" is also the wrong
word.

Later in the section, "(Note that a leading U+002E ("."), if present,
is ignored even though that character is not permitted by the subdomain
production in [RFC1034].)" This needs to be rephrased so it refers to
the legal syntax of the attribute, otherwise it is unclear whether it
is conforming to send it.

The subsequent "Warning" needs to explain how and why the described be-
haviour is erroneous.

Later, "The user agent will reject cookies (refuse to store them in the
cookie store) unless the Domain attribute specifies a scope for the
cookie that would include the origin server.  For example, the user
agent will accept a Domain attribute ..." This needs to be rephrased, it
starts out saying the whole cookie will be rejected, but then only re-
fers to the Domain attribute being rejected.

In section 4.1.2.4, "If the server omits the Path attribute, the user
agent will use the directory of the request-uri's path component as the
default value." It is not clear what the "directory" of a path component
is, RFC 3986 for example does not define it.

The second paragraph needs a forward reference to where the matching is
explained. The forward reference in the third paragraph should be more
precise, section 8 is rather long.

I note that section 4.1.2.6 does not mention that some user agents do
not support the attribute. I am not sure if it should.

In section 4.2.1, "The user agent returns stored cookies ..." this
should probably not use "return" but something like "pass" as the cookie
may come from a different source than the server. (This also applies to
section 4.2.2).

"If the server conforms to the requirements in Section 4.1, the
requirements in the Section 5 will cause the user agent to return a
Cookie header that conforms to the following grammar" This seems some-
what confused. If the user agent is free to send whatever it wants in
the header, perhaps because the `Set-Cookie` it processed was invalid,
then this should be clearly stated here, more so if it is required to.

In section 5.1.1, "If the found-day-of-month flag is not set ..." Up to
this point it has not been explained what this flag is. The same goes
probably for all "flags". Definitions need to be added.

The surprising requirements in this section need to be accompanied by
some rationale, for example, accepting 1601 as year but not 1600 is
surprising and so is the 68/69 boundary.

In section 5.1.2 it needs to be explained what is supposed to happen if
the normalization process fails, e.g. when ToASCII fails. There should
be a reminder that using non-ASCII characters directly is prohibited.

There needs to be a discussion, I am not sure this is the right place,
on the handling of permissable HTTP "hosts" (non-DNS names, IP-Literals
and IPv4 addresses, for instance); some questions are, is it intentional
that the Domain attribute disallows setting the attribute to a IPv4
address, would a leading '.' in the attribute still be stripped, if such
host specifications are disallowed, must all cookies coming from them be
rejected? With a Set-Cookie from 127.0.0.1 with no Domain attribute, is
it incorrect to imply Domain=127.0.0.1? (I don't care, the specification
however needs to discuss those questions).

In section 5.1.3, there is a reference to `abs_path` without explaining
where it comes from (RFC 3986 no longer specifies that symbol).

The draft needs to explain how to handle a Set-Cookie header in a reply
to an `OPTIONS *` request. Right now it seems `uri-path` would be unde-
fined in that case, but the draft does not say what the consequence is.
There should be an explicit mention of the case, not just adjusted algo-
rithms.

In section 5.2, if the draft takes the position that HTTP header values
are sequences of octets and not Unicode code points, then it needs to be
explained at the beginning of this section how those octets are turned
into characters (at least so long as it refers directly to the header
and not some pre-processed value based on it, and so long it refers to
characters instead of octets).

In section 5.2.1 and subsequent sections the term "case-insensitively
matches" is not defined; this should probably be defined in the termi-
nology section.

Also in 5.2.1, using "last" and "first" in the context of time strikes
me as poor form.

I doubt the draft should require implementations to use the "last" Date
header, there are plenty of HTTP implementations where that is not
possible as the headers are only accessible in folded form. Section
4.1.2.1 also should be updated to clearly express that even though the
expiry time is set in an absolute form, it'll be handled as relative
value, as that is rather surprising.

regards,
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Fri, 28 May 2010 12:55:20 -0700

> until the specified date has passed." `Max-Age` does not specify a date,

Fixed.

> rather than a note,

Changed to a NOTE.

> and the text needs to clarify that support for the

That requirement is contained in the section entitled User Agent Requirements.

> to it, not just contain it as substring.

Fixed.

> And "will" is also the wrong word.

What word do you suggest instead?

> is conforming to send it.

I've changed this to the following:

[[
            Note that a leading
            U+002E ("."), if present, is ignored even though that character is
            not permitted.
]]

The "." is not permitted, but many places that talk about cookies
believe in the cargo cult of a leading "." character, hence the note.

> The subsequent "Warning" needs to explain how and why the described be-

I don't understand this comment.  It's a warning that some existing
user agents behave incorrect w.r.t. the description that section.
That behavior is erroneous: it is in error.

> fers to the Domain attribute being rejected.

Fixed.

> is, RFC 3986 for example does not define it.

I've put directory in quotes.  The definition is in Section 5.

> precise, section 8 is rather long.

I'm sure we need a forward reference.  This is all just informative text anyway.

> not support the attribute. I am not sure if it should.

There exist, of course, user agents with all manner of bugs and lack
of support.  We've only called out issues that are prevalent and/or
particularly problematic.

> may come from a different source than the server. (This also applies to

Changed to "sends".

> then this should be clearly stated here, more so if it is required to.

Section 4 does not contain any requirements on the user agent.  All
the user agent requirements are contained in Section 5.  This sentence
just states some consequences of those requirements.

> probably for all "flags". Definitions need to be added.

They don't mean anything other than what the text says.  There's
nothing to define.

> surprising and so is the 68/69 boundary.

The reason is just that's how the protocol works.  It's odd, but
that's how it works.

> the normalization process fails, e.g. when ToASCII fails.

Can you provide a test case that illustrates behavior that you think
is not defined?

> There should

Who is prohibited from using non-ASCII characters for what?  I don't
understand your request.

> however needs to discuss those questions).

Can you provide a test case that illustrates behavior that you think
is not defined?

> where it comes from (RFC 3986 no longer specifies that symbol).

Fixed.

> There should be an explicit mention of the case, not just adjusted algo-

The draft says:

[[
            <t>If the uri-path is empty or if first character of the uri-path
            is not a U+002F ("/") character, output U+002F ("/") and skip the
            remaining steps.</t>
]]

Doesn't this fall under the case of the uri-path being empty?

> characters instead of octets).

The octets are never turned into characters.

> matches" is not defined; this should probably be defined in the termi-

I'll be happy to add this if you can provide the definition you'd like
to see in the terminology section.

> Also in 5.2.1, using "last" and "first" in the context of time strikes

Fixed.

> possible as the headers are only accessible in folded form.

As far as I can tell, that's how the protocol works.  Can you provide
a test case that shows otherwise?

> Section

There are lots of details explained in Section 5 that are not
presented in Section 4.  It's unclear why this particular detail
merits inclusion.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 29 May 2010 01:05:47 +0200

>That requirement is contained in the section entitled User Agent Requirements.

The point is that the text does not say those user agents are non-con-
forming, so the casual reader may well assume support is optional. It
should say something like "Some older user agents do not support it",
or whatever, so long as it is clear that they are not compliant.

>What word do you suggest instead?

I don't have a ready proposal how to rephrase it, but the point is that
they may not do so and still conform to the specification.

>That behavior is erroneous: it is in error.

A remedy would be to state in the note what the correct behavior is,
rather than simply say it is wrong. Obviously the developers of the 
misbehaving user agents did not think it wrong.

>I've put directory in quotes.  The definition is in Section 5.

This probably works if you include the forward reference to the re-
levant sub-section of section 5.

>just states some consequences of those requirements.

The text draws attention to a condition, either it matters that the
server conforms to the requirements in section 4.1 for the user agent
to meet the requirements in section 5, then it needs to be explained
what the implications are of the server not doing so, or the user agent
will behave according to the requirements in section 5 regardless of
whether the server conforms to its requirements, then no attention
should be drawn to the condition.

>They don't mean anything other than what the text says.  There's

There are two occurrences in the draft of `found-day-of-month`, the
first is "If the found-day-of-month flag is not set and ..."; the other
is in "Abort these steps and fail to parse if ... at least one of the
found-day-of-month, found-month, found-year, or found-time flags is not
set"; I translate that into:

  bool foundDayOfMonth;
  ...
  if (foundDayOfMonth && ...) { ... }
  ...
  if (!foundDayOfMonth || ...) { ... }

The variable is read twice but never written to. You might try to read
the draft after replacing all occurences of "found-day-of-month" with
"squirrel0815"; that is, in effect, what I do. I could guess that maybe 
`found-day-of-month` is supposed to be set when the `day-of-month` sym-
bol is used when parsing the input, but the draft does not say that, so
my guess may very well be wrong.

>The reason is just that's how the protocol works.  It's odd, but

I think it is unlikely that surprising requirements will be widely im-
plemented unless the reasoning behind them is explained. For example,
it seems unlikely to me that implementations will treat the year 69 as
1969 rather than 2069 just because the draft says so; they would rather
assume this is an error as the 19XXs usually start with 1970. Similarily
it seems quite reasonable to me that some implementations that cannot
represent the year 1601 will simply fail to parse the date and then ig-
nore the attribute, rather than succeed in parsing and then clamp it
later, unless they are given a good reason to do otherwise.

I also note that the draft fails to address handling impossible dates
like the 31st of february, but that is obviously an error.

>Can you provide a test case that illustrates behavior that you think

For instance, ToASCII(ToASCII(x)) fails for any `x`, and the draft does
not discuss this failure.

>Who is prohibited from using non-ASCII characters for what?  I don't

A `domain-value` cannot contain non-ASCII characters directly.

>Can you provide a test case that illustrates behavior that you think

I believe I did.

>Doesn't this fall under the case of the uri-path being empty?

Since `uri-path` is "the path portion", and "*" is not a path, we do not
know what `uri-path` is.

>The octets are never turned into characters.

You eventually pass some of the octets to the ToASCII function which
only accepts characters, not octets. So, either you do convert, or you
cannot use ToASCII.

>to see in the terminology section.

There should be plenty of specifications to steal the text from, you
could say "In this specification two strings are said to case-insen-
sitively match each other if and only if they are equivalent under the
i;ascii-casemap collation defined in RFC 4790". I am sure you can find
more appropriate text.

>a test case that shows otherwise?

A quick reading of the libwww-perl code suggests it does not use the
"Date" header at all when determining the expiration time. Qt would
seem to be another example.

>presented in Section 4.  It's unclear why this particular detail

It is common for cookie processing library to operate only on the values
of the Set-Cookie and Cookie headers, not on whole request and response
objects or the value of the headers plus the value of a Date header; for
example, QNetworkCookie::parseCookies only takes a byte array as input.
Obviously the requirement has not been anticipated or has been ignored.
That should be sufficient reason to draw special attention to it.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Daniel Stenberg <[email-masked]>
Date: Sat, 29 May 2010 20:15:42 +0200 (CEST)

On Fri, 28 May 2010, Bjoern Hoehrmann wrote:

> header, there are plenty of HTTP implementations where that is not possible 
> as the headers are only accessible in folded form. Section 4.1.2.1 also 
> should be updated to clearly express that even though the expiry time is set 
> in an absolute form, it'll be handled as relative value, as that is rather 
> surprising.

I agree. I don't think we've discussed this properly on the list. (Or if we 
did I might've missed it.)

Why would the 'expires' attribute be treated relative like 5.2.1 describes? 
Expires is an absolute time, and I don't see why clients shouldn't try to use 
exactly that time in time comparisons. If the server/client clocks are off in 
any significant way, every response-with-cookies without Date: header would 
then be subject to flaws and I expect that a majority of cookie-sending server 
responses are done without Date:

I also would expect that lots of implmentations treat that time as an absolute 
and not a relative unconditionally.

Are the big-5 doing it this way?

-- 

  / daniel.haxx.se

---

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Sat, 29 May 2010 20:37:34 +0200

On Sat, 29 May 2010 20:15:42 +0200, Daniel Stenberg <[email-masked]> wrote:

> I agree. I don't think we've discussed this properly on the list. (Or if  
> we did I might've missed it.)
>
> Why would the 'expires' attribute be treated relative like 5.2.1  
> describes? Expires is an absolute time, and I don't see why clients  
> shouldn't try to use exactly that time in time comparisons. If the  
> server/client clocks are off in any significant way, every  
> response-with-cookies without Date: header would then be subject to  
> flaws and I expect that a majority of cookie-sending server responses  
> are done without Date:
>
> I also would expect that lots of implmentations treat that time as an  
> absolute and not a relative unconditionally.
>

Opera is using the Expires attribute as an absolute UTC date and time.  
Frankly, I think it should stay that way.

Servers should keep their clock well in sync, so should clients,  
particularly since quite a few security features in SSL depends on a  
reasonably accurate clock.

The situation where a slightly off clock can cause problems is for  
shortlived cookies, and those are better handled through maxage.

My guess is that this has been adopted from the cache lifetime definitions  
in 2616 and 2616-bis.

I have no idea whether other clients use relative time at present, but if  
they don't, then all the clients that could be adapted to it would already  
support maxage.

-- 
Sincerely,
Yngve N. Pettersen

********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Daniel Stenberg <[email-masked]>
Date: Sat, 29 May 2010 22:43:15 +0200 (CEST)

> shortlived cookies, and those are better handled through maxage.

Well, one could imagine a similar concept for maxage, like for the case where 
the server wants the cookie to be alive 10 seconds, but the mere transfer of 
the cookie to the client takes 5... I would expect most client implementations 
to consider the maxage time to start ticking the moment the cookie is actually 
parsed and not from the Date: of the http response.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Sun, 30 May 2010 14:22:00 -0700

[[
              Some legacy user agents do not support the Max-Age
              attribute.
]]

Hopefully that should be sufficient to convey that this is not desired.

> they may not do so and still conform to the specification.

I've removed the word "will."

> misbehaving user agents did not think it wrong.

On the contrary, even the developers of the misbehaving user agents
agree that the behavior is wrong, at least according to their public
statements on the matter.

> levant sub-section of section 5.

Done.

> should be drawn to the condition.

The consequent follows only if both of the antecedents are satisfied.
There's no entanglement between the antecedents.

I've changed the structure of this sentence to be more parallel to
reflect the parallelism in the situation.  I've also demoted the
mention of Section 5 to a parenthetical.

[[
          <t>The user agent sends stored cookies to the origin server in the
          Cookie header. If the server conforms to the requirements in <xref
          target="sane-set-cookie" /> (and the user agent conforms to the
          requirements in the <xref target="ua-requirements" />), the user
          agent will send a Cookie header that conforms to the following
          grammar:</t>
]]

> set";

Ah, I understand your confusion.  You missed the third occurrence of
the term in which the flag is actually set.  :)

> my guess may very well be wrong.

I think the text still makes sense if read with squirrel0815:

[[
              <t>If the squirrel0815 flag is not set and the date-token
              matches the day-of-month production, set the squirrel0815
              flag and ...</t>

              [...]

              <t>at least one of the squirrel0815, found-month,
              found-year, or found-time flags is not set, ...</t>
]]

> plemented unless the reasoning behind them is explained.

That requirement is already widely implemented, so we have nothing to fear here.

> like the 31st of february, but that is obviously an error.

Fixed.  We not fail explicitly when no such date exists.

> A `domain-value` cannot contain non-ASCII characters directly.

Do you mean there's a certain sequence of octets that the server is
prohibited from sending?  Please read the draft more carefully.  The
server is permitted to send any sequence of octets it pleases.  We
recommend against sending some crazy ones, but none are prohibited by
this document.

>>Can you provide a test case that illustrates behavior that you think

I should be more clear.  In this case, I'm asking for a sequence of
octets exchanged by the server and the user agent that causes the
protocol to arrive in a situation that you believe is not defined.
You have not provided any such sequence of octets and so I cannot use
them to test the protocol definition.

> Since `uri-path` is "the path portion", and "*" is not a path, we do not

I've changed the text to say explicitly that uri-path is empty if the
path portion of the request-uri does not exist.  In that case, the
default-path will be computed as "/", which is correct.

> only accepts characters, not octets. So, either you do convert, or you

What algorithm should I use instead?  It's quite difficult to work
with underlying specifications that can't provide such elementray
concepts as the URL of an HTTP request or the host name of a URL.

> i;ascii-casemap collation defined in RFC 4790". I am sure you can find

[[
   The "i;ascii-casemap" collation is a simple collation that operates
   on octet strings and treats US-ASCII letters case-insensitively.
]]

That does appear to be what we'd like.  Thanks.

> seem to be another example.

See below.

> That should be sufficient reason to draw special attention to it.

See below.

> did I might've missed it.)

We discussed it briefly in the context of time zones.  The current
text is based on the Firefox behavior.

> Why would the 'expires' attribute be treated relative like 5.2.1 describes?

Dunno.

> Expires is an absolute time, and I don't see why clients shouldn't try to
> use exactly that time in time comparisons. If the server/client clocks are
> off in any significant way, every response-with-cookies without Date: header
> would then be subject to flaws and I expect that a majority of
> cookie-sending server responses are done without Date:
>

I will test in more detail.  Hopefully this is a Firefox-only
behavior, in which case I'd be in favor of removing it because its
very subtle.

On Sat, May 29, 2010 at 11:37 AM, Yngve N. Pettersen (Developer Opera
Software ASA) <[email-masked]> wrote:
>> response-with-cookies without Date: header would then be subject to flaws
>> and I expect that a majority of cookie-sending server responses are done
>> without Date:
>>

Thanks, this is useful data.

> reasonably accurate clock.

There are plenty of non-SSL sites that we need to worry about as well.

> shortlived cookies, and those are better handled through maxage.

Indeed, but that doesn't stop existing sites from using Expires.

> My guess is that this has been adopted from the cache lifetime definitions

The text in the spec was adopted from studying how Firefox behaves at
the suggestion of dwitte.  I don't know the origin of the behavior in
Firefox.

> they don't, then all the clients that could be adapted to it would already

Considerations about Max-Age are irrelevant.  We're worried about
already deployed servers who are using Expires instead of Max-Age.

On Sat, May 29, 2010 at 1:43 PM, Daniel Stenberg <[email-masked]> wrote:
> Well, one could imagine a similar concept for maxage, like for the case
> where the server wants the cookie to be alive 10 seconds, but the mere
> transfer of the cookie to the client takes 5... I would expect most client
> implementations to consider the maxage time to start ticking the moment the
> cookie is actually parsed and not from the Date: of the http response.

I'll add this to my testing matrix.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 31 May 2010 19:34:59 +0200

>statements on the matter.

By your own account there have been several developers who have made the
same mistake; could you explain how the specification would become worse
if it briefly mentioned why the mistake actually is a mistake, and what
would have been correct instead?

>          agent will send a Cookie header that conforms to the following

What you really seem to be trying to say is that user agents may send
bad Cookie headers if and only if the Set-Cookie header they got also
did not conform to the specification. That would be important to say,
but that a user agent that conforms to the specification will behave in
a conforming manner when dealing with conforming input is not important.

>the term in which the flag is actually set.  :)

I see. Please use &nbhy; (U+2011) instead of the hyphen, xml2rfc will
then not wrap identifiers around, that way you can actually locate them.
Alternatively you could quote them or use the underscore instead.

>That requirement is already widely implemented, so we have nothing to fear here.

Well, the draft requires the 19XXs to start with 1969, Gecko/20100529
and Opera 10.51 use 1970, Internet Explorer 6 uses 1980. The draft re-
quires to ignore the Expires attribute if the year is below 1601, and
Opera 10.51 rather treats all years before 1900 as 2036 (on my system
anway). That is not my understanding of "widely implemented".

>them to test the protocol definition.

Well, to pick an example, "Set-Cookie from 127.0.0.1 with no Domain
attribute" should be entirely sufficient to make a test case if there
is something to test. Similarily, you should be able to answer, "is it
intentional that the Domain attribute disallows setting the attribute to
a IPv4 address" (in the so-called well-behaved profile, if I need to add
that) without observing implementation behavior. I do not see how a hex
dump of IP packets would really help here.

>default-path will be computed as "/", which is correct.

I would need to know your reasoning behind not mentioning `OPTIONS *`
explicitly to tell whether that is sufficient.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Mon, 31 May 2010 10:48:00 -0700

> would have been correct instead?

I'm not sure it was a mistake when they made it.  They just made a
different choice than other folks in the absence of a specification.
In any case, you're just asking for a footnote to a footnote.  It
doesn't seem important.

> a conforming manner when dealing with conforming input is not important.

What is a "bad" Cookie header?  The document defines no such thing.
The text means exactly what it says.  If the two antecedents are true,
then the consequent follows.  Nothing more.  Nothing less.  That
statement neither imposes nor relaxes any normative requirements on
user agents, which you can detect by the lack of 2119 keywords.

> Alternatively you could quote them or use the underscore instead.

Ok, now you're nit picking which characters I'm using in the names of
opaque identifiers.  This is ridiculous.

> anway). That is not my understanding of "widely implemented".

Do you have test cases that demonstrate these claims?  If so, I'd be
happy to add them to the test suite.

> is something to test.

Thanks.  I'll test this case.

> Similarily, you should be able to answer, "is it
> that) without observing implementation behavior.

Yes.  Doing so is meaningless at best and harmful at worse.  Setting
the Domain attribute to an IPv4 address would make the cookie
available to all "subdomains" of that IP address, which is a concept
that doesn't make sense.  Supplying no Domain attribute, however, is
perfectly sensible as it means the user agent should return the cookie
to that IP address alone.

> I do not see how a hex dump of IP packets would really help here.

I'm just asking that you be precise.  The more precise we are in our
discussions, the less likely we are to misunderstand each other.  For
example, now that I know you're worried about Set-Cookie headers
received by the user agent from IPv4 addresses without a Domain
attribute, I can do something concrete with your feedback.

> explicitly to tell whether that is sufficient.

OPTIONS is but one of many cases in which this situation can occur.
Rather than trying to enumerate every possible case, I've just written
the requirements to handle the general case.

Adam