Subject: [http-state] I-D Action:draft-ietf-httpstate-cookie-12.txt

From: [email-masked]
Date: Thu, 16 Sep 2010 03:00:02 -0700 (PDT)

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the HTTP State Management Mechanism Working Group of the IETF.

	Title           : HTTP State Management Mechanism
	Author(s)       : A. Barth
	Filename        : draft-ietf-httpstate-cookie-12.txt
	Pages           : 40
	Date            : 2010-09-16

This document defines the HTTP Cookie and Set-Cookie header fields.
These header fields can be used by HTTP servers to store state
(called cookies) at HTTP user agents, letting the servers maintain a
stateful session over the mostly stateless HTTP protocol.  Although
cookies have many historical infelicities that degrade their security
and privacy, the Cookie and Set-Cookie header fields are widely used
on the Internet.Editorial Note (To be removed by RFC Editor)

If you have suggestions for improving this document, please send
email to <mailto:[email-masked]>.  Suggestions with test cases
are especially appreciated.  Further Working Group information is
available from <https://tools.ietf.org/wg/httpstate/>.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-httpstate-cookie-12.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 16 Sep 2010 17:07:25 -0700

> http://www.ietf.org/internet-drafts/draft-ietf-httpstate-cookie-12.txt

For the most part, this spec looks well done and ready to progress.
Good work, Adam.

I do have some editorial concerns:

1) use of U+nnnn notation in the prose, e.g., U+002F ("/")

  HTTP parsers must be octet-based because Unicode character
  processing becomes unpredictable on invalid multi-octet sequences.
  Everything else about the spec is ASCII, including the ABNF notation,
  so UTF-16 constants in hex are not appropriate for the prose.

  I personally prefer to use the ASCII character names, as in

       slash ("/")
       comma (",")

  or just the character symbol and the ASCII notation in parens, as in

       slash (%x2F)
       comma (%x2C)

  or if you really want to be formal, include the following ABNF in
  section 2.2 (Syntax Notation):

       comma         = ","  ; %x2C
       dot           = "."  ; %x2E
       slash         = "/"  ; %x2F
       semicolon     = ";"  ; %x3B
       equals        = "="  ; %x3D
       question-mark = "?"  ; %x3F

  and then just refer to them by name in the prose instead of repeating
  the Unicode double-wide hex code points everywhere.

2) I found the date parsing algorithm for user agents (5.1.1) to be
   unreadable.  Do user agents actually implement this, or do they
   send the extracted string to a standard date parser?  The reason
   I ask is that I don't know of any parser that actually works this
   way -- most will pattern-match the string to see if it matches
   one of the four or so most-common forms and then directly grab
   the fields -- and I have no idea if the "results" will be the
   same for those more-standard date parsers.

   Some parsers will return an invalid (zero) date if there is no
   standard match, while others try again with the kind of heuristic
   scan that removes identifiable strings (timezone, day-of-week, a
   4-digit year, etc.) first and then looks for day and month.

   And the whole [ mystery ] thing makes the grammar useless.
   I am not sure if user agents currently handle (skip) day-of-week
   as well, i.e. ( "Mon" / "Monday" / "Tue" / "Tuesday" / ... )

   Personally, I'd prefer a requirement that user agents MUST be
   able to read the three standard HTTP dates (plus the Netscape
   Cookie Spec date format) and MAY try to guess if it doesn't match
   one of those.  But if the user agent implementers actually want
   to implement this algorithm, then I have no objection.

3) expires (5.2.1) vs max-age (5.2.2) -- if both are given which
   one wins (sets the expiry date)?  I suggest that max-age should
   win because it minimizes problems due to local clock skew.
   Others might claim that it should be whichever one results in
   the earlier expiry time.  The spec should choose one based on
   what the user agents do.

4) Informative References should include the following patent:

    [USP-670] Montulli, L. "Persistent client state in a hypertext
              transfer protocol based client-server system",
              United States Patent 5,774,670: filed Oct 6, 1995;
              issued Jun 30, 1998.

Cheers,

Roy T. Fielding                            <http://roy.gbiv.com/>
Chief Scientist, Day Software              <http://www.day.com/>

---

From: Adam Barth <[email-masked]>
Date: Fri, 17 Sep 2010 01:34:14 -0700

> For the most part, this spec looks well done and ready to progress.

Thanks.

>  the Unicode double-wide hex code points everywhere.

I've changed these to use the %xNN notation.

>   send the extracted string to a standard date parser?

Cookie dates are sufficiently obtuse that many user agents have
dedicated parsers for cookie dates.  I don't believe the algorithm in
the document matches any user agent in every detail, however it should
be extremely close to both Firefox and Chrome.

>   The reason

A cookie date parser implemented in this way would not interoperate
well with many existing servers.  For some data on the frequency of
various date formats, please see this note:

http://github.com/abarth/http-state/blob/master/notes/2009-08-11-Dan-Winship.txt

>   as well, i.e. ( "Mon" / "Monday" / "Tue" / "Tuesday" / ... )

If you're interested in these questions, I'd encourage you to test
popular user agents.

>   to implement this algorithm, then I have no objection.

I understand that this is a cultural divide between browser
implementors and server implementors.  Such a specification is not
nearly precise enough for browser implementors.  Browser implementors
desire byte-for-byte interoperability.  Falling short of that goal
leads to years of pain and misery.  Phrasing the requirements as you
suggest is unlikely to lead to interoperable implementations.

>   the earlier expiry time.  The spec should choose one based on

It already does:

          <t>If the cookie-attribute-list contains an attribute with an
          attribute-name of "Max-Age":
          <list style="empty">
            <t>Set the cookie's persistent-flag to true.</t>

            <t>Set the cookie's expiry-time to attribute-value of the last
            attribute in the cookie-attribute-list with an attribute-name of
            "Max-Age".</t>
          </list>
          Otherwise, if the cookie-attribute-list contains an attribute with
          an attribute-name of "Expires" (and does not contain an attribute
          with an attribute-name of "Max-Age"):
          <list style="empty">
            <t>Set the cookie's persistent-flag to true.</t>

            <t>Set the cookie's expiry-time to attribute-value of the last
            attribute in the cookie-attribute-list with an attribute-name of
            "Expires".</t>
          </list>
          Otherwise:
          <list style="empty">
            <t>Set the cookie's persistent-flag to false.</t>

            <t>Set the cookie's expiry-time to the latest representable
            date.</t>
          </list>
          </t>

>              United States Patent 5,774,670: filed Oct 6, 1995;

In what context do you propose citing this reference?

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 17 Sep 2010 12:16:54 +0200

>   same for those more-standard date parsers.

A while ago I used http://cutycapt.sf.net to load all Alexa 1000 web
sites into a WebKit derived browser and recorded all the generated
traffic. All values of the non-empty expires attribute that did not
match the expression

  /(Monday|Mon|...)?
   ,?
   \s*
   \d{1,2}
   [ -]
   (January|Jan|...)
   [ -]
   \d{2,4}
   \s
   (\d{1,2}:\d{1,2}:\d{1,2}(\sGMT)?)/x;

were of the form "Tue Jun 08 23:00:00 2010 GMT" from four different
sources (various Amazon sites, target.com, base.de, urating.com) of
over 3,000 different hosts and over 2,750 expires attributes in total.

http://www.ietf.org/mail-archive/web/http-state/current/msg00128.html
observes roughly the same thing (although with a two or so cases I
did not encounter).
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 18 Sep 2010 15:39:16 -0700

> http://github.com/abarth/http-state/blob/master/notes/2009-08-11-Dan-Winship.txt

I don't understand.  A date parser like the one I described will
always determine the correct date if it is in one of the normal
forms (all of the ones listed by Dan are normal, as far as date
parsing goes), and if it isn't one of those normal forms it can
then be described as an algorithm.

> suggest is unlikely to lead to interoperable implementations.

No, they would interoperate just fine.

My concern is that the algorithm in the spec is both unreadable and
wrong.  If all of the user agents implement the wrong algorithm, that's
one thing.  If you just made up this algorithm and are now claiming
that all user agents need to break their implementation (and fail to
interoperate with servers as a result), then that's a problem.

5.1.1.  Dates

   The user agent MUST use an algorithm equivalent to the following
   algorithm to parse a cookie-date.  Note that the various boolean
   flags defined as a part of the algorithm are initially "not set".

   1.  Using the grammar below, divide the cookie-date into date-tokens.

   cookie-date     = *delimiter date-token-list *delimiter
   date-token-list = date-token *( 1*delimiter date-token )
   delimiter       = %x09 / %x20-2F / %x3B-40 / %x5B-60 / %x7B-7E
   date-token      = day-of-month / month / year / time / mystery
   day-of-month    = 1*2DIGIT
   month           = "jan" [ mystery ] / "feb" [ mystery ] /
                     "mar" [ mystery ] / "apr" [ mystery ] /
                     "may" [ mystery ] / "jun" [ mystery ] /
                     "jul" [ mystery ] / "aug" [ mystery ] /
                     "sep" [ mystery ] / "oct" [ mystery ] /
                     "nov" [ mystery ] / "dec" [ mystery ]
   year            = 1*4DIGIT
   time            = time-field ":" time-field ":" time-field
   time-field      = 1*2DIGIT
   CTLwoHTAB       = %x00-08 / %x0A-1F / %x7F
                     ; CTL except HTAB
   mystery         = CTLwoHTAB / ":" / ALPHA / DIGIT / %x80-FF
                     ; any OCTET except a delimiter

Note that mystery is part of both date-token and month, consists
of a single octet, and accepts both single digits and single letters.
My guess is that you meant 1*mystery as a sort of any-word.

Try this grammar on for size:

   cookie-date     = [ DQUOTE ] *WSP date-token-list *WSP [ DQUOTE ]
   date-token-list = date-token *( 1*delimiter date-token )
   delimiter       = WSP / %x21 / %x23-2F / %x3B-40 / %x5B-60 / %x7B-7E
   date-token      = month / time / numeric / other
                     ; ambiguous: other must be given least precedence
   month           = ( "jan" / "feb" / "mar" / "apr" /
                       "may" / "jun" / "jul" / "aug" /
                       "sep" / "oct" / "nov" / "dec" ) *ALPHA
   time            = time-field ":" time-field ":" time-field
   time-field      = 1*2DIGIT
   numeric         = 1*4DIGIT

   other           = 1*( %x00-08 / %x0A-1F / ":" /
                         DIGIT / ALPHA / %x7F-FF )
                     ; word of any OCTET except a delimiter or DQUOTE

The algorithm can then disambiguate the grammar in the same way that
you currently define, but the steps need to be listed in precedence.
In other words,

   2.  Process each date-token sequentially in the order the date-tokens
       appear in the cookie-date:

       a.  If the found-month flag is not set and the date-token
           matches the month production, set the found-month flag and
           set the month-value to the month denoted by the date-token.
           Skip the remaining sub-steps and continue to the next
           date-token.

       b.  If the found-time flag is not set and the date-token
           matches the time production, set the found-time flag and
           set the hour-value, minute-value, and second-value to
           the numbers denoted by the time-field digits in the
           date-token, respectively.  Skip the remaining sub-steps
           and continue to the next date-token.

       c.  If the date-token matches the numeric production, then
           do one of:

           i) if the found-day-of-month flag is not set and the
           decimal value of the date-token is in the range [1-31],
           then set the found-day-of-month flag and set the
           day-of-month-value to the number denoted by the
           date-token.  Skip the remaining sub-steps and continue
           to the next date-token.

           ii) if the found-year flag is not set, set the found-year
           flag and set the year-value to the number denoted by
           the date-token.  Skip the remaining sub-steps and
           continue to the next date-token.

           iii) otherwise, discard this numeric date-token and
           continue to the next date-token.

        d. otherwise, discard this date-token and continue to
           the next date-token.

Note that the difference between the above and what is currently
in the spec is that it doesn't allow DQUOTE in the token list,
the date-tokens can be arbitrary words (what I guess was meant
by mystery), and the unambiguous matches are tried first.  Also,
if it does happen to encounter a date that begins with a four-digit
year instead of the day-of-month, then it will correctly parse the
date.

What it won't handle are dates where the month is given as digits,
as in 2010-09-18, but I gather that doesn't occur in practice and
so is not a concern even if user agents can currently read those
dates (because they use an OS routine that will do so).

>            <t>Set the cookie's expiry-time to the latest representable

Sorry, I misunderstood the meaning of "expiry-time" in the
definition sections.  You appear to be using that as a local
variable instead of a definition.  Don't you find that confusing?

> In what context do you propose citing this reference?

In section 1:

   Prior to this document, there were at least three descriptions of
   cookies: the so-called "Netscape cookie specification" [Netscape],
   RFC 2109 [RFC2109], and RFC 2965 [RFC2965].  However, none of these
   documents describe how the Cookie and Set-Cookie headers are actually
   used on the Internet (see [Kri2001] for historical context).  This
   document attempts to specify the syntax and semantics of these
   headers as they are actually used on the Internet.

replace it with

   Prior to this document, there were at least four descriptions of
   cookies: the so-called "Netscape cookie specification" [Netscape],
   Netscape patent [USP-670], RFC 2109 [RFC2109], and RFC 2965 [RFC2965].
   However, current practice differs from each of those prior documents
   (see [Kri2001] for historical context).  This document specifies the
   syntax and semantics of the Cookie and Set-Cookie header fields as
   they are currently used on the Internet.

....Roy

---

From: Adam Barth <[email-masked]>
Date: Sat, 18 Sep 2010 16:25:56 -0700

> then be described as an algorithm.

The problem with your proposal is that states:

"MAY try to guess if it doesn't match one of those."

That's not sufficiently precise.  What does it mean to try to guess?
How will user agents that wish to interoperate on all inputs be able
to implement that requirement in a way that achieves their
interoperability goals?

I understand that byte-for-byte interoperability is not a goal for
HTTP (at least as construed in the HTTP working group).  However, it's
a common goal for many implementors of cookies.

> No, they would interoperate just fine.

How can they interoperate if you're telling them to guess?  We have no
reason to believe that every implementation will guess in the same
way.  Rather than leave interoperability up to guesswork, we're
defining precisely the result of parsing every sequence of octets.

> My guess is that you meant 1*mystery as a sort of any-word.

Thanks for pointing out the typo w.r.t. to mystery.  I've changes
these to *mystery, which was my original intent.

>   cookie-date     = [ DQUOTE ] *WSP date-token-list *WSP [ DQUOTE ]

Woah, where did DQUOTE come from?  We don't need that in the grammar
in this way.  DQUOTE gets consumed as a delimiter.

>   delimiter       = WSP / %x21 / %x23-2F / %x3B-40 / %x5B-60 / %x7B-7E

Why did you take %x22 out of delimiters?

>                       "sep" / "oct" / "nov" / "dec" ) *ALPHA

Why did you restrict the garbage characters after month names to ALPHA?

>                     ; word of any OCTET except a delimiter or DQUOTE

What about "!" ?  That doesn't seem to be in your grammar anywhere.
Your grammar doesn't appear to match every sequence of octets.  That
means your date parser doesn't define how to parse every sequence of
octets, which is contrary to our goal of defining how to parse a
cookie-date from every sequence of octets.  Here's are some examples
strings that don't appear to be parsed at all:

""Mon, 10-Dec-2007 17:02:24 GMT""
Mon, 10-Dec-2007! 17:02:24 GMT
"Sat, 15-Apr-17\"21:01:22\"GMT"

> by mystery), and the unambiguous matches are tried first.

That's not what's meant by mystery.  Mystery captures all non-delimiter octets.

> Also,

This statement doesn't make any sense because there is no external
notion of correctness.  Can you give an example date that your
algorithm parses more like popular browsers than the algorithm
contained in the draft?

> dates (because they use an OS routine that will do so).

That's apparently not required for compatibility.

In any case, thanks for pointing out the typo.  That's helpful.

I think it's unlikely we'll twiddle the parser as you propose because
you haven't presented any evidence that your parser achieves better
compatibility or better interoperability than the current parser.  We
have a mountain of implementation experience with the current parser
that indicates that it's sufficiently compatible with current Internet
practice as to be shippable in popular user agents.

> variable instead of a definition.

Yes.  It's a local variable in that context.

> Don't you find that confusing?

Nope.

>   they are currently used on the Internet.

I don't see what value that adds to the document.  The current text is
accurate.  I'm sure there are many more descriptions of cookies around
(e.g., in text books, scrawled on bathroom walls).  We certainly don't
need to reference them all.  Moreover, I don't know whether USP-670 is
or is not a description of cookies (I haven't looked at it), nor do I
know whether or to what extent it matches or differs from current
practice on the Internet.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 18 Sep 2010 22:04:30 -0700

> to implement that requirement in a way that achieves their

I don't care.  It is how user agents actually read a Set-Cookie
and interpret its values.  You know, like it says in the spec that
this is going to specify how actual implementations work on the
internet.

> a common goal for many implementors of cookies.

That's great.  When they all implement the same algorithm, then you
can say that algorithm is the standard.

> way.

Yes we do.  There are only so many ways to parse a date using
system supplied routines and nobody (including user agent
developers) cares what the result is when the server supplies
an invalid date because cookie expiration is advisory.

>  Rather than leave interoperability up to guesswork, we're

Except that you don't have the implementations to show that it
works, let alone that it is universal.  So you don't have any
interoperability with this rather unique parser either.

> these to *mystery, which was my original intent.

You don't want date-token to match the empty string, so at least
the first one would have to be 1*mystery.

> in this way.  DQUOTE gets consumed as a delimiter.

Because you don't want to consume the entire header field if the
date is quoted.  If that isn't a concern, then never mind that part.

> Why did you take %x22 out of delimiters?

Because it occasionally appears as a delimiter of cookie-date.  Again,
if you don't think that ever happens in real life and you want to read
the entire header field as if it were part of the cookie-date (or
require that expires always be last), then you can put it back in.

> Why did you restrict the garbage characters after month names to ALPHA?

So that it would not match garbage.  *mystery (or other) is okay.
I assume that implementations only do a 3 character prefix match
on the token anyway.

> What about "!" ?  That doesn't seem to be in your grammar anywhere.

See %x21 in delimiter.

> "Sat, 15-Apr-17\"21:01:22\"GMT"

The middle one will parse.  The rest nobody cares about, though you can
parse them too by putting DQUOTE back in delimiter.  I only took it out
because I don't think real implementations continue parsing past them.

> That's not what's meant by mystery.  Mystery captures all non-delimiter octets.

So does other.  It just isn't a mystery.

> algorithm parses more like popular browsers than the algorithm

Of course there is an external notion of correctness.  Dates of
the form "2007 Dec 10 17:02:24 GMT" are far more likely to occur than
your bogus examples above.  Have you tested that on browsers?

> practice as to be shippable in popular user agents.

Please, Adam, don't sell me that nonsense.  We have no implementation
experience with the parser in the current draft.  If we did, then it
would not have required someone like me to point out the errors.
If you want to copy and paste the current Mozilla date parser in C++
into the spec and call that golden, then you can claim it has more
implementation experience.

>> Don't you find that confusing?

*shrug*

> practice on the Internet.

I think a patent defining the entire scope of this specification is
somewhat more relevant than the bathroom walls, even when its claims
have been licensed under the NPL/MPL.  If you don't want to look at it,
then you can take my word for it.  Your opinion is not necessary
in this matter.

....Roy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 19 Sep 2010 07:53:18 +0200

>interoperate with servers as a result), then that's a problem.

The draft just says you tokenize the string with the greedy rules

  time  = 1*2DIGIT ":" 1*2DIGIT ":" 1*2DIGIT ; hh:mm:ss
  month = "jan" / "feb" / ...
  num34 = 3*4DIGIT
  num12 = 1*2DIGIT
  delim = %x09 / ...
  other = <any octet not matched otherwise>

then you ignore any token that is not separated by a delim token from
the preceding time, month, num34, num12 token (if any), and then take
the first time token as the time, the first month token as the month,
the first num12 token as the day, and the first num34 token or the
second num12 token (whichever comes first) as the year.

(I note that the rule for delimiters and special casing num12/34 in-
stead of simply looking for 1*DIGIT makes the algorithm more compli-
cated and less like what's implemented in Internet Explorer.)

The draft expresses this extremely badly, it's not at all like what
is implemented in IE6 or Firefox last I checked, and while only `day
month year time` and `month day time year` seem to be used, this
requires support for all 24 permutations (which is inefficient); but
other than that there does not seem much wrong with it.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sat, 18 Sep 2010 23:00:39 -0700

> can say that algorithm is the standard.

How can they achieve interoperability without a standard?  By reverse
engineering each other?  That's the status quo we seek to improve.

> an invalid date because cookie expiration is advisory.

I can assure you that a number of user agent implementors care very
deeply about how invalid dates are parsed.

> interoperability with this rather unique parser either.

In fact, this is one of the problems we've set out to solve.  One
could say the same thing about HTML parsing, but now the major user
agents are converging on a single HTML parsing algorithm, which is a
huge boon, especially for handling invalid documents.

> the first one would have to be 1*mystery.

In fact, I do want date-token to match the empty string.  That's how
the parser deals with sequential delimiters.

> date is quoted.  If that isn't a concern, then never mind that part.

That won't occur given on the Set-Cookie header field is parse in the
specification.  The range of octets that could possibly represent the
date are already extracted from the rest of the header field before we
attempt to parse them as a date.  Therefore, it's impossible for the
date parsing algorithm to consume the entire header field.

> require that expires always be last), then you can put it back in.

I'm not sure you're understanding how Set-Cookie is handled in the
specification.  The thing you're worried about cannot occur.

> I assume that implementations only do a 3 character prefix match

Indeed.  That's why we need to the optional octets after the prefix to
match anything (that's not a delimiter) and not be limited to just
ALPHA.

> because I don't think real implementations continue parsing past them.

It's quite presumptuous of you to assume you know what everyone in the
world cares about.  In fact, the last of these examples is a test case
from the Chromium cookie date parsing test suite:

http://src.chromium.org/viewvc/chrome/trunk/src/net/base/cookie_monster_unittest.cc?view=markup

I assure you that the author of that test suite cares how that string is parsed.

> your bogus examples above.  Have you tested that on browsers?

Here's how the spec parses that string:

["2007", "Dec", "10", "17:02:24", "GMT"]

Token #1: "2007"

   (Does not match the day-of-month production.)
   (Does not match the month production.)
   Does match the year production and the found-year flag is not set.
           Set the found-year flag and set the
           year-value to the number denoted by the date-token.  Skip the
           remaining sub-steps and continue to the next date-token.

Token #2: "Dec"

   (Does not match day-of-month production.)
   Does match the month production and the found-month flag is not set.
           Set the found-month flag and set the
           month-value to the month denoted by the date-token.  Skip the
           remaining sub-steps and continue to the next date-token.

Token #3: "10"

   Does match the day-of-month production and the found-day-of-month
flag is not set.

[...]

It appears, then, that this string will parse as Mon, 10 Dec 2007 17:02:24 GMT.

> implementation experience.

Thank you for point out the typo.  I'm not sure what other "errors"
you're referring to.

> then you can take my word for it.  Your opinion is not necessary

I'm not a lawyer, and I have very little interested in interacting
with that document.

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 18 Sep 2010 23:52:57 -0700

Those *delimiter and 1*delimiter are sufficient to handle sequential
delimiters.  Allowing date-token to match the empty string is the
grammatical equivalent of an infinite loop.

> I assure you that the author of that test suite cares how that string is parsed.

I don't care.

> Here's how the spec parses that string:

Sorry, I chose the wrong example.  I was thinking of the ambiguity
between 1*2DIGIT and 1*4DIGIT, as in "98-Dec-10 7:02:24 GMT", but
I agree that it just doesn't matter for expiry until 2032 or so.

Since you are intent on defining this as an algorithm, I suggest you
separate the ABNF for parsing from the ABNF for matching.  Such as

   cookie-date     = *delimiter date-token-list *delimiter
   date-token-list = date-token *( 1*delimiter date-token )
   date-token      = 1*non-delimiter

   delimiter       = %x09 / %x20-2F / %x3B-40 / %x5B-60 / %x7B-7E
   non-delimiter   = %x00-08 / %x0A-1F / DIGIT / ":" / ALPHA / %x7F-FF

   day-of-month    = 1*2DIGIT
   month           = ( "jan" / "feb" / "mar" / "apr" /
                      "may" / "jun" / "jul" / "aug" /
                      "sep" / "oct" / "nov" / "dec" ) *non-delimiter
   year            = 1*4DIGIT
   time            = time-field ":" time-field ":" time-field
   time-field      = 1*2DIGIT

At least then the ABNF is not ambiguous, fits the prose, and
isn't a mystery.

> I'm not a lawyer, and I have very little interested in interacting

Then don't.  Just paste the text I gave you into the spec.

....Roy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 19 Sep 2010 09:05:21 +0200

>be extremely close to both Firefox and Chrome.

Well, the code in the draft and the code in Chromium even use the same
variable names, so I suppose they are similar, but for all I can tell,
how Firefox parses the value of the Expires attribute is nothing like
that, at all. Which version did you verify this with?
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sun, 19 Sep 2010 00:25:35 -0700

>> I assure you that the author of that test suite cares how that string is parsed.

Great.  Then I'll leave it as is.

> I agree that it just doesn't matter for expiry until 2032 or so.

The definition of the parsing algorithm in the specification is not
ambiguous.  It defines the behavior for all input strings.  If we have
problems in 2032, we might need to update the document at some point
in the next 20 years.

> At least then the ABNF is not ambiguous, fits the prose, and

Done.  I'm not sure why you think that's any different than the
current text, but I'm happy to use that presentation if it makes you
happier.

> Then don't.  Just paste the text I gave you into the spec.

No thanks.

Adam

---

From: Adam Barth <[email-masked]>
Date: Sun, 19 Sep 2010 00:28:36 -0700

> that, at all. Which version did you verify this with?

The date parsing algorithm is based on research done by Dean McNamee.
He spent an inordinate amount of time reverse engineering the cookie
date processing behavior of the major browsers and studying
compatibility data.  The algorithm has been shipping in Chrome since
the first release.  We have received zero reports of compatibility
problems.  AFAIK, it's the highest quality cookie date parsing
algorithm available to us, which is why its in the spec.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Sun, 19 Sep 2010 10:38:11 +0200

On 19.09.2010 09:28, Adam Barth wrote:
> ...
> ...

Hmm.

That makes it sound as if an important part of the spec actually 
originates from somebody who, as far as I can tell, is not a member of 
this Working Group.

It would be really good to confirm that Dean is Ok with it being 
published as IETF work.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Sun, 19 Sep 2010 10:53:28 -0700

On Sun, Sep 19, 2010 at 1:38 AM, Julian Reschke <[email-masked]> wrote:
> That makes it sound as if an important part of the spec actually originates
> from somebody who, as far as I can tell, is not a member of this Working
> Group.
>
> It would be really good to confirm that Dean is Ok with it being published
> as IETF work.

Ok, I'll ask him.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 20 Sep 2010 09:09:09 +0200

>algorithm available to us, which is why its in the spec.

It would seem that Firefox and Chromium use the NSPR parser to parse the
date in the Date header, and Firefox uses the same parser to parse the
date in the Set-Cookie header, suggesting that if Chromium dropped the
dedicated parser and simply called the normal HTTP date parser, it would
reduce code size, make the behavior more consistent with Firefox and
other implementations using the Firefox parser, and make date parsing
more consistent in general. If that analysis is not mistaken, what web
site compatibility data (or other data) is there to suggest having the
dedicated cookie date parser is actually well-justified?
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 00:14:53 -0700

> dedicated cookie date parser is actually well-justified?

HTTP date parsing is woefully underspecified.  I think a better
solution is to write a patch for Firefox that implements the
cookie-date parser from this specification.  The impact on code size
is minuscule.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Sep 2010 09:33:23 +0200

> cookie-date parser from this specification.  The impact on code size

HTTP date parsing is well-specified, except for invalid dates.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 00:51:12 -0700

> HTTP date parsing is well-specified, except for invalid dates.

Exactly.

Adam

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:11:03 +0200

> cookie-date parser from this specification.  The impact on code size

Well yes, but if we can have one algorithm instead of two would that not  
be desirable? Just like have one algorithm for parsing URLs, less  
algorithms for parsing dates would be great.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 01:14:32 -0700

On Mon, Sep 20, 2010 at 1:11 AM, Anne van Kesteren <[email-masked]> wrote:
> Well yes, but if we can have one algorithm instead of two would that not be
> desirable? Just like have one algorithm for parsing URLs, less algorithms
> for parsing dates would be great.

That's an issue to take up with HTTPbis.  However, I suspect you'll
encounter resistance trying to convince that working group to
specifying a date parser that actually meets the needs of browsers.
Someday I expect we'll need to specify all the details of HTTP that
are missing from the current specs.  However, that's a bigger job that
I'd like to bite off at the moment.

Adam

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:31:22 +0200

> I'd like to bite off at the moment.

Right, I agree we should do that some day.

What I meant is that I prefer user agents using one (for now) undefined  
algorithm than one (for now) undefined algorithm and another defined  
algorithm that we know differ. Well, I suppose I am not opposed if you  
think they can be reconciled in the future somehow.

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 01:33:55 -0700

On Mon, Sep 20, 2010 at 1:31 AM, Anne van Kesteren <[email-masked]> wrote:
> algorithm that we know differ. Well, I suppose I am not opposed if you think
> they can be reconciled in the future somehow.

Do you have a concrete example of a string that cookie-date parses in
a way that violates HTTP?  Put another way, is there a reason
implementations can't use the cookie-date parser as the one defined
algorithm for both purposes?

Adam

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:47:49 +0200

> algorithm for both purposes?

If it is expected that Chromium, Safari, Firefox, will implement it this  
way it would be nice if that was indicated somehow (maybe with a note?).  
It was just not clear to me that is the plan.

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 02:02:51 -0700

On Mon, Sep 20, 2010 at 1:47 AM, Anne van Kesteren <[email-masked]> wrote:
> If it is expected that Chromium, Safari, Firefox, will implement it this way
> it would be nice if that was indicated somehow (maybe with a note?). It was
> just not clear to me that is the plan.

I can ask around to see if that's what folks would like to do.  From
my perspective, I think that makes the most sense unless there
compatibility issues that prevent implementations from using the same
date parser for the Date header and for cookie dates.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Mon, 20 Sep 2010 18:40:43 +0200 (CEST)

On Mon, 20 Sep 2010, Adam Barth wrote:

> I can ask around to see if that's what folks would like to do.  From my 
> perspective, I think that makes the most sense unless there compatibility 
> issues that prevent implementations from using the same date parser for the 
> Date header and for cookie dates.

The cookie date parsers are a lot more liberal in what they accept than what a 
strict parser for the Date: header needs to be, so just adopting the cookie 
date parser will then loosen the syntax requirements for Date.

This said, curl has always used the same date parser for Date and cookie-date 
and it never caused us any trouble...

-- 

  / daniel.haxx.se

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 21 Sep 2010 00:22:10 +0200

>cookie-date parser from this specification.  The impact on code size

The values that are actually used in the wild and work reasonably well
across implementations are easily parsed by a short regular expression.
Compared to that, what the draft requires is exceedingly complicated.
What the draft requires maps to a regular expression as follows.

First you have patterns for a year, month, day, and a time string. Then
one for a delimiter, and finally one for junk, which is zero or more of
"something that is not a year, month, day, or time, followed by a delim-
iter". If `_` is junk, `$` a delimiter, and `y`, `m`, `d`, `t` the other
tokens, then you have the alternatives:

  _ d $ _ m $ _ t $ _ y |
  _ d $ _ m $ _ y $ _ t |
  _ d $ _ t $ _ m $ _ y |
  ...

with the additional constraint that `y` must have 3 or 4 digits if it
occurs before `d` in the pattern. Note that you cannot have junk after
the d, t, and y and only a subset of junk after m. That means you have
to actually look ahead and check for junk before you recognize a token,
something that Chromium does not actually implement.

The complexity of the algorithm rules out a good number of simpler and
more efficient implementation strategies. You would have a hard time to
map the description to a regular expression or the equivalent automaton,
for instance. You can basically either implement it very literally,
where you probably end up with bugs like the one just mentioned, or you
can implement something else entirely.

If nobody has actually studied the differences in parsing between the
draft and Chrome and the major implementations, and how they relate to
actual cookie use on the web, yet we are to fully specify a single al-
gorithm which will then be implemented in the implementations of inter-
est (which is a bold assumption), then I fail to see why it should be
the algorithm in the draft. It's not a good algorithm considering what
it has to process in practise.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 15:49:33 -0700

On Mon, Sep 20, 2010 at 3:22 PM, Bjoern Hoehrmann <[email-masked]> wrote:
> Note that you cannot have junk after the d, t, and y and only a subset of junk after m.

Thanks for pointing that out.  Fixed.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 21 Sep 2010 02:44:43 +0200

>Thanks for pointing that out.  Fixed.

Well, previously Chromium would take "2d" as a day value, but the draft
would not. Now the draft would take "2dd" as a day value, but Chromium
would not. Similarily, when a time precedes a day value, Chromium and
the draft would treat it as a time value, now the draft treats the time
as a day value.

There are a bunch of other differences, like parsing "1:+1:1" as a time,
or accepting years with five digits (with leading zeros they do actually
occur in the wild); wouldn't it be better to match Chromium there, too?
Are we sure Chromium's implementation is one of the most widely used
variations?

(As an aside, it seems Chromium would reach a "NOTREACHED" when you use
a minus in place of a plus in the time value above, maybe that's a bug.)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 17:53:33 -0700

> a minus in place of a plus in the time value above, maybe that's a bug.)

As I wrote earlier in this thread, the algorithm in the spec doesn't
match any particular implementation in every detail.  That's not
necessary (or even desirable).  For example, I'm not aware of any
cases that require 1:+1:1 to be parsed or not parsed as an hms-time.

I made the change above because subsetting the admissible junk after
digits is added complexity that we don't need, as you so kindly
pointed out.

Thanks,
Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 21 Sep 2010 15:22:57 +0200

>digits is added complexity that we don't need, as you so kindly

I have absolutely no idea what "we need". Just because certain things do
not show up in a trivial data set doesn't mean there are no applications
that set malformed expiry dates that your proposal would parse as a date
in the past, but that are rejected by the comparatively strict parsers
in major implementations. Users typically do not appreciate if they can
no longer configure their router because their browser vendor decided it
is best to parse cookie dates more like Chrome, but not really quite
like Chrome, than as it parsed them for the past decade or so.

Applications that break because a cookie did not expire quickly enough
are rare because unsynchronized clocks are common, so a simple parser
that only tolerates minor variations of the proper HTTP date formats may
lead to inconvenience (users have to get a new cookie for each session).
A complex parser like the one you are proposing that treats a lot more
things as a date than major implementations currently do may, however,
lead to applications actually breaking, because the cookies they attempt
to set aren't set anymore.

So your proposal is not only rather complicated, it is also more risky
to implement than simpler alternatives. That is why I asked you for the
data you have in support of adopting your proposal, which unfortunately
you did not provide. Without such data I rather doubt your proposal will
be implemented by the major implementations.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Tue, 21 Sep 2010 12:05:03 -0700

> be implemented by the major implementations.

Thanks for your input.  However, the above is just your opinion.  Do
you have any data to support your claims?  If not, I'm inclined to
stick with the current draft.

Adam