Subject: Re: Ticket 162, was: Ambiguity in ABNF for comment

From: Julian Reschke <[email-masked]>
Date: Fri, 01 May 2009 15:30:02 +0200

Julian Reschke wrote:
> Thomas Nordin wrote:
>> The ABNF in 
>> http://www.ietf.org/internet-drafts/draft-ietf-httpbis-p1-messaging-06.txt 
>> (section 4.2) for "comment" seems to have the same problem as 
>> mentioned in Ticket #31 for "quoted-string". I.e. "ctext" includes the 
>> "\" character and thus "(\)" is ambiguous, either "\)" is a 
>> "quoted-pair" and the comment is still open or it is a "ctext" 
>> backslash followed by ")" thus ending the comment.
>>
>> Proposed solution: exclude "\" from ctext.
> 
> Agreed. I have created ticket 162 
> (<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/162>), and will 
> change ctext to exclude "\" soonish.

Proposed patch; change ctext to say:

       ctext          = *( OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-text )

(see also 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/162/162.diff>)

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 1 May 2009 20:12:21 -0500

>        ctext          = *( OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-

It makes perfect sense, once one realizes that x28, x29, and x5C are "(",
")", and "\" respectively. 

Nitpick: The Kleene star is unnecessary and inconsistent with obs-text. This
is enough:

    ctext = OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-text

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 02 May 2009 11:31:42 +0200

> ")", and "\" respectively. 

Indeed.

>     ctext = OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-text

Correct.

So how about:

      quoted-string  = DQUOTE *( qdtext / quoted-pair ) DQUOTE
      qdtext         = OWS / %x21 / %x23-5B / %x5D-7E / obs-text
                     ; OWS / VCHAR except DQUOTE / obs-text
      obs-text       = %x80-FF

and

      comment        = "(" *( ctext / quoted-pair / comment ) ")"
      ctext          = OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-text
                     ; OWS / VCHAR except "(", ")" and "\" / obs-text

then? See also 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/162/162.diff>.

Furthermore, am I missing something, or can't we also remove "comment" 
from the right hand side of the "comment" production?

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 02 May 2009 12:06:32 +0200

Julian Reschke wrote:
 > ...

Got an explanation in private mail: this is to allow nested comments.

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Sat, 2 May 2009 22:55:29 -0500

>                      ; OWS / VCHAR except DQUOTE / obs-text

It should be 'VCHAR except DQUOTE and "\"'.

>                      ; OWS / VCHAR except "(", ")" and "\" / obs-text

It should be 'VCHAR except "(", ")", and "\"' (one more comma), based on the
style used in the rest of the document.

> from the right hand side of the "comment" production?

IMO, nested comments are a bad idea because it makes comment (and thus Via
and User-Agent) non-regular. They should be treated as obsolete (like
obs-text), especially since those headers are very frequently parsed with
regular expressions.

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 03 May 2009 11:21:42 +0200

> It should be 'VCHAR except DQUOTE and "\"'.

Oh well. Thanks for catching this one.

> style used in the rest of the document.

Ack. New text:

      quoted-string  = DQUOTE *( qdtext / quoted-pair ) DQUOTE
      qdtext         = OWS / %x21 / %x23-5B / %x5D-7E / obs-text
                     ; OWS / <VCHAR except DQUOTE and "\"> / obs-text

      comment        = "(" *( ctext / quoted-pair / comment ) ")"
      ctext          = OWS / %x21-27 / %x2A-5B / %x5D-7E / obs-text
                     ; OWS / <VCHAR except "(", ")", and "\"> / obs-text

See 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/162/162.2.diff>.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 03 May 2009 18:49:06 +0200

> <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/162/162.2.diff>. 

Change applied with 
<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/574>.

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 May 2009 15:35:19 +1000

I've never seen an nested comment, and agree that they're nasty to  
handle.

So, +1 to deprecating.

--
Mark Nottingham     http://www.mnot.net/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 04 May 2009 07:49:32 +0200

>I've never seen an nested comment, and agree that they're nasty to  

I have plenty of those in my logs, for example, in the User-Agent
header,

  Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.0.8)
    Gecko/2009032811 Iceweasel/3.0.7

  GoogleEarth/5.0.11337.1968(Windows;Microsoft Windows Vista
    (Service Pack 0);no;kml:2.2;client:Free;type:default)

  Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0;
    GTB5; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; ...
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 May 2009 15:52:37 +1000

A pity. Is discouraging this in future implementations a bad thing?

---

From: David Morris <[email-masked]>
Date: Sun, 3 May 2009 23:24:46 -0700 (PDT)

At this stage, its gonna be pretty hard to discourage ... given the amount 
of programming in this space done by example. I'm not convinced that the
User Agent header is parsed in this fashion ... I suspect that most/all
servers do something like find the first non-whitespace following the 
colon and take everything through the last non-whitespace before the end 
of the line. So the user-agent header may not be the best example, but
I think deprecating nesting would be inviting future breakage when a
recipient decides that deprecated means they don't have to accept the
construct some number of years from now.

Having grown up on languages which allow nested comments, I find the 
concept useful and natural and find it frustrating when comments don't 
nest. So I'd rather just let them be.

Dave Morris

On Mon, 4 May 2009, Mark Nottingham wrote:

>> -- Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 05 May 2009 17:37:55 +0200

>A pity. Is discouraging this in future implementations a bad thing?

It might make sense to define that the User-Agent header takes a free
form string and that the recommended format is as currently specified.
That might match reality better, and move the nesting out of the ABNF.

Discouraging "nested comments" does not strike me as very useful, when
someone wants to nest things, they could then just use [...] or encode
everything in a quoted string, with little benefit to anyone.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 05 May 2009 17:56:51 +0200

> everything in a quoted string, with little benefit to anyone.

The other places seem to be "Via" and "Server". I assume that all of 
these should be placed the same way, so it seems keeping the ABNF the 
way it is makes sense...

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Tue, 5 May 2009 12:17:16 -0500

> these should be placed the same way, so it seems keeping the ABNF the

I agree, since some implementations are already using nested comments. We'll
just have to accept that parsing "Via" is tedious. Luckily, it almost never
has to be done.

- Brian