Subject: Ambiguities in header-field rules (p1-messaging)

From: Frank Mertens <[email-masked]>
Date: Wed, 17 Aug 2011 19:58:21 +0200

Hi,

I played around with the ABNF published by this WG and stumbled
over some rough edges.

Current rules:

OWS = *( [ obs-fold ] WSP )
header-field = field-name ":" OWS [ field-value ] OWS
field-value = *( field-content / OWS )
field-content = *( WSP / VCHAR / obs-text )

Problems:

  - field-value and field-content match the empty symbol,
    which requires searching for the longest match, which is costly
    (and confusing for the human reader)
  - because field-value matches the empty symbol claiming it optional
    in header-field allows ambiguous productions of same length
    (with or without field-value of zero length?)

Suggested improvement:

field-value = 1*( field-content OWS )
field-content = 1*( VCHAR / WSP / obs-text )

Best Regards,
Frank Mertens.

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 18 Aug 2011 15:16:38 +1200

> field-content = 1*( VCHAR / WSP / obs-text )

 The OWS on header-field remains ambiguous as well.

 Also, with WSP being in field-content there is the possibility of 
 header-field matching:

   field-name ":" [ obs-fold ] 1*( WSP OWS ) OWS

 Nasty. But section 3.2 comes to the rescue:
  "The field value does not include any leading or trailing white space"
 and
  "HTTP/1.1 senders MUST NOT produce messages that include line folding"

 So OWS in the field-value ABNF appears to be invalid in several ways 
 going by the text.

 Perhapse this would be better:

  header-field = field-name ":" [ WSP ] BWS [ field-value ]
  field-value = 1*( field-content BWS )
  field-content = 1*( VCHAR / WSP / obs-text )

 Nit: section 1.2.2 currently says:

 "Multiple OWS octets that occur within field-content
    SHOULD be replaced with a single SP before interpreting the field
    value or forwarding the message downstream."
 ...
 "Multiple RWS octets that occur within field-content SHOULD be
    replaced with a single SP before interpreting the field value or
    forwarding the message downstream.
 "

 When there is no OWS or RWS in the field-content ABNF.

 I think both should say header-field instead of field-content. Or maybe 
 drop the "within field-content" condition to make it general.

 AYJ

---

From: Frank Mertens <[email-masked]>
Date: Thu, 18 Aug 2011 07:58:55 +0000

On 08/18/2011 05:16 AM, Amos Jeffries wrote:
> Also, with WSP being in field-content there is the possibility of header-field matching:
>
> So OWS in the field-value ABNF appears to be invalid in several ways going by the text.
>
>
> I think both should say header-field instead of field-content. Or maybe drop the "within field-content" condition to make it general.
>
>
> AYJ
>
>

Yes, maybe we should also have a strict version of the grammar.
But for now, I'm happy with a working tolerant one;)
Replacing OWS by BWS would also disable support for line folding.

FM

---

From: Frank Mertens <[email-masked]>
Date: Thu, 18 Aug 2011 10:09:07 +0200

> I think both should say header-field instead of field-content. Or maybe drop the "within field-content" condition to make it general.

Yes, we should also have a strict version of the grammar.
But for now, I'm happy with a working tolerant one;)
Replacing OWS by BWS would also disable support for line folding.

FM

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 18 Aug 2011 20:35:48 +1200

On 18/08/11 19:58, Frank Mertens wrote:
>> I think both should say header-field instead of field-content. Or
>> maybe drop the "within field-content" condition to make it general.
>>
>>
>> AYJ
>>
>>
>

BWS accepts input which is folded. Look at its description in section 1.2.2

section 3.2 already prohibits folding with a MUST NOT which I quoted.

AYJ

---

From: Frank Mertens <[email-masked]>
Date: Thu, 18 Aug 2011 15:10:01 +0200

> section 3.2 already prohibits folding with a MUST NOT which I quoted.

YES, you are right and I fully agree with your ABNF rules.
I was misreading BWS as WS yesterday. Anyone around here, that can make the
changes?

FM

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 18 Aug 2011 10:11:44 -0700

> I was misreading BWS as WS yesterday. Anyone around here, that can make the

p1 is currently being rewritten, so all of those rules will be changing soon.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Sun, 21 Aug 2011 17:10:36 +0200

> Replacing OWS by BWS would also disable support for line folding.

We have discussed this years ago and as far as I recall decided to keep 
things simple by having a single ABNF, and add additional producer 
constraints into prose.

At this stage, I'm very reluctant to change this.

Best regards, Julian