Subject: Re: [Technical Errata Reported] RFC7230 (4189)

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Apr 2015 15:13:34 +0200

On 2015-02-06 23:32, Roy T. Fielding wrote:
> ...
> I looked at this originally and thought it made sense, but figured
> I would have to check carefully first before responding ... and
> then lost it.
>
> It should be verified as technical, but (like Julian) I think the
> fix should be limited to field-content and obs-fold:
>
> Section: 3.2
>
> Original Text
> -------------
>       field-name     = token
>       field-value    = *( field-content / obs-fold )
>       field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]
>       field-vchar    = VCHAR / obs-text
>
>       obs-fold       = CRLF 1*( SP / HTAB )
>                      ; obsolete line folding
>                      ; see Section 3.2.4
>
> Corrected Text
> --------------
>       field-name     = token
>       field-value    = *( field-content / obs-fold )
>       field-content  = field-vchar [ 1*( SP / HTAB / field-vchar ) field-vchar ]
>       field-vchar    = VCHAR / obs-text
>
>       obs-fold       = OWS CRLF RWS
>                      ; obsolete line folding
>                      ; see Section 3.2.4
>
> This fixes the problem examples and keeps obs-fold separate from field-content.
> It would be best if some other folks could confirm the above before making
> the errata official.

Looks right to me.

The one thing I'd change is to change

   obs-fold       = OWS CRLF RWS

to

   obs-fold       = OWS CRLF 1*( SP / HTAB )

to minimize the diff from the RFC.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Apr 2015 15:15:41 +0200

On 2015-02-07 13:57, Zhong Yu wrote:
> ...
> Roy's proposal still leaves leading/trailing obs-folds inside
> field-value, which Simon's proposal tries to address.
>
> The original text, which appears more complicated than necessary, was
> probably trying to trim white spaces as well from core elements, a
> practice that's consistently followed throughout the RFC.
>
> Zhong Yu
> ...

That is true, but I'd like to constrain the erratum to the bare minimum 
(and then discuss a better ABNF once we get to RFC 7231bis).

Best regards, Julian

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 22 Apr 2015 01:46:29 +1200

> to minimize the diff from the RFC.

Seems to me that neither version allows matching of field-content
consisting of exactly 2 characters. eg. "Content-Length:10\r\n"

If I am reading that right the part inside the [] always matches 2+
characters at a time. So we have 1 character, or 3+ characters.

This pattern would fix:
 field-vchar [ *( SP / HTAB / field-vchar ) field-vchar ]

Amos

---

From: Zhong Yu <[email-masked]>
Date: Tue, 21 Apr 2015 10:24:47 -0500

Another question about obs-fold before we proceed with the formal
definitions. Consider the following example

foo: bar<CRLF>
<SP><CRLF>
...

It won't be surprising if some parser mistakes the 2nd line as an
"empty line" that terminates the headers. Visually it *is* an empty
line.

In spirit, obs-fold should be followed by visible chars, otherwise
it's very confusing and problematic.

RFC 822 $3.2 appears to suggest the same thing, that obs-fold can only
appear between two non-empty segments.

Zhong

---

From: Zhong Yu <[email-masked]>
Date: Tue, 21 Apr 2015 10:48:53 -0500

I'll suggest the following fix:

     header-field   = field-name ":" OWS field-value OWS

     field-name     = token
*    field-value    = [ field-content  *( obs-fold field-content ) ]
*    field-content  = field-vchar [ *( SP / HTAB / field-vchar ) field-vchar ]
     field-vchar    = VCHAR / obs-text

*    obs-fold       = OWS CRLF 1*( SP / HTAB )

"field-content" spans a whole line, without leading/trailing spaces,
and contains at least one visible char.

This will force message producers to apply folding reasonably.

And if by using this stricter grammar, a message parser rejects a
production that was previously accepted, it might not be a bad thing
either - the production is practically ambiguous and invites different
interpretations.

Zhong

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 22 Apr 2015 06:31:25 +0200

> it's very confusing and problematic.

I disagree, a parser doesn't "see" characters, it consumes them. Here
you have a space after a CRLF, so it's a continuation of a folded header,
that's as simple as that. And it's important that it's properly defined
so that it's not abused by senders trying to put parsers in a situation
which is not well defined.

> appear between two non-empty segments.

And what is the parser supposed to do if it receives something which does
not match this rule ? That's always the problem when adding exceptions to
well-defined rules, it requires more work on the recipient side to properly
handle the situation. In short, it *adds* more risks of confusion.

Regards,
Willy

---

From: Zhong Yu <[email-masked]>
Date: Wed, 22 Apr 2015 08:55:06 -0500

Well, Willy you are right that we cannot change a rule that has been
in effect for 20 years. If a parser doesn't follow the rule, it is a
bug, and it needs to be fixed.

Out of curiosity, I constructed the following response, and tested on
5 major browsers

HTTP/1.1 200 OK\r\n
Connection: close\r\n
Content-Type: text/plain;charset=UTF-8\r\n
<SP>\r\n
Server: test-folding\r\n
\r\n
123456789

IE displays the response as

Server: test-folding\r\n
\r\n
123456789

That doesn't seem right.

Zhong Yu
bayou.io

---

From: Barry Leiba <[email-masked]>
Date: Wed, 22 Apr 2015 10:05:44 -0400

Interesting, and thanks for testing that.

Regardless of whether the change you propose is a good idea or not, I
think it does not fall under "errata".  It'd be a change we'd have to
consider as we advance the HTTP 1.1 specs to Internet Standard... but
in this regard, the current documents do say what they were intended
to.

I'm going to go ahead and mark this errata report "Verified" with
Julian's latest edit.  And thanks, everyone, for taking the time to
sort this out.

Barry

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 22 Apr 2015 17:03:29 +0200

> bug, and it needs to be fixed.

Yes. And RFC7230 says that we should not emit folded header fields anymore
IIRC so we don't need to recommend how to emit them either.

> IE displays the response as

Indeed, that deserves a bug report. I don't know if that can be
used as a content smuggling attack to send improper responses in
data that have not been validated by an upstream proxy for example
and which would be parsed as the response to the second request.
That said, for those to appear like a valid response, the upstream
proxy would need to accept something with spaces in the header
names, which is unlikely to happen if it behaves properly on
header folding (in general they're either very lax or very strict).

Thanks,
Willy

---

From: "Walter H." <[email-masked]>
Date: Wed, 22 Apr 2015 21:41:43 +0200

> IE displays the response as
always think of what is expected;
maybe the browser (in this case IE) expected nothing, because the
HTTP reply header

HTTP/1.1 200 OK\r\n
Connection: close\r\n

would be a complete header;

try different browsers by sending just this "mistake" reply

HTTP/1.1 200 OK\r\n
<SP>\r\n
Connection: close\r\n

I'd say just this HTTP reply header

HTTP/1.1 200 OK\r\n\r\n

is not complete ...