Subject: Re: CRLF requirement

From: Zhong Yu <[email-masked]>
Date: Tue, 1 Jul 2014 09:36:01 -0500

On Mon, Jun 30, 2014 at 6:35 AM, Willy Tarreau <[email-masked]> wrote:
> Hi Anne,
>
> On Mon, Jun 30, 2014 at 01:23:50PM +0200, Anne van Kesteren wrote:
>> Why does https://tools.ietf.org/html/rfc7230 still require CRLF while
>> most implementations handle either CR or LF fine? As far as I can tell
>> line parsing in HTTP is interoperable with text/plain, text/html,
>> text/css, etc. It's newline = CR / LF / CRLF.
>
> Please take a look at 3.5. "Message Parsing Robustness" :
>
>   "Although the line terminator for the start-line and header fields is
>    the sequence CRLF, a recipient MAY recognize a single LF as a line
>    terminator and ignore any preceding CR."

The text in RFC2616 was
http://tools.ietf.org/html/rfc2616#section-19.3
> we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore **the** leading CR.

The change from [CR] LF to *CR LF is dangerous, IMO. Previously,
CRCRLF must be rejected, now, in rfc7230, it MAY be accepted as one
line break. (And then Anne wanted it to be accepted as two line
breaks).

It's always troubling if different agents MAY interpret the same
message differently. We cannot add any more leniency to parsing. If
any new software generates anything but CRLF in 2014, it's just sloppy
programming.

Zhong Yu
bayou.io

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 1 Jul 2014 16:52:23 +0200

Hi,

On Tue, Jul 01, 2014 at 09:36:01AM -0500, Zhong Yu wrote:
> line break.

I don't read it this way, here above I read "any CR" as "the CR if it
exists" just like in "anyone", with "CR" still being used in singular
and not plural form. I guess we would have come up with "any and all"
or "any series of CR" or something like this if there was an intent
to indicate more than one. But at least now that you raise this, I
admit it can be confusing.

> (And then Anne wanted it to be accepted as two line
> breaks).
> 

In fact it's still common to see that in shell scripts. I've seen
some monitoring scripts use echo $REQUEST | netcat $dest.

Willy

---

From: Zhong Yu <[email-masked]>
Date: Tue, 1 Jul 2014 10:00:17 -0500

> exists" just like in "anyone", with "CR" still being used in singular

I see, thanks for the explanation.

> some monitoring scripts use echo $REQUEST | netcat $dest.

Yes, we have to accept single LFs. But that's it... We cannot accept
LFCR as two line breaks; the only sane thing to do is outright
rejection.

Zhong Yu
bayou.io

---

From: Anne van Kesteren <[email-masked]>
Date: Tue, 1 Jul 2014 17:05:07 +0200

> LFCR as two line breaks; the only sane thing to do is outright

Did you test that? That is not what happens in the most popular clients today.

-- 
http://annevankesteren.nl/

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 1 Jul 2014 17:29:52 +0200

On 01 Jul 2014, at 17:05, Anne van Kesteren <[email-masked]> wrote:

> That is not what happens in the most popular clients today.

I’m still curious why you think that’s relevant.

Browsers are soup-tolerant because they compete, among other things, on soup-tolerance.
That doesn’t turn soupiness into a general recommendation in a protocol specification.

I’m much more interested in data that shows that any bare-CR problem is still extant in the wild, than data that shows traces of somebody thinking a long time ago that it might be a good idea to guard against such a problem, might it ever occur.

Grüße, Carsten

---

From: Anne van Kesteren <[email-masked]>
Date: Tue, 1 Jul 2014 17:35:40 +0200

> I’m still curious why you think that’s relevant.

Network effects. Content will end up relying on quirks of dominant
players, especially if those quirks are stable and interoperable among
a class of clients such as browsers.

> Browsers are soup-tolerant because they compete, among other things, on soup-tolerance.

o_O

---

From: Julian Reschke <[email-masked]>
Date: Tue, 01 Jul 2014 17:54:11 +0200

> a class of clients such as browsers.

That's true when the "bug" is likely to happen, such as creating broken 
filename parameters in Content-Disposition by just using string 
concatenation (causing whitespace and non-ASCII to appear where they do 
not belong).

But here? Why would anybody do this, even by accident?

> ...

Best regards, Julian

---

From: Anne van Kesteren <[email-masked]>
Date: Tue, 1 Jul 2014 17:55:46 +0200

> But here? Why would anybody do this, even by accident?

If you have something like Apache's mod_asis it seems quite plausible.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 01 Jul 2014 18:18:09 +0200

> If you have something like Apache's mod_asis it seems quite plausible.

That would be somebody being able to use mod_asis, *and* using a pre-OSX 
MacOS, right?

Best regards, Julian

---

From: "Martin Nilsson" <[email-masked]>
Date: Wed, 02 Jul 2014 01:18:17 +0200

> But here? Why would anybody do this, even by accident?

With compliments from our bug database:

"HTTP/1.1 200 \r"
"Content-Type: image/gif\r"
"nnCoection: close\r"
"Content-Length: 42\r"
"Cache-Control: no-cache\r"
"Pragma: no-cache\r"
"\n"

(followed by 42 bytes of gif)

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Jul 2014 01:38:36 +0200

On 02 Jul 2014, at 01:18, Martin Nilsson <[email-masked]> wrote:

> bug

Indeed.

(Curious: Which vintage was that bug report?
And was the bug introduced by the proxy or was that from the origin server?)

Grüße, Carsten

---

From: "Martin Nilsson" <[email-masked]>
Date: Wed, 02 Jul 2014 02:01:15 +0200

> And was the bug introduced by the proxy or was that from the origin  

The vintage is 2009. I would assume that the proxy (that scrambled the  
connection header), received a header block composed with \r and \r\r. It  
parsed it, modified it and then glued it together and botched the header  
termination sequence.

I just checked, and the URL in question responds with sane HTTP now.

/Martin Nilsson

---

From: David Krauss <[email-masked]>
Date: Wed, 2 Jul 2014 09:22:47 +0800

On 2014–07–02, at 8:01 AM, Martin Nilsson <[email-masked]> wrote:

> The vintage is 2009. I would assume that the proxy (that scrambled the connection header), received a header block composed with \r and \r\r. It parsed it, modified it and then glued it together and botched the header termination sequence.

It looks to me like the proxy received valid input (\r\n) but at some point assumed that newline is a single character — it treated the socket as if it were in text mode. The header content was extracted correctly. Perhaps the author noticed a bug when taking \n as the line terminator. But, to re-add the newline, it used byte-oriented C string processing and only got the \r instead.

On the last line, he intended to add two bytes to get \n\n, hence we get a single \r\n. Which is the correct sequence to introduce a new header, but instead the content follows.

Also, somehow the word Connection got munged to nnCoection. Never underestimate the expressive power of old-school C — I’d wager that this is somehow the result of a typo where the backslash was omitted from “\n” in the source.

It’s totally reasonable to ask others to handle this sort of bug. Whether or not my intuitive inferences are correct, this is not the output of a production-quality program.

---

From: Glen Knowles <[email-masked]>
Date: Tue, 1 Jul 2014 19:18:35 -0700

On 2014–07–02, at 8:01 AM, Martin Nilsson <[email-masked]> wrote:
>Also, somehow the word Connection got munged to nnCoection. Never
underestimate the expressive power of old-school C — I’d wager that this is
somehow the result of a typo where the backslash was omitted from “\n” in
the source.

Changing "Connection: close" to "nnCoection: close" is something I have
seen some "zero copy" proxies do when they wanted to remove the connection
header. The theory is you just load a 32 bit "Conn", rotate it 16 bits to
"nnCo", put it back, and rely on the upstream server to ignore it as an
unrecognized header.

That said, it's also something that I haven't seen in a long time, 2009
sounds about right.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 2 Jul 2014 12:32:38 +1000

On 2 Jul 2014, at 12:18 pm, Glen Knowles <[email-masked]> wrote:

> Changing "Connection: close" to "nnCoection: close" is something I have seen some "zero copy" proxies do when they wanted to remove the connection header. The theory is you just load a 32 bit "Conn", rotate it 16 bits to "nnCo", put it back, and rely on the upstream server to ignore it as an unrecognized header.
> 
> That said, it's also something that I haven't seen in a long time, 2009 sounds about right.

Is still very common, from what I see...

--
Mark Nottingham   https://www.mnot.net/

---

From: David Krauss <[email-masked]>
Date: Wed, 2 Jul 2014 10:35:29 +0800

> Changing "Connection: close" to "nnCoection: close" is something I have seen some "zero copy" proxies do when they wanted to remove the connection header. The theory is you just load a 32 bit "Conn…

Hmm, if it was a convention among several implementations at the time, perhaps it should be recorded in the standard. /s

Also, such bitwise operations for string manipulation suggest unaligned memory access, which is a uniquely risky way of parsing. The same effect would be achieved by substituting the first character. Lemmings.

---

From: "Martin Nilsson" <[email-masked]>
Date: Wed, 02 Jul 2014 10:27:38 +0200

On Wed, 02 Jul 2014 04:35:29 +0200, David Krauss <[email-masked]> wrote:

>> seen some "zero copy" proxies do when they wanted to remove the  
>> connection header. The theory is you just load a 32 bit "Conn", rotate  
>> it 16 bits to "nnCo", put it back, and rely on the upstream server to  
>> ignore it as an unrecognized header.
>
> Hmm, if it was a convention among several implementations at the time,  
> perhaps it should be recorded in the standard. /s
>

I have more often seen it overwritten. Xonnection: close or XXXXXXXXXX:  
close.

/Martin Nilsson