Subject: CRLF requirement

From: Anne van Kesteren <[email-masked]>
Date: Mon, 30 Jun 2014 13:23:50 +0200

Why does https://tools.ietf.org/html/rfc7230 still require CRLF while
most implementations handle either CR or LF fine? As far as I can tell
line parsing in HTTP is interoperable with text/plain, text/html,
text/css, etc. It's newline = CR / LF / CRLF.

-- 
http://annevankesteren.nl/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Jun 2014 13:34:20 +0200

> most implementations handle either CR or LF fine? As far as I can tell

Assuming that the statement above is true: because there are some 
existing implementations that require CRLF and which would be broken by 
just sending CR or LF.

This would be a non-backwards-compatible change.

> ...

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 30 Jun 2014 13:35:23 +0200

Hi Anne,

> text/css, etc. It's newline = CR / LF / CRLF.

Please take a look at 3.5. "Message Parsing Robustness" :

  "Although the line terminator for the start-line and header fields is
   the sequence CRLF, a recipient MAY recognize a single LF as a line
   terminator and ignore any preceding CR."

Also, I've seen some servers on some sites which do not accept plain LFs
(probably some WAFs installed there), and my experience with plain CR has
never been good, at least LF was needed on all servers tested.

Regards,
Willy

---

From: Anne van Kesteren <[email-masked]>
Date: Mon, 30 Jun 2014 13:40:12 +0200

On Mon, Jun 30, 2014 at 1:34 PM, Julian Reschke <[email-masked]> wrote:
> Assuming that the statement above is true: because there are some existing
> implementations that require CRLF and which would be broken by just sending
> CR or LF.

It seems for robustness you'd want to parse using newline (per initial
email), but require CRLF for transmission. (The tight coupling the
HTTP specification has with parsing and format has always seemed like
a bug.)

(Willy, I recommend testing sending e.g. CR-separated fields to a
client rather than a server.)

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Jun 2014 13:50:37 +0200

> HTTP specification has with parsing and format has always seemed like

Do you have any evidence that accepting anything but CRLF is needed in 
practice? If yes, we'd probably want to add something to the prose about 
the message format.

Best regards, Julian

---

From: Michael Sweet <[email-masked]>
Date: Mon, 30 Jun 2014 07:56:22 -0400

Julian,

On Jun 30, 2014, at 7:50 AM, Julian Reschke <[email-masked]> wrote:
> Do you have any evidence that accepting anything but CRLF is needed in practice? If yes, we'd probably want to add something to the prose about the message format.

In my experience, many servers and clients accept just "LF".  And for CUPS we always generate CR LF but accept both CR LF and LF alone.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Jun 2014 14:37:41 +0200

On 2014-06-30 14:23, Anne van Kesteren wrote:
> On Mon, Jun 30, 2014 at 1:50 PM, Julian Reschke <[email-masked]> wrote:
>> practice? If yes, we'd probably want to add something to the prose about the
>> message format.
>
> I thought the party line was rough consensus and running code? The
> most widely deployed clients exhibit this behavior. I doubt they're
> willing to change that, but I guess you can try make them during
> conformance testing.

Can you please clarify what kind of change you want?

The spec already says:

Do you think we need to allow single CRs as well?

Best regards, Julian

---

From: Anne van Kesteren <[email-masked]>
Date: Mon, 30 Jun 2014 14:23:37 +0200

---

From: Anne van Kesteren <[email-masked]>
Date: Mon, 30 Jun 2014 14:42:58 +0200

On Mon, Jun 30, 2014 at 2:37 PM, Julian Reschke <[email-masked]> wrote:
> The spec already says:
>
> "Although the line terminator for the start-line and header fields is the
> sequence CRLF, a recipient MAY recognize a single LF as a line terminator
> and ignore any preceding CR."
>

Yes. I would also expect that to be a MUST. Otherwise new clients not
paying attention have to learn these lessons again.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 30 Jun 2014 15:09:04 +0200

> paying attention have to learn these lessons again.

CR as a line terminator fell out of favor at the beginning of this century.

I’d sure like to know where this still exists in the wild.

The reality for the line terminator has been [CR] LF for a long time.
RFC 7230 (§ 3.5, 3rd para) reflects this nicely.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Mon, 30 Jun 2014 15:12:51 +0200

> paying attention have to learn these lessons again.

You are asking for a backwards-incompatible change on a spec that *just* 
was published.

Do you have any *evidence* that accepting single CRs is actually required?

Best regards, Julian

---

From: Anne van Kesteren <[email-masked]>
Date: Mon, 30 Jun 2014 15:16:35 +0200

On Mon, Jun 30, 2014 at 3:12 PM, Julian Reschke <[email-masked]> wrote:
> You are asking for a backwards-incompatible change on a spec that *just* was
> published.

I'm not sure why you think publishing has any merit. As long as the
running code is different, it's just a bug.

> Do you have any *evidence* that accepting single CRs is actually required?

Given that all major clients do it, I think the evidence should be
sought elsewhere.

---

From: David Krauss <[email-masked]>
Date: Mon, 30 Jun 2014 22:34:24 +0800

> running code is different, it's just a bug.

CR doesn’t appear in the grammar, so it has no well-defined meaning. Accepting and sanitizing malformed input is only sometimes considered a bug.

> Given that all major clients do it, I think the evidence should be

Such formatting is probably found most often in MacRoman files. (This encoding was deprecated at the beginning of the century, as mentioned.) Yet, the latest Eclipse editor bizarrely defaults to it on OS X. I forget, though, whether it defaults to CR endings or only to the ASCII extensions. Anyway, such behavior is completely anachronistic.

Yes, a lot of low-level code in popular products is more than 12 years old. No, that’s not a bad thing. Yet, it doesn’t imply that nothing should ever be deprecated. Especially this.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Jun 2014 11:36:55 -0700

> Given that all major clients do it, I think the evidence should be

Do you have evidence of an implementation that sends CR as a line terminator?

I know that this text has been very carefully checked, as with much of
the specification.  So I think that evidence of promiscuity on the
part of some implementations is only one half of the evidence we
require to declare this an outright bug.

I shouldn't have to remind you that not every one wants to be like a
browser.  For one, servers are highly relevant too.