Subject: [Technical Errata Reported] RFC7230 (4412)

From: RFC Errata System <[email-masked]>
Date: Thu, 9 Jul 2015 03:44:01 -0700 (PDT)

The following errata report has been submitted for RFC7230,
"Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing".

--------------------------------------
You may review the report below and at:
http://www.rfc-editor.org/errata_search.php?rfc=7230&eid=4412

--------------------------------------
Type: Technical
Reported by: Rick Taylor <[email-masked]>

Section: 3.3.3

Original Text
-------------
If a Transfer-Encoding header field is present in a response and
the chunked transfer coding is not the final encoding, the
message body length is determined by reading the connection until
it is closed by the server.  If a Transfer-Encoding header field
is present in a request and the chunked transfer coding is not
the final encoding, the message body length cannot be determined
reliably; the server MUST respond with the 400 (Bad Request)
status code and then close the connection.

Corrected Text
--------------
Either:

If a Transfer-Encoding header field is present in a response and
the chunked transfer coding is not the final encoding, the
message body length is determined by reading the connection until
it is closed by the server.

Or:

If a Transfer-Encoding header field is present in a request and 
the chunked transfer coding is not the final encoding, the 
message body length cannot be determined 
reliably; the server MUST respond with the 400 (Bad Request) 
status code and then close the connection.

Notes
-----
The paragraph has two apparently contradictory rules.  It is unclear which is the correct behaviour.

Instructions:
-------------
This erratum is currently posted as "Reported". If necessary, please
use "Reply All" to discuss whether it should be verified or
rejected. When a decision is reached, the verifying party (IESG)
can log in to change the status and edit the report, if necessary. 

--------------------------------------
RFC7230 (draft-ietf-httpbis-p1-messaging-26)
--------------------------------------
Title               : Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing
Publication Date    : June 2014
Author(s)           : R. Fielding, Ed., J. Reschke, Ed.
Category            : PROPOSED STANDARD
Source              : Hypertext Transfer Protocol Bis APP
Area                : Applications
Stream              : IETF
Verifying Party     : IESG

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 9 Jul 2015 18:53:45 +0800

Reject; these are two separate requirements, on responses and requests, respectively.

Sent from my iPhone

> On 9 Jul 2015, at 6:44 pm, RFC Errata System <[email-masked]> wrote:
>

---

From: Zhong Yu <[email-masked]>
Date: Thu, 9 Jul 2015 09:45:12 -0500

> it is closed by the server.

Does it actually happen in practice that a Transfer-Encoding is applied on
a response without "chunked"?

Zhong Yu
bayou.io

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 9 Jul 2015 17:08:04 +0200

> a response without "chunked"?

It does not because the spec says that chunked must be the last one in case
of T-E.

Willy

---

From: Zhong Yu <[email-masked]>
Date: Thu, 9 Jul 2015 13:33:46 -0500

The spec does allow a response like

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: gzip
Connection: close

I tested this response with five major browsers. All of them seem to know
to consume the response body till connection EOF. ( None of them knows how
to deal with the 'gzip' encoding; and if EOF arrives prematurely resulting
in a corrupted  'gzip' body, none of them detects it as a problem. )

So it does work, sort of. Although, it seems preposterous; no server would
emit a response like that, right?

PS. If the response is "Transfer-Encoding: gzip, chunked" on a keep-alive
connection, the browsers know to demarcate the response body by chunk
framing. (Again, they don't deal with the "gzip")

Zhong Yu
bayou.io

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 9 Jul 2015 22:43:31 +0200

> The spec does allow a response like

No it does not allow it as chunked is not last. It's in 3.3.1 :

   If any transfer coding
   other than chunked is applied to a request payload body, the sender
   MUST apply chunked as the final transfer coding to ensure that the
   message is properly framed.

What it says is how to process it if it is received :

> in a corrupted  'gzip' body, none of them detects it as a problem. )

Because they comply with the spec :-)

> emit a response like that, right?

That's exactly the point. The spec took a great care at explaining what must
not be sent at all, and then how to deal with violations. The "message body
length" section is 2-3 times longer than the equivalent one in 2616 for the
exact same protocol, just because it aimed to cover all situations even the
forbidden ones to make sure implementations don't run into vulnerabilities
by missing something.

Willy

---

From: Zhong Yu <[email-masked]>
Date: Thu, 9 Jul 2015 16:11:10 -0500

>    message is properly framed.

This text talk about "REQUEST" only :) The next sentence talks about
"RESPONSE" -

> If any transfer coding other than chunked is applied to a response
payload body, the sender MUST either apply chunked as the final transfer
coding or terminate the message by closing the connection.

Apparently, the text is prune to misreading.

Zhong Yu
bayou.io

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 9 Jul 2015 23:29:35 +0200

> This text talk about "REQUEST" only :) The next sentence talks about

Hmm yes, good catch, sorry!

> Apparently, the text is prune to misreading.

Indeed, I didn't pay attention, I tend to consider that the rule
applies to both directions, though the strict control in haproxy
clearly is only for the request, just as specified.

So yes, the spec allows it and also specifies how to process it.
I doubt any server find it fun to play this game given that we
all know that connection-delimited bodies are prone to silent
truncation, but that doesn't mean no server does it.

Regards,
Willy

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 10 Jul 2015 21:15:26 +1200

CC'ing Mark to make sure this is on his radar.

This same text seems to keep coming up as errata (this is the third time
IIRC) and as below it can even catch some of us WG regulars who know
what its about.

To me it seem that the core issue is the words 'request' and 'response'
being semi-hidden in the long descriptive text.

I wonder if it is worth putting in an eratta to have the paragraph split
into two simpler paragraphs starting with words like "For requests ..."
and "For responses ..." ?

This is after all two different sets of handling criteria for different
message types. It makes a fair bit of sense to have them in different
clauses as the rest of the documents text generally does for similar things.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 15 Jul 2015 14:13:04 +0200

I'm reluctant to open the door for wordsmithing the specs for readability in errata; everyone tends to have an opinion, and few actually read the errata, which leads me to believe it's not a good use of WG time.

That said, if you want to make a concrete suggestion, go ahead (before and after).

Cheers,

> On 10 Jul 2015, at 11:15 am, Amos Jeffries <[email-masked]> wrote:
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 15 Jul 2015 15:14:05 +0200

On 2015-07-15 14:13, Mark Nottingham wrote:
> ...

Shouldn't we collect things like this over here: 
<https://github.com/httpwg/http11bis/issues>?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 15 Jul 2015 15:18:32 +0200

Indeed, that would be a great place for it to end up — especially since we've talked about taking these docs to full Standard in the not-absurdly-distant future.

Cheers,

> On 15 Jul 2015, at 3:14 pm, Julian Reschke <[email-masked]> wrote:
> 
> Shouldn't we collect things like this over here: <https://github.com/httpwg/http11bis/issues>?
> 
> Best regards, Julian