Subject: Re: P1: Content-Length SHOULD be sent

From: Mark Nottingham <[email-masked]>
Date: Tue, 4 Dec 2012 15:40:49 +1100

On 28/11/2012, at 11:10 AM, Martin Thomson <[email-masked]> wrote:

> On 27 November 2012 14:58, Mark Nottingham <[email-masked]> wrote:
>> When a message is allowed to contain a body, does not have a Transfer-Encoding header field, and has a payload body length that is known to the sender before the message header section has been sent, the sender should send a Content-Length header field to indicate the length of the payload body as a decimal number of octets, unless the message is a request and the method does not define any meaning for the body (in which case the Content-Length header MAY be sent).
> 
> IANAL, so this was hard to parse for me.  How about:
> 
> --8<--
> The Content-Length header indicates the size of a message body as a
> decimal number of octets.  The sender SHOULD set Content-Length if:
>   o  the message is allowed to contain a body,
>   o  the Transfer-Encoding header field is not included, and
>   o  the size of the message body is known at the time that headers are sent.
> Messages that do not define a meaning for the body MAY either omit the
> Content-Length header field or include a value of 0.
> -->8--

I think that's an improvement, delta some adjustments to align with terminology used elsewhere and a few other editorial tweaks. Read very strictly, there's a question of precedence between the requirements, but the intent is clear enough.

I'll set this as the proposal in the ticket.

> I'm leery of the "SHOULD" still.  It seems weak.  I'd almost go with
> MUST include and a fourth condition: the message size is unknown and
> the end of the body is signaled by closing the connection (in which
> case the Connection header MUST include the 'close' token).

I know what you're saying. However, I'm not sure the improvement is great enough to justify the ensuing discussion and back-and-forth...

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 00:35:58 -0600

>> Content-Length header field or include a value of 0.

Previously, a GET request should not contain a Content-Length header
(unless the client mischievously bundles a message body in a GET
request).

A server can legitimately reject a GET request with a Content-Length
header, since it is out of the norm, and the server isn't prepared for
it - dealing with request body is not trivial.

Now, if the proposed text is in effect, a programmer of a new client
reads the proposed text and sees that it's OK to add Content-Length:0
in a GET request; even worse, he'll probably reason that it is helpful
to set Content-Length:0, just to be clear. Yet such requests will be
rejected by that server.

So the proposed change could actually harm interoperability, since it
undermines the established practice and belief that GETs shouldn't
contain Content-Length.

The current policy seems pretty good already: if there's no message
body, don't set Content-Length (with a few exceptions like in 304/HEAD
responses). A client programmer will not benefit anyway from the
ability to set Content-Length:0 when there's no message body.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Dec 2012 10:09:36 +0100

Hi,

> undermines the established practice and belief that GETs shouldn't

I don't see why you're saying this, because one of the point is :

If the GET message is not allowed to contain a body, then content-length:0
is not needed. Also, I remember that somewhere it was indicated that no
message body length indication in requests means there is no body. Ah,
found it, it's in 3.3 :

   The rules for when a message body is allowed in a message differ for
   requests and responses.

   The presence of a message body in a request is signaled by a a
   Content-Length or Transfer-Encoding header field.  Request message
   framing is independent of method semantics, even if the method does
   not define any use for a message body.

=> a client sending "content-length:0" with a GET request explicitly
says that there is a body in the request (which is empty). So it must
be prepared to have its request rejected by some servers.

> responses).

Warning, you can't do that for responses, otherwise it means that the
close determines the end of the message !

> A client programmer will not benefit anyway from the

Except in responses where this is needed if he wants to keep the connection
alive.

Regards,
Willy

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 09:42:47 -0600

Yes a GET request can contain Content-Length, but people don't really
do that. Based on that observation, a server may reject any GET with
Content-Length, reasoning that it's simpler to program, and it won't
lose any legitimate requests.

However the proposed change may encourage new programmers to add
Content-Length:0 to GET requests. (Actually it seems to be the sole
motivation for the change)

This change will disturb the existing practices, without adding any value.

(My real problem with the change is the semantics. A server API may
want to distinguish between a request with an empty entity body and a
request without an entity. Previously this can be done uniformly for
requests, per RFC2616.)

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Dec 2012 17:45:08 +0100

> motivation for the change)

As I said, I don't think it encourages this due to the paragraph above
explicitly saying that requests without body don't need to send a c-l.
But maybe this could be reinforced in the text, I don't know.

> request without an entity. Previously this can be done uniformly for

This distinction -if it exists at all- cannot be made based on the presence
of the content-length header field precisely because this one has been
optional for ages due to the compatibility with the close mode. My
understanding has always been that no entity body is an empty body and
conversely.

Regards,
Willy

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 12:32:04 -0600

> optional for ages due to the compatibility with the close mode. My

I was referring to requests only.

> understanding has always been that no entity body is an empty body and

An entity with an empty body can be non-trivial, because it can carry
meanings in entity headers. It is different from not having an entity
in a request.

For example, this is a legitimate request

POST /action HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 0

It is a form post with 0 parameters.

This is an illegitimate request:

POST /action HTTP/1.1
Content-Type: application/x-www-form-urlencoded

It's a POST request without a body. Most web servers probably will reject it.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Dec 2012 22:30:44 +0100

> It's a POST request without a body. Most web servers probably will reject it.

OK, so then if I understand you, we have two distinct cases :

  - responses always contain a body (except 1xx, 204, 304, HEAD) no matter
    what header we have ;

  - requests generally don't contain a body unless explicitly stated
    otherwise (eg: c-l: 0, or t-e: chunked + 0CRLF).

>From a pure messaging standpoint, that makes no difference, as both allow
the message parsers to keep in sync, but for a server consuming a request,
there is a semantical difference between saying "I'm not sending a body"
and "look, this is my body, and it's empty". That makes sense.

I think that everything is summarized in this part :

   6.  If this is a request message and none of the above are true, then
       the message body length is zero (no message body is present).

It ensures that all parties get the messaging right, but it does not
necessarily consider a different behaviour for a server. It's hard to
find how to change this to avoid introducing new issues or confusion.
Maybe the following sentence could simply be appended at the end of
the existing one :

       Some servers may require a message body with certain requests (for
       example, with POST or PUT methods). When a client knows that it is
       sending a request with an empty message body, it SHOULD explicitly
       state that a message body is present and of length zero in order to
       satisfy the server's requirements.

What do you think ?

Willy

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 17:05:55 -0600

>        the message body length is zero (no message body is present).

I think it should simply state

    6.  If this is a request message and none of the above are true, then
        the message contains no body.

>        satisfy the server's requirements.

I don't think such clarification is necessary. It is clearly implied
that, if a request does not contain a body, it must not set
Content-Length or Transfer-Encoding; it a request contains a body, it
must set Content-Length or Transfer-Encoding, even if the body is
empty.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 4 Dec 2012 15:08:01 -0800

>         the message contains no body.

Is it really useful to distinguish between no body and body with no
content?  I can't imagine a use for such a distinction.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 5 Dec 2012 00:11:38 +0100

Hi Martin,

> content?  I can't imagine a use for such a distinction.

I think the example with the POST that is rejected without a content-length
is valid, I have already observed this one, though I don't remember on
what server.

Willy

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Wed, 05 Dec 2012 00:19:33 +0000

------ Original Message ------
From: "Willy Tarreau" <[email-masked]>
>is valid, I have already observed this one, though I don't remember on
maybe that's a bug in that server?

Adrien

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 18:25:10 -0600

>> content?  I can't imagine a use for such a distinction.

At least, it is easier to distinguish the two, than to make them equivalent.

The former is already what RFC2616 does.

The latter requires new effort, introduces incompatible changes. And
what does it achieve? It'll allow client programmers to add a
frivolous Content-Length:0 to GET requests, and to omit
Content-Length:0 from POST requests. But why would any programmers
need that?

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Dec 2012 18:31:54 -0600

> maybe that's a bug in that server?

The server conforms to RFC2616:

the request has no Content-Length/Transfer-Encoding
=> the request has no message body
=> the request has no entity body
=> the POST request contains no entity
=> the request is unintelligible.

My prev quotes from RFC2616

=====

(HTTP messages) consist of ... possibly a message-body.

The message-body (if any) of an HTTP message is used to carry the entity-body

The presence of a message-body in a request is signaled by the
inclusion of a Content-Length or Transfer-Encoding header field in the
request's message-headers.

messages MAY transfer an entity

An entity consists of entity-header fields and an entity-body,
although some *responses* will only include the entity-headers.

An entity-body is only present in a message when a message-body is present

====

We can conclude that

If a request does not have "Content-Length" or "Transfer-Encoding", it
does not contain a message body, it does not contain an entity body,
it does not contain an entity.

If a request has "Content-Length: 0", it contains a message body of 0
bytes, it contain an entity body of 0 bytes, it contain an entity.

This is true regardless of the request method.

---

From: Jonathan Ballard <[email-masked]>
Date: Tue, 4 Dec 2012 16:43:34 -0800

To me, the bug appears in use of the word octets, as it implies HTTPS
without any Transfer-Encoding: header.

I think it is easier known if length is determinate or indeterminate on the
wire; otherwise, the numeric value is an assertion in HTTP.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 5 Dec 2012 07:48:02 +0100

Hi Adrien,

> maybe that's a bug in that server?

not necessarily, don't forget that we're both reading this with our
intermediary author hat on, and we're mostly interested in getting
messaging right. But for application servers, some subtilities may
very well make a difference. Especially considering what was said in
2616 about how to detect presence of a message body and the requirement
for POST requests to carry a message body.

Willy

---

From: Jonathan Ballard <[email-masked]>
Date: Thu, 6 Dec 2012 15:41:47 -0800

Maybe media suffixes SHOULD be allowed on Content-Length: without type.

Content-Length: 1234+ic

Multiplication of values are symmetric to octets with such suffix. That
would require no random order on Content-Type:.

Solved?

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 7 Dec 2012 10:45:55 +1100

Jonathan / Dzontatas,

Your contributions to this discussion are (yet again) disruptive.

As such, this is a public warning, as per BCP94. If you continue to disrupt the work, your posting privileges will again be suspended.

Regards,

On 07/12/2012, at 10:41 AM, Jonathan Ballard <[email-masked]> wrote:

> Multiplication of values are symmetric to octets with such suffix. That would require no random order on Content-Type:.
> 
> Solved?
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 7 Dec 2012 10:47:32 +1100

Trying to wrap this discussion up --

Roy has made a proposed edit here:
  http://trac.tools.ietf.org/wg/httpbis/trac/changeset/2034#file1
 
Please have a look and speak up if you can't live with that approach.

Regards,

---

From: Jonathan Ballard <[email-masked]>
Date: Thu, 6 Dec 2012 16:36:00 -0800

Mark, that looked like your auto-responder, if I didn't remove you from
reply-list. It seemed out of order to this public resolution.

On Thursday, December 6, 2012, Mark Nottingham wrote:

> As such, this is a public warning, as per BCP94. If you continue to
> disrupt the work, your posting privileges will again be suspended.
>
> Regards,
>
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 7 Dec 2012 11:42:22 +1100

No, that was a direct message to you, written by me (the WG chair).

On 07/12/2012, at 11:36 AM, Jonathan Ballard <[email-masked]> wrote:

> Mark, that looked like your auto-responder, if I didn't remove you from reply-list. It seemed out of order to this public resolution.
>

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 7 Dec 2012 07:37:16 +0100

> Please have a look and speak up if you can't live with that approach.

I like it, really. It clearly explains the difference between the
message and the representation and helps developers decide what to do.

+1 for me.

Willy