Subject: NEW ISSUE: Transfer-Encoding in 1.0 messages

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 22 Nov 2007 10:44:45 +0100

Hi,

  I could not find in RFC 2616 or the current drafts a discussion how to
handle a Transfer-Encoding header in HTTP/1.0 messages. Implementations
appear to either ignore it (Internet Explorer, Firefox) or process it as
if this was a HTTP/1.1 message (libwww-perl, Opera, Apache 2.2). There
may be other differences if there is also a Content-Length header. This
is a bit of a problem since this might cause some applications (e.g. a
crude intrusion detection system) to see different messages. Can we add
a note or perhaps recommendation on dealing with this situation?

regards,
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 23 Nov 2007 08:50:19 +0100

> a note or perhaps recommendation on dealing with this situation?

Who is sending transfer-encoding in HTTP/1.0 messages? Have never seen
that, and quite outside specifications.

I have seen transfer-encoding in response to HTTP/1.0 requests however,
even if that is a MUST NOT..

Regards
Henrik

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 23 Nov 2007 11:20:07 +0100

>even if that is a MUST NOT..

I do not know any current setup that would cause this, and that browsers
do not agree how to handle this is a good hint that there are none. What
I can easily imagine though is that in ancient times broken servers with
broken proxies would cause this, and that exploits might try to use this
to bypass crude security measures.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 23 Nov 2007 12:34:59 +0100

> to bypass crude security measures.

I don't think we need to do anything more about this in the specs. It's
quite clear how it should be parsed by a HTTP/1.1 client (chunked), and
also quite clear that it's not allowed to be sent so if seen on the wire
then it's fishy, i.e. either a broken implementation or someone trying
nasty things.

It's not the point of the spec to enumerate every possible protocol
violation that may be seen outside of the specifications or how
recipients is to deal with each such case. Attempts to do so would
seriously encumber the usability of the specification.

We could add a general recommendation that recipients SHOULD reject
obviously malformed messages where the sender has violated MUST
requirements or MUST NOT restrictions. But I seriously doubt many
implemeters would care to do this even if explicitly recommended by the
specifications as there is too much broken crap out there, and users do
not want to know about others broken crap, they just want things to work
at best effort. So I prefer to leave this up to each implementer to
judge how strict they want to be in their parsers.

Regards
Henrik

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 23 Nov 2007 13:01:54 +0100

>then it's fishy, i.e. either a broken implementation or someone trying

Could you say where the specification makes this quite clear? As I said,
I could not find a section that discusses this; both behaviors seem
quite reasonable to me, and that you find both in up to date implemen-
tations seems to indicate that this is not as clear as you say it is.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 23 Nov 2007 13:31:04 +0100

> tations seems to indicate that this is not as clear as you say it is.

4.4 Message Length

Regards
Henrik

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 23 Nov 2007 13:47:27 +0100

>> tations seems to indicate that this is not as clear as you say it is.

That is not clear enough for me then. If that is the intended meaning,
we could simply add "This applies to HTTP/1.1 and HTTP/1.0 messages.",
and perhaps note "Existing implementations vary in how they handle the
Transfer-Encoding header in HTTP/1.0 messages" where apperopriate.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 23 Nov 2007 13:58:40 +0100

> Transfer-Encoding header in HTTP/1.0 messages" where apperopriate.

The whole spec applies to HTTP/1.1 clients and servers. HTTP/1.1 clients
or servers receiving an HTTP/1.0 message is supposed to parse this per
the rules in the HTTP/1.1 specs, not the obsolete HTTP/1.0
specifications.

This is why there is certain areas of the HTTP/1.1 specs mention
HTTP/1.0 restrictions where the client or server need to act differently
if the message was HTTP/1.0.

The version number in an HTTP message is the compliance level of the
sender. The receiver should parse it per it's own compliance level, not
the message protocol level.

Regards
Henrik

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 23 Nov 2007 18:23:12 +0100

>the message protocol level.

Your reading of the specification requires to assume that it does not
explicitly discuss this case for a reason. However, it might just not
discuss it because the case has been overlooked, and some would see
the conflicting implementation behavior as security problem. I disagree
that doing nothing about this in the specification is the best course of
action.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 23 Nov 2007 23:17:39 +0100

> that doing nothing about this in the specification is the best course of

I am fine with adding yet another comment mentioning that for
interoperbility reasons HTTP/1.1 implementations SHOULD follow the
HTTP/1.1 specifications even if seeing HTTP/1.0 messages except where
specifically noted in the HTTP/1.1 specifications.  HTTP/1.1 clients or
servers SHOULD NOT downgrade their implementation to the informal
HTTP/1.0 specification when seeing an HTTP/1.0 message but continue
processing that message according to the rules defined in the HTTP/1.1
specifications.

Would that solve your concerns in terms of specifications?

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 Nov 2007 15:38:53 -0800

We've also briefly discussed folding RFC2145 into 2616bis, or at  
least expanding upon the reference to make it more prominent and give  
more context.

--
Mark Nottingham     http://www.mnot.net/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 26 Nov 2007 15:59:56 -0800

On Nov 26, 2007, at 3:38 PM, Mark Nottingham wrote:

> least expanding upon the reference to make it more prominent and  
> give more context.

I am pretty sure that 2145 was already folded into 2616.  Maybe  
something
was left out, but I see no reason to add more redundant specification.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 Nov 2007 22:57:57 -0800

I often have to point people to 2145, because a) they're not aware of  
it, and b) 2616 doesn't go into much detail about this topic.