Subject: Re: [#95] Multiple Content-Lengths

From: Maciej Stachowiak <[email-masked]>
Date: Fri, 08 Oct 2010 11:11:41 -0700

On Sep 20, 2010, at 3:35 PM, Roy T. Fielding wrote:

> On Sep 20, 2010, at 1:06 AM, Anne van Kesteren wrote:
> 
>> On Mon, 20 Sep 2010 07:57:56 +0200, Mark Nottingham <[email-masked]> wrote:
>>> Two replied that they have concerns about displaying errors to end users. If they come on-list and discuss their concerns, we can discuss this more.
>> 
>> I was one of those if I remember correctly. End users just do not understand such error messages. And since they cannot do anything with them either showing error messages to end users will just make them confused, which gives them a bad product experience. It is exactly the same problem with asking users if they are okay with doing the request using the method CHICKEN to this other location. They'll just go "WTF" hit "OK" and hope it works, which is not that great really. Or worse, terminate the browser and start over.
> 
> Please, let's focus on what is actually written for the spec and
> not on vague generalities of what an error message might be.
> The spec draft 11 says
> 
>      If this is a
>      response message received by a user-agent, the message-body
>      length is determined by reading the connection until it is
>      closed; an error SHOULD be indicated to the user.
> 
> It does not say that an error is displayed in a dialog box that
> the user must click OK upon.  It does not describe a user interface
> requirement in any way, shape, or form.  How a user agent indicates
> anything to a user is dependent on the type and style of the UA.
> For example, the tiny little conformance violation icon in the
> corner of Firefox's window is sufficient to indicate an error.
> So is an error log.
> 
> The reason to indicate the error is because the user agent has
> no way to know whether it is actually rendering the message as
> desired by the server or possibly several responses or possibly
> a corrupted or truncated response.  Failure to note such things
> as errors can result in very unsafe results for some applications
> of the Web, such as medical information systems.

If the situation is so dangerous, then the UA should not present the response at all and instead should fail with a hard error. Presenting the response and then also showing an error message sends confusing mixed signals to the user. It does not provide an effective defense against security violations. A hidden error message (e.g. in the error log) will also not protect the user in the case of either a security attack or a garbled message - how would they know to look? Users don't check the error log periodically to see if they have been hacked. Furthermore, the damage may have already been done if the response is presented at all, and users will not be helped by being told about it after the fact.

Therefore, processing the response and also signaling an error is my least preferred option. Either processing the response is dangerous, in which case it should be completely rejected, or it is not all that dangerous, in which case there should be no mandated warning or error. UAs may wish to signal the problem in developer tools for debugging purposes, but that is a quality-of-implementation issue, and should be up to the judgment of the tool developers.

Regards,
Maciej

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 8 Oct 2010 20:23:52 +0200

> Therefore, processing the response and also signaling an error is my least preferred option. Either processing the response is dangerous, in which case it should be completely rejected, or it is no…

Well, I think that has been said in an earliser mail, but multiple
content lengths in requests are used by attackers, but in responses
they provide little value, so when that happens (if at all), it only
results from bugs. I've only observed them in requests forwarded by
buggy components BTW.

Regards,
Willy

---

From: William Chan (陈智昌) <[email-masked]>
Date: Fri, 8 Oct 2010 11:55:30 -0700

FWIW, on Windows Chrome 7.0.536.2 (a recent dev channel release), .0009% of
main frames (where an error would result in a user-visible Chrome network
error page, which we don't show for subresources) have responses with
multiple content-lengths.

On Tue, Sep 21, 2010 at 12:01 AM, William Chan (陈智昌)
<[email-masked]>wrote:

> On Mon, Sep 20, 2010 at 11:44 PM, Adam Barth <[email-masked]> wrote:
>
>> On Mon, Sep 20, 2010 at 10:28 PM, William Chan (陈智昌)
>> <[email-masked]> wrote:
>> > From the brief discussion amongst the Chrome network developers, we plan
>> to
>> > discard the response and display an error.
>>
>> What sort of error message are you planing to display in the case that
>> Mark asked about (a CSS stylesheet with multiple Content-Length
>> headers)?
>>
>> Adam
>>
>
> I missed that comment.  That's an interesting point.  Unless bug reports /
> user metrics indicate this merits special handling, or we have confidence
> this is an attack rather than a buggy server, I'd have no current plans to
> treat it differently from any other network error, which gets logged and
> results in a resource load failure for WebKit.
>

---

From: Adam Barth <[email-masked]>
Date: Fri, 8 Oct 2010 13:41:50 -0700

Great.  At that frequency of occurrence, we should just fail the
request with a fatal error and close the socket.  I don't see any
reason to partially render the page.

Adam

On Fri, Oct 8, 2010 at 11:55 AM, William Chan (陈智昌)
<[email-masked]> wrote:
> On Tue, Sep 21, 2010 at 12:01 AM, William Chan (陈智昌) <[email-masked]>
> wrote:
>>
>>> > From the brief discussion amongst the Chrome network developers, we
>>> > plan to

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Oct 2010 14:56:53 +0200

On 08.10.2010 22:41, Adam Barth wrote:
> ...

The current text 
(<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-11.html#rfc.section.3.3.p.8>) 
says:

"...If this is a response message received by a user-agent, the 
message-body length is determined by reading the connection until it is 
closed; an error SHOULD be indicated to the user."

Given the experiment with Chrome, we need to allow that behavior as 
well, but are we ready to require it?

So, how about (for now):

"...If this is a response message received by a user-agent, it either 
SHOULD be treated as error, or otherwise the message-body length SHOULD 
be determined by reading the connection until it is closed; an error 
SHOULD be indicated to the user."

?

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Mon, 11 Oct 2010 10:18:08 -0700

On Mon, Oct 11, 2010 at 5:56 AM, Julian Reschke <[email-masked]> wrote:
> "...If this is a response message received by a user-agent, the message-body
> length is determined by reading the connection until it is closed; an error
> Given the experiment with Chrome, we need to allow that behavior as well,
> but are we ready to require it?
>
> So, how about (for now):
>
> "...If this is a response message received by a user-agent, it either SHOULD
> be treated as error, or otherwise the message-body length SHOULD be
> determined by reading the connection until it is closed; an error SHOULD be
> indicated to the user."
>
> ?

I would say:

"...If this is a response message received by a user-agent, it MUST
be treated as error."

(or at the SHOULD-level if you're scared of MUST-level requirements).
IMHO, reading to the end of the connection is neither a good idea for
security nor necessary for compatibility (given that this case occurs
sufficiently infrequently).  I've also removed the user interface
requirements because I don't think the spec should have any user
interface requirements.

Adam

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 11 Oct 2010 15:55:36 -0700

Any idea of how many of those were duplicates (ie not mismatching)?

On 08/10/2010, at 11:55 AM, William Chan (陈智昌) <[email-masked]> wrote:

> FWIW, on Windows Chrome 7.0.536.2 (a recent dev channel release), .0009% of main frames (where an error would result in a user-visible Chrome network error page, which we don't show for subresources) have responses with multiple content-lengths.
> 
> On Tue, Sep 21, 2010 at 12:01 AM, William Chan (陈智昌) <[email-masked]> wrote:
> > From the brief discussion amongst the Chrome network developers, we plan to
> I missed that comment.  That's an interesting point.  Unless bug reports / user metrics indicate this merits special handling, or we have confidence this is an attack rather than a buggy server, I'd have no current plans to treat it differently from any other network error, which gets logged and results in a resource load failure for WebKit.
>

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 11 Oct 2010 15:57:02 -0700

No, I did not gather that information.

On Mon, Oct 11, 2010 at 3:55 PM, Mark Nottingham <[email-masked]> wrote:

> On Tue, Sep 21, 2010 at 12:01 AM, William Chan (陈智昌) <<[email-masked]>
> [email-masked]> wrote:
>
>> On Mon, Sep 20, 2010 at 11:44 PM, Adam Barth < <[email-masked]>
>> [email-masked]> wrote:
>>
>>> < <[email-masked]>[email-masked]> wrote:

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Oct 2010 14:07:39 +0200

> (or at the SHOULD-level if you're scared of MUST-level requirements).

Well, "MUST be treated as error" isn't really helpful; it doesn't 
require any observable behavior.

That a response message like this *is* broken is a statement of fact; 
the question is whether we want to require any specific handling. So, 
for instance, do we want to forbid any of the behaviors we see today? 
(use the first value/use the second value/use until end of connection)?

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 12 Oct 2010 09:13:26 -0700

On Tue, Oct 12, 2010 at 5:07 AM, Julian Reschke <[email-masked]> wrote:
>>> SHOULD
> Well, "MUST be treated as error" isn't really helpful; it doesn't require
> any observable behavior.
>
> That a response message like this *is* broken is a statement of fact; the
> question is whether we want to require any specific handling. So, for
> instance, do we want to forbid any of the behaviors we see today? (use the
> first value/use the second value/use until end of connection)?

I see.  I meant that the user agent MUST close the socket and ignore
the response, or whatever the HTTP spec idiom is for instructing the
user agent to treat this response as a fatal error.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Oct 2010 18:21:41 +0200

> user agent to treat this response as a fatal error.

Yes. As a matter of fact, my proposal wasn't better in that aspect :-)

So...

"If this is a response message received by a user-agent, it SHOULD be 
treated as in error by ignoring the message and closing the connection."

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 12 Oct 2010 09:38:57 -0700

> treated as in error by ignoring the message and closing the connection."

SGTM.

Adam

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 12 Oct 2010 20:40:07 +0200

> > treated as in error by ignoring the message and closing the connection."

I'm realizing that this is very similar to the chunked-encoding with
content-length. I wonder whether we have numbers on the cases where
this still happens, because the risks of bad handling are exactly the
same, even though the spec says that chunking must be assumed in this
case. It would be nice if numbers showed that we could simply consider
that situation as an error.

Regards,
Willy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 12 Oct 2010 21:52:15 +0200

>that situation as an error.

In http://lists.w3.org/Archives/Public/ietf-http-wg/2010JulSep/0375.html
this did not happen at all. Like I said there, the interaction between
user-written code and web server infrastructure has been improved a lot
in the past couple of years, eliminating many of the problem sources.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 12 Oct 2010 21:59:16 +0200

> in the past couple of years, eliminating many of the problem sources.

Thanks Björn, I missed that previous mail of yours. That fuels the idea
that we might now make chunk+content-length invalid.

Regards,
Willy

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 13 Oct 2010 09:47:26 +1300

On 13/10/2010 7:40 a.m., Willy Tarreau wrote:
>> On Tue, Oct 12, 2010 at 9:21 AM, Julian Reschke<[email-masked]>  wrote:

I agree.  The current spec presumes it's an innocent error that a 
Content-Length header sneaked in, and really it's meant to be chunked if 
there's a Transfer-Encoding: chunked header.

However for any other problem scenario, this leads to other issues which 
show up as malformed chunks if you're lucky.

I'm struggling to see how this could be used in an attack though.

regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 12 Oct 2010 22:56:05 +0200

>I'm struggling to see how this could be used in an attack though.

If all participants in a HTTP communication agree that the messages are
delimited by the lengths indicated for the chunks (or by the length in
the Content-Length header) then there is no problem, but if some use the
Content-Length headers while others use the chunks the framing is broken
and an attacker may be able to get some of the participants to treat the
entity body of a message as request or response.

Using both is already forbidden and what to do if you do not want to
abort the connection is also well-defined. That HTTP implementations may
also abort a connection if they feel like it is already clear, so there
does not seem to be anything that needs changing (and a requirement to
abort the connection would probably be widely ignored at least for some
time).
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "William A. Rowe Jr." <[email-masked]>
Date: Tue, 12 Oct 2010 21:05:50 -0500

> I'm struggling to see how this could be used in an attack though.

Familiarize yourself with the Watchfire HTTP Smuggling attack vector.
http://svn.apache.org/repos/asf/httpd/docs-build/trunk/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 13 Oct 2010 05:27:16 +0200

On Wed, Oct 13, 2010 at 09:47:26AM +1300, Adrien de Croy wrote:
(...)
> I'm struggling to see how this could be used in an attack though.

I'm thinking about requests. Since the Apache chunked encoding exploits
a few years ago, we realized that chunked encoding is almost unused in
requests. Some products even consider that as an attack on the old Apache
vulnerability. So I suspect that we might find products which simply
ignore chunked encoding in a request, or maybe ignore it if there's a
content-length.

Also, chunking appeared with HTTP/1.1. If we send an HTTP/1.0 request to
a component with a "transfer-encoding: chunked" header, I guess that we'll
observe various behaviours. Some products might consider chunking despite
the version, others might ignore it due to the version.

Now, for responses, my guess is that if we start from an example Adam has
suggested on hybi concerning shared hosting environments, we could imagine
an attacker's server building a response that's interpreted by a browser as
a short one, but by intermediaries as a larger one, then we can have the
browser reuse the connection for a second request to the victim's site,
which will use as a response the payload of the first one. For instance :

GET / HTTP/1.1
Host: attacker.com

	HTTP/1.1 302 Found
	Location: http://victim.com/	(same ip:port as attacker)
	Content-length: 200		(covers next fake response for intermediary)
	Transfer-encoding: chunked

	0				(end of data for spec-compliant browser)
	HTTP/1.1 302 Found
	Location: /			(reconnect to clean up the mess)
	Connection: close
	Set-Cookie: sessionid=123456; path=/
	Content-length: 0

GET / HTTP/1.1
Host: victim.com

browser may then use the unread data above just after the '0', and use it
as a valid response, believing it's really connected to victim.com.

Maybe this is stupid and cannot work for some reasons I've overlooked,
but at first glance I can't see a failure here. It only relies on an
intermediary to ignore the Transfer-encoding header (eg: a pre-1.1
proxy, we'd probably have to explicitly set "Connection: keep-alive"
in the request then).

While it might be used accidentely from a browser, we could also imagine
that an attacker could be on the client side too, and forge such requests
in order to populate an intermediary's cache with forged contents for the
victim's site. In such a case, I fear that there's no solution if the
cache is the only intermediary and it's not spec-compliant :-/

Regards,
Willy

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 13 Oct 2010 13:57:00 -0700

First bug report wrt to this change:
http://code.google.com/p/chromium/issues/detail?id=59077.  Indeed, this is a
duplicate Content-Length (not mismatching).

On Mon, Oct 11, 2010 at 3:57 PM, William Chan (陈智昌)
<[email-masked]>wrote:

>> FWIW, on Windows Chrome 7.0.536.2 (a recent dev channel release), .0009%
>> of main frames (where an error would result in a user-visible Chrome network
>>> I missed that comment.  That's an interesting point.  Unless bug reports
>>> / user metrics indicate this merits special handling, or we have confidence

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 14 Oct 2010 05:18:04 +0200

> duplicate Content-Length (not mismatching).

Indeed! So now we have one good reason to suggest relaxing the test in order
to accept exact duplicates.

The algorithm I used to check for non-duplicates multiple entries was simple :
compute the min and the max of the values found. If min != max, then error.

Thank you for this useful report of a live anomaly.

Willy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Oct 2010 10:27:38 +0200

>> treated as in error by ignoring the message and closing the connection."

I've made it say:

"If this is a response message received by a user-agent, it SHOULD be 
treated as an error by discarding the message and closing the connection."

(<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1031>)

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Oct 2010 10:29:06 +0200

> to accept exact duplicates.

I think we should wait a bit more on that.

That being said, the "SHOULD" that we now have already allows 
implementations to be permissive with respect to identical lengths...

Best regards, Julian

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 14 Oct 2010 19:28:43 -0700

On Oct 14, 2010, at 1:27 AM, Julian Reschke wrote:

> "If this is a response message received by a user-agent, it SHOULD be treated as an error by discarding the message and closing the connection."
> 

Is there a reason for this to be a SHOULD instead of a MUST? I know Adam already asked that, but I don't recall seeing an answer.

Regards,
Maciej

---

From: "Eric J. Bowman" <[email-masked]>
Date: Thu, 14 Oct 2010 21:26:03 -0600

Maciej Stachowiak wrote:
> 
> Julian Reschke wrote:
> 
> > Adam Barth wrote:
> >> ...
> >>> "If this is a response message received by a user-agent, it
> >>> SHOULD be treated as in error by ignoring the message and closing
> >>> the connection."
> >> 
> > 
> > I've made it say:
> > 
> > "If this is a response message received by a user-agent, it SHOULD
> > be treated as an error by discarding the message and closing the
> > connection."
> > 
> Is there a reason for this to be a SHOULD instead of a MUST? I know
> Adam already asked that, but I don't recall seeing an answer.
> 

While MUST would be correct for a browser-conformance specification, it
isn't in scope for HTTP to dictate what user-agents do with responses,
only how to interpret them.  If I'm debugging a server using curl, I
want to see duplicate or incorrect Content-Length headers instead of
total failure.

This isn't a security problem in my development environment, access to
which isn't possible over the Internet, so I'd rather be able to debug
without resorting to wireshark because all user-agents are required to
be as paranoid as a desktop browser.  My preferred wording:

"If this is a response message received by a user-agent, the message-
body length MAY be determined by reading the connection until it is
closed; an error SHOULD be indicated to the user."

To me, that implies that it's up to the user-agent to decide whether to
discard the message, or allow it to complete; either way is an error.

I understand the security problem, but something which in reality
occurs in an undetermined subset of responses (different Content-
Lengths) which altogether only account for .001% of traffic (duplicate
Content-Lengths) is not justification for SHOULD discard -- which would
downgrade debugging tools like curl to being only conditionally
conformant, an outcome which just isn't justified.

-Eric

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 14 Oct 2010 22:17:18 -0700

On Oct 14, 2010, at 8:26 PM, Eric J. Bowman wrote:

> only how to interpret them.

Requiring clients to close the connection seems in scope to me. Though to be fair, the requirement for the very explicit "Connection: close" directive is only SHOULD-level.

> If I'm debugging a server using curl, I

Debugging is a special case. I would not be inclined to wrap the whole spec around it.

That being said, it seems like a debugging tool would be entitled to report the error, including multiple Content-Lengths, but it would also be required to close the connection immediately and to not present the message as if it were a normal valid response.

> closed; an error SHOULD be indicated to the user."

Phrasing it like that sounds like a recipe for interoperability failure. Furthermore, it doesn't even list the option that we seem to have rough consensus is the best behavior for browsers.

> conformant, an outcome which just isn't justified.

The security risk is not proportional to the frequency of the problem on the live internet. However, the need for debugging is proportional to the frequency. So if the problem is very rare, the tradeoffs should be in favor of security. (That's assuming the security risk is indeed serious; I do not know enough about the relevant attacks to judge).

Regards,
Maciej

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Oct 2010 08:58:11 +0200

> Is there a reason for this to be a SHOULD instead of a MUST? I know Adam already asked that, but I don't recall seeing an answer.

We can't simply break formerly-conforming implementations.

What's important is IMHO that the problem is explained, and that we 
advise implementations what do to. Which keyword we use is unlikely to 
have a big effect on what implementations will do in practice.

Best regards, Julian

---

From: "Eric J. Bowman" <[email-masked]>
Date: Fri, 15 Oct 2010 01:46:11 -0600

Maciej Stachowiak wrote:
> > While MUST would be correct for a browser-conformance
> > specification, it isn't in scope for HTTP to dictate what
> > user-agents do with responses, only how to interpret them.
> 
> Requiring clients to close the connection seems in scope to me.
> Though to be fair, the requirement for the very explicit "Connection:
> close" directive is only SHOULD-level.
> 

Exactly my point.  HTTPbis won't be finished for decades if every last
aspect of it must be rewritten to detail how all user agents should
behave, based on the requirements of one specific class of user agent.

I don't have a problem with browser vendors collaborating on a
specification for how browsers, specifically, should behave in specific
situations.  But such a spec should be supplemental to HTTP, not part
of it.

> Debugging is a special case. I would not be inclined to wrap the

How does the explanation of reading to the end of the content to
determine length, while also stating that's an error, "wrap the whole
spec around" debugging?  I gave one concrete example of a use case
which precludes a MUST... how many more do I need to provide, in order
to make my point that HTTP shouldn't be wrapped around the specific
needs of *any* class of user agent?

>
> That being said, it seems like a debugging tool would be entitled to
> report the error, including multiple Content-Lengths, but it would
> also be required to close the connection immediately and to not
> present the message as if it were a normal valid response.
> 

Well, that's a SHOULD.  Have you ever seen a descriptive error message
from curl in such a case?  Kinda pointless for a tool meant to show
exactly what the server output is, because the error in one header
could be the result of a missing cr/lf in the previous header, etc...
I'd rather see the results of the code I'm working on than have to
decipher exactly what caused the user agent error I'm seeing.

But, any debate about the debugging example is pointless.  The cardinal
reason that HTTP doesn't dictate user agent behavior, is because there
are more use cases than could possibly be accounted for.  The protocol
cannot base itself on what's best for a single class of user agent.  If
that isn't good enough for any class of user agent, the developers of
user agents of that class are free to create a supplementary spec where
their precise needs *are* in-scope.

> 
> > This isn't a security problem in my development environment, access
> > to which isn't possible over the Internet, so I'd rather be able to
> > debug without resorting to wireshark because all user-agents are
> > required to be as paranoid as a desktop browser.  My preferred
> > wording:
> > 
> > "If this is a response message received by a user-agent, the
> > message- body length MAY be determined by reading the connection
> > until it is closed; an error SHOULD be indicated to the user."
> 
> Phrasing it like that sounds like a recipe for interoperability
> failure. Furthermore, it doesn't even list the option that we seem to
> have rough consensus is the best behavior for browsers.
> 

If 99.999% of implementations get it correct, then this entire phrasing
is irrelevant to interoperability.  In general, my gripe is that
HTTPbis will never be finished if every speck of wording is going to go
through the consensus grinder based on .001% edge cases based on broken
implementations which can't be *expected* to interoperate, let alone in
a secure fashion.

> > Lengths) which altogether only account for .001% of traffic
> > (duplicate Content-Lengths) is not justification for SHOULD discard
> > -- which would downgrade debugging tools like curl to being only
> > conditionally conformant, an outcome which just isn't justified.
> 
> The security risk is not proportional to the frequency of the problem
> on the live internet. However, the need for debugging is proportional
> to the frequency. So if the problem is very rare, the tradeoffs
> should be in favor of security. (That's assuming the security risk is
> indeed serious; I do not know enough about the relevant attacks to
> judge).
> 

Whatever security risk there is, comes from not following the standard.
That's to be expected in the general case.  It really isn't necessary
for that standard to go into detail about how to handle every specific
case of nonconformant syntax found in the wild.  I don't need to be
told that nonconformant syntax is an undefined security issue, as it
goes without saying, and may not be relevant to what I'm doing anyway,
so please don't impose any paranoid requirements on all user agents
based on FUD and extreme edge cases.  Put those in your supplemental
browser-conformance spec, they don't need to be in HTTP because desktop/
mobile browsing isn't the only use case of HTTP.

-Eric

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Oct 2010 14:17:09 +1100

> We can't simply break formerly-conforming implementations.

We can if it's a security issue.

> What's important is IMHO that the problem is explained, and that we advise implementations what do to. Which keyword we use is unlikely to have a big effect on what implementations will do in practice.

I tend to agree, SHOULD vs. MUST here isn't worth a tremendous amount of time. However, if we get agreement among UA implementers on MUST, that does seem the way to go.

Julian, have you put in any text about duplicate content-length values yet?

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Eric J. Bowman" <[email-masked]>
Date: Sun, 17 Oct 2010 23:00:43 -0600

> We can if it's a security issue.

The security issue in question is "HTTP request smuggling" which is an
attack vector which always takes the form of a malicious request from a
user-agent.  All it is the other way around, is a broken server putting
itself at risk.  There's no justification for a MUST even if there is
consensus for it.

I thought the consensus the WG was after, was whether or not to discard
all but the first C-L or the last C-L.  The current proposed language
says read to connection close, instead.  This makes loads of sense to
me, instead of MUST fail hard based on what concern, exactly?

-Eric

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Oct 2010 16:03:12 +1100

http://www.securiteam.com/securityreviews/5CP0L0AHPC.html

Technique #2.

---

From: "Eric J. Bowman" <[email-masked]>
Date: Sun, 17 Oct 2010 23:31:51 -0600

OK, that's definitely a browser vulnerability if the first C-L is used.
But how is this *still* a vulnerability under the current wording?  In
the presence of multiple C-L headers, if length is determined by reading
until connection close, then no splitting occurs.

-Eric

Mark Nottingham wrote:
> > The security issue in question is "HTTP request smuggling" which is
> > an attack vector which always takes the form of a malicious request
> > from a user-agent.  All it is the other way around, is a broken
> > server putting itself at risk.  There's no justification for a MUST
> > even if there is consensus for it.
> > 
> > I thought the consensus the WG was after, was whether or not to
> > discard all but the first C-L or the last C-L.  The current
> > proposed language says read to connection close, instead.  This
> > makes loads of sense to me, instead of MUST fail hard based on what
> > concern, exactly?
> > 
> > -Eric
> 
> --

---

From: Julian Reschke <[email-masked]>
Date: Mon, 18 Oct 2010 09:25:35 +0200

> Julian, have you put in any text about duplicate content-length values yet?

No. We wouldn't need it if we stick with SHOULD, right?

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 18 Oct 2010 09:29:37 +0200

On Mon, Oct 18, 2010 at 09:25:35AM +0200, Julian Reschke wrote:
> >I tend to agree, SHOULD vs. MUST here isn't worth a tremendous amount of 
> >time. However, if we get agreement among UA implementers on MUST, that 
> >does seem the way to go.
> >

The fact that that sometimes happens might cause some browsers to loosen
the check due to negative user feedback. Specifically focusing the control
on different values will help developers satisfy users demand and security
controls.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Oct 2010 11:21:44 +1100

The attack is also on intermediaries, which don't evolve as fast as browsers. By causing a hard error in the UAs, your'e reducing the effectiveness of the cache poisoning attack as well. 

Regards,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Oct 2010 17:28:15 +1100

+1; if we're going to allow duplicates to be ignored, we should be explicit about it, not rely on a SHOULD.

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 19 Oct 2010 01:11:15 -0700

I agree with that as well. If there are known exceptions to the rule that we wish to have interoperably implemented, they should be documented. That's so even if the requirement is SHOULD-level, potentially allowing further exceptions.

Regards,
Maciej

---

From: Julian Reschke <[email-masked]>
Date: Tue, 19 Oct 2010 11:20:13 +0200

>> +1; if we're going to allow duplicates to be ignored, we should be explicit about it, not rely on a SHOULD.

OK. But I still hear an "If" here.

My understanding is that Chromium is experimenting with even treating 
duplicate lengths as error; I think we should wait for more results, and 
at least one additional UA follow that.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Oct 2010 11:23:54 +1100

The current mozilla patch ignores duplicates; see
  https://bugzilla.mozilla.org/show_bug.cgi?id=597706

On 19/10/2010, at 8:20 PM, Julian Reschke wrote:

> My understanding is that Chromium is experimenting with even treating duplicate lengths as error; I think we should wait for more results, and at least one additional UA follow that.
> 
> Best regards, Julian