Subject: Issues addressed in the -10 and -11 drafts

From: Mark Nottingham <[email-masked]>
Date: Thu, 2 Sep 2010 15:29:53 +1000

The editors believe that the following issues have been addressed by the -10 and -11 rounds of drafts. 

Please have a look. Barring objections, we'll close the relevant issues. 

Issues are linked from:
  http://trac.tools.ietf.org/wg/httpbis/trac/report/15

General issues
#69 - Clarify "Requested Variant" (-11)
#109 - Clarify entity / representation / variant terminology (-11)
#199 - Status codes and caching (-11)

p1-messaging
#73 - Clarification of the term "deflate" (-10)
#83 - OPTIONS * and proxies (-10)
#165 - Case-sensitivity of HTTP-date (-10)
#90 - Delimiting messages with multipart/byteranges (-11)
#95 - Handling multiple Content-Length headers (-11)

p2-semantics
#139 - Methods and Caching (-11)
#171 - Are OPTIONS and TRACE safe? (-11)

p3-payload
#155 - Content Sniffing (-10)
#183 - "requested resource" in content-encoding definition  (-11)

p5-range
#85 - Custom Ranges (-11)

p6-cache
#29	Age calculation
#174 - Caching authenticated responses (-10)
#208 - IANA registry for cache-control directives (-10)
#211 - Heuristic caching of URLs with query components (-10)
#216 - No-cache definition (-11)
#223 - Allowing heuristic caching for new status codes (-11)
#235 - Cache Invalidation only happens upon successful responses (-11)

--
Mark Nottingham     http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 2 Sep 2010 08:00:09 +0200

> #90 - Delimiting messages with multipart/byteranges (-11)

Good point it went away. I too am an author of an implementation that
does not support it.

> #95 - Handling multiple Content-Length headers (-11)

I reported a few cases of duplicated content-lengths I already observed
and the fact that I decided to only allow multiple content-lengths if
they are all exactly the same. You said that for this specific case this
was probably acceptable. Do you think relaxing the rule for this specific
case merits a a small add-on to the spec or should we simply enfore the
test and wait for code responsible for duplication to get fixed when
facing the errors (after all, since the content-length is not defined as
a comma-separated list, it should never appear more than once, but you
know the difference between what we say and what we observe) ?

Thanks,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 14 Sep 2010 15:31:14 +1000

On 02/09/2010, at 4:00 PM, Willy Tarreau wrote:

> they are all exactly the same.

How common is this in your experience? 

> You said that for this specific case this

I think there are two concerns here:
  1) making the spec more complex
  2) making code implementing the spec more complex

Is it going to cause your implementation problems to refuse messages with two matching content-length headers?

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 14 Sep 2010 07:50:11 +0200

Hi Mark,

> How common is this in your experience? 

Very rare, I would say that I might have observed them 2-3 times in
10 years but I was not specifically looking at that, so it might be
slightly more common. It was always caused by a poorly written
application somewhere. But you know how it works : "my application
worked before you plugged your system there, so your system is buggy
and must be fixed". It is even more complex when you install your
system at a customer's and it's one of their customers which
experiences the issue (I've had that several times with forbidden
characters in header names).

> Is it going to cause your implementation problems to refuse messages with two matching content-length headers?

No, it's even easier, I can remove a test in the code ! But I
suspect that I'll have to relax the check again in a few months
to a few years because of the effect mentionned above. Maybe we
could add a single sentence at the end of point 3 of 3.3 :

 "An implementation MAY accept exact duplicates of valid Content-Length
  header fields as a single one, though this practice is discouraged".

At least it will suggest how to handle the case for implementations
that have no other choice when facing the problem. The fact that
several implementations already make a special case of this indicates
to me that the need was encountered at least once.

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 14 Sep 2010 12:02:26 +0200

On 14.09.2010 07:50, Willy Tarreau wrote:
> ...

We currently have:

"If a message is received without Transfer-Encoding and with either 
multiple Content-Length header fields or a single Content-Length header 
field with an invalid value, then the message framing is invalid and 
MUST be treated as an error to prevent request or response smuggling."

We can't have a MAY relax a MUST; that makes it a "SHOULD".

We could relax this edge case from MUST to SHOULD, but it would 
complicate the spec a lot, and I'm not convinced it's really needed.

It would be great to have more data on this.

 > ...

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 14 Sep 2010 22:35:21 +0200

On Tue, Sep 14, 2010 at 12:02:26PM +0200, Julian Reschke wrote:
> >>Is it going to cause your implementation problems to refuse messages 
> >>with two matching content-length headers?
> >

Indeed, I see your point. Alternatively, couldn't we keep the must
and slightly adjust the text :

 "If a message is received without Transfer-Encoding and with either 
 multiple differing Content-Length header fields or any Content-Length
 header field with an invalid value, then the message framing is invalid
 and MUST be treated as an error to prevent request or response smuggling."

> It would be great to have more data on this.

I agree, though that's not easy to catch on purpose :-(

I have seen a type of component which tend to create duplicate headers,
though I've not observed that with the content-length particularly. The
type of component is in the class of all the application-level proxies
developped by some companies to perform various checks on application
contents that are to be forwarded inside or outside, and that rely on
a combination of a web server API + an HTTP client library. Eg: FCGI +
libcurl or anything like this. The issue is that the HTTP client will
automatically set some headers (eg: content-length when posting data)
while the application code will blindly copy headers provided by the
web server's API. It's the same practice that sometimes leads to duplicate
cookies or other such headers. But once again, it's not easy to find
such crappy components (that generally remain on private networks) and
to audit them :-/

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 15 Sep 2010 13:44:15 +1000

If lots of implementations are breaking in obvious ways because of duplicate headers, it will encourage those broken intermediaries to fix themselves.

However, we should discuss how UAs will display such errors to users. 

Currently, the language implies that the content is displayed to the user by the UA, as well as (SHOULD) an error message. Will that SHOULD get implemented? 

If not, intermediaries will get bug reports because a site sending two C-L headers will work with browsers, but won't work when an intermediary is interposed -- incenting them not to implement.

A few possible fixes:
  0) status quo -- UA vendors are happy to display an error to the user, because this type of error is so rare. (UA vendors?)
  1) UA required NOT to display / make available content with multiple C-L headers, giving parity with intermediaries (if they implement).
  2) Change requirement to focus on not using any more messages after this one in the connection, as they're tainted. 

Thoughts?

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 15 Sep 2010 07:33:15 +0200

> If lots of implementations are breaking in obvious ways because of duplicate headers, it will encourage those broken intermediaries to fix themselves.

In general it's the last installed component which is declared faulty and
which must adapt, at least for the time required to fix the first one. I
had to do that in haproxy several times (relax header names checks, extend
cookie parsing, etc...). But I tend to agree with the goal of cleaning up
existing implementations.

>   2) Change requirement to focus on not using any more messages after this one in the connection, as they're tainted. 

That last point is a bit dangerous IMHO. My fears are that some people who
implement intermediaries decide to do that as an acceptable fallback solution.
If their implementation forwards the message assuming the max of the two as
the body length, the next hop may very well accept the shorter length and
ignore the rule, thus effectively process two messages.

I think it is reasonable to distinguish between two cases :

  - duplicate C-L
  - different C-L

The former may result from stupid programming but should be harmless,
because whatever the C-L the implementation uses (first, last, max, min),
the result remains the same.

The second may result from attacks or from early bugs. But it already does
not work equally through various components. For instance, firefox takes
the last one. From memories, Squid takes the max. Apache takes the first
one (just checked). So the end result is that such an anomaly can't live
long. I think that's why all times I encountered two C-L, they were
duplicates that remained unnoticed.

In my opinion, we should make it mandatory to reject a message with multiple
different C-L headers. It's too dangerous and has no valid reason to be met.
The case of the duplicates can then either be rejected because it's nothing
but as a special case of the multiple C-L, or be specifically accepted
because it's harmless, if the implementation wishes to perform this specific
test.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 20 Sep 2010 15:57:56 +1000

I've been talking to a few browser vendors about this off-list (and encouraging them to get involved on-list).

Two have indicated that they intend to implement as specified, although there's some agreement that a duplicate C-L can be ignored, as Willy has implemented.

There have also been questions about the text seeming to encourage implementers to display the content when there are mismatching C-L's; although it's written so they're not required to do so, it might be good to refine this a bit.

Two replied that they have concerns about displaying errors to end users. If they come on-list and discuss their concerns, we can discuss this more.

One hasn't replied.

Based on those discussions, I'm reopening issue #95, and making a straw-man proposal to kick off discussion:

Current (-11): 

   3.  If a message is received without Transfer-Encoding and with
       either multiple Content-Length header fields or a single Content-
       Length header field with an invalid value, then the message
       framing is invalid and MUST be treated as an error to prevent
       request or response smuggling.  If this is a request message, the
       server MUST respond with a 400 (Bad Request) status code and then
       close the connection.  If this is a response message received by
       a proxy or gateway, the proxy or gateway MUST discard the
       received response, send a 502 (Bad Gateway) status code as its
       downstream response, and then close the connection.  If this is a
       response message received by a user-agent, the message-body
       length is determined by reading the connection until it is
       closed; an error SHOULD be indicated to the user.

Proposed:

   3.  If a message is received without Transfer-Encoding and with
       either multiple Content-Length header fields indicating different
       lengths or a single Content-
       Length header field with an invalid value, then the message
       framing is invalid and treated as an error to prevent
       request or response smuggling.  If this is a request message, the
       server MUST respond with a 400 (Bad Request) status code and then
       close the connection.  If this is a response message received by
       a proxy or gateway, the proxy or gateway MUST discard the
       received response, send a 502 (Bad Gateway) status code as its
       downstream response, and then close the connection.  If this is a
       response message received by a user-agent, it SHOULD NOT
       be used, but if it is the message-body
       length is determined by reading the connection until it is
       closed, and the user SHOULD be informed of the problem. 
       Subsequent responses on the connection MUST NOT
       be used.

Also, FYI:
  https://bugzilla.mozilla.org/show_bug.cgi?id=597706

Cheers,

> ignore the rule, thus effectively process two messages.

I was speaking just about user-agents, not other implementations.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Sep 2010 08:06:01 +0200

Hi Mark,

you proposal looks fine to me. I just have a minor comment :

>        be used, but if it is the message-body length is determined by
                              ^
Please add a comma here, between "is" and "the", I had to parse the sentence
three times to figure it was here and not anywhere else.

Thanks!
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 20 Sep 2010 14:58:08 +0800

Clarity suggestion only:

>        ...  If this is a response message received by
>        downstream response, and then close the connection.  ...

The construction of the sentence could be (if I were being especially dense) mistaken to mean that the connection between intermediary and client is closed, when you really mean that the connection between intermediary and server is closed.

   If this is a response message received by an intermediary, the intermediary MUST discard the response and close the upstream connection, and send a 502 status code as its downstream response.

I don't think that there needs to be any implied order to these actions, as implied by the "then".

--Martin

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 20 Sep 2010 17:02:48 +1000

How about:

... If this is a response message received by a proxy or gateway, the proxy or gateway MUST discard the received response and close its connection, and MUST send a 502 (Bad Gateway) status code to the client. ...

Although that second MUST gives me pause; there may be situations where it can use a stale response, or it may want to retry.

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 20 Sep 2010 15:11:41 +0800

> ... If this is a response message received by a proxy or gateway, the
> proxy or gateway MUST discard the received response and close its
> connection, and MUST send a 502 (Bad Gateway) status code to the
> client. ...
> 
> Although that second MUST gives me pause; there may be situations where
> it can use a stale response, or it may want to retry.

Agreed, you don't want to be overly specific about the behaviour, where there is already plenty of guidance about what an intermediary might do in this case.

  ...and MUST treat this as a failed request, which might result in a 502 (Bad Gateway) status code being send to the client.  

I'm sure that you can manage to turn this into English.

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:06:17 +0200

On Mon, 20 Sep 2010 07:57:56 +0200, Mark Nottingham <[email-masked]> wrote:
> Two replied that they have concerns about displaying errors to end  
> users. If they come on-list and discuss their concerns, we can discuss  
> this more.

I was one of those if I remember correctly. End users just do not  
understand such error messages. And since they cannot do anything with  
them either showing error messages to end users will just make them  
confused, which gives them a bad product experience. It is exactly the  
same problem with asking users if they are okay with doing the request  
using the method CHICKEN to this other location. They'll just go "WTF" hit  
"OK" and hope it works, which is not that great really. Or worse,  
terminate the browser and start over.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 20 Sep 2010 18:09:25 +1000

Thanks, Anne.

Any thoughts on the most recent proposal (same e-mail)? It only requires that the user SHOULD be informed of the problem, which IMO could be satisfied by being made visible in a developer's error console, etc.

Regards,

On 20/09/2010, at 6:06 PM, Anne van Kesteren wrote:

> I was one of those if I remember correctly. End users just do not understand such error messages. And since they cannot do anything with them either showing error messages to end users will just make them confused, which gives them a bad product experience. It is exactly the same problem with asking users if they are okay with doing the request using the method CHICKEN to this other location. They'll just go "WTF" hit "OK" and hope it works, which is not that great really. Or worse, terminate the browser and start over.
> 
> 
> -- 
> Anne van Kesteren

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 01:24:16 -0700

I don't think a wire-level network protocol should impose normative
requirements on the user interface.  A non-normative recommendation
might be more appropriate.

Adam

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:27:46 +0200

>        Subsequent responses on the connection MUST NOT

I'm not quite familiar with our code here, but if I understand the bug  
report below most (if not all) browsers do not implement any of the above.  
That does not seem good. Also, a SHOULD seems way too strong; even if we  
would report HTTP errors in an error console in most cases the user will  
not be informed at all. If I remember correctly, HTML5 typically uses MAY  
for such cases and a MUST for conformance checkers.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Sep 2010 10:46:57 +0200

On 20.09.2010 10:27, Anne van Kesteren wrote:
> report below most (if not all) browsers do not implement any of the
> above. That does not seem good. Also, a SHOULD seems way too strong;
> even if we would report HTTP errors in an error console in most cases
> the user will not be informed at all. If I remember correctly, HTML5
> typically uses MAY for such cases and a MUST for conformance checkers.
> ...

"MAY" is useless here, of course UAs "MAY" inform the user about just 
anything.

The reason why this is discussed at all (*) is that it's a *security* 
issue, and also recovery from this kind of problem isn't really possible.

Best regards, Julian

(*) as compared to a broken date, for instance.

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 20 Sep 2010 20:46:54 +1200

I think we need to balance the reluctance to assault users with error 
messages with the potential for mischief such responses could contain.

In the past there may not have been many real smuggling exploits that 
had any real damaging effect.  It would only take one vulnerability to 
be exploited on a wide scale to change user attitudes towards such 
things, and have users baying for blood - "you knew about this before, 
why didn't you do something about it!!!??!!!!"

I'm no PR person, but how you present it makes an enormous difference.

One could adopt the approach google takes with suspicious sites - post a 
big red page stating the site appears to be doing something potentially 
dangerous (the truth).  If the user really wishes, give them a button 
where they can proceed ignoring the warning.

Doing it this way provides a point of difference ("see we protect you 
from this as well") as opposed to yet another pointless nag dialog.

Plus it provides an enormous incentive for site operators to fix the 
problem.

On 20/09/2010 8:06 p.m., Anne van Kesteren wrote:
>> users. If they come on-list and discuss their concerns, we can 
>> discuss this more.
>
> using the method CHICKEN to this other location. They'll just go "WTF" 
> hit "OK" and hope it works, which is not that great really. Or worse, 

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 20 Sep 2010 10:50:22 +0200

> (*) as compared to a broken date, for instance.

What exactly is the security issue then? Before I was told it's a  
potential security issue. If it's a security issue then the specification  
should probably not define recovery at all and user agents would have  
outstanding security advisories.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 20 Sep 2010 19:06:10 +1000

As long as the browser does the right thing with the response, it doesn't matter whether the user is made aware. 

I.e., if we require the UA not to display / use the response (upgrade the first SHOULD to a MUST), the second becomes irrelevant, and as Anne says we can reduce the second part to advisory text. 

The question is whether there's a legitimate case for ignoring the fact that response smuggling could be happening. Given that two browser vendors already don't seem to think there is, I think this is a good direction to go in.

Cheers,

P.S. As an aside, it may be good to tighten up our language around user interaction, if we still are left with any requirements for it at the end.

On 20/09/2010, at 6:46 PM, Julian Reschke wrote:

> "MAY" is useless here, of course UAs "MAY" inform the user about just anything.
> 
> The reason why this is discussed at all (*) is that it's a *security* issue, and also recovery from this kind of problem isn't really possible.
> 
> Best regards, Julian
>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Sep 2010 11:10:34 +0200

On 20.09.2010 10:50, Anne van Kesteren wrote:
> potential security issue. If it's a security issue then the
> specification should probably not define recovery at all and user agents
> would have outstanding security advisories.

The security issue is that with conflicting length information, 
different recipients (proxies, user agents) may extract different 
payloads, *and* may differ in what part of the stream they use for the 
next message.

See <http://www.google.de/search?q=http+request+smuggling>.

The spec currently says that a message like this is broken. Origin 
servers are required to respond with 400 + closing the connection, 
clients are required to read until EOF + close the connection, plus 
signal an error.

The client behavior is some kind of recovery, and I'd be totally happy 
if we agreed that clients MUST NOT display the message. However I have 
my doubts that we can convince the browser implementers of that.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 02:19:57 -0700

> The question is whether there's a legitimate case for ignoring the fact that response smuggling could be happening. Given that two browser vendors already don't seem to think there is, I think this…

In general, making this kinds of decision is very mechanical.  We run
an experiment to assess the compatibility impact of making the change.
 We then compare the compatibility impact with the severity of the
issue we'd mitigate by making this change.  In this case, the severity
is somewhere between "moderate" and "low" according to our usual
severity guidelines.  That means we'd like to see a compatibility
impact of something like < 0.001% of HTTP responses.

We have nice infrastructure for running these experiments, both in the
Chromium project and in Firefox.  I'm sure if you ask someone at
Mozilla, they'd be happy to run the experiment using TestPilot.

Adam

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 20 Sep 2010 19:27:32 +1000

I was thinking along similar lines -- if Google (etc.) could check across their repositories (assuming that the information is collected).

I'll ask them if they can run a test.

Thanks,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 20 Sep 2010 15:35:48 -0700

> I was one of those if I remember correctly. End users just do not understand such error messages. And since they cannot do anything with them either showing error messages to end users will just ma…

Please, let's focus on what is actually written for the spec and
not on vague generalities of what an error message might be.
The spec draft 11 says

      If this is a

It does not say that an error is displayed in a dialog box that
the user must click OK upon.  It does not describe a user interface
requirement in any way, shape, or form.  How a user agent indicates
anything to a user is dependent on the type and style of the UA.
For example, the tiny little conformance violation icon in the
corner of Firefox's window is sufficient to indicate an error.
So is an error log.

The reason to indicate the error is because the user agent has
no way to know whether it is actually rendering the message as
desired by the server or possibly several responses or possibly
a corrupted or truncated response.  Failure to note such things
as errors can result in very unsafe results for some applications
of the Web, such as medical information systems.

....Roy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 20 Sep 2010 15:51:32 -0700

>       Subsequent responses on the connection MUST NOT

I don't have a problem with the first change (duplicate headers
being allowed) or "Content-Length: n, n".

The latter changes don't work.  "it" is ambiguous, and
"SHOULD NOT be used" is a new requirement that no current browser
supports AFAIK.  And there are no subsequent responses on the
connection because the framing no longer works -- they all get
folded into that one response because the message length is
determined by connection close [in practice].

I did not specify any specific choice of how a user agent
might inspect the stream and make a safe choice about which
of the lengths actually work for that response because, AFAIK,
none of them do so now.  I'd love to be proven wrong about that
by a sudden desire of user agents to conform to a single behavior.
I don't think we can require anything more than it SHOULD be
indicated as an error.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 21 Sep 2010 12:14:43 +1000

On 21/09/2010, at 8:51 AM, Roy T. Fielding wrote:
> supports AFAIK.

Yes, but the feedback from them about making this change is positive, overall. 

Making breaking changes to fix security issues is allowed by our charter.

> And there are no subsequent responses on the

Can you suggest a rewrite?

> by a sudden desire of user agents to conform to a single behavior.

I think it's worth discussing. As it is, it looks to me like the preference is to require that the response not be used at all, rather than trying to figure out how to salvage it.

> I don't think we can require anything more than it SHOULD be

I read the discussion so far as leaning towards getting rid of that requirement.

Cheers,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 20 Sep 2010 22:00:01 -0700

...

> I read the discussion so far as leaning towards getting rid of that requirement.

Excuse me, but ... WTF?

So two browser developers insist that they can't possibly indicate
an error message, but they are willing to refuse to use the response
at all?  Implying, of course, that they will indicate an error message,
since otherwise the user has no idea what happened to their request
(or inline image/object/stylesheet/script).

I'd really like to encourage that we stop asking people's opinions
and instead ask them what they are going to implement (or already
have implemented).  Everyone has an opinion and I'd prefer that
thoughtful answers be provided instead of knee-jerk slogans.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 21 Sep 2010 15:10:35 +1000

> (or inline image/object/stylesheet/script).

This is why I said we need to clarify the terminology around user interaction. They read "inform the user" as printing a message (possibly in a modal dialogue box, status bar, etc.). You appear to be reading it as "show a broken image box."

It would be more helpful if you explained how you envision this requirement being fulfilled. E.g., how will a browser that receives a CSS stylesheet with two content-length headers inform the user of the problem? Does simply not displaying the stylesheet qualify?

> thoughtful answers be provided instead of knee-jerk slogans.

I've been told (albeit privately) what they are willing to implement, and have communicated that back. I again encourage other implementers (thanks, Anne) to engage in the discussion directly. 

<chair hat>

Roy, characterising other people's opinions *or* intent as "knee-jerk slogans" doesn't move the discussion forward, it makes implementers more reluctant to engage openly on-list. Please refrain.

Thanks,

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 21 Sep 2010 07:28:24 +0200

Hello Roy,

> thoughtful answers be provided instead of knee-jerk slogans.

It's important to ask people what they think about breaking changes, because
we already know they don't always respect the standards. Instead of believing
that we're writing a law they're forced to apply, we should try to understand
what in that law causes them issues. For instance, if they get enough negative
feedback from users because they implement the required change, they will
simply revert it or soften it, and we'll get again several different
implementations. Some will ignore the test, others will take the min, others
the max, etc... So let's have their opinion on the subject to draw the line
between what is mandatory for security reasons and what is not because it's
just their own business.

Regards,
Willy

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 20 Sep 2010 22:28:36 -0700

>From the brief discussion amongst the Chrome network developers, we plan to
discard the response and display an error.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 20 Sep 2010 22:37:17 -0700

On Sep 20, 2010, at 10:28 PM, William Chan (陈智昌) wrote:

> From the brief discussion amongst the Chrome network developers, we plan to discard the response and display an error.

Thank you.  That is a very sensible solution and I am more
than happy to spec it that way if we can get rough consensus
(and hopefully some running code).

....Roy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 20 Sep 2010 22:50:50 -0700

> It would be more helpful if you explained how you envision this requirement being fulfilled. E.g., how will a browser that receives a CSS stylesheet with two content-length headers inform the user…

I can't do that Mark -- that would be making a UI decision
for the user agents that I am not capable of making from the
perspective of the HTTP interface.  Why should the HTTP spec
tell browsers how to handle a 404 response on a stylesheet?
It is the same problem.

> Roy, characterising other people's opinions *or* intent as "knee-jerk slogans" doesn't move the discussion forward, it makes implementers more reluctant to engage openly on-list. Please refrain.

Mark, if you are going to have private discussion with
a few browser developers and then bring them to this
list as if they were WG consensus, then you aren't being chair.

I did not dissuade *implementers* from engaging openly on this
list.  I dissuaded uninformed opinions from being the basis of
our decisions, rather than asking the real implementers of the
HTTP protocol stack within those very same groups what they
intend to implement so that we can have rough consensus and
running code.  That is the IETF way.

....Roy

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 20 Sep 2010 22:55:50 -0700

On Mon, Sep 20, 2010 at 10:37 PM, Roy T. Fielding <[email-masked]> wrote:

> > From the brief discussion amongst the Chrome network developers, we plan
> to discard the response and display an error.
>
I can probably check in the code this week or the next.
http://code.google.com/p/chromium/issues/detail?id=56344.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 21 Sep 2010 16:00:14 +1000

> tell browsers how to handle a 404 response on a stylesheet?

I wasn't asking you to specify their behaviour, merely to speculate on how they might conform -- as it seems like your understanding of the requirement is different from others -- so that the discussion can move forward. 

When I wrote that message, I hadn't seen yours from earlier today (my time), possibly because I've been CC:ed on some of these messages, not others. Apologies; it looks like you have answered that question.

> list as if they were WG consensus, then you aren't being chair.

I didn't declare consensus; I'm only trying to get a reading of the Working Group -- including implementers -- so we can get there. 

Regards,

---

From: Adam Barth <[email-masked]>
Date: Mon, 20 Sep 2010 23:44:58 -0700

> discard the response and display an error.

What sort of error message are you planing to display in the case that
Mark asked about (a CSS stylesheet with multiple Content-Length
headers)?

Adam

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 21 Sep 2010 00:01:03 -0700

On Mon, Sep 20, 2010 at 11:44 PM, Adam Barth <[email-masked]> wrote:

> to

I missed that comment.  That's an interesting point.  Unless bug reports /
user metrics indicate this merits special handling, or we have confidence
this is an attack rather than a buggy server, I'd have no current plans to
treat it differently from any other network error, which gets logged and
results in a resource load failure for WebKit.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 22 Sep 2010 14:19:42 +0200

> (and hopefully some running code).

I just did a few tests with current versions IE/FF/Op/Saf/Chrome.

Observations:

1) some pick the first Content-Length header (Op/Chr/Saf/IE), FF picks 
the second

2) some close the connection (Op/IE), some do not

3) most parse multiple lenghts in a single header just like multiple 
headers, except for FF which then ignores the header and reads until EOF

4) all are ok with multiple header instances having the same value

I think this is good news in that there's no interop for broken 
messages, thus whatever we decide to do is unlikely to break existing 
content.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Sep 2010 23:13:23 +1000

FWIW, I used this:
  http://gist.github.com/580078

On 22/09/2010, at 10:19 PM, Julian Reschke wrote:

> 1) some pick the first Content-Length header (Op/Chr/Saf/IE), FF picks the second
> 
> 3) most parse multiple lenghts in a single header just like multiple headers, except for FF which then ignores the header and reads until EOF
> 
> I think this is good news in that there's no interop for broken messages, thus whatever we decide to do is unlikely to break existing content.
> 
> Best regards, Julian
> 
> 
> 
>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 22 Sep 2010 21:28:24 +0200

On 22.09.2010 20:57, Adam Barth wrote:
> On Wed, Sep 22, 2010 at 5:19 AM, Julian Reschke<[email-masked]>  wrote:
> Did you test with three Content-Length headers?  I suspect FF is using

No.

> the last header (not the second) because that's what they usually do.

I agree that this is the likely reason.

> IE usually uses the first header, so that's not surprising.  Chrome
> usually matches Firefox (in using the last header), so there might be
> some specific reason it uses the first one here.  We could ask Darin
> whether he had a specific reason.

My wild guess is: it doesn't matter. Either there's no content out there 
relying on this, or both variations exist, and one approach breaks one 
kind, and the other approach breaks the other kind.

> This sounds worth adding to the spec.  There isn't a security problem
> if all the headers agree.

Agreed, but it makes the spec more complex, and it's not sure whether 
it's worth the special case (that's why statistical data would be useful).

>> I think this is good news in that there's no interop for broken messages,
>> thus whatever we decide to do is unlikely to break existing content.
>
> I wouldn't necessarily draw that conclusion from the data above.  The
> fact that Chrome matches IE here instead of Firefox is a hint that
> there might be something more going on.

Well, you could ask.

On the other hand, has anybody complained that FF's behavior breaks 
anything?

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 22 Sep 2010 22:17:13 +0200

> I agree that this is the likely reason.

I did the test some time ago but unfortunately I lost the results. The
test was quite simple :

  $ printf "HTTP/1.1 200 OK\r\nContent-length: 6\r\nContent-length: 2\r\nContent-length: 8\r\nContent-length: 4\r\n\r\n0123456789" | nc -lp80

Then send your browser at it. If it returns 2 bytes, it takes the min. If it
returns 4, it takes the last. If it returns 6, it takes the first. If it
returns 8, it takes the max, if it returns 10, it takes everything.

And yes, I remember about FF taking the last one.

> On the other hand, has anybody complained that FF's behavior breaks 

in my opinion, these differences don't break anything, they just make
browser-specific smuggling possible in some circumstances, that's all.
Multiple different content-length headers produce so widely different
behaviours across intermediaries that no sane production can stay long
enough in that situation without noticing and fixing.

Regards,
Willy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 22 Sep 2010 22:22:34 +0200

>it's worth the special case (that's why statistical data would be useful).

Earlier this week I had http://cutycapt.sf.net/ load a bunch of URLs
(random selection of pages linked from the german Wikipedia, at most
three per host) resulting in about 125,000 requests. There was not a
single response with both Transfer-Encoding and Content-Length, or two
or more Content-Length headers, or Content-Length headers with any-
thing but digits and white space (especially no commas). Neither does
this happen for the Alexa 1000. Obviously this does not include certain
things like responses resulting from user interaction (say, posting to
a forum).

It's a lot harder these days to get webservers send out messages with
broken framing, thanks to improvements to script interpreters, web
servers, and the interfaces between them, than it was a few years ago.

(The usual strange things are included, like headers that end up as
a reason phrase in the status line of the response ala "HTTP/1.0 200
OKContent-Type: text/html; char..."; portugese appears to be the most
popular language second to english in the reason phrase by the way,
followed by german and serbian...)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 22 Sep 2010 23:48:09 +0200

On 22.09.2010 23:33, Adam Barth wrote:
> ...
> As a control, did you look for multiple Content-Type headers?  I know
 > ...

Separate issue, thus changing the subject line.

> that these occur frequently enough that it's important to use the last
> one if you're not IE (i.e., if you actually use the Content-Type
> header for something).

I note that IE can be configured over the registry to at least report 
cases where sniffing occurred and reduce the amount of sniffing (don't 
recall the details right now).

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Wed, 22 Sep 2010 14:33:20 -0700

Adam

---

From: Adam Barth <[email-masked]>
Date: Wed, 22 Sep 2010 15:06:53 -0700

On Wed, Sep 22, 2010 at 2:48 PM, Julian Reschke <[email-masked]> wrote:
> I note that IE can be configured over the registry to at least report cases
> where sniffing occurred and reduce the amount of sniffing (don't recall the
> details right now).

Sure, but the number of folks who will do that are one or two orders
of magnitude less than the number of folks who use IE without that
registry key set.

More generally, I'd like to see HTTPbis say that the last Content-Type
header is the one that matters, but I understand that adding
requirements like that might be more specific than the rest of the
working group would like to see.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 23 Sep 2010 00:16:06 +0200

>one if you're not IE (i.e., if you actually use the Content-Type

No, of course not, but I can check now for you. In the Wikipedia derived
set are 32 responses with two Content-Type headers, all but 2 specified
the same type, and the other two had just "text/html" as one value and
"text/html; charset=iso-8859-1" as the other, both from the same server.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Wed, 22 Sep 2010 11:57:51 -0700

Adam

---

From: Dan Winship <[email-masked]>
Date: Thu, 23 Sep 2010 09:36:25 -0400

> things like responses resulting from user interaction (say, posting to

In my experience, egregiously broken HTTP responses tend to involve
either (a) embedded HTTP servers (wifi hub config servers, etc), or (b)
non-2xx responses. (Or both at once, as in
https://bugzilla.gnome.org/show_bug.cgi?id=588658 and
https://bugzilla.gnome.org/show_bug.cgi?id=588660.)

-- Dan