Subject: chunking without chunking

From: Adrien de Croy <[email-masked]>
Date: Wed, 17 Jun 2009 18:40:10 +1200

Hi all

sorry, today seems to be a run of issues.  We found a website that sends 
Transfer-Encoding: chunked, but then doesn't send the data chunked.  
This means the first line of content is parsed as a chunk header, which 
resolves to a chunk length of 0, since the first line is non-numeric.

OK, so this is obviously seriously broken.  And it broke our proxy, 
since we decided we were getting a 0 chunk right up followed by a bunch 
of extraneous data.

However, it works in IE and Chrome.  Firefox spins its wheels indefinitely.

So, IE (8) and Chrome (2) can scavenge the data out of a stream that is 
supposed to be chunked but isn't.  This is possibly a security problem 
for these browsers.

The URL I was hitting is

http://www.nrhrehab.org/Locations/Locations_Page.aspx?id=48

It loads a bunch of JS pages, which exhibit this issue.

The server is reporting itself as IIS 6.  Is this a common problem with 
IIS, or is this likely to be a bug in their ASPX, e.g. inserting the 
Transfer-Encoding header from script, but not actually doing the 
chunking, and IIS 6 not trapping this?

Handling this cleanly is difficult.  We get complaints from customers 
that the proxy breaks the site, because their browser works fine going 
direct to the site, so it must be the proxy's fault.

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 17 Jun 2009 17:12:01 +1000

That's really broken.

I note that the transfer-encoding header uses "Chunked" -- capital C  
-- as the value. While this is allowed (transfer-codings are case- 
insensitive), it's atypical; I wonder as well if they've got some  
extension or library that's inserting that.

My library just gives up when it can't parse the chunk size on that  
URI. How do you get a chunk size of 0 out of that?

The forgiving thing to do would be to consider a chunk size parsing  
error as a trigger to fall back to content-length or connection close  
(whichever happens first) and consider the connection dirty (i.e.,  
don't reuse it); I imagine this is what IE and Chrome are doing. As  
you point out, there are some security (and probably interop) issues  
here, though. I wouldn't do it, personally.

Anybody from IE or Chrome care to comment here? IMO this is an area  
where browsers shouldn't be forgiving.

Cheers,

On 17/06/2009, at 4:40 PM, Adrien de Croy wrote:

>
> Hi all
>
> sorry, today seems to be a run of issues.  We found a website that  
> sends Transfer-Encoding: chunked, but then doesn't send the data  
> chunked.  This means the first line of content is parsed as a chunk  
> header, which resolves to a chunk length of 0, since the first line  
> is non-numeric.
>
> since we decided we were getting a 0 chunk right up followed by a  
> bunch of extraneous data.
>
> So, IE (8) and Chrome (2) can scavenge the data out of a stream that  
> is supposed to be chunked but isn't.  This is possibly a security  
> problem for these browsers.
>
> The URL I was hitting is
>
> The server is reporting itself as IIS 6.  Is this a common problem  
> with IIS, or is this likely to be a bug in their ASPX, e.g.  
> inserting the Transfer-Encoding header from script, but not actually  
> doing the chunking, and IIS 6 not trapping this?
>
> Handling this cleanly is difficult.  We get complaints from  
> customers that the proxy breaks the site, because their browser  
> works fine going direct to the site, so it must be the proxy's fault.
>
> Adrien
>
> -- 

--
Mark Nottingham     http://www.mnot.net/

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 18 Jun 2009 12:15:34 +0200

> resolves to a chunk length of 0, since the first line is non-numeric.

Which is an invalid chunk.. malformed HTTP response etc..

> since we decided we were getting a 0 chunk right up followed by a bunch 

Correct. Both broken. Site broken in sending a malformed HTTP response,
proxy broken in incorrectly misreading it as a valid response...

> However, it works in IE and Chrome.  Firefox spins its wheels indefinitely.

Heh..

Specs is quite dim on how clients should deal with malformed responses,
so no surprise agents differ in how they react.

> supposed to be chunked but isn't.  This is possibly a security problem 

Would be very interesting to see results if you mix Content-Length with
this, and pipelined responses...

> direct to the site, so it must be the proxy's fault.

As usual...

Tried contacting the site owners/authors/technical webmaster? It's not a
very common server brokeness and the company owning the site in question
seem to be both responsible and serious..

Regards
Henrik