Subject: Dealing with bad server chunking

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 15 Mar 2013 04:44:30 +0000

Hi all

we have recently had issues with a site where the server sends chunked 
responses back but closes the TCP connection prior to sending any 0 
chunk (in fact we never see a packet with this).

WinGate detects this as an abortive close, and if there were any filters 
processing the stream, they are reset, and the data may not go to the 
client.

However, client browsers typically "forgive" this transgression without 
any sort of warning.  Should we be making more forceful suggestions 
about this in the specs?

It seems slightly dangerous that a browser would consider content 
perfectly fine to use when the chunked transfer was aborted.

Is this actually a security issue, or should we be more tolerant?  The 
customer of course just wants to be able to go to the page, and chances 
of getting it fixed seem slim.   Is this also a known bug in IIS6.0?

Regards

Adrien

GET / HTTP/1.1
Host: www.smemsc.org
Cache-Control: no-cache
Pragma: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 
(KHTML, like Gecko) Chrome/25.0.1364.97 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Connection: Keep-Alive

HTTP/1.1 200 OK
Connection: close
Date: Fri, 15 Mar 2013 04:29:44 GMT
Server: Microsoft-IIS/6.0
x-server: sjl03
X-AspNet-Version: 2.0.50727
Transfer-Encoding: chunked
Cache-Control: private
Content-Type: text/html; charset=utf-8

1f9
<html><head>
<title></title></head>
<!-- Redirection Services SJL01WRED03 H1 -->
<frameset rows='100%, *' frameborder=no framespacing=0 border=0>
<frame src="http://www.smemsc.comcastbiz.net" name=mainwindow 
frameborder=no framespacing=0 marginheight=0 marginwidth=0></frame>
</frameset>
<noframes>
<h2>Your browser does not support frames.  We recommend upgrading your 
browser.</h2><br><br>
<center>Click <a href="herehttp://www.smemsc.comcastbiz.net">here</a> to 
enter the site.</center>
</noframes></html>

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 15 Mar 2013 07:44:46 +0100

Hi Adrien,

On Fri, Mar 15, 2013 at 04:44:30AM +0000, Adrien W. de Croy wrote:
> Is this actually a security issue, or should we be more tolerant?

I don't think this is a security issues, however this voids some of the
benefits of content-length or transfert-encoding, as the client has no
way to know if the object is complete. A cache should never cache such
a thing. For the reasons you explained above, in haproxy, when we compress
a chunk-encoded message, if the message is truncated, we also truncated it
on output so that the client can decide what to do with it and knows we
didn't get it full (eg: a cache can decide not to store it).

I have seen at least one well-known security gateway try to "fix" incorrectly
chunked messages, by skipping unaligned chunks etc... *this* is a security
issue in my opinion, as it allows messages to be accidentely merged. But I
don't think that accepting a message without 0 CRLF CRLF is a big deal.

Regards,
Willy

---

From: David Morris <[email-masked]>
Date: Fri, 15 Mar 2013 00:27:16 -0700 (PDT)

> don't think that accepting a message without 0 CRLF CRLF is a big deal.

I would think it a good idea to distinguish an incomplete last chunk and
missing 0 CRLF CRLF following a completed chunk.

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 15 Mar 2013 11:13:55 +0100 (CET)

On Fri, 15 Mar 2013, Adrien W. de Croy wrote:

> responses back but closes the TCP connection prior to sending any 0 chunk 
> (in fact we never see a packet with this).
>
> However, client browsers typically "forgive" this transgression without any 
> sort of warning.  Should we be making more forceful suggestions about this 
> in the specs?

IMHO, a broken transfer is a broken transfer. How can you know it is only a 0 
chunk that is missing and not any further chunks?

If browsers don't warn about broken transfers then I think that's their choice 
but it is not saying that it was a fine transfer as far as the actual HTTP 
transfer goes.

(lib)curl will return an error for this case.

  / daniel.haxx.se

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Mar 2013 11:38:36 +0100

On 2013-03-15 11:13, Daniel Stenberg wrote:
>> WinGate detects this as an abortive close, and if there were any
>> filters processing the stream, they are reset, and the data may not go
>> to the client.
>>
>> However, client browsers typically "forgive" this transgression
>> without any sort of warning.  Should we be making more forceful
>> suggestions about this in the specs?
>
> IMHO, a broken transfer is a broken transfer. How can you know it is
> only a 0 chunk that is missing and not any further chunks?
>
> If browsers don't warn about broken transfers then I think that's their
> choice but it is not saying that it was a fine transfer as far as the
> actual HTTP transfer goes.
> ...

One thing are truncated web pages (might be detectable by the user), one 
thing are truncated downloads (very bad).

That being said, Chrome and Firefox recently have become more picky with 
respect to malformed header fields (such as Content-Length), so if they 
get this wrong now, there's chance it might get fixed.

This obviously needs test cases.

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 15 Mar 2013 11:39:56 +0100

On Fri, Mar 15, 2013 at 11:13:55AM +0100, Daniel Stenberg wrote:
> >any sort of warning.  Should we be making more forceful suggestions about 
> >this in the specs?
> 
> IMHO, a broken transfer is a broken transfer. How can you know it is only a 
> 0 chunk that is missing and not any further chunks?
> 

Indeed and I can confirm that curl's strict checking helped us a lot to
fix compression in haproxy :-)

Willy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 15 Mar 2013 13:50:48 +0100

>of getting it fixed seem slim.   Is this also a known bug in IIS6.0?

If a network attacker can cause this to happen this can change how the
content is interpreted; for instance, `"a b"` and `"a b` without the
closing quote can be interpreted very differently in many formats and
protocols, so this might, for instance, open up XSS vulnerabilities.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Osama Mazahir <[email-masked]>
Date: Fri, 15 Mar 2013 17:55:37 +0000

>> WinGate detects this as an abortive close
Are you seeing a TCP FIN or RST?

From: Adrien W. de Croy [mailto:[email-masked]]
Sent: Thursday, March 14, 2013 9:45 PM
To: IETF HTTP Working Group
Subject: Dealing with bad server chunking

Hi all

we have recently had issues with a site where the server sends chunked responses back but closes the TCP connection prior to sending any 0 chunk (in fact we never see a packet with this).

WinGate detects this as an abortive close, and if there were any filters processing the stream, they are reset, and the data may not go to the client.

However, client browsers typically "forgive" this transgression without any sort of warning.  Should we be making more forceful suggestions about this in the specs?

It seems slightly dangerous that a browser would consider content perfectly fine to use when the chunked transfer was aborted.

Is this actually a security issue, or should we be more tolerant?  The customer of course just wants to be able to go to the page, and chances of getting it fixed seem slim.   Is this also a known bug in IIS6.0?

Regards

Adrien

GET / HTTP/1.1
Host: www.smemsc.org<http://www.smemsc.org>
Cache-Control: no-cache
Pragma: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.97 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Connection: Keep-Alive

HTTP/1.1 200 OK
Connection: close

1f9
<html><head>
<title></title></head>
<!-- Redirection Services SJL01WRED03 H1 -->
<frameset rows='100%, *' frameborder=no framespacing=0 border=0>
<frame src="http://www.smemsc.comcastbiz.net" name=mainwindow frameborder=no framespacing=0 marginheight=0 marginwidth=0></frame>
</frameset>
<noframes>
<h2>Your browser does not support frames.  We recommend upgrading your browser.</h2><br><br>
<center>Click <a href="http://www.smemsc.comcastbiz.net">here</a<http://www.smemsc.comcastbiz.net%22%3ehere%3c/a>> to enter the site.</center>
</noframes></html>

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sun, 17 Mar 2013 21:56:14 +0000

FIN from server.

Regards

Adrien

------ Original Message ------
From: "Osama Mazahir" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "IETF HTTP Working Group" <[email-masked]>
Sent: 16/03/2013 6:55:37 a.m.
Subject: RE: Dealing with bad server chunking
><center>Click <a href="http://www.smemsc.comcastbiz.net">here</a> to 
>enter the site.</center>
></noframes></html>
>
>
>
>
>

---

From: Osama Mazahir <[email-masked]>
Date: Thu, 21 Mar 2013 05:43:02 +0000

I suspect some IIS plugin generated a malformed response.  You could try posting your question to www.iis.net<http://www.iis.net>.

From: Adrien W. de Croy [mailto:[email-masked]]
Sent: Sunday, March 17, 2013 2:56 PM
To: Osama Mazahir
Cc: IETF HTTP Working Group
Subject: Re: Dealing with bad server chunking

FIN from server.

Regards

Adrien

------ Original Message ------
From: "Osama Mazahir" <[email-masked]<mailto:[email-masked]>>
To: "Adrien W. de Croy" <[email-masked]<mailto:[email-masked]>>
Cc: "IETF HTTP Working Group" <[email-masked]<mailto:[email-masked]>>
Sent: 16/03/2013 6:55:37 a.m.
Subject: RE: Dealing with bad server chunking

From: Adrien W. de Croy [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, March 14, 2013 9:45 PM
To: IETF HTTP Working Group
Subject: Dealing with bad server chunking

Hi all

Regards

Adrien

GET / HTTP/1.1

HTTP/1.1 200 OK
Connection: close

1f9
<html><head>
<title></title></head>