Subject: RFC 1979 Deflate / zlib warning

From: James Carlson <[email-masked]>
Date: Fri, 19 May 2000 10:24:42 -0400 (EDT)

I've found a potentially serious bug in all popular versions of zlib
(0.99, 1.0.4, and the latest 1.1.3) by Jean-loup Gailly and Mark
Adler.  This library is used to implement RFC 1979 Deflate
compression.

The short version:

	If the deflate window size is set to 8, zlib will corrupt
	memory and (depending on your implementation) cause a kernel
	panic.  The recommended fix is to reply with Configure-Nak if
	the peer the Window parameter set to 0000 (size 8) in its
	Configure-Request and ignore Configure-Nak with Window set to
	0000.

The long version:

	The problem is that s->strstart gets set to a very large
	positive integer when wsize (local copy of s->w_size) is
	subtracted in deflate.c:fill_window().  This happens because
	MAX_DIST(s) resolves as a negative number when the window size
	is 8 -- MAX_DIST(s) is defined as s->w_size-MIN_LOOKAHEAD in
	deflate.h.  MIN_LOOKAHEAD is MAX_MATCH+MIN_MATCH+1, and that
	is 258+3+1 or 262.  Since a window size of 8 gives s->w_size
	256, MAX_DIST(s) is 256-262 or -6.

	This results in read_buf() writing over memory outside of
	s->window, and a crash.

	I tried experimenting with the definition of MAX_MATCH,
	MAX_LOOKAHEAD, and MAX_DIST(s) using cargo-cult techniques
	without much success.  I was able to get deflate() (the
	compression call) to avoid crashing, but I rewarded with
	either "invalid stored block lengths" or "oversubscribed
	dynamic bit lengths tree" on calling inflate() on the
	resulting compressed data, and I wasn't able to fix this.

Patches:

	I've posted patches for ANU PPP and a short example program
	that crashes zlib to my Sun web site:

		http://playground.sun.com/~carlsonj/

-- 
James Carlson, Internet Engineering       <[email-masked]>
SUN Microsystems / 1 Network Drive         71.234W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.497N   Fax +1 781 442 1677
"PPP Design and Debugging" --- http://people.ne.mediaone.net/carlson/ppp