Subject: Re: [hybi] deflate-stream and masking

From: "Bob Gezelter" <[email-masked]>
Date: Mon, 20 Jun 2011 07:09:15 -0700

As I have noted before, Greg's observation about the interaction of
masking and compression is undoubtedly correct.

To be effective, compression must be done when the steam of data is most
strongly correlated. In short, this should be BEFORE multiplexing,
encryption, and masking. Data on different multiplexed streams cannot be
assumed to be correlated, and correlation is needed for effective
compression. 

Encrypting or masking data to create randomness similarly negates the
effectiveness of compression. 

When dealing with correlated data (e.g., the contents of a printable
file, or even a PostScript file), I have seen compression factors of
100:1. If the data had first been encrypted, or randomly masked, the
ratio would have been likely 2:1 (what is often achieved by disk or tape
compression facilities), or even less than 1:1 (worst case).

Compression should be done within each sub-stream (assuming
multiplexing, which is not in the current specification), and in any
event, before encryption and masking.

- Bob Gezelter, http://www.rlgsc.com

---

From: "Arman Djusupov" <[email-masked]>
Date: Mon, 20 Jun 2011 17:51:17 +0300

I don't think we should take the "mux" extension into account when
discussing the deflate-stream extension. Compressing "mux" channels is a
special case that should be covered by the "mux" extension specification. 

Combining all multiplexed streams into a single deflate channel is a good
idea because it is actually very cheap memory wise, otherwise compressing
each sub-channel with a separate DEFLATE stream could end up being very
expensive memory wise. You can expect over 200KB of memory needed per each
sub-channel, this basically kills the whole purpose of sub-channels which
are actually supposed to decrease the load on the server side. Two-three
thousands of compressed sub-channels would be enough to put the server on
its knees.

I consider the deflate-stream extension just as basic transport compression
which is offered "out of the box" by a WebSocket implementation,
irrespective of whether it supports multiplexing or not. 

With best regards,
Arman

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Bob Gezelter
> Sent: Monday, June 20, 2011 5:09 PM
> To: [email-masked]
> Subject: Re: [hybi] deflate-stream and masking
> 
masking
> and compression is undoubtedly correct.
> 
encryption,
> and masking. Data on different multiplexed streams cannot be assumed to
> be correlated, and correlation is needed for effective compression.
> 
> When dealing with correlated data (e.g., the contents of a printable file,
or
> even a PostScript file), I have seen compression factors of 100:1. If the
data
> had first been encrypted, or randomly masked, the ratio would have been
> likely 2:1 (what is often achieved by disk or tape compression
facilities), or
> even less than 1:1 (worst case).
> 
event,
> before encryption and masking.
> 
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi