Subject: [hybi] deflate-stream vs. frame-by-frame

From: "Bob Gezelter" <[email-masked]>
Date: Wed, 22 Jun 2011 06:00:58 -0700

Arman,

With regards to:

>I don't think we should take the "mux" extension into account when
>discussing the deflate-stream extension. Compressing "mux" channels is a
>special case that should be covered by the "mux" extension specification.
>
>Combining all multiplexed streams into a single deflate channel is a good
>idea because it is actually very cheap memory wise, otherwise compressing
>each sub-channel with a separate DEFLATE stream could end up being very
>expensive memory wise. You can expect over 200KB of memory needed per each
>sub-channel, this basically kills the whole purpose of sub-channels which
>are actually supposed to decrease the load on the server side. Two-three
>thousands of compressed sub-channels would be enough to put the server on
>its knees.
>
>I consider the deflate-stream extension just as basic transport compression
>which is offered "out of the box" by a WebSocket implementation,
>irrespective of whether it supports multiplexing or not.

There are several issues that are co-mingled in this thread.

First, there is the question of compression and masking. Compressing
masked text is of dubious utility, efficacy, and efficiency. Masking
tends to randomize text, random text is a poor candidate for compression
(in some cases, attempting to compress random text can actually cause
the "compressed" text to be larger than the original text). Taken solely
on this basis, ANY compression should precede masking.

Second, John Tamplin's draft multiplexing proposal well notes that
multiplexing opens up possibilities for various traffic management
devices (e.g., de-aggregators) at the WebSocket protocol level. Thus, a
collection of sub-channels within a WebSocket connection may end up at
different destination nodes. If the entire stream is compressed as a
whole, this will require the entire stream to be de-compressed and then
the individual sub-channels re-compressed before ongoing transmission.
This realizes little benefit over compression at the sub-channel level
(further noting that there may indeed be compression done in any event
at lower levels of the protocol stack).

Lastly, compression on a sub-channel by sub-channel basis (generally
meaning per-frame basis) does not imply a need for 200KB of buffer space
per sub-channel. Rather, the amount of buffer space required per
sub-channel depends upon the frame-size and the compression scheme used.
Some compression schemes can function with minimal context, others may
need more. In any event, compression works best with a high degree of
correlation between successive data elements, thus there is a logical
division between the frames being transmitted within different
sub-channels to (presumably) different applications.

- Bob Gezelter, http://www.rlgsc.com

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 22 Jun 2011 15:54:34 +0200

> on this basis, ANY compression should precede masking.

I expect this is obvious for everyone (or should be). Will not WS
draft be adapted to fulfill this obvious reality?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Arman Djusupov" <[email-masked]>
Date: Wed, 22 Jun 2011 17:49:50 +0300

> tends to randomize text, random text is a poor candidate for 
> compression (in some cases, attempting to compress random text can 
> actually cause the "compressed" text to be larger than the original 
> text). Taken solely on this basis, ANY compression should precede masking.

Agree, compression should be applied before masking or not applied at all. 
 
> meaning per-frame basis) does not imply a need for 200KB of buffer 
> space per sub-channel. Rather, the amount of buffer space required per
> sub- channel depends upon the frame-size and the compression scheme used.

Yes, there are solutions on how to efficiently compress mux channels but these do not need to be defined in the current spec. IMO a mux extension should offer its own, independent compression extension which could be more efficient with mux frames.

The "deflate-stream" should either be removed or adapted to be applied prior to masking to be good for general purpose transport compression. But in any case, since "deflate-stream" is optional it is not required to be ideal for mux frames compression.

With best regards,
Arman