Subject: Re: [hybi] Call for Consensus: draft-tyoshino-hybi-websocket-perframe-deflate as WG Item

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 1 Mar 2012 14:59:37 -0500

On Tue, Feb 28, 2012 at 09:05, John Tamplin <[email-masked]> wrote:
>
> I think it needs some significant changes (in particular, if it is going
> to allocate a reserved frame bit it needs to be extensible to other
> compression algorithms, which also implies renaming it to compress-frame or
> similar), but I think it is a fine starting point.
>

Do you think drastic abstraction is better? I.e. should we put texts
describing generic per-frame compression spec and texts describing how to
use it with DEFLATE into separate sections, or even into separate specs?

If it's sufficient to make it clear that the bit allocated in this spec is
also available for any other per-frame compression method, I'd replace
"COMP" bit in the spec with, say, "Per-frame compressed" bit, and change
the registration section to:

7.2.  Registration of the "Per-frame compressed" WebSocket Framing Header
Bit

   This section describes a WebSocket framing header bit registration in
   the WebSocket Framing Header Bits Registry.  [RFC6455]

   Header Bit
      RSV1

   Common Name
      Per-frame compressed

   Meaning
      Per-frame compression is applied to the frame or not.

   Reference
      Section 4 of this document.

   The "Per-frame compressed" framing header bit is used to indicate whether
   any negotiated per-frame compression extension is applied to the frame
or not.

   When this bit is used by this per-frame DEFLATE extension, it means
whether
   "Application data" part of the frame contains octets generated using
   DEFLATE or not.

   Any other compression extension MAY define
   its own use of this bit.  Note that this per-frame DEFLATE
   extension and that extension will be incompatible in such case.

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 1 Mar 2012 15:04:23 -0500

On Tue, Feb 28, 2012 at 18:10, Greg Wilkins <[email-masked]> wrote:

>
>
> +1 to have it adopted as a base line for further discussion and refinement.
>
> I share Johns concern about the allocation of a reserved bit to the
> extension and think that we should go through a process of exhaustively
> considering all the alternatives before allocation of such a scarce
> resource.
>

Yes.

I also wonder if it's useful to make it able to negotiate whether to use
compression bit or not on opening handshake.

---

From: John Tamplin <[email-masked]>
Date: Thu, 1 Mar 2012 17:25:49 -0500

> use it with DEFLATE into separate sections, or even into separate specs?

I think it is fine to have it in the same spec.  It can describe the
general use of per-frame compression, and then supply the specific case of
using deflate as the algorithm.  I don't think any more is required than
reordering pieces of the doc and some name changes, as well as adding some
way of selecting the compression algorithm (and supplying parameters to it
if necessary).

>    extension and that extension will be incompatible in such case.

I would prefer a bit more generic definition of the bit, and rather than
saying that this is the per-frame deflate extension it is a general-purpose
per-frame compression extension, and deflate is merely a required option
among potentially many different compression algorithms.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: John Tamplin <[email-masked]>
Date: Thu, 1 Mar 2012 17:26:33 -0500

> compression bit or not on opening handshake.

Personally, I think that complicates things too much.  When we reserved the
bit, one of the things we were reserving it for was per-frame compression.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 2 Mar 2012 11:11:23 +1100

On 2 March 2012 09:26, John Tamplin <[email-masked]> wrote:

> On Thu, Mar 1, 2012 at 3:04 PM, Takeshi Yoshino <[email-masked]>wrote:
>
> Personally, I think that complicates things too much.  When we reserved
> the bit, one of the things we were reserving it for was per-frame
> compression.

I agree - we should either use a reserve bit or not - optional use is
probably the worst of both worlds.

The options I see we have are:

a) Just use the bit in this extension.  Any other extension that uses the
same bit will be incompatible and we leave it undefined how we work that
out.
b) Nominate the bit as some kind of general purpose per frame compression
bit, available to multiple compression extensions.
c) Come up with a system of dynamically allocated reserved bits. The
extension requests to use a bit and the server allocates one and indicates
which in the handshake response.  This would allow multiple reserve bit
using extensions to co-exist... up until our limit of reserved bits.
d) Don't use a reserved bit.  Use the first byte of the payload to indicate
if the frame is compressed or not.
e) Don't use a reserved bit. Compress all frames.

I see c) as probably too complex and too limited to be worth the effort.
So failing dynamic reserve bit allocation, I think if we don't allocate a
reserved bit for frame compression, then we will never allocate them for
anything.
If we don't allocate a reserved bit, I think there is enough previous
discussions to show that e) compress everything is not optimal.   d) is not
too bad, specially if meanings could be assigned to other bits in that byte
(eg a reset dictionary bit, a use dictionary but don't update it bit etc.)

So I think b) or d) are viable options

cheers

---

From: Brian <[email-masked]>
Date: Thu, 1 Mar 2012 16:58:48 -0800

I like option "d."  One framing byte of overhead is practically nothing and
enables lots of opportunities for tuning and control.  If the byte of
overhead is too much for some people, perhaps there could be a
configuration parameter during the handshake to allow the user to opt into
across-the-board compression settings that would eliminate the byte and
just apply to every frame.  That would allow users with
consistently compressible data to eliminate the one byte overhead.

Alternatively, we could define a control frame to configure the compression
extension that could turn the per-frame control byte on or off.

Brian

On Thu, Mar 1, 2012 at 4:11 PM, Greg Wilkins <[email-masked]> wrote:

> d) Don't use a reserved bit.  Use the first byte of the payload to
> indicate if the frame is compressed or not.
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 1 Mar 2012 20:10:38 -0500

On Thu, Mar 1, 2012 at 7:58 PM, Brian <[email-masked]> wrote:

> I like option "d."  One framing byte of overhead is practically nothing
> and enables lots of opportunities for tuning and control.  If the byte of
> Alternatively, we could define a control frame to configure the
> compression extension that could turn the per-frame control byte on or off.
>

The options of knowing what to compress are:
1) compress everything on a connection, configured at handshake time.  We
know compressing incompressible data is
a loss, so we would need to expose a way in the JS API to indicate whether
to use compression or not.  Applications that mix compressible and
incompressible data would use multiple connections (made less costly via
multiplexing).

2) compress only compressible data in a frame, leaving incompressible data
in uncompressed frames.  The choice of how to determine compressibility:
  2a) the API says whether it is compressible
  2b) the API gives a type of data being supplied, and an implementation
chooses whether to compress it based on the type
  2c) heuristic choices, such as "compress all text frames, leave binary
frames uncompressed" (though earlier experiments with GwtQuake over
WebSockets showed that even its binary data substantially benefited from
compression)
  2d) try and compress the data, and send it compressed only if it reduces
the size

Another question is about sharing compression state -- if we compress each
frame individually, we lose the benefit of redundancy with other frames.
 For example, the WS binary frames in GwtQuake saw little improvement at
all if each frame was compressed individually, but there was a lot of
redundancy across frames.  To take advantage of this, we need a compression
scheme that maintains a compression dictionary for the stream (which might
complicated 2d above depending on the compression library available).

As Greg said, if we aren't going to use a reserved bit for per-frame
compression, where we hope to reduce the payload to a small number of bytes
already so an extra byte to indicate compression may be expensive, what
exactly will we use it for?

My opinion is we should allocate a reserved bit for per-frame compression,
create an extension to indicate that is to be used for compression and a
way to select the compression algorithm, and define deflate with no state
shared across frames as a required compression algorithm to be supported if
no other algorithm is negotiated.  We can then define more effective
compression algorithms that provide ways of sharing compression state
across frames, for example, though we still have to work out the above
questions though.

---

From: Brian <[email-masked]>
Date: Thu, 1 Mar 2012 17:19:43 -0800

> across frames, for example, though we still have to work out the above

That's true enough.  If ever there were a perfect use for the reserved bits
it would be this.  It's just such a rare resource and it's still early days
for WebSockets in general that the gut reaction is to keep them reserved at
all costs until some mythical extension suddenly jumps out as worthy enough
to have one of the sacred bits ceremonially bestowed upon it from on high.

I'm good with a reserved bit being allocated for per frame compression.

Brian

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 02 Mar 2012 17:25:45 +0000

On 28/02/2012 14:05, John Tamplin wrote:
> On Tue, Feb 28, 2012 at 2:42 AM, Salvatore Loreto 
> <[email-masked] <mailto:[email-masked]>> 
> wrote:
>
>     the first optimization/extension (or milestone if you prefer)
>     in the new approved HyBi charter (see
>     http://tools.ietf.org/wg/hybi/charters )
>     is about:
>
>         A per-frame compression extension to improve bandwidth
>         usage
>
>
>     Takeshi Yoshino has been working on a draft proposal since April 2011.
>     The draft has been discussed extensively in the mailing list,
>     it  has already received comments, suggestions and feedback;
>     and  Takeshi has produced several new versions to address the
>     raised issues.
>
>     The current version is:
>     http://tools.ietf.org/html/draft-tyoshino-hybi-websocket-perframe-deflate-05
>
>
>     Gabriel and I, as chairs, want to check the consensus to adopt the
>     draft as
>     baseline for the working group item that will fulfill the first
>     Goal in the current charter:
>
>        Feb 2012 - Adopt a WG item for the per-frame compression extension
>
> I think it needs some significant changes (in particular, if it is 
> going to allocate a reserved frame bit it needs to be extensible to 
> other compression algorithms, which also implies renaming it to 
> compress-frame or similar), but I think it is a fine starting point.
+1.

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 02 Mar 2012 10:34:43 -0800

+1

On 2/27/2012 11:42 PM, Salvatore Loreto wrote:
> baseline for the working group item that will fulfill the first Goal 
> in the current charter:

---

From: "Arman Djusupov" <[email-masked]>
Date: Tue, 6 Mar 2012 09:49:16 +0200

I also agree on this. Creating an extension that would allow the use of various compression algorithms and reserving this bit for per-frame compression would be wise. In general, frame compression is a useful feature of the transport that deserves reserving one bit for it.

With best regards,

Arman

From: [email-masked] [mailto:[email-masked]] On Behalf Of John Tamplin
Sent: Friday, March 02, 2012 3:11 AM
To: Brian
Cc: [email-masked]
Subject: Re: [hybi] Call for Consensus: draft-tyoshino-hybi-websocket-perframe-deflate as WG Item

…

My opinion is we should allocate a reserved bit for per-frame compression, create an extension to indicate that is to be used for compression and a way to select the compression algorithm, and define deflate with no state shared across frames as a required compression algorithm to be supported if no other algorithm is negotiated.  We can then define more effective compression algorithms that provide ways of sharing compression state across frames, for example, though we still have to work out the above questions though.

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 13 Mar 2012 06:53:00 +0200

Hi there,

this call is now over.
Thanks to all of you for the feedback and the good discussion on the topic.

Based on the fact that nobody objected,
Gabriel and I, as chairs, declare consensus on the adoption of the
draft-tyoshino-hybi-websocket-perframe-deflate as wg item.

cheers
Salvatore

-- 
Salvatore Loreto, PhD
www.sloreto.com

On 2/28/12 9:42 AM, Salvatore Loreto wrote:
> Hi there,
> and  Takeshi has produced several new versions to address the raised 
> issues.
>
> The current version is:
> This consensus call will run until Monday March 12th, 2012.
>
>
> Ciao
> Salvatore and Gabriel
> -- 
> Salvatore Loreto
> www.sloreto.com
>