Subject: Re: [hybi] Payload only compression extension, again

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 27 Apr 2011 13:54:06 +0900

Hi,

As the core spec is going to include only deflate-stream (make sure the core
spec includes built-in compression, and also have some extension example),
I've put together the per-frame compression idea we've been discussing into
a separate I-D.
http://www.ietf.org/id/draft-tyoshino-hybi-websocket-perframe-deflate-00.txt

<http://www.ietf.org/id/draft-tyoshino-hybi-websocket-perframe-deflate-00.txt>
Thanks,
Takeshi

On Thu, Mar 10, 2011 at 05:36, Greg Wilkins <[email-masked]> wrote:

> On 9 March 2011 23:17, Takeshi Yoshino <[email-masked]> wrote:
> > Thanks for feedback.
> > On Tue, Mar 1, 2011 at 16:01, Greg Wilkins <[email-masked]> wrote:
> >>
> >> On 1 March 2011 01:51, Takeshi Yoshino <[email-masked]> wrote:
> >> > Hi,
> >
> >> > - Want not to mess up dictionary by compressing incompressible
> messages
> >>
> >> Is that to say that you don't want to compress fragments?
> >> I'm cautious about this as we don't want to have to buffer up all
> >> fragments before we can decompress.
> >>
> >
> > Sorry, could you elaborate your question?
> > I put this in the requirement since IIRC some people said that they want
> not
> > to get some incompressible contents (mixed with compressible contents in
> a
> > single connection) through the compressor. Like this.
> > | WS header | BTYPE=01 <compressible contents> | WS header | BTYPE=00
> > <incompressible contents> | ...
>
> thanks - that answers my question fine.
>
> cheers
>

---

From: Brodie Thiesfield <[email-masked]>
Date: Thu, 28 Apr 2011 09:18:08 +0900

> http://www.ietf.org/id/draft-tyoshino-hybi-websocket-perframe-deflate-00.txt

Since you have spent the effort to create this spec, which is much
better than the current deflate-stream method that we have at the
moment, wouldn't it be worthwhile to also take over one of the
reserved bits and use it to specify if the frame is compressed or not?

That way we get the data only compression with an uncompressed data
option. Pretty much ticks all of the boxes.

Regards,
Brodie

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: John Tamplin <[email-masked]>
Date: Wed, 27 Apr 2011 21:22:37 -0400

> option. Pretty much ticks all of the boxes.

This was exactly what was proposed a year ago when we were talking about
framing, and we couldn't come to consensus on exactly how it would work
then.  I am not sure there is any reason to believe it is more likely to
achieve consensus now.  Among other considerations, you have to define how
an implementation can choose when to send a compressed frame versus an
uncompressed frame.  Do you allow/require the sender to try and compress it
first and then send it uncompressed if that is smaller?  Doing so might
impose unwelcome headaches for some implementations if they can't easily
save the compression state and revert to a previous checkpoint.

My gut feeling is at this point, it is better to get the base spec released,
experiment with compression options in private-use extensions, and then have
a formal proposal for a real compression extension once we get that
experience.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Brodie Thiesfield <[email-masked]>
Date: Thu, 28 Apr 2011 10:38:28 +0900

> uncompressed frame.  Do you allow/require the sender to try and compress it

No, this does not have to be defined. Only the ability to send
uncompressed frames needs to be defined. It is completely up to the
implementation whether it makes use of that ability, and if it does,
then how. The protocol just needs to enable it.

In any case, at the present time, this proposal is an extension and
not part of the base spec. It therefore has no affect on the release
of the base spec.

On an unrelated point, I would love to search for these discussions in
the archive, but I haven't found a way to do so. Do you know if there
is a searchable archive of the hybi mail list? It's not linked to from
the main list archive info.

Regards,
Brodie

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 28 Apr 2011 11:42:43 +1000

> not part of the base spec. It therefore has no affect on the release

Well currently there is a proposal to replace the deflate-stream
proposal with a deflate-frame.  So it could effect the release of the
base spec.... but I think only in a positive way.

I agree that we do not need to specify how the extension decides what
frames to compress - only that it has the ability to decide and to
indicate the result in the reserved bit.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 28 Apr 2011 11:49:42 +1000

> a formal proposal for a real compression extension once we get that

Can we apply that logic to deflate-stream?

It should be punted to a private-use extension until the many
identified problems with it are solved.

---

From: John Tamplin <[email-masked]>
Date: Wed, 27 Apr 2011 22:03:36 -0400

> identified problems with it are solved.

It was added as a better-than-nothing example of an extension, but as we
have added masking since then I would not be opposed to removing it.

---

From: John Tamplin <[email-masked]>
Date: Wed, 27 Apr 2011 22:15:45 -0400

> indicate the result in the reserved bit.

You do have to specify whether or not uncompressed frames affect the
compression state of the connection.  I think you also have to specify
things like window size/etc (see
https://groups.google.com/forum/#!topic/spdy-dev/369JhUL6x5I for a
discussion of the implications of having lots of compressed connections on
the server).  I think trying to squeeze in getting consensus on such
definitions just before last call is unlikely, and rushing through something
ill-considered is only likely to define an extension that will never
actually be used.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 27 Apr 2011 22:20:45 -0400

compression is a proven win on the web. I don't want to support a 
protocol without it defined and the web has waited way too long for 
websockets already, so I'm not willing to support a delay in the spec 
over refining it when there is an extension mechanism defined for 
getting experience with and deploying other approaches.

individual messages will still compress fine with stream compression 
even in the presence of the rotating mask (though I support the per 
payload compression draft moving forward independently) - the rotating 
mask just damages the backreferences between different messages. This is 
effectively the same level of impact content-encodings give you in http, 
which is significant for messages of reasonable size. That isn't the 
only use case for websockets, but it is a meaningful one.

Furthermore the mask is unidirectional making the downstream data flow 
unimpacted.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 27 Apr 2011 22:40:50 -0400

compression is a proven win on the web. I don't want to support a 
protocol without it defined. The web has waited way too long for 
websockets already, so I'm not willing to support a delay in the spec 
over refining it when there is an extension mechanism defined for 
getting experience with and deploying other approaches as needed.

Furthermore, the mask is unidirectional making the downstream data flow 
unimpacted by masking.