Subject: [hybi] permessage-deflate: add a few more parameters to avoid processing non-compressible contents?

From: Takeshi Yoshino <[email-masked]>
Date: Mon, 27 Oct 2014 22:02:08 +0900

(Branched from the thread "Closing the hybi working group" [1])

Joakim suggested that we should add a few more parameters to
permessage-deflate in this thread [1]. Here's his proposal revised by me.
Everyone please share your thoughts about this proposal by replying to this
thread.

----

* server_skip_binary = 1*DIGIT

If the "agreed parameters" contain the "server_skip_binary" extension
parameter, the server MUST NOT compress binary messages with original size
smaller than the size specified by the parameter value.

* server_skip_text = 1*DIGIT

If the "agreed parameters" contain the "server_skip_text" extension
parameter, the server MUST NOT compress text messages with original size
smaller than the size specified by the parameter value.

* client_skip_binary = 1*DIGIT

If the "agreed parameters" contain the "client_skip_binary" extension
parameter, the client MUST NOT compress binary messages with original size
smaller than the size specified by the parameter value.

* client_skip_text = 1*DIGIT

If the "agreed parameters" contain the "client_skip_text" extension
parameter, the client MUST NOT compress text messages with original size
smaller than the size specified by the parameter value.

----

Any suggestion about max value of the parameters? Max of uint32? uint16
would suffice?

[1] http://www.ietf.org/mail-archive/web/hybi/current/msg10518.html

---

From: Peter Thorson <[email-masked]>
Date: Mon, 27 Oct 2014 17:02:39 -0400

I am curious what the use case for having these options baked in at the WebSocket protocol level is? This seems like the wrong level of the stack to be addressing the question “will this message benefit from compression”.. Would this not be better done at the application level where you have more information about the actual messages?

> On Oct 27, 2014, at 09:02 , Takeshi Yoshino <[email-masked]> wrote:
> 
> Joakim suggested that we should add a few more parameters to permessage-deflate in this thread [1]. Here's his proposal revised by me. Everyone please share your thoughts about this proposal by replying to this thread.
> 
> ----
> 
> If the "agreed parameters" contain the "server_skip_binary" extension parameter, the server MUST NOT compress binary messages with original size smaller than the size specified by the parameter value.
> 
> If the "agreed parameters" contain the "server_skip_text" extension parameter, the server MUST NOT compress text messages with original size smaller than the size specified by the parameter value.
> 
> If the "agreed parameters" contain the "client_skip_binary" extension parameter, the client MUST NOT compress binary messages with original size smaller than the size specified by the parameter value.
> 
> If the "agreed parameters" contain the "client_skip_text" extension parameter, the client MUST NOT compress text messages with original size smaller than the size specified by the parameter value.
> 
> ----
> 
> Any suggestion about max value of the parameters? Max of uint32? uint16 would suffice?
> 
> 
> 
> [1] http://www.ietf.org/mail-archive/web/hybi/current/msg10518.html <http://www.ietf.org/mail-archive/web/hybi/current/msg10518.html>_______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 28 Oct 2014 17:11:11 +0900

Peter,

I introduced extension to the WHATWG WebSocket API to address the needs at
API level [1].

Joakim provided some justification for introduction of the new parameters
at protocol level in [2] and the last paragraph of [3].

Joakim, could you please tell us your thoughts about whether API level
support in the future suffices or not?

[1] http://www.ietf.org/mail-archive/web/hybi/current/msg10540.html
[2] http://www.ietf.org/mail-archive/web/hybi/current/msg10531.html
[3] http://www.ietf.org/mail-archive/web/hybi/current/msg10534.html

On Tue, Oct 28, 2014 at 6:02 AM, Peter Thorson <[email-masked]>
wrote:

> I am curious what the use case for having these options baked in at the
> WebSocket protocol level is? This seems like the wrong level of the stack
> to be addressing the question “will this message benefit from
> compression”.. Would this not be better done at the application level where
> you have more information about the actual messages?
>
>  _______________________________________________
> hybi mailing list
> [email-masked]

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 28 Oct 2014 05:36:47 -0700

> Joakim suggested that we should add a few more parameters to permessage-deflate in this thread [1]. Here's his proposal revised by me. Everyone please share your thoughts about this proposal by rep…

My view: adding more parameters to the extension negotiation to prohibit compression of messages that would not profit from compression .. based simply on message size .. mmh, not really convincing. Compression can be quite effective even for very small messages when compression context takeover is active.

On that other hand, apps naturally might have knowledge about which message is compressible or not. IOW: extend APIs for allowing to prohibit compression when sending a message.

Instead of more extension parameters, I'd rather have a browser API like

[Constructor(DOMString url, optional (DOMString or DOMString[]) protocols)]
interface WebSocket : EventTarget {
  void send(DOMString data, boolean doNotCompress = false);
  ...
};

and control compression of individual messages from within the app.

Cheers,
/Tobias

---

From: Peter Thorson <[email-masked]>
Date: Tue, 28 Oct 2014 09:37:14 -0400

The proposals defined in [1] for the javascript api seem sufficient to me to allow flexibility in not compressing incompressible things. My own WebSocket library that implements permessage-deflate has a similar API for application level code to specify a compression strategy more complicated than all/none. If clients and servers need to coordinate between each other they can do so offline or via their own application level messages.

The main use case I see described below for compression strategy as a protocol level parameter would be the case of a server that wants to force a crude compression strategy on a client that was not yet updated to use the compression api or for some other reason declined to use it. This doesn’t seem compelling enough to me to justify an entirely new subsystem to negotiate and manage compression strategies at the protocol level.

> On Oct 28, 2014, at 04:11 , Takeshi Yoshino <[email-masked]> wrote:
> 
> Peter,
> 
> I introduced extension to the WHATWG WebSocket API to address the needs at API level [1].
> 
> Joakim provided some justification for introduction of the new parameters at protocol level in [2] and the last paragraph of [3].
> 
> Joakim, could you please tell us your thoughts about whether API level support in the future suffices or not?
> 
> 
> [1] http://www.ietf.org/mail-archive/web/hybi/current/msg10540.html <http://www.ietf.org/mail-archive/web/hybi/current/msg10540.html>
> [2] http://www.ietf.org/mail-archive/web/hybi/current/msg10531.html <http://www.ietf.org/mail-archive/web/hybi/current/msg10531.html>
> [3] http://www.ietf.org/mail-archive/web/hybi/current/msg10534.html <http://www.ietf.org/mail-archive/web/hybi/current/msg10534.html>
> 
> 
> On Tue, Oct 28, 2014 at 6:02 AM, Peter Thorson <[email-masked] <mailto:[email-masked]>> wrote:
>> On Oct 27, 2014, at 09:02 , Takeshi Yoshino <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/hybi <https://www.ietf.org/mailman/listinfo/hybi>
> 
>

---

From: Adam Rice <[email-masked]>
Date: Fri, 31 Oct 2014 12:48:19 +0900

I think Joakim's suggestion seems like a way to tune the heuristics used by
the browser to decide whether to compress a particular message. There are a
few issues with this approach I see:

* This constrains the design space of heuristics that a browser may use.
Even if a browser has implemented some amazing machine-learning algorithm
for deciding whether or not to compress a message, it still has to
implement message-size heuristics to implement these parameters.
* It doesn't make sense to me for the browser to be telling the server what
heuristics to use to decide whether or not to compress messages. The
browser has no application-specific knowledge, whereas the server
presumably knows what the application is and when compression is
appropriate.
* The server may not know the size of a message when it starts to send it.
The browser currently always knows, but future developments (I'm
particularly thinking of integration with Streams) could change that.

Personally, I feel the ratio between the number of permessage-deflate
parameters and the number of permessage-deflate implementations is
currently way too high. I don't want to do anything that further raises the
barrier to implementation.

On 28 October 2014 22:37, Peter Thorson <[email-masked]> wrote:

> The proposals defined in [1] for the javascript api seem sufficient to me
> to allow flexibility in not compressing incompressible things. My own
> WebSocket library that implements permessage-deflate has a similar API for
> application level code to specify a compression strategy more complicated
> than all/none. If clients and servers need to coordinate between each other
> they can do so offline or via their own application level messages.
>
> The main use case I see described below for compression strategy as a
> protocol level parameter would be the case of a server that wants to force
> a crude compression strategy on a client that was not yet updated to use
> the compression api or for some other reason declined to use it. This
> doesn’t seem compelling enough to me to justify an entirely new subsystem
> to negotiate and manage compression strategies at the protocol level.
>