Subject: [hybi] Closing the hybi working group

From: Barry Leiba <[email-masked]>
Date: Fri, 3 Oct 2014 14:15:12 -0400

It's vey clear that HyBi is no longer a working group, and hasn't been
for some time.  Takeshi last updated the compression doc in May, and
there were no responses.  In general, I have seen no discussion at all
about the document -- no messages other than from Takeshi, except for
a good review by Joakim Erdfelt in October, and a discussion a year
ago June, though there continue to be regular updates to the document
-- nine versions since that June 2013 set of messages.

So, again: HyBi is no longer a working group -- there are no participants.  And
the compression doc is not a working group document, it's Takeshi's
alone, as far as I can tell.  No one seems to be clamouring for this,
no one seems eager to implement it.  And there's no way we can really
say that there's a meaningful version of working group consensus
behind what's there now.

I intend to close the HyBi working group and abandon the compression
document.  I intend to do this on 24 Oct.  If anyone would like to
discuss this before it happens, feel free to do that in response to
this message.

Barry, Applications AD

---

From: Kevin <[email-masked]>
Date: Fri, 03 Oct 2014 14:36:12 -0400

On 10/3/2014 2:15 PM, Barry Leiba wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
I second that.  I see know reason for a group to be open if nobody 
contributes.

-- 
Kevin

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 3 Oct 2014 11:36:34 -0700

I'm not sure how the process works, but would this group, or ietf-http-wg
be the place for discussing websocket over http/2? (heck, is there overlap
in the groups now?)

--
Joakim Erdfelt <[email-masked]>
webtide.com <http://www.webtide.com/> - intalio.com/jetty
Expert advice, services and support from from the Jetty & CometD experts
eclipse.org/jetty - cometd.org

---

From: Wenbo Zhu <[email-masked]>
Date: Fri, 3 Oct 2014 11:40:54 -0700

IMO:

if the interest is in the WS API, it goes to W3C etc
If the interest is in preserving the wire compatibility with rfc-6455, then
the discuss probably should stay here (there is nothing wrong to have an
inactive WG if everything is sufficient)
if the interest is in layering WS API or whatever on top of HTTP/2, then
http-wg may be a better place.

---

From: Tobias Oberstein <[email-masked]>
Date: Sat, 4 Oct 2014 03:51:07 -0700

> It's vey clear that HyBi is no longer a working group, and hasn't been for some
> time.  Takeshi last updated the compression doc in May, and there were no
> responses.  In general, I have seen no discussion at all about the document --
> no messages other than from Takeshi, except for a good review by Joakim
> Erdfelt in October, and a discussion a year ago June, though there continue
> to be regular updates to the document
> the compression doc is not a working group document, it's Takeshi's alone, as
> far as I can tell.  No one seems to be clamouring for this, no one seems eager
> to implement it.  And there's no way we can really say that there's a
> meaningful version of working group consensus behind what's there now.
> 
> document.  I intend to do this on 24 Oct.  If anyone would like to discuss this
> before it happens, feel free to do that in response to this message.

Hello Barry,

Actually, permessage-deflate is already implemented and deployed by different implementors - and others are just starting.

The AutobahnTestsuite (a WebSocket protocol testsuite) has a whole section that tests permessage-deflate extensions, e.g. section 12 here:

http://autobahn.ws/testsuite/reports_20140314/clients/index.html

What is missing to move the document ("permessage-deflate") to a proper RFC?

===

Then, I am planning to take up work again on

https://github.com/oberstet/permessage-priority

which is another WebSocket extension.

Yet another area is WebSocket subprotocols: e.g. for WAMP (http://wamp.ws/), which is an open WebSocket subprotocol
with multiple, interoperable implementations. I am planning to rewrite the WAMP spec into a RFC.

In short: if this group is closed, where should we discuss new WebSocket extensions and subprotocols?
And where should we discuss future revisions of WebSocket itself?

Doing all this on the HTTP2 list seems suboptimal for various reasons.

Probably we should adjust the group charter, but please keep this list open ...

Cheers,
/Tobias

---

From: Tobias Oberstein <[email-masked]>
Date: Sat, 4 Oct 2014 03:56:34 -0700

Hi Wenbo,

yes, I agree.

The WS API has always been living on W3C anyway. The "WS/whatever" layering on top of HTTP/2 has been discussed here, but moved to http-wg as far as I know. Don't know if that has lead to anything concrete, but it's not happening here anymore.

But rgd WebSocket, there are multiple areas that "need a home":

1. possible clarifications / revisions of RFC6455 itself
2. WebSocket extensions
3. WebSocket subprotocols

If this group is closed, where should discussions related to 1. - 3. happen?

Cheers,
/Tobias

Von: hybi [mailto:[email-masked]] Im Auftrag von Wenbo Zhu
Gesendet: Freitag, 3. Oktober 2014 20:41
An: Joakim Erdfelt
Cc: [email-masked]; Barry Leiba
Betreff: Re: [hybi] Closing the hybi working group

IMO:

if the interest is in the WS API, it goes to W3C etc
If the interest is in preserving the wire compatibility with rfc-6455, then the discuss probably should stay here (there is nothing wrong to have an inactive WG if everything is sufficient)
if the interest is in layering WS API or whatever on top of HTTP/2, then http-wg may be a better place.

On Fri, Oct 3, 2014 at 11:36 AM, Joakim Erdfelt <[email-masked]<mailto:[email-masked]>> wrote:
I'm not sure how the process works, but would this group, or ietf-http-wg be the place for discussing websocket over http/2? (heck, is there overlap in the groups now?)

--
Joakim Erdfelt <[email-masked]<mailto:[email-masked]>>
webtide.com<http://www.webtide.com/> - intalio.com/jetty<http://intalio.com/jetty>
Expert advice, services and support from from the Jetty & CometD experts
eclipse.org/jetty<http://eclipse.org/jetty/> - cometd.org<http://cometd.org/>

On Fri, Oct 3, 2014 at 11:15 AM, Barry Leiba <[email-masked]<mailto:[email-masked]>> wrote:

Barry, Applications AD

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

---

From: Alexey Melnikov <[email-masked]>
Date: Sat, 04 Oct 2014 16:43:21 +0100

Hi,

On 04/10/2014 11:56, Tobias Oberstein wrote:
>
> Hi Wenbo,
>
> yes, I agree.
>
> The WS API has always been living on W3C anyway. The "WS/whatever" 
> layering on top of HTTP/2 has been discussed here, but moved to 
> http-wg as far as I know. Don't know if that has lead to anything 
> concrete, but it's not happening here anymore.
>
I suspect the mailing list is going to remain open(*), so discussions 
can continue here.

(*) - at least this is true for all other mailing lists for closed IETF WGs.
>
> Cheers,
>
> /Tobias
>
> *Von:*hybi [mailto:[email-masked]] *Im Auftrag von *Wenbo Zhu
> *Gesendet:* Freitag, 3. Oktober 2014 20:41
> *An:* Joakim Erdfelt
> *Cc:* [email-masked]; Barry Leiba
> *Betreff:* Re: [hybi] Closing the hybi working group
>
> IMO:
>
> If the interest is in preserving the wire compatibility with rfc-6455, 
> then the discuss probably should stay here (there is nothing wrong to 
> have an inactive WG if everything is sufficient)
>
>     I'm not sure how the process works, but would this group, or
>     ietf-http-wg be the place for discussing websocket over http/2?
>     (heck, is there overlap in the groups now?)
>
>
>     --
>
>     Joakim Erdfelt <[email-masked] <mailto:[email-masked]>>
>
>     <http://intalio.com/jetty>
>
>     eclipse.org/jetty <http://eclipse.org/jetty/> - cometd.org
>     <http://cometd.org/>
>
>     On Fri, Oct 3, 2014 at 11:15 AM, Barry Leiba
>     <[email-masked] <mailto:[email-masked]>> wrote:
>

---

From: "Arman Djusupov" <[email-masked]>
Date: Mon, 6 Oct 2014 13:52:24 +0300

Hello Barry,

The permessage-compression draft is supported by several implementations
including our own PowerWebSockets product. It is deployed in production use,
interop is there, no problems showed up to discuss. There is no discussion
about the draft mostly because we consider that the work on the spec has
been concluded and the current draft is the final.

With best regards,
Arman

-----Original Message-----
From: hybi [mailto:[email-masked]] On Behalf Of Barry Leiba
Sent: Friday, October 3, 2014 9:15 PM
To: [email-masked]
Subject: [hybi] Closing the hybi working group

It's vey clear that HyBi is no longer a working group, and hasn't been for
some time.  Takeshi last updated the compression doc in May, and there were
no responses.  In general, I have seen no discussion at all about the
document -- no messages other than from Takeshi, except for a good review by
Joakim Erdfelt in October, and a discussion a year ago June, though there
continue to be regular updates to the document
-- nine versions since that June 2013 set of messages.

So, again: HyBi is no longer a working group -- there are no participants.
And the compression doc is not a working group document, it's Takeshi's
alone, as far as I can tell.  No one seems to be clamouring for this, no one
seems eager to implement it.  And there's no way we can really say that
there's a meaningful version of working group consensus behind what's there
now.

I intend to close the HyBi working group and abandon the compression
document.  I intend to do this on 24 Oct.  If anyone would like to discuss
this before it happens, feel free to do that in response to this message.

Barry, Applications AD

---

From: "Alexander Philippou" <[email-masked]>
Date: Mon, 6 Oct 2014 14:18:35 +0300

+1, fully agree with Tobias.

Compression was discussed extensively in 2010/2011/2013. By mid-2013 the spec was practically done. Any changes since then have been mostly of the "cosmetic" type. Takeshi tried to accommodate all feedback and thus published several IDs even a year after the discussion between him and most major contributors had been happily concluded. The fact that most of us did not comment on the latest ID should by no means be taken as an indication of indifference or absence of implementation. On the contrary, we were quiet because we were afraid that commenting in any way could fuel further discussions that might go on to eternity. So what was perceived as silence was actually eagerness to see the spec become an RFC. WS is an RFC, WS needs compression, please let's have the RFC for WS compression too.

BTW, *thank you* Takeshi for all the work you've done, the spec has been concluded well, we appreciate your effort.

Alexander

> From: hybi [mailto:[email-masked]] On Behalf Of Tobias Oberstein
> Sent: Saturday 4 October 2014 13:51
> To: Barry Leiba; [email-masked]
> Subject: Re: [hybi] Closing the hybi working group
> 
> some
> And
> this
> Actually, permessage-deflate is already implemented and deployed by
> different implementors - and others are just starting.
> 
> The AutobahnTestsuite (a WebSocket protocol testsuite) has a whole section
> that tests permessage-deflate extensions, e.g. section 12 here:
> 
> Yet another area is WebSocket subprotocols: e.g. for WAMP
> (http://wamp.ws/), which is an open WebSocket subprotocol

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 7 Oct 2014 08:33:36 +0900

Thanks for the comments about the permessage-deflate. I also think the I-D
is ready for resuming the process and it's still worth putting it into an
RFC.

We shipped it on Chrome by default from Chrome 32 released in Nov 2013.
Ilya published an article introducing the extension:
https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/.
Our user metrics analysis has been showing gradual increase of usage count
indicating more and more adoption.

Searching on the web for permessage-deflate or searching on GitHub also
show me implementations of it. In addition to ones already listed in this
thread, e.g. Tomcat, Crossbar.io, Mojolicious. Jetty has but not turned on
by default?

----

One topic I'm still interested in but has been away from is standardization
of liveness-detection (or keep-alive) API. As Wenbo suggested, it could be
done at W3C/WHATWG unless we need any extension to the ping/pong stuff.

Takeshi

On Mon, Oct 6, 2014 at 8:18 PM, Alexander Philippou <[email-masked]> wrote:

> Compression was discussed extensively in 2010/2011/2013. By mid-2013 the
> spec was practically done. Any changes since then have been mostly of the
> "cosmetic" type. Takeshi tried to accommodate all feedback and thus
> published several IDs even a year after the discussion between him and most
> major contributors had been happily concluded. The fact that most of us did
> not comment on the latest ID should by no means be taken as an indication
> of indifference or absence of implementation. On the contrary, we were
> quiet because we were afraid that commenting in any way could fuel further
> discussions that might go on to eternity. So what was perceived as silence
> was actually eagerness to see the spec become an RFC. WS is an RFC, WS
> needs compression, please let's have the RFC for WS compression too.
>
> BTW, *thank you* Takeshi for all the work you've done, the spec has been
> concluded well, we appreciate your effort.
>
> Alexander
>
> for
> > > responses.  In general, I have seen no discussion at all about the
> document
> > --
> alone,
> > as
> > > far as I can tell.  No one seems to be clamouring for this, no one
> seems
> > eager
> discuss
> > with multiple, interoperable implementations. I am planning to rewrite
> the
> > WAMP spec into a RFC.
> >

---

From: Joakim Erdfelt <[email-masked]>
Date: Mon, 6 Oct 2014 17:17:51 -0700

Jetty has an implementation, but has chosen to leave it as disabled, but
configurable for any endpoint the developer wants it on for.

We had it enabled, by default for many versions of Jetty, But we've just
experienced too many warts with the various implementations to leave it on
for serious work, just too many different interpretations of the spec for
it to be practical. (these issues have been brought up before here)
We pass most of the autobahn tests just fine, just not the ones related to
window bits. (also been discussed before on this mailing list).

We only really recommend it for environments where the endpoints are
tightly controlled (such as mobile apps to server).

We continue to evaluate it periodically, and perform large scale
experiments with groups that heavily use websocket in production, as of mid
July it still not ready to be turned on permanently.

The most common feedback surrounding permessage-deflate:

1) lack of ability to manage / configure extensions and their parameters
from the javascript websocket client api
2) lack of extension parameter to specify a minimum frame (or message) size
to compress
3) lack of features in websocket api to limit (from the sender) which
messages should get compressed or not

On Mon, Oct 6, 2014 at 4:33 PM, Takeshi Yoshino <[email-masked]> wrote:

> One topic I'm still interested in but has been away from is
> standardization of liveness-detection (or keep-alive) API. As Wenbo
> suggested, it could be done at W3C/WHATWG unless we need any extension to
> the ping/pong stuff.
>
> Takeshi
>

---

From: Barry Leiba <[email-masked]>
Date: Tue, 7 Oct 2014 13:01:47 -0400

I'm pleased with the responses here, which seem to be saying that
there is interest in the compression doc, and that it *is* ready for
publication request -- the last time it came to me, I found it very
hard to read and needing a lot of editing.  If we think it's ready
this time, let's get it out -- far better to close the working group
*after* finishing the document, rather than without it.

Barry, still hopeful...

---

From: Patrick McManus <[email-masked]>
Date: Tue, 7 Oct 2014 16:47:17 -0400

fwiw per-mesage deflate is on the firefox roadmap.. we haven't spent any
cycles really looking at the doc, but the presence of the draft in the
standards pipeline has a lot to do with us pursuing it..

---

From: Joakim Erdfelt <[email-masked]>
Date: Wed, 8 Oct 2014 06:07:45 -0700

Arman,

In experimentation with real world products, this is what we've found with
permessage-deflate on the server side. (excluding implementation
differences)

In regards to memory:
Memory use: any use of permessage-deflate is expensive, with or without
context takeover.  (this is mainly due to the nature of decompress, where
we have to work with the received network buffers, the deflate context
buffers, and the uncompressed buffers).  (note: the memory differences
between context takeover choices is so negligible that it is actually
difficult to measure in real world applications)
Even so, a small server, on modest hardware (think raspberry pi) can handle
20k websocket connections all with permessage-deflate negotiated easily.
The fastest way to impact performance on this system is rate of messages,
using lots of smaller messages (sub 2k decompressed in size).  If we test
it differently with less rate, but larger message sizes (in the 3k to 100k
decompressed range) then we can saturate the network and not see a server
issue (testing switches from becoming memory limited to being I/O limited)

In regards to "is compressible":
the permessage-deflate existence, and design, along with the lack of
control over the compression from the many websocket apis make this a red
flag for many deployments.  This is related to the general need for control
over if compression should be applied on a per-message basis.  With
permessage-deflate its not "is this compressible?" which is important, its
more of a question of "is it worth compressing this message?" as many
websocket messages are small enough that the the costs of the compression
in regards to latency and memory are not worth the impact it has on the
system.
This means we should expand our discussion on this subject from types of
data (eg: text vs image data) to a larger scope including "is the hit on
performance worth compressing this small text data?", this should really be
exposed to the developer in the websocket apis, either as a controllable
extension parameter to upgrade with, or as a configurable on the api send
behavior ( api.setMinCompressSize(bytes) ), or as optional advanced
configuration on each send method ( api.send(msg,compress=false) )

A simple minimum message size before compression goes a long way for these
concerns, as it basically compresses larger messages that would have taken
a latency hit anyway.

On Wed, Oct 8, 2014 at 2:14 AM, Arman Djusupov <[email-masked]> wrote:

> Hello Joakim,
>
>
>
> In general I think it is wise to keep permessage-compression disabled by
> default on the server side. Compression costs memory, it is up to the
> developers and administrators to decide whether a given endpoint should
> have compression enabled based on the estimated number of concurrent
> clients.
>
>
>
> 1)      As far as I understand this forces the client side to use the default
> configuration in most of the cases? IMO the server must be ready to
> impose a constrained compression configuration on the client and the clients
> must be ready to accept the constrained configuration from the server, since
> in most cases the server is more resource constrained than the clients.
> This means that browser clients (since they represent millions of users)
> must always be ready to accept client_no_context_takeover,
> server_no_context_takeover, and server_max_window_bits parameters from
> the server. Otherwise it will be impossible for the server to accept the permessage-compression
> extension in some cases, because the default configuration costs a
> significant amount of memory which limits the maximum number of client
> connections that the server can handle concurrently.
>
>
>
> One of the solutions to this problem would be to change the default
> configuration to make context-takeover disabled by default, so
> client_context_takeover and server_context_takeover would be optional
> parameters. Not having to keep client and server context in memory in the default
> configuration would make it much easier for the server to accept a large
> number of client connections.
>
>
>
> 2)      It’s up to the sender to decide on whether an outbound message is
> worth being compressed. The receiving side must allocate resources and be
> ready to decompress an inbound message anyway, so exchanging such an extension
> parameter would not prevent the required amount of resources from being
> allocated.
>
>
>
> 3)      True, some messages can be completely uncompressible or simply
> not worth being compressed.
>
>
>
> With best regards,
>
> Arman
>
>
>
>
>
> *From:* hybi [mailto:[email-masked]] *On Behalf Of *Joakim Erdfelt
> *Sent:* Tuesday, October 7, 2014 3:18 AM
> *To:* Takeshi Yoshino
> *Subject:* Re: [hybi] Closing the hybi working group
>
>
>
> 2) lack of extension parameter to specify a minimum frame (or message)
> size to compress
>

---

From: "Arman Djusupov" <[email-masked]>
Date: Thu, 9 Oct 2014 11:59:22 +0300

Hello Joakim,

In general I think it is wise to keep permessage-compression disabled by default on the server side. Compression costs memory, it is up to the developers and administrators to decide whether a given endpoint should have compression enabled based on the estimated number of concurrent clients.

1)      As far as I understand this forces the client side to use the default configuration in most of the cases? IMO the server must be ready to impose a constrained compression configuration on the client and the clients must be ready to accept the constrained configuration from the server, since in most cases the server is more resource constrained than the clients. This means that browser clients (since they represent millions of users) must always be ready to accept client_no_context_takeover, server_no_context_takeover, and server_max_window_bits parameters from the server. Otherwise it will be impossible for the server to accept the permessage-compression extension in some cases, because the default configuration costs a significant amount of memory which limits the maximum number of client connections that the server can handle concurrently. 

One of the solutions to this problem would be to change the default configuration to make context-takeover disabled by default, so client_context_takeover and server_context_takeover would be optional parameters. Not having to keep client and server context in memory in the default configuration would make it much easier for the server to accept a large number of client connections.

2)      It’s up to the sender to decide on whether an outbound message is worth being compressed. The receiving side must allocate resources and be ready to decompress an inbound message anyway, so exchanging such an extension parameter would not prevent the required amount of resources from being allocated. 

3)      True, some messages can be completely uncompressible or simply not worth being compressed.

With best regards,

Arman

From: hybi [mailto:[email-masked]] On Behalf Of Joakim Erdfelt
Sent: Tuesday, October 7, 2014 3:18 AM
To: Takeshi Yoshino
Cc: [email-masked]; Barry Leiba
Subject: Re: [hybi] Closing the hybi working group

Jetty has an implementation, but has chosen to leave it as disabled, but configurable for any endpoint the developer wants it on for.

We had it enabled, by default for many versions of Jetty, But we've just experienced too many warts with the various implementations to leave it on for serious work, just too many different interpretations of the spec for it to be practical. (these issues have been brought up before here)
We pass most of the autobahn tests just fine, just not the ones related to window bits. (also been discussed before on this mailing list).

We only really recommend it for environments where the endpoints are tightly controlled (such as mobile apps to server).

We continue to evaluate it periodically, and perform large scale experiments with groups that heavily use websocket in production, as of mid July it still not ready to be turned on permanently.

1) lack of ability to manage / configure extensions and their parameters from the javascript websocket client api

2) lack of extension parameter to specify a minimum frame (or message) size to compress

3) lack of features in websocket api to limit (from the sender) which messages should get compressed or not

--

eclipse.org/jetty <http://eclipse.org/jetty/>  - cometd.org <http://cometd.org/> 

Thanks for the comments about the permessage-deflate. I also think the I-D is ready for resuming the process and it's still worth putting it into an RFC.

We shipped it on Chrome by default from Chrome 32 released in Nov 2013. Ilya published an article introducing the extension: https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/. Our user metrics analysis has been showing gradual increase of usage count indicating more and more adoption.

Searching on the web for permessage-deflate or searching on GitHub also show me implementations of it. In addition to ones already listed in this thread, e.g. Tomcat, Crossbar.io, Mojolicious. Jetty has but not turned on by default?

----

One topic I'm still interested in but has been away from is standardization of liveness-detection (or keep-alive) API. As Wenbo suggested, it could be done at W3C/WHATWG unless we need any extension to the ping/pong stuff.

Takeshi

Alexander

---

From: "John A. Tamplin" <[email-masked]>
Date: Thu, 9 Oct 2014 11:09:14 -0400

On Wed, Oct 8, 2014 at 9:07 AM, Joakim Erdfelt <[email-masked]> wrote:

> Even so, a small server, on modest hardware (think raspberry pi) can
> handle 20k websocket connections all with permessage-deflate negotiated
> easily.  The fastest way to impact performance on this system is rate of
> messages, using lots of smaller messages (sub 2k decompressed in size).  If
> we test it differently with less rate, but larger message sizes (in the 3k
> to 100k decompressed range) then we can saturate the network and not see a
> server issue (testing switches from becoming memory limited to being I/O
> limited)
>

See https://www.ietf.org/mail-archive/web/hybi/current/msg01810.html posted
here in 2010 - even small, binary messages benefit from compression if you
maintain compression state across frames.

With only per-message compression, then having a simple size threshold for
compression may make sense as a default if we can't rely on the sender
telling us whether to compress or not, but can we find a threshold that
makes sense across all applications?

-- 
John A. Tamplin

---

From: Joakim Erdfelt <[email-masked]>
Date: Thu, 9 Oct 2014 08:34:53 -0700

Regarding the "should compress" part. (been working on organizing the
feedback)

There's been a few proposals to us about it.
Mostly in the area of pre-configuration, at the extension negotiation level.

Stated in terms of extension parameters, it would look like this.

# Should specific message type even compress at all? (default is true)
server-compress-text = (bool)
server-compress-binary = (bool)
client-compress-text = (bool)
client-compress-binary = (bool)
# What is the minimum size (in bytes) before compression kicks in? (default
is 0)
server-min-compressible-binary = (number in bytes)
server-min-compressible-text = (number in bytes)
client-min-compressible-binary = (number in bytes)
client-min-compressible-text = (number in bytes)

With these options the server endpoint could be the one with the knowledge
of use for this endpoint.
The server could provide these values back to the client to indicate
optimum compression behavior for the conversation.

While this looks noisy at face value, there could be some valid technique
to combine the concept of "min compress size in bytes" and "on/off".

We have valid scenarios where a client endpoint will open 1 websocket
connection, using one form of message type (text or binary) for larger
infrequent important messages, and the other message type for smaller more
frequent / lower priority / time sensitive / messages.  In this sort of
setup, the client (browser) will always compress everything, but the server
has some knowledge prior to the negotiation to know that is non-optimal,
but has no control over how the client should behave.

On Thu, Oct 9, 2014 at 8:09 AM, John A. Tamplin <[email-masked]> wrote:

>> In experimentation with real world products, this is what we've found
>> with permessage-deflate on the server side. (excluding implementation
>> differences)
>>
>> In regards to memory:
>> A simple minimum message size before compression goes a long way for
>> these concerns, as it basically compresses larger messages that would have
>> taken a latency hit anyway.
>>
>

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 9 Oct 2014 11:27:20 -0700

I agree with John that permessage-deflate can achieve significant compression levels even with modest parameters (e.g. JSON payload).

Of course there is a tradeoff between memory/cpu usage and wire-level efficiency.

FWIW, this is what we recommend to users:

https://github.com/crossbario/crossbar/wiki/WebSocket%20Compression#production-settings

And that is based on work done by Peter:

http://www.ietf.org/mail-archive/web/hybi/current/msg10222.html

Unfortunately, current browsers will waste a lot of wire bytes when running over TLS again

http://tavendo.com/blog/post/dissecting-websocket-overhead/

The latter is an area with significant potential for improvement (IMO) - both with and without permessage-deflate.

Cheers,
/Tobias

Von: hybi [mailto:[email-masked]] Im Auftrag von John A. Tamplin
Gesendet: Donnerstag, 9. Oktober 2014 17:09
An: Joakim Erdfelt
Cc: [email-masked]
Betreff: Re: [hybi] Closing the hybi working group

On Wed, Oct 8, 2014 at 9:07 AM, Joakim Erdfelt <[email-masked]> wrote:
In experimentation with real world products, this is what we've found with permessage-deflate on the server side. (excluding implementation differences)

In regards to memory:
Memory use: any use of permessage-deflate is expensive, with or without context takeover.  (this is mainly due to the nature of decompress, where we have to work with the received network buffers, the deflate context buffers, and the uncompressed buffers).  (note: the memory differences between context takeover choices is so negligible that it is actually difficult to measure in real world applications)
Even so, a small server, on modest hardware (think raspberry pi) can handle 20k websocket connections all with permessage-deflate negotiated easily.  The fastest way to impact performance on this system is rate of messages, using lots of smaller messages (sub 2k decompressed in size).  If we test it differently with less rate, but larger message sizes (in the 3k to 100k decompressed range) then we can saturate the network and not see a server issue (testing switches from becoming memory limited to being I/O limited)

In regards to "is compressible":
the permessage-deflate existence, and design, along with the lack of control over the compression from the many websocket apis make this a red flag for many deployments.  This is related to the general need for control over if compression should be applied on a per-message basis.  With permessage-deflate its not "is this compressible?" which is important, its more of a question of "is it worth compressing this message?" as many websocket messages are small enough that the the costs of the compression in regards to latency and memory are not worth the impact it has on the system.
This means we should expand our discussion on this subject from types of data (eg: text vs image data) to a larger scope including "is the hit on performance worth compressing this small text data?", this should really be exposed to the developer in the websocket apis, either as a controllable extension parameter to upgrade with, or as a configurable on the api send behavior ( api.setMinCompressSize(bytes) ), or as optional advanced configuration on each send method ( api.send(msg,compress=false) )

A simple minimum message size before compression goes a long way for these concerns, as it basically compresses larger messages that would have taken a latency hit anyway.

See https://www.ietf.org/mail-archive/web/hybi/current/msg01810.html posted here in 2010 - even small, binary messages benefit from compression if you maintain compression state across frames.

With only per-message compression, then having a simple size threshold for compression may make sense as a default if we can't rely on the sender telling us whether to compress or not, but can we find a threshold that makes sense across all applications?

-- 
John A. Tamplin

---

From: Barry Leiba <[email-masked]>
Date: Thu, 16 Oct 2014 13:49:06 -0400

And so...
Anything here?

Barry

On Tuesday, October 7, 2014, Barry Leiba <[email-masked]> wrote:

> On Fri, Oct 3, 2014 at 2:15 PM, Barry Leiba <[email-masked]
> <javascript:;>> wrote:
> > [email-masked] <javascript:;>

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 16 Oct 2014 18:21:21 +0000

I will review the draft next-next week (starting October 27 after the IETF submission deadline)
 to check the status and evaluate its readiness for publication.

I will work (via mail or phone) with Takeshi in case I find that some editing still needs to be done.
I hope it works for Takeshi as well.

Berry, does it works for you?

best regards
Salvatore

On Oct 16, 2014, at 8:49 PM, Barry Leiba <[email-masked]<mailto:[email-masked]>>
 wrote:

And so...
Anything here?

Barry

On Tuesday, October 7, 2014, Barry Leiba <[email-masked]<mailto:[email-masked]>> wrote:

Barry, still hopeful...

On Fri, Oct 3, 2014 at 2:15 PM, Barry Leiba <[email-masked]<javascript:;>> wrote:
> [email-masked]<javascript:;>

---

From: Barry Leiba <[email-masked]>
Date: Thu, 16 Oct 2014 16:24:45 -0400

> Berry, does it works for you?

It does.  Thanks.

Barry

On Thursday, October 16, 2014, Salvatore Loreto <
[email-masked]> wrote:

>  On Oct 16, 2014, at 8:49 PM, Barry Leiba <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>>
>  wrote:
>
> And so...
> Anything here?
>
>  Barry
>
> On Tuesday, October 7, 2014, Barry Leiba <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>> wrote:
>
> [email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 17 Oct 2014 22:19:19 +0900

Sorry for delay. I was on biztrip and got sick after that...

> permessage-deflate on the server side. (excluding implementation

Thanks for the feedback.

> difficult to measure in real world applications)

no_context_takeover allows you to have some pool of deflate contexts and
share them among connections. Unless the time spent for each message is not
so long, you should be able to finish one message and return the context
used for it to the pool (of course, after zero clear not to leak info).
This may face some DoS attack vector such as delaying finish of message
sending, but such attack could be detected. This might not be practical and
needs some smart mechanism to ensure the peak memory usage low, but this is
the motivation of introduction of no_context_takeover. Did your experiment
include this kind of optimization?

> server issue (testing switches from becoming memory limited to being I/O

Hmm, what does this imply? Using smaller messages required more memory
spent for compression context?

> extension parameter to upgrade with,
>

Any control regarding temporary on/off of compression per message can be
addressed by extending the API without big loss. It's basically good to
reduce complex negotiation mechanism at the protocol level not to inflate
the spec.

What we have in our mind is adding a dictionary parameter that includes a
directive to control compression.

  ws.send(message, {compress: true});

Passing some algorithm such as the compress threshold you showed might be
better since it's expected to be faster than passing a parameter for each
send() call as above. This is also useful if we want to on/off compression
based on information only the user-agent knows (how many bytes can be saved
for a certain message, etc.).

  ws.send(message, {compressIfRatioLE: 0.5});

Or we could provide sysctl like API.

  // send() call beyond this point won't get compressed
  ws.configExtension("permessage-deflate", {compress: false});

If the control is not temporary and the user expect reduction of memory
used for compression context, this also be the reason to have this kind of
API to tell the user-agent to do so.

  ws.configExtension("permessage-deflate", {command: "shutdown"});
  ...
  ws.configExtension("permessage-deflate", {command: "init"});

anyway the detail of the API should be discussed at WHATWG/W3C. We should
discuss if we need any protocol level support to realize that.

Even no_context_takeover and max_window_size can be also implemented in
this way. E.g. client sending a request to stop context takeover using a
special message on the application level. The server sends an
acknowledgement message on the application level with its protocol stack
using the old config, and then config its protocol stack to use
no_context_takeover to apply the config for the following messages. Once
the client receives the acknowledgement, it's safe for the client to stop
taking over compression context.

We need to expose and API to allow the JavaScript to probe the capability
of the user-agent so that the client can reply to the request with
rejection message. This must be done before committing the config change by
calling configExtension() so that we can reply to the request on the
application level with the protocol stack set to use the old config.

And we need to start with no-compression state, and then transition to
compressed state.

Even if we implement these options at the protocol level, acknowledgement
mechanism is required (to allow changing them after WebSocket handshake).

The only concern is that intermediaries need to interpret the
application-level application-specific message if they want to scan the
compressed data. E.g. if we allow increasing the window size after
WebSocket handshake, intermediaries cannot assume that the data exchanged
over the connection is compressed using the value in the extension
negotiation. IMO, we don't need to care this.

For better/more intelligent compression, we could also choose to just move
the place to work to WHATWG/W3C and build a compression API. If we set such
a roadmap, I think we should just release the very basic one at the
protocol level.

So, I guess that the issues you listed can be addressed reasonably at the
API level. But if your proposals are critical to make the extension
successful, we could introduce them as well as we introduced
no_context_takeover and max_window_size based on John's analysis.

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 17 Oct 2014 22:35:56 +0900

Joakim,

Do you think we want to enforce the implementations to follow the requests?
I.e. should we let the receiver to disconnect the connection when the one
receives a message violating the request?

On Fri, Oct 10, 2014 at 12:34 AM, Joakim Erdfelt <[email-masked]> wrote:

> # What is the minimum size (in bytes) before compression kicks in?
> (default is 0)

How about parameters as follows?

server_skip_binary
server_skip_text
client_skip_binary
client_skip_text

Have integer value. An endpoint must leave messages with original length
smaller than the specified value uncompressed (RSV1=0). Specify 0 to skip
everything.

>> With only per-message compression, then having a simple size threshold
>> for compression may make sense as a default if we can't rely on the sender

Takeshi

On Fri, Oct 10, 2014 at 12:34 AM, Joakim Erdfelt <[email-masked]> wrote:

> # What is the minimum size (in bytes) before compression kicks in?
> (default is 0)
>> With only per-message compression, then having a simple size threshold
>> for compression may make sense as a default if we can't rely on the sender

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 17 Oct 2014 22:54:49 +0900

Hi Arman,

On Thu, Oct 9, 2014 at 5:59 PM, Arman Djusupov <[email-masked]> wrote:

> server_no_context_takeover,
>
Nitpick: server_no_context_takeover is unnecessary. If it's sent
unsolicitedly (without one from the client), it's just a hint that the
server doesn't takeover context. The client doesn't have to do anything
unless it wants.

> and server_max_window_bits parameters from the server. Otherwise it will
> be impossible for the server to accept the permessage-compression
> I'd like to just encourage people to use no_context_takeover by default
sacrificing 19+ octets on each connection than renaming the parameter in
the spec now.

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 18 Oct 2014 02:17:37 +0900

Thanks Sal, Barry.

I'll work with all to ensure the I-D is ready for the process. Given the
suggestion by Joakim, we might make a little addition to it but I think
it'll be small not affecting the organization of the document.

Takeshi

On Fri, Oct 17, 2014 at 5:24 AM, Barry Leiba <[email-masked]>
wrote:

>>  On Oct 16, 2014, at 8:49 PM, Barry Leiba <[email-masked]>
>>  wrote:
>>
>> And so...
>> Anything here?
>>
>>  Barry
>>

---

From: "Arman Djusupov" <[email-masked]>
Date: Mon, 20 Oct 2014 15:00:50 +0300

Hello Takeshi,

Using no_context_takeover by default as you suggested is fine with me.

With best regards,

Arman

From: Takeshi Yoshino [mailto:[email-masked]] 
Sent: Friday, October 17, 2014 4:55 PM
To: Arman Djusupov
Cc: [email-masked]; Barry Leiba
Subject: Re: [hybi] Closing the hybi working group

Hi Arman,

Hello Joakim,

1)      As far as I understand this forces the client side to use the default configuration in most of the cases? IMO the server must be ready to impose a constrained compression configuration on the client and the clients must be ready to accept the constrained configuration from the server, since in most cases the server is more resource constrained than the clients. This means that browser clients (since they represent millions of users) must always be ready to accept client_no_context_takeover, server_no_context_takeover,

Nitpick: server_no_context_takeover is unnecessary. If it's sent unsolicitedly (without one from the client), it's just a hint that the server doesn't takeover context. The client doesn't have to do anything unless it wants. 

and server_max_window_bits parameters from the server. Otherwise it will be impossible for the server to accept the permessage-compression extension in some cases, because the default configuration costs a significant amount of memory which limits the maximum number of client connections that the server can handle concurrently. 

I'd like to just encourage people to use no_context_takeover by default sacrificing 19+ octets on each connection than renaming the parameter in the spec now.