Subject: WebSocket2

From: Van Catha <[email-masked]>
Date: Fri, 30 Sep 2016 17:16:37 -0400

Continuing from the Google group
https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/WIBN7HqHIY0
.

I drafted up a rough RFC to support WebSocket2
https://github.com/vans163/websocket2-drafts/blob/master/websocket2-over-http2.txt
.

There are some pain points that I hope can be sorted out; if of course this
is of interest.

To answer some questions that were asked on the Google group:

Daniel Stenberg asked:
-- The compression negotiation seems very complicated, do clients/servers
really need that extreme flexibility with levels and bits?

For lz4, really only level 1 and 9 is needed.  But supporting in between
adds some flexibility without really compromising anything.  Removal of
support for comma separated values is indeed complex. It has been removed.

Lz4=1 uses barely any CPU and provides solid reduction in final size.
Lz4=9 uses about as much CPU as deflate (under best_compression) with
substantial reduction in final size.

For deflate, the sliding window makes a huge difference. From
https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/
you can see the amount of memory taken.

        compressor = (1 << (windowBits + 2)) + (1 << (memLevel + 9))
      decompresser = (1 << windowBits) + 1440 * 2 * sizeof(int)
             total = compressor + decompresser

So with a window size of 15 bits you need at least 131kb, with compression
level 8 another 131kb, plus decompresser. Your looking at around 300kb per
connected peer.  With a window of 8 bits and memory level 1 the memory
requirements significantly drop to 15kb~ per peer, also you can tune the
memory level on the server without breaking the client. So you can have 20
times more peers.

This is the difference between taking 30GB of ram for 100,000 peers, and
30GB of ram for 2m peers.  Tuning in between will allow you to find a sweet
middle.

To make the compression even simpler its possible to only allow lz4=1,
lz4=9 (aka lz4hc), but for deflate sliding windows, it really needs to be
customizable IMO.

A service like WhatsApp would be interested in worst compression ratio but
more long living concurrent connections per server, so for them a 8 bit
deflate window might be good.

A real time game for mobile like Clash Royal would be interested in a
deflate window of 15 bits, as they need to make sure data is as compressed
as possible so it arrives as quickly as possible on low bandwidth channels
like 3G.

A service like Dropbox would be interested in lz4=9, as this would greatly
speed upload and download of uncompressed files such as raw images and text
docs.

A service transmitting pixels and input under low latency such as
TeamViewer would be interested in lz4=1 as the compression throughput is
very high and ratio is often at least 2x.

-- A 501 response code for not accepted feels weird as I don't think its
the server's fault. I would expect a 4xx code. Maybe just 403?

The response codes returned really need work. 403 does look appropriate. I
am thinking the response should carry an extra header with the 403 if an
error occurred, websocket2-error perhaps?  To give details on the exact
error.

Lucas Pardue asked:
-- What is the benefit of the payload size field defined in section 4?

This is only for when HTTP/2 is also the transport layer.  HTTP/2 has a max
DATA frame size an endpoint is willing to receive set by the end points
during negotiation (SETTINGS_MAX_FRAME_SIZE), it has a minimum value of
16,384 and max of 16,777,215.  A websocket frame must be presented as 1
full message to the client API.  The client API has no way to know the true
final size of the full message.  Using QUIC I believe you can set a frame
length up to a 64 bit unsigned long with no restricted maximum, so this is
open to debate on the best way to do it.

---

From: Kari hurtta <[email-masked]>
Date: Sat, 1 Oct 2016 08:16:59 +0300 (EEST)

https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0615.html

> https://github.com/vans163/websocket2-drafts/blob/master/websocket2-over-http2.txt

( Very quick reading )

I'm not sure that is this problem or not 

> 4.  Data Framing
> 
>    Once a handshake has been successfully completed the remote endpoints
>    can begin to send data to each other.  Data is sent using the HTTP/2
>    transport layer fully adhering to DATA Frames, Section 6.1 [RFC7540].
>    WebSocket2 has its own encapsulated framing protocol that is not to
>    be confused with HTTP/2 DATA Frames.

So these websockect frames are inside of HTTP/2 DATA frame
payload.

> 3.1.  Client Handshake Request

and

> 3.2. Server Handshake Reply

These use HTTP/2 HEADERS frame.

1) So consider situation

+---------------+                     +-----------+                   +--------+
|  Web          |  ---- HTTP/2 ---->  |  proxy    | ---- HTTP/2 ----> | origin |  
|  browser      |                     |           |                   | server |
+---------------+                     +-----------+                   +--------+

"Web server" and "origin server" supports 
"WebSocket2 over HTTP/2" and handshake 
with headers frames succeed.

"proxy" does not know about
"WebSocket2 over HTTP/2".

There there start using of HTTP/2 DATA
frames. These usage however differ
very much from usage of http -request
and response.

Perhaps "proxy" tries cache response.

Perhaps there should be some method
to verify that also "proxy" supports
"WebSocket2 over HTTP/2".

One method is to use HTTP/2 SETTINGS -frame.
(Another method is to use new frame
 type on HTTP/2. )

2) One note

> 3.2. Server Handshake Reply

If "origin server"  doe snot know about
"WebSocket2 over HTTP/2", this looks like 
regular request for page "/ws2".
This can still succeed.

>   The server MUST send ONLY ONE of the advertised compression methods
>   or exclude the websocket2-compression header from the reply,
>   signaling that no compression will be used.

Wgen websocket2-compression is excluded, client
gets /ws2 -resource with
:status: 200

and client does not know is this supporting 
"WebSocket2 over HTTP/2" or not.

Or have I missed something?

/ Kari Hurtta