Subject: Re: WebSocket2

From: Van Catha <[email-masked]>
Date: Sat, 1 Oct 2016 14:20:38 -0400

Correct. Because HTTP2/ DATA frames essentially provide flow control only
and WebSocket2 frames need to be delivered in full to the underlying client
API. A single WebSocket2 frame could be broken up into multiple HTTP/2 Data
frames depending how large the MAX negotiated size is by the endpoints.

The proxy problem circles around back to the implementation. Perhaps a
header in the request could be included saying to not cache anything, if
the proxy caches things well its the proxies fault.  Also if the proxy is
not aware of WebSocket2 this should not matter, the proxies job is to
forward everything as it came.  As long as the proxy would forward the
websocket2-[version|compression] headers to the server and forward what the
server replies with there should be no problems.  Again if the proxy is
"smart" and decides to cache the response (which did not specify any
headers related to caching) its the proxies fault.  To be more direct the
response may be forced to include headers specifically instructing nothing
should be cached.  Does this work?

I am thinking using SETTINGS frames would be too complex, as that would
require baking WebSocket2 directly into HTTP/2, the way it is now,
WebSocket2 should run over HTTP/2 with minimal resistance since we do not
introduce new settings or HTTP/2 frame types.  HTTP/2 was designed from the
very beginning to not support 2 way streaming like websocket provides
currently for HTTP/1.1.  I think the resistance would be great if adding
WebSocket2 requires adding to the actual HTTP/2 specification.

A side option is to make WebSocket2 its own connection, for example ws2 as
the ALPN type when connecting to a server and to create the connection the
server would send a Alt-Svc with the host+port+other params.  This method
was proposed already, I recall reading it in an RFC.  I think the problem
with this method is now you will have more than 1 connection per server as
well as tons of duplication during the peer handshake. The benefit to this
method is it would be totally separate from HTTP/2, and you would be able
to have a server that strictly only speaks WebSocket2.  I think the
complexity of adoption to do it this way is much higher.

If origin server does not know about WebSocket2 and the request succeeds
that is indeed a problem.  Perhaps the server can reply with the error
header always like websocket2-error: okay; in the case when the WebSocket2
negotation was a success. This way an origin server without WebSocket2 will
reply with 200, and the client will see there is no websocket2-error: okay;
header and promptly notify the client that WebSocket2 negotiation failed.

One problem I find is I have not come across anything in HTTP/2 to make the
handshake smooth except for Alt-Svc header, but what Alt-Svc was designed
for does not fit the use case for WebSocket2.  For example Alt-Svc can only
be sent by a server, a client cannot request an Alt-Svc.

On Sat, Oct 1, 2016 at 1:16 AM, Kari hurtta <[email-masked]>
wrote:

> https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0615.html
>
> > I drafted up a rough RFC to support WebSocket2
> > https://github.com/vans163/websocket2-drafts/blob/master/
> websocket2-over-http2.txt
>
>
> ( Very quick reading )
>
> I'm not sure that is this problem or not
>
> https://github.com/vans163/websocket2-drafts/blob/master/
> websocket2-over-http2.txt
>
> > 4.  Data Framing
> >
> >    Once a handshake has been successfully completed the remote endpoints
> >    can begin to send data to each other.  Data is sent using the HTTP/2
> >    transport layer fully adhering to DATA Frames, Section 6.1 [RFC7540].
> >    WebSocket2 has its own encapsulated framing protocol that is not to
> >    be confused with HTTP/2 DATA Frames.
>
>
> So these websockect frames are inside of HTTP/2 DATA frame
> payload.
>
> > 3.1.  Client Handshake Request
>
> and
>
> > 3.2. Server Handshake Reply
>
> These use HTTP/2 HEADERS frame.
>
>
>
> 1) So consider situation
>
> +---------------+                     +-----------+
>  +--------+
> |  Web          |  ---- HTTP/2 ---->  |  proxy    | ---- HTTP/2 ----> |
> origin |
> |  browser      |                     |           |                   |
> server |
> +---------------+                     +-----------+
>  +--------+
>
>
> "Web server" and "origin server" supports
> "WebSocket2 over HTTP/2" and handshake
> with headers frames succeed.
>
> "proxy" does not know about
> "WebSocket2 over HTTP/2".
>
>
> There there start using of HTTP/2 DATA
> frames. These usage however differ
> very much from usage of http -request
> and response.
>
> Perhaps "proxy" tries cache response.
>
> Perhaps there should be some method
> to verify that also "proxy" supports
> "WebSocket2 over HTTP/2".
>
> One method is to use HTTP/2 SETTINGS -frame.
> (Another method is to use new frame
>  type on HTTP/2. )
>
>
> 2) One note
>
> > 3.2. Server Handshake Reply
>
> If "origin server"  doe snot know about
> "WebSocket2 over HTTP/2", this looks like
> regular request for page "/ws2".
> This can still succeed.
>
> >   The server MUST send ONLY ONE of the advertised compression methods
> >   or exclude the websocket2-compression header from the reply,
> >   signaling that no compression will be used.
>
> Wgen websocket2-compression is excluded, client
> gets /ws2 -resource with
> :status: 200
>
> and client does not know is this supporting
> "WebSocket2 over HTTP/2" or not.
>
>
> Or have I missed something?
>
> / Kari Hurtta
>
>
>

---

From: Ilari Liusvaara <[email-masked]>
Date: Sat, 1 Oct 2016 21:47:30 +0300

> should be cached.  Does this work?

Is there request header to request no caching? There is certainly a
response header to request no caching.

Or perhaps use a dedicated method? It would seem pretty obivous that
if you see a unknown method, you shouldn't assume very much about what
it is.

> WebSocket2 requires adding to the actual HTTP/2 specification.

Unfortunately, HTTP/2 does not have strict scheme handling like I
proposed. With it, one could just have directly used the wss scheme
(or ws for oppsec) and be done with it.

> header and promptly notify the client that WebSocket2 negotiation failed.

It seems to me that using https:// GET here is rather dangerous. Even with
extra custom headers.

-Ilari

---

From: Van Catha <[email-masked]>
Date: Sat, 1 Oct 2016 15:18:49 -0400

> response header to request no caching.

I believe there is no request that can specify "don't cache", but I may be
wrong.

> if you see a unknown method, you shouldn't assume very much about what

I think adding/using an unconventional method will be way beyond the scope
of what
is presented.  I do not think anyone will implement that?

> (or ws for oppsec) and be done with it.

:scheme is perfect! Wow.  If we could pass ws/wss for example as the scheme
that
fits perfectly. Looking at https://tools.ietf.org/html/rfc3986#section-3.1
the spec for schemes
it seems ws and wss are perfectly valid schemes to use and are registered;
http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml.

If we wanted to pass ws2, we would have to register the scheme which I think
should not be a problem.  As ws2 will not be backwards compatible with
ws/wss.  Would wss2 need to be included as well in this case?

So in the current state of affairs, using wss as the scheme over HTTP/2
will default to using
WebSocket2. Using ws as the scheme will default to WebSocket2 as well (for
backwards compat).
So if the client API uses WebSocket("ws://my-http2-server.com/channel") or
wss, both should
go to WebSocket2.

Where is the problem in HTTP/2 that would disallow schemes different from
http and https, I do not see
anything related to this?

> It seems to me that using https:// GET here is rather dangerous. Even with

Any alternative suggestion?

---

From: Kari hurtta <[email-masked]>
Date: Sat, 1 Oct 2016 22:36:35 +0300 (EEST)

Van Catha <[email-masked]>: (Sat Oct  1 21:20:38 2016)
<...>
> WebSocket2 requires adding to the actual HTTP/2 specification.
<...>

I interpreter Settings to be extension point. It does not touch HTTP/2
specification.

Hypertext Transfer Protocol Version 2 (HTTP/2)
https://tools.ietf.org/html/rfc7540

5.5.  Extending HTTP/2
https://tools.ietf.org/html/rfc7540#section-5.5

|   This document doesn't mandate a specific method for negotiating the
|   use of an extension but notes that a setting (Section 6.5.2) could be
|   used for that purpose.  If both peers set a value that indicates
|   willingness to use the extension, then the extension can be used.  If
|   a setting is used for extension negotiation, the initial value MUST
|   be defined in such a fashion that the extension is initially
|   disabled.

11.3.  Settings Registry
https://tools.ietf.org/html/rfc7540#section-11.3

|   This document establishes a registry for HTTP/2 settings.  The
|   "HTTP/2 Settings" registry manages a 16-bit space.  The "HTTP/2
|   Settings" registry operates under the "Expert Review" policy
|   [RFC5226] for values in the range from 0x0000 to 0xefff, with values
|   between and 0xf000 and 0xffff being reserved for Experimental Use.

How something (registeration) which is just "Expert Review" can be considered
to be change of actual HTTP/2 specification ?

/ Kari Hurtta

---

From: Van Catha <[email-masked]>
Date: Sat, 1 Oct 2016 17:21:19 -0400

> to be change of actual HTTP/2 specification ?

Well in the case that it will provide no resistance. The only possible
value of adding a Setting
I can see is to advertise what the headers currently advertise (is
WebSocket2 available, can we compress, other possible other things).

Keep in mind different paths on the authority can use different compression
methods.

The resistance I see is that it will have to be "Expert Reviewed" before
being included, and a part of that 16bit space would have to
be reserved for WebSocket2 extension settings. First, I doubt it would
happen in a reasonable time frame and second
I just do not see the necessity.  We have custom headers in HTTP/1.1 for a
long time and no one has had a problem with it.

---

From: Ilari Liusvaara <[email-masked]>
Date: Sun, 2 Oct 2016 08:08:16 +0300

> is presented.  I do not think anyone will implement that?

Well, if one uses https://, then Websockets connections definitely have
unconventional semantics.
 
> http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml.

The problem with :scheme is that without the strict handling, you can
get oddball HTTP responses for unknown schemes.

But I guess it could be OK if you had extra response header to indicate
that the server supports WS and intends to establish one (no need for
request header, as the :scheme already impiles intent to establish a WS
connection).

> ws/wss.  Would wss2 need to be included as well in this case?

I would say that WS2 would be a bad idea here, and one should reuse the
already-defined Websockets schemes.
 
> anything related to this?

Well, it doesn't disallow other schemes, just that servers might do odd
things with them (like e.g. ignore the scheme, treating it as https://).

> Any alternative suggestion?

Well, two above.

-Ilari

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 2 Oct 2016 18:39:40 +1300

> should be cached.  Does this work?

If the WebSocket message transmits over HTTP and does not prohibit
caching it can expect to be cached as per the default cacheability of
the method used. Caching is a fundamental part of HTTP that has to be
accounted for and looking at DATA frames alone is not sufficient for that.

We had CONNECT method baked into the WS v1 handshake for exactly this
reason.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 2 Oct 2016 18:48:28 +1300

> happen in a reasonable time frame and second

I believe the relevant expert(s) are reading this thread already. So if
a clear need for such extension is found and a definition document
created (eg the WebSocket2 RFC / Draft) it wont take much longer than
getting that spec defiend to use it in the first place. :-)

Implementation rollout will be what it will be. Regardless of any Draft
review.

> long time and no one has had a problem with it.

That is the more likely reason a negotiated extension is unlikely to
happen. If it is not needed at all, then it might be easier not to bother.

Amos

---

From: Kari Hurtta <[email-masked]>
Date: Sun, 2 Oct 2016 10:13:07 +0300 (EEST)

Van Catha <[email-masked]>: (Sun Oct  2 00:21:19 2016)
> long time and no one has had a problem with it.

My reason was:

| There [you] start using of HTTP/2 DATA
| frames. These usage however differ
| very much from usage of http -request
| and response.

Your use is not that send all request body
on DATA frames and then send response body
on DATA frames to different direction.

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0006.html

>             HTTP/2 was designed from the
>  currently for HTTP/1.1.  

Yes, that is reason why I suspect that you need negotate this.

8.1.  HTTP Request/Response Exchange
https://tools.ietf.org/html/rfc7540#section-8.1

|   A client sends an HTTP request on a new stream, using a previously
|   unused stream identifier (Section 5.1.1).  A server sends an HTTP
|   response on the same stream as the request.
|
|   An HTTP message (request or response) consists of:
|
|   1.  for a response only, zero or more HEADERS frames (each followed
|       by zero or more CONTINUATION frames) containing the message
|       headers of informational (1xx) HTTP responses (see [RFC7230],
|       Section 3.2 and [RFC7231], Section 6.2),
|
|   2.  one HEADERS frame (followed by zero or more CONTINUATION frames)
|       containing the message headers (see [RFC7230], Section 3.2),
|
|   3.  zero or more DATA frames containing the payload body (see
|       [RFC7230], Section 3.3), and
|
|   4.  optionally, one HEADERS frame, followed by zero or more
|       CONTINUATION frames containing the trailer-part, if present (see
|       [RFC7230], Section 4.1.2).
|
|   The last frame in the sequence bears an END_STREAM flag, noting that
|   a HEADERS frame bearing the END_STREAM flag can be followed by
|   CONTINUATION frames that carry any remaining portions of the header
|   block.

and so on

|   An HTTP request/response exchange fully consumes a single stream.  A
|   request starts with the HEADERS frame that puts the stream into an
|   "open" state.  The request ends with a frame bearing END_STREAM,
|   which causes the stream to become "half-closed (local)" for the
|   client and "half-closed (remote)" for the server.  A response starts
|   with a HEADERS frame and ends with a frame bearing END_STREAM, which
|   places the stream in the "closed" state.

First question: Is there http/2 proxies at all?

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0006.html

|                    Again if the proxy is
| "smart" and decides to cache the response (which did not specify any
| headers related to caching) its the proxies fault. 

I found at least on there

  :method: GET

https://github.com/vans163/websocket2-drafts/blob/master/websocket2-over-http2.txt

I also note that required :schema was missing, If it was

  :schema: http
or
  :schema: https

then it is cachable.

Hypertext Transfer Protocol (HTTP/1.1): Caching
https://tools.ietf.org/html/rfc7234

2.  Overview of Cache Operation
https://tools.ietf.org/html/rfc7234#section-2

|   Each cache entry consists of a cache key and one or more HTTP
|   responses corresponding to prior requests that used the same key.
|   The most common form of cache entry is a successful result of a
|   retrieval request: i.e., a 200 (OK) response to a GET request, which
|   contains a representation of the resource identified by the request
|   target (Section 4.3.1 of [RFC7231]).  However, it is also possible to
|   cache permanent redirects, negative results (e.g., 404 (Not Found)),
|   incomplete results (e.g., 206 (Partial Content)), and responses to
|   methods other than GET if the method's definition allows such caching
|   and defines something suitable for use as a cache key

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0008.html

> http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml.

Using :schema other than http or https, may be negotation what you need
with proxy.

> anything related to this?

My viewpoint is just opposite. ( I suppose that you do not want
negotiate with proxy. )

You need negotiate your usage with next hop. Hopefully using your scheme
is recognised as negotation by proxies.

( That idea of using :scheme ws or wss was also on

WebSocket over HTTP/2
https://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01
August 12, 2014

but it used also

SETTINGS frame with SETTINGS_WEBSOCKET_CAPABLE

That draft is expired. 
)

8.1.2.3.  Request Pseudo-Header Fields
https://tools.ietf.org/html/rfc7540#section-8.1.2.3

|   o  The ":scheme" pseudo-header field includes the scheme portion of
|      the target URI ([RFC3986], Section 3.1).
|
|      ":scheme" is not restricted to "http" and "https" schemed URIs.  A
|      proxy or gateway can translate requests for non-HTTP schemes,
|      enabling the use of HTTP to interact with non-HTTP services.

Other schemas are allowed.

Hopefully

8.  HTTP Message Exchanges
https://tools.ietf.org/html/rfc7540#section-8

then does not apply.

/ Kari Hurtta

---

From: Kari hurtta <[email-masked]>
Date: Sun, 2 Oct 2016 11:00:29 +0300 (EEST)

> anything related to this?

So your client does HTTP/2
request with 

  :scheme=ws
  :authority=my-http2-server.com
  :path=/channel

• If proxy does not support "ws" then it respons with error page,
  so you are negotating Websockect over HTTP/2 (as I hope).

  (A)

• If proxy supports "ws", and next hop does not support HTTP/2
  then proxy uses 

The WebSocket Protocol
https://tools.ietf.org/html/rfc6455

• If proxy supports "ws", and next hop supports HTTP/2,
then proxy does HTTP/2
request with

  :scheme=ws
  :authority=my-http2-server.com
  :path=/channel

   If next hop (probably origin server), does not
   support "ws" then it respons with error page

   (B)

(A,B) Is there http  error code which tells that
that it was unsuppoted :scheme (and not other 
error, for example wrong :path) ?

That is needed that client (A) orPbroxy (B) can
switch Websocket (RFC 6455) negotated over
HTTP/1.1

/ Kari Hurtta

---

From: Ilari Liusvaara <[email-masked]>
Date: Sun, 2 Oct 2016 13:15:48 +0300

> switch Websocket (RFC 6455) negotated over

I don't think there is (and that's a part of the problem) with current
scheme handling.

There are probably quite a bit of servers that just plain ignore the
scheme in request.

-Ilari

---

From: Kari Hurtta <[email-masked]>
Date: Sun, 2 Oct 2016 14:10:32 +0300 (EEST)

Ilari Liusvaara <[email-masked]>: (Sun Oct  2 13:15:48 2016)
> There are probably quite a bit of servers that just plain ignore the

Hmm. Was reason why

used SETTINGS frame with SETTINGS_WEBSOCKET_CAPABLE parameter ?
Seems not mention that reason. 

That means that server needs send SETTINGS_WEBSOCKET_CAPABLE = 1
to client indicating that server handles :scheme = ws
( and on that draft SETTINGS_WEBSOCKET_CAPABLE = 1
  was sent from client to server. )

Hypertext Transfer Protocol (HTTP) Status Code Registry
http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

gives 

----------------------------------------------
    Value            Description                             Reference
     100   Continue                        [RFC7231, Section 6.2.1]
     101   Switching Protocols             [RFC7231, Section 6.2.2]
     102   Processing                      [RFC2518]
   103-199 Unassigned
     200   OK                              [RFC7231, Section 6.3.1]
     201   Created                         [RFC7231, Section 6.3.2]
     202   Accepted                        [RFC7231, Section 6.3.3]
     203   Non-Authoritative Information   [RFC7231, Section 6.3.4]
     204   No Content                      [RFC7231, Section 6.3.5]
     205   Reset Content                   [RFC7231, Section 6.3.6]
     206   Partial Content                 [RFC7233, Section 4.1]
     207   Multi-Status                    [RFC4918]
     208   Already Reported                [RFC5842]
   209-225 Unassigned
     226   IM Used                         [RFC3229]
   227-299 Unassigned
     300   Multiple Choices                [RFC7231, Section 6.4.1]
     301   Moved Permanently               [RFC7231, Section 6.4.2]
     302   Found                           [RFC7231, Section 6.4.3]
     303   See Other                       [RFC7231, Section 6.4.4]
     304   Not Modified                    [RFC7232, Section 4.1]
     305   Use Proxy                       [RFC7231, Section 6.4.5]
     306   (Unused)                        [RFC7231, Section 6.4.6]
     307   Temporary Redirect              [RFC7231, Section 6.4.7]
     308   Permanent Redirect              [RFC7538]
   309-399 Unassigned
     400   Bad Request                     [RFC7231, Section 6.5.1]
     401   Unauthorized                    [RFC7235, Section 3.1]
     402   Payment Required                [RFC7231, Section 6.5.2]
     403   Forbidden                       [RFC7231, Section 6.5.3]
     404   Not Found                       [RFC7231, Section 6.5.4]
     405   Method Not Allowed              [RFC7231, Section 6.5.5]
     406   Not Acceptable                  [RFC7231, Section 6.5.6]
     407   Proxy Authentication Required   [RFC7235, Section 3.2]
     408   Request Timeout                 [RFC7231, Section 6.5.7]
     409   Conflict                        [RFC7231, Section 6.5.8]
     410   Gone                            [RFC7231, Section 6.5.9]
     411   Length Required                 [RFC7231, Section 6.5.10]
     412   Precondition Failed             [RFC7232, Section 4.2]
     413   Payload Too Large               [RFC7231, Section 6.5.11]
     414   URI Too Long                    [RFC7231, Section 6.5.12]
     415   Unsupported Media Type          [RFC7231, Section 6.5.13][RFC7694, Section 3]
     416   Range Not Satisfiable           [RFC7233, Section 4.4]
     417   Expectation Failed              [RFC7231, Section 6.5.14]
   418-420 Unassigned
     421   Misdirected Request             [RFC7540, Section 9.1.2]
     422   Unprocessable Entity            [RFC4918]
     423   Locked                          [RFC4918]
     424   Failed Dependency               [RFC4918]
     425   Unassigned
     426   Upgrade Required                [RFC7231, Section 6.5.15]
     427   Unassigned
     428   Precondition Required           [RFC6585]
     429   Too Many Requests               [RFC6585]
     430   Unassigned
     431   Request Header Fields Too Large [RFC6585]
   432-450 Unassigned
     451   Unavailable For Legal Reasons   [RFC7725]
   452-499 Unassigned
     500   Internal Server Error           [RFC7231, Section 6.6.1]
     501   Not Implemented                 [RFC7231, Section 6.6.2]
     502   Bad Gateway                     [RFC7231, Section 6.6.3]
     503   Service Unavailable             [RFC7231, Section 6.6.4]
     504   Gateway Timeout                 [RFC7231, Section 6.6.5]
     505   HTTP Version Not Supported      [RFC7231, Section 6.6.6]
     506   Variant Also Negotiates         [RFC2295]
     507   Insufficient Storage            [RFC4918]
     508   Loop Detected                   [RFC5842]
     509   Unassigned
     510   Not Extended                    [RFC2774]
     511   Network Authentication Required [RFC6585]
   512-599 Unassigned
----------------------

:status =  501

is probably that what should be expected with unsupported :scheme

( and HTTP/2 Error Code registry 
http://www.iana.org/assignments/http2-parameters/http2-parameters.xhtml#error-code

gives

Code 	Name 	Description 	Reference
0x0 	NO_ERROR 	Graceful shutdown 	[RFC7540, Section 7]
0x1 	PROTOCOL_ERROR 	Protocol error detected 	[RFC7540, Section 7]
0x2 	INTERNAL_ERROR 	Implementation fault 	[RFC7540, Section 7]
0x3 	FLOW_CONTROL_ERROR 	Flow-control limits exceeded 	[RFC7540, Section 7]
0x4 	SETTINGS_TIMEOUT 	Settings not acknowledged 	[RFC7540, Section 7]
0x5 	STREAM_CLOSED 	Frame received for closed stream 	[RFC7540, Section 7]
0x6 	FRAME_SIZE_ERROR 	Frame size incorrect 	[RFC7540, Section 7]
0x7 	REFUSED_STREAM 	Stream not processed 	[RFC7540, Section 7]
0x8 	CANCEL 	Stream cancelled 	[RFC7540, Section 7]
0x9 	COMPRESSION_ERROR 	Compression state not updated 	[RFC7540, Section 7]
0xa 	CONNECT_ERROR 	TCP connection error for CONNECT method 	[RFC7540, Section 7]
0xb 	ENHANCE_YOUR_CALM 	Processing capacity exceeded 	[RFC7540, Section 7]
0xc 	INADEQUATE_SECURITY 	Negotiated TLS parameters not acceptable 	[RFC7540, Section 
7]
0xd 	HTTP_1_1_REQUIRED 	Use HTTP/1.1 for the request 	[RFC7540, Section 7]
0xe-0xffffffff 	Unassigned 		

but unknown :scheme is not HTTP/2 error anyway.
)

/ Kari Hurtta

---

From: Ilari Liusvaara <[email-masked]>
Date: Sun, 2 Oct 2016 15:43:46 +0300

>   was sent from client to server. )
 
Well, I think the following would work and avoid SETTINGS:

-> :method ws2
-> :scheme wss
-> :authority foo.example
-> :path /bar
-> <optional extra parameters, e.g. compression support>
<- :status 200
<- sec-ws2-ack 1
<- <optional negotiated extras>

That is, include one header where server acknowledges that it is
Websockets2 capable. No need for crypto in negotiation because the
target is already known to be HTTP/2 capable, instead of just known to
be TCP capable as in case with Websockets(1).

And unsuccessful request would elict a HTTP response code:
- 405 for endpoint not supporting Websockets2
- 404 for endpoit does not exist
- 403 for "I don't want to talk to you"
- 401 for "identify yourself".
<And possibly others>

As for 301/302/307/308 responses, redirect across schemes would be
error (channel open failed). And one would need to be very careful about
redirects out of server's authority (probably channel open failed).

-Ilari

---

From: Kari Hurtta <[email-masked]>
Date: Sun, 2 Oct 2016 16:40:11 +0300 (EEST)

Ilari Liusvaara <[email-masked]>: (Sun Oct  2 15:43:46 2016)
> <- <optional negotiated extras>

If we can also assume that proxy does not ignore
   :method = ws2
   :scheme = ws
then this may work.
   
( if :scheme is wss, then proxy gets
  
  :method = CONNECT
  :authority = foo.example:443

  and there is no :scheme or :path
)

But on general forward proxies (also http/1 proxies)
look about scheme  (ie  "http" or "ftp" usually).
So :scheme = ws as unknown scheme may correctly generate
error.

They can otherwise classify :method to four classes:
•   CONNECT,
•   HEAD,
•   GET and other cachable methods, and
•   all other known and unknown methods
I think. So proxies can probably treat 
:method = ws2 as unknown, not cachable method.

I do not know about reverse proxies (load balancers (°)
or cdn).  But these are selected by :authority,
so they usually are not concern.

(  :method = CONNECT already gives that DATA frames
   are not releated to http messaging, so that
   :method = ws2, :scheme = ws or :scheme = wss
   can also be same effect.

https://tools.ietf.org/html/rfc7540#section-8.3

|   After the initial HEADERS frame sent by each peer, all subsequent
|   DATA frames correspond to data sent on the TCP connection.  The
|   payload of any DATA frames sent by the client is transmitted by the
|   proxy to the TCP server; data received from the TCP server is
|   assembled into DATA frames by the proxy.  Frame types other than DATA
|   or stream management frames (RST_STREAM, WINDOW_UPDATE, and PRIORITY)
|   MUST NOT be sent on a connected stream and MUST be treated as a
|   stream error (Section 5.4.2) if received.

)

/ Kari Hurtta

(°) Load balancer can terminate to it

   ∙ IP -protocol  (that make them some kind routers), 
   ∙ TCP -protocol (that make them TCP proxies), or
   ∙ HTTP -protocol (that make them HTTP proxies)

   Only HTTP -proxies are concern on here.

---

From: Ilari Liusvaara <[email-masked]>
Date: Sun, 2 Oct 2016 20:19:05 +0300

> If we can also assume that proxy does not ignore

Oh yeah, that only works against dodgily implemented origins, not WS2-
unsupporting proxies (that do something else than just realtime
forwarding of unknown methods).

If one is worried about the latter, one would need the SETTING then
(one only needs to have server end signal support, since it is a
capability server has that client may or may not use)..

-Ilari

---

From: Kari Hurtta <[email-masked]>
Date: Mon, 3 Oct 2016 07:40:03 +0300 (EEST)

Ilari Liusvaara <[email-masked]>: (Sun Oct  2 20:19:05 2016)
> capability server has that client may or may not use)..

If forward proxy (= proxy configured on browser) supports both http and ftp, 
then it is checking :scheme. If it supports only http (and tunneling with 
CONNECT), then it can be lazy.

Reverse proxies (which DNS gives from :authority) are more likely
to be lazy.  These are either http/2 ⇒ http/1.1 or http/2 ⇒ http/2.
 
Reverse proxies may be also TLS termination point. So wss is not tunneled.

"Transparent" proxies; then perhaps SETTINGS does not work either.

/ Kari Hurtta

---

From: Van Catha <[email-masked]>
Date: Mon, 3 Oct 2016 12:39:34 -0400

Amos Jeffries

> I believe the relevant expert(s) are reading this thread already.
>

I was not aware using a custom method was allowed, if that is the case,
CONNECT, UPGRADE, WS2, etc all work.

Kari Hurtta

> redirects out of server's authority (probably channel open failed).

This looks really optimal.

>   and there is no :scheme or :path

Again a good option.

--------------

About Proxies:
~
I assumed the concern was with forward / reverse proxies like NGINX
forwarding http/2 to http.

Afaik HTTP/2 browser only allow using TLS, so a HTTP transparent proxy will
not be able to "proxy" anything unless the reverse proxy serves a MITM
certificate.  I do not think this is a common enough use case.
~

About Settings frame:
~
If the idea behind this is to make WebSocket2 compatible over HTTP/1.1 then
that is part of the reasons why I advocate to avoid SETTINGS frame.

If WebSocket2 can be negotiated and used in such a way to avoid locking to
the transport layer, it can easily be used in HTTP/1.1 as well.

An option is to define a WebSocket2 spec RFC, then define individual RFCs
for WebSocket2 over X|Y|Z.
~

About New HTTP/2 Frame Type:
~
If a new frame type is to be introduced it severly complicates things, as
anything WebSocket2 related to this frame is not applicable. This new frame
type would fulfill the role of data streaming.  It should be applicable not
only to websockets2 but also to HTTP/1.1's Chunked-Encoding.

There would be no standard on using this new frame type, and
implementations that build off HTTP/2 such as QUIC would be left in the
dark on how to use it.

Also HTTP/1.1 would have no chance at getting WebSocket2. If such a need
were to ever exist, currently I am not considering it at any priority level.
~

About Payload Length optimizations:
~
There is a particular use case where if you sent lots of small messages
that end up in a single HTTP/2 DATA frame or a single system level packet,
there is an overhead of 3 extra bytes and 6 for LZ4 compressed payloads.

Using flags to specify how many bits the payload size is we can remove some
flexibily but also decrease final system level packet size:

RSV now becomes 6 bits.
TYPE now becomes 2 bits. We have room for 1 more frame type.

Last 2 RSV bits are Payload Length. 0 for no length (error frame), 1 for 1
byte, 2 for 2 byte, 3 for 4 byte.

Second last 2 RSV bits are for compression.   0 is no compression, 1 is
lz4, 2 is deflate.  3 is reversed.

First 2 RSV bits are decompressed payload size in the case of LZ4 or
reserved for future use by compression.

This also has the benefit of not requiring to keep a serverside state for
LZ4 compressed payloads; for anyone that would have a sane usecase for that
:)

Before a small LZ4 Text frame looked like this:
b00000000 0x00 0x00 0x00 0x09 0x00 0x00 0x00 0x05 0xAE 0xB4

Now it can look like:
b01010100 0x09 0x05 0xAE 0xB4

>From 11 bytes down to 5.

The reason I think a 32bit UINT max is a good top value is that gives you
about 4G~ of a maximum payload.  If you have a payload greater than 4G
there is no sane way a client API can process that. Even 4G is way too much
~

Where to go from here:
~
What do you think if I update the RFC with contributors for the new
handshaking we discussed and possibly this new way to do the payload if we
all like it?

A possible approach is to break up the WebSocket2 framing into its own RFC,
which
will cover compression methods and framing.

A seperate RFC would cover transporting WebSocket2 over HTTP/2, now this
seperate RFC can involve custom HTTP/2 frame types, header based
authentication, negotiating compression, custom settings frames, proxy
support, etc?
~

---

From: Kari Hurtta <[email-masked]>
Date: Mon, 3 Oct 2016 22:24:19 +0300 (EEST)

Van Catha <[email-masked]>: (Mon Oct  3 19:39:34 2016)
> certificate.  I do not think this is a common enough use case.

You can avoid complications of forward proxy if you define 
WebSocket2 only for wss: -scheme (which implies TLS).

On case of ws: -scheme request of HTTP/2 is 
interpreted by forward proxy.  Forward proxy 
uses :scheme, and :authority to determine what 
host and port to connect and which protocol to 
use.

( If browsers use HTTP/2 only for https connections,
  then ws: means HTTP/1.1 for Websockect protocol 
  negation I guess. )

I do not know if any browser is using HTTP/2
for connection to configured proxy. I guess
that they use HTTP/1.1 on here. Is there
HTTP/2 forward proxies either? (‡)

( Some browsers can use TLS on connection
  to configured proxy. )

Anyway these parts which process HTTP/2 frames
must understand that DATA -frames are not
HTTP request / response body. Instead they
behave same way than with :method = CONNECT
(and no :scheme or :path).

Situation what I was looking was

    +---------------------------+
    | Web browser or other      |
    | WebSockect client         |
    +---------------------------+
                 ⇓
                 ⇓
    +---------------------------+
    | forward proxy configure   |     ( client uses CONNECT
    | on client                 |       tunnel if encypted scheme )
    +---------------------------+
                 ⇓
                 ⇓
    +---------------------------+
    | reverse proxy for         |   ( may include TLS offloading,
    | :authority                |     if encryption is used )
    +---------------------------+
                 ⇓
                 ⇓
    +---------------------------+
    | origin server             |
    |                           |
    +---------------------------+

I think that is is common network
architecture.

I'm ignoring transparent proxies
and MITM of TLS here.

/ Kari Hurtta

(‡) nghttp2 as forward proxy
    needs http/1.1 proxy which it
    uses for proxy functionality

    nginx seems mention only
    reverse proxy.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 3 Oct 2016 15:52:08 -0600

On 10/03/2016 01:24 PM, Kari Hurtta wrote:
> Is there HTTP/2 forward proxies

Yes, at least in the active development stage. AFAIK, the popular HTTP
proxies do not support full HTTP/2 forwarding yet, but I suspect that
will change as well.

>From the real world compatibility point of view, if WebSocket2 are going
to use TCP port(s) commonly used for HTTP and HTTPS traffic, it would be
great if an HTTP proxy (any HTTP version) can easily identify and tunnel
that WebSocket2 traffic without implementing WebSocket2.

Thank you,

Alex.

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 4 Oct 2016 07:19:08 +0300 (EEST)

> CONNECT, UPGRADE, WS2, etc all work.

• Using CONNECT makes this

 ":scheme"    = "wss"  (or  "ws"  on forward proxy scenario )
 ":authority" = "foo.example"
 ":method"    = "CONNECT"
 ":path"      = "/bar"

RFC of Websocket2  updates RFC 7540 (HTTP/2)

This make clear negation for WebSocket2

|   o  The ":method" pseudo-header field is set to "CONNECT".
|
|   o  The ":scheme" and ":path" pseudo-header fields MUST be omitted.
|
|   o  The ":authority" pseudo-header field contains the host and port to
|      connect to (equivalent to the authority-form of the request-target
|      of CONNECT requests (see [RFC7230], Section 5.3)).
|
|   A CONNECT request that does not conform to these restrictions is
|   malformed (Section 8.1.2.6).
 
https://tools.ietf.org/html/rfc7540#section-8.1.2.6

|   Intermediaries that process HTTP requests or responses (i.e., any
|   intermediary not acting as a tunnel) MUST NOT forward a malformed
|   request or response.  Malformed requests or responses that are
|   detected MUST be treated as a stream error (Section 5.4.2) of type
|   PROTOCOL_ERROR.
|
|   For malformed requests, a server MAY send an HTTP response prior to
|   closing or resetting the stream.  Clients MUST NOT accept a malformed
|   response.  Note that these requirements are intended to protect
|   against several types of common attacks against HTTP; they are
|   deliberately strict because being permissive can expose
|   implementations to these vulnerabilities.

1) If origin server, forward proxy or reserve proxy
   supports HTTP/2 and pay attention and does not
   support WebSocket2, then it produces RST_STREAM 
   or GOAWAY frame with PROTOCOL_ERROR.

   RST_STREAM is cheap but GOAWAY is expensive.

2) If origin server notices ":method", but ignores
   ":scheme" and ":path", this produces 
   ":status" = "405"   (Method Not Allowed) most likely.

3) If reverse proxy for ":authority" notices ":method", 
   but ignores ":scheme" and ":path", then it perhaps just
   forwards request (maybe dropping ":scheme" and ":path")

4) If forward proxy configured on browser notices ":method", 
   but ignores ":scheme" and ":path", then it perhaps tries connect port
   443 of ":authority" or perhaps it notices that port is missing
   from ":authority".

• Using WS2 makes this

 ":scheme"    = "wss"  (or  "ws"  on forward proxy scenario )
 ":authority" = "foo.example"
 ":method"    = "WS2"
 ":path"      = "/bar"

RFC of Websocket2 registers new method. 

( http://www.iana.org/assignments/http-methods/http-methods.xhtml#methods )
( Registration Procedure(s): IETF Review
  RFCs (of IETF working group) and some other RFCs 
  fills that procedure. )

This is similar than PRI method registeration of
RFC 7540 (HTTP/2)

https://tools.ietf.org/html/rfc7540#section-11.6

|   Related information:  This method is never used by an actual client.
|      This method will appear to be used when an HTTP/1.1 server or
|      intermediary attempts to parse an HTTP/2 connection preface.

WS2 is not used as actual http -method but it is used on negotiation.

If origin server, forward proxy or reserve proxy does
not pay attention to ":scheme" and ":method", it 
probbaly assumes that DATA frames form HTTP request
and response body.

1) If origin server does not pay attention to ":scheme"
   it  produces ":status" = "405"  (Method Not Allowed) most likely.

2) If origin server notices that it does not support
   ":scheme" it can also produce ":status" = "501" (Not Implemented).

3) Reverse proxy for ":authority" probably forwards request
   but may assume that DATA frames form HTTP request
   and response body if it does not know about 
   RFC of Websocket2.

4) forward proxy configured on browser proabably
   notices that it does not support ":scheme"
   and returns ":status" = "501" (Not Implemented)
   (in case it does not support or know about
    RFC of Websocket2).

• Using UPGRADE is same case than WS2

RFC of Websocket2 registers new method.

/ Kari Hurtta

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Oct 2016 20:12:25 +0900

Hi Van, Kari,

On Tue, Oct 4, 2016 at 1:39 AM, Van Catha <[email-masked]> wrote:

<snip>

> > Well, I think the following would work and avoid SETTINGS:

Yutaka's proposal required the SETTINGS_WEBSOCKET_CAPABLE SETTINGS
parameter because we planned to use special framing (use of HTTP2 framing
level features not used for HTTP1.1/h2 layering). We wanted allow endpoints
to make sure that the h2 intermediaries between them are able to forward
WS/HTTP2's special framing correctly.

So, for the current proposal by Van where only the DATA frame is used as
well as HTTP1.1/h2 layering, it's unnecessary.

> > <- <optional negotiated extras>
:scheme is needed, yes.

Not sure about need for "sec-ws2-ack". For WebSocket/TCP, we employed the
Sec-WebSocket-Key/Accept challenge/response in order to prevent the
WebSocket protocol from being abused for cross protocol attacks e.g. SMTP.
If all the intermediaries and servers correctly investigate the :scheme
header and don't get confused, "sec-ws2-ack" is unnecessary. Regarding
ws/h2-RFC6455 bridging, a correctly implemented intermediary facing ws/h2
capable node and ws/h2 non-capable node would just perform RFC 6455
handshake as Kari suggested. So, no problem.

We can also consider taking care of bad proxies who don't check :scheme and
just converts data between h2 and HTTP/1.1/TCP. Then, including a special
header for capability validation like this "sec-ws2-ack" might be a good
choice. I understand that people are feeling that the
Sec-WebSocket-Key/Accept got to be too much complex.

But "sec-ws2-ack" doesn't work for the signaling what Yutaka wanted to
realize by SETTINGS_WEBSOCKET_CAPABLE.

<snip>

About Proxies:
> Afaik HTTP/2 browser only allow using TLS, so a HTTP transparent proxy
> will not be able to "proxy" anything unless the reverse proxy serves a MITM

Are you suggesting that the WS2/h2 should be indistinguishable from
HTTP1.1/h2 from intermediaries and server without knowledge of the
websocket2- headers or application level knowledge? Or just suggesting that
we can reduce complexity by ignoring all the v2-v1 bridging at proxies?

Do you have any strong opinion about how WS2 should be exposed on the web
platform? The WebSocket API, some brand new API? Or are you only interested
in defining a standardized messaging protocol to be layered over HTTP2 and
QUIC to utilize their power?

> About Settings frame:
> ~
> If the idea behind this is to make WebSocket2 compatible over HTTP/1.1
> then that is part of the reasons why I advocate to avoid SETTINGS frame.
>

This comment also strikes me that you want a standardized messaging
protocol over arbitrary bidirectional byte streams.

<snip>

> were to ever exist, currently I am not considering it at any priority level.

Got it.

> there is an overhead of 3 extra bytes and 6 for LZ4 compressed payloads.

How does this 3 break down?

Reading the following revised header proposal, I looks the minimum is 2
octets.
- 2 LZ4 decompressed payload length field's length bits
- 2 compression bits
- 2 payload length field's length bits
- 2 type bits
- 1 octet at minimum for text and binary message

and 2 + 4 (decompressed byte size) = 6 for LZ4?

> Using flags to specify how many bits the payload size is we can remove
> some flexibily but also decrease final system level packet size:
>
> RSV now becomes 6 bits.

To me, 4 octet fixed-length payload length header looks competitive for its
simplicity, but this variable length doesn't compared to the RFC 6455
length encoding. I understand it ended up to look weird, but it has good
representation power and efficiency, and there're existing code resources
developed for implementing RFC 6455. What do you think about just using RFC
6455's length format?

> lz4, 2 is deflate.  3 is reversed.

Unless there we expect needs for switching between the two or more
compression algorithms in a single connection, we don't need to give a
dedicated bit for each algorithm.

The first 2 RSV bits could be also defined as generic compression parameter
field.

> there is no sane way a client API can process that. Even 4G is way too much

I understand your rationale though when dealing with very large data, the
overhead of the length header is relatively small and doesn't matter so
much.

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 4 Oct 2016 18:20:06 +0300 (EEST)

Takeshi Yoshino <[email-masked]>: (Tue Oct  4 14:12:25 2016)
> well as HTTP1.1/h2 layering, it's unnecessary.

I contemplated SETTINGS as:

∙ SETTINGS frame with SETTINGS_WEBSOCKET_CAPABLE = 1
  from HTTP/2 server ⇒  HTTP/1 client direction

  ∘ Promises that HTTP/2 server checks ":scheme"
    (and specially "wss" and "ws")

  ∘ Acknowledges that DATA frames for
    ":scheme" = "wss" and "ws" are handled
    as bidirectional traffic (similar than
    ":method" = "CONNECT").

    In other words these DATA drames do
    NOT form HTTP request body from client 
    and DATA drames do not form
    HTTP response body from server
    (on cases ":scheme" = "wss" and "ws").
   
  ∘ SETTINGS_WEBSOCKET_CAPABLE = 1 does
    not promise that HTTP/2 server
    accepts ":scheme" = "wss" or "ws".

  (※)

∙ Promise of HTTP/1 client (⇒ HTTP/2 server)
  is implied with ":scheme" = "wss" or "ws". 

> handshake as Kari suggested. So, no problem.

Ilari Liusvaara suggested sec-ws2-ack for check that
":scheme" is not ignored (if I remember correctly).

/ Kari Hurtta

(※)  Yes, on my network picture there is 
     many HTTP/2 servers and clients:

    +---------------------------+
    | Web browser or other      |
    | WebSockect client         |
    +---------------------------+   [ HTTP/2 client ]
                 ⇓
                 ⇓
    +---------------------------+   [ HTTP/2 server ]
    | forward proxy configure   |     ( client uses CONNECT
    | on web blowser            |       tunnel if encypted scheme )
    +---------------------------+  [ HTTP/2 client ]
                 ⇓
                 ⇓
    +---------------------------+  [ HTTP/2 client ]
    | reverse proxy for         |   ( may include TLS offloading,
    | :authority                |     if encryption is used )
    +---------------------------+  [ HTTP/2 client ] 
                 ⇓
                 ⇓
    +---------------------------+  [ HTTP/2 server ]
    | origin server             |
    |                           |
    +---------------------------+

SETTINGS frame is for one hop between
HTTP/2 server and HTTP/2 client.

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 4 Oct 2016 20:26:43 +0300 (EEST)

Alex Rousskov <[email-masked]>: (Tue Oct  4 00:52:08 2016)
> that WebSocket2 traffic without implementing WebSocket2.

Is this

RFC 7639: The ALPN HTTP Header Field
https://tools.ietf.org/html/rfc7639
 
> Alex.

Except that here is no ALPN  token for websocket.

Application-Layer Protocol Negotiation (ALPN) Protocol IDs
http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids

Protocol 	Identification Sequence 				Reference
HTTP/1.1 	0x68 0x74 0x74 0x70 0x2f 0x31 0x2e 0x31 ("http/1.1") 	[RFC7230]
SPDY/1 		0x73 0x70 0x64 0x79 0x2f 0x31 ("spdy/1") 		[http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft1]
SPDY/2 		0x73 0x70 0x64 0x79 0x2f 0x32 ("spdy/2") 		[http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2]
SPDY/3 		0x73 0x70 0x64 0x79 0x2f 0x33 ("spdy/3") 		[http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3]
Traversal Using Relays around NAT (TURN) 	0x73 0x74 0x75 0x6E 0x2E 0x74 0x75 0x72 0x6E ("stun.turn") 	[RFC7443]
NAT discovery using Session Traversal Utilities for NAT (STUN) 	0x73 0x74 0x75 0x6E 0x2E 0x6e 0x61 0x74 0x2d 0x64 0x69 0x73 0x63 0x6f 0x76 0x65 0x72 0x79 ("stun.nat-discovery") 	[RFC7443]
HTTP/2 over TLS 	0x68 0x32 ("h2") 				[RFC7540]
HTTP/2 over TCP 	0x68 0x32 0x63 ("h2c") 				[1][RFC7540]
WebRTC Media and Data 	0x77 0x65 0x62 0x72 0x74 0x63 ("webrtc") 	[RFC-ietf-rtcweb-alpn-04]
Confidential WebRTC Media and Data 	0x63 0x2d 0x77 0x65 0x62 0x72 0x74 0x63 ("c-webrtc") 	[RFC-ietf-rtcweb-alpn-04]
FTP 		0x66 0x74 0x70 ("ftp") 					[RFC959][RFC4217]

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Wed, 5 Oct 2016 12:26:08 +1100

> Except that here is no ALPN  token for websocket.

That's relatively easy to correct.  Note that it doesn't make sense to
register a token for the existing specification since that is upgraded
from "http/1.1"; at the point that the connection is established, it's
HTTP/1.1 all the way.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 5 Oct 2016 12:58:59 +0900

I asked the questions about layering and encoding since it seems what
you're proposing is very similar to what we (Wenbo from our team) presented
at the HTTP workshop in July. It's currently developed at
https://github.com/bidiweb/wish/blob/master/draft-yoshino-wish-00.txt and
reviewed by interested parties.

We're finalizing it and plan to post it soon when ready.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 5 Oct 2016 14:28:23 +0900

Hi Kari,

I just remembered that you gave some feedback to Yutaka's proposal in 2014.
Thanks.

On Wed, Oct 5, 2016 at 12:20 AM, Kari Hurtta <[email-masked]>
wrote:

<snip>

>     accepts ":scheme" = "wss" or "ws".

Yutaka's proposal tried to represent the same guarantee by using the
combination of the client to server direction SETTINGS and the response
status code.

I chatted with Yutaka offline. He don't remember the background fully, but
we guess we chose to let the client send SETTINGS because we want to start
sending WebSocket handshake before waiting for response from the server to
reduce the initial latency. Such an attempt may result in failure, but
worth doing for some latency sensitive applications.

There're two approaches to realize this:
(a) let the server send SETTINGS and let the client send handshake
speculatively without waiting for the SETTINGS
(b) let the client send SETTINGS and then send handshake speculatively, and
let the server determine the response status code based on whether or not
it has received SETTINGS as specified in Yutaka's I-D.

(a) still gives the client path check result before receiving the WebSocket
handshake response, but it's not good that the server cannot know whether
the path was good or not before accepting the WebSocket handshake.

One more thing to note is that we were investigating whether we could relax
the requirement of the WebSocket API that messages cannot be sent before
receiving handshake response, when h2 is used. The server needs to process
these speculative messages without determining whether the the path was
good for WS/h2 if we adopt (a).

One small benefit of (b) is that there wouldn't be any
SETTINGS_WEBSOCKET_CAPABLE traffic if the client doesn't want to use WS.

Now I think combining (a) and (b) is the best given the concern that the
server may forget to investigate the SETTINGS correctly but still reply to
a WebSocket handshake with non 501 status code.

<snip>

> ":scheme" is not ignored (if I remember correctly).

Yeah. I understood so and therefore I said "If all the intermediaries and
servers correctly investigate the :scheme header and don't get confused,".

<snip>

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 5 Oct 2016 10:03:02 +0300 (EEST)

Takeshi Yoshino <[email-masked]>: (Wed Oct  5 08:28:23 2016)
> >     (on cases ":scheme" = "wss" and "ws").

Quote from WiSH ( https://github.com/bidiweb/wish/blob/master/draft-yoshino-wish-00.txt )
gives more backgroud:

|   responses.  Since proxies may buffer response body, communication
|   over WiSH may experience extra latency compared to WebSocket.  When
|   HTTPS is used, response buffering by proxies is less likely an issue.

Even when proxy does not cache ":scheme" or ":method",
it may buffer assumed HTTP request or response body.

Idelly DATA frames for ":scheme" = "wss" and "ws" want to
be sent immediately when underline protocol stack is empty
(TCP window have space, TCP/Sockect write buffers are empty,
 possible TLS write buffers are empty) and HTTP/2 flow
control windows for that sream have space and HTTP/2
priority tree say that that stream is correct for writing.

( same apply for ":method" = "CONNECT" )

> the path was good or not before accepting the WebSocket handshake.

yet one more possibility:

It is also possible that HTTP/2 server sets SETTINGS_WEBSOCKET_CAPABLE = 1
on initial SETTINGS which is part of server greeting.  This add little
overhead on case where client does not use websockect2. 

Origin server need send that only of there at least one websocket service 
configured.

HTTP/2 server which is HTTP/2 proxy may need set SETTINGS_WEBSOCKET_CAPABLE = 1 
on initial SETTINGS always when it supports ":scheme" = "wss" or "ws".

<removed>

> servers correctly investigate the :scheme header and don't get confused,".

On another context Mike Bishop wrte ( 
https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0037.html )

¤ Should we just define SETTINGS_MIXED_SCHEME_PERMITTED and call it a day?

I realize that ":scheme" = "wss" or "ws" sent on same HTTP/2 connection
than what is used for ":scheme" = "https", is also just one case of
for mixed scheme.

I wrote ( https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0044.html )

| connection apply probably for several origins. TLS connection
| may be terminated by reverse proxy. And different origins
| are served by different processes or servers behind of
| reverse proxy.
| 
| I guess that SETTINGS_MIXED_SCHEME_PERMITTED is too wide.

This may apply also for SETTINGS_WEBSOCKET_CAPABLE. However confusing
between ":scheme" = "http" and "https" is more dangerous than between
:scheme" = "wss" and "https".

/ Kari Hurtta

---

From: Van Catha <[email-masked]>
Date: Wed, 5 Oct 2016 12:29:05 -0400

Kari and other participants, which way do you think would be better:

OR

Using it this way you proposed would it require adding
SETTINGS_WEBSOCKET_CAPABLE
as well?

I think the SETTINGS_WEBSOCKET_CAPABLE can be included in the RFC and if
a middle box does not implement support for it, as long as it forwards
everything correctly
AND the response contains the sec-ws2-error header, there should not be
problems?

> > ":scheme" is not ignored (if I remember correctly).

If the websocket2 response returns the custom error header, this is similar
to the sec-ws2-ack.
Example on success we MUST return OKAY|SUCCESS.

sec-ws2-error: OKAY

Without this error header how can the client know what went wrong during
ws2 negotiation?

> Are you suggesting that the WS2/h2 should be indistinguishable from
HTTP1.1/h2 from
> intermediaries and server without knowledge of the websocket2- headers or
> application level knowledge? Or just suggesting that we can reduce
complexity by
> ignoring all the v2-v1 bridging at proxies?
>
platform?
> The WebSocket API, some brand new API? Or are you only interested in
defining a
> standardized messaging protocol to be layered over HTTP2 and QUIC to
utilize their
> power?

Standardized protocol or not is irrelevant.  I was simply considering the
options
and what would work.  QUIC uses different negotiation and transport frames
then HTTP/2
does (also QUIC does not have HTTP/2 SETTINGS frames). I was proposing
ideas at
tackling the problem to make WebSocket2 work across different transport
layers.

> > There is a particular use case where if you sent lots of small messages
that end
> > up in a single HTTP/2 DATA frame or a single system level packet, there
is an
> > overhead of 3 extra bytes and 6 for LZ4 compressed payloads.

octets.

> b01010100 0x09 0x05 0xAE 0xB4

Should be b01010100 0x03 0x05 0xAE 0xB4

The old proposal used 32 bits for payload length and if LZ4 32 bits for
decompressed
size. The new way I proposed specifies how many bits those will be by
setting bits
in the 8 bit header. So instead of forcing 32 bits always, we can have
variable
8,16,32.

Value of 0 in the payload length (bxxxx00xx) signifies no payload length.

The minimum size of a frame is 1 byte.  Which would be a text or binary
frame and
look like:

b00000000
OR
b00000001

> simplicity, but this variable length doesn't compared to the RFC 6455
length
> encoding. I understand it ended up to look weird, but it has good
representation
> power and efficiency, and there're existing code resources developed for
> implementing RFC 6455. What do you think about just using RFC 6455's
length format?

On all accounts RFC 6455 payload length is great.  It also would free
up 2 bits in the header.

Since the MASK bit is gone (?), maybe it can gain an extra bit.  Also a
64bit
max payload size I think is too much, will we ever want to send any message
larger
then 4.3~ gigabyte? I think a MAX_SIZE should be imposed, and one way to do
this
without imposing a MAX_SIZE spec is to use a defined amount of bits for the
lengths.

Both the LZ4 decompressed size and Payload Length can be replaced with the
RFC 6455 way
if the mask bit can be freed.

> > Second last 2 RSV bits are for compression.   0 is no compression, 1 is
lz4, 2 is
deflate.  3 is reversed.

> Unless there we expect needs for switching between the two or more
compression
> algorithms in a single connection, we don't need to give a dedicated bit
for each
> algorithm.

> The first 2 RSV bits could be also defined as generic compression
parameter field.

I thought about this, and it does add a nice amount of flexibility at the
cost of 2
bits.  For example if your streaming LZ4 compressed data which is often
1000 bytes+ per message but
occasionally get a less than 20 byte message (that gets framed), specifying
to not
compress that frame would see a performance/+size benefit.

I think we can all agree we do not want to handle decompression in the
client
Javascript.

On Wed, Oct 5, 2016 at 1:28 AM, Takeshi Yoshino <[email-masked]> wrote:

> Hi Kari,
>
> I just remembered that you gave some feedback to Yutaka's proposal in
> 2014. Thanks.
>
> On Wed, Oct 5, 2016 at 12:20 AM, Kari Hurtta <[email-masked]
> > wrote:
>
> <snip>
>
>
> (b) let the client send SETTINGS and then send handshake speculatively,
> and let the server determine the response status code based on whether or
> not it has received SETTINGS as specified in Yutaka's I-D.
>
> (a) still gives the client path check result before receiving the
> WebSocket handshake response, but it's not good that the server cannot know
> whether the path was good or not before accepting the WebSocket handshake.
>
> One more thing to note is that we were investigating whether we could
> relax the requirement of the WebSocket API that messages cannot be sent
> before receiving handshake response, when h2 is used. The server needs to
> process these speculative messages without determining whether the the path
> was good for WS/h2 if we adopt (a).
>

---

From: Kari hurtta <[email-masked]>
Date: Fri, 7 Oct 2016 19:40:22 +0300 (EEST)

> AND the response contains the sec-ws2-error header, there should not be

• Assuming

  ( This may need: RFC of Websocket2  updates RFC 7540 (HTTP/2) )

  ‣ benefit

    ∘ If http/2 component does not support Websocket2, this produces
      http/2 error code PROTOCOL_ERROR  (because ":scheme" and
      ":path" are not allowed).

    ∘ ":method" = "CONNECT" have already semantic
      which implies that DATA frames are not
      collected for request or response body
      and DATA frame traffic is bidirectional.

  ‣ disadvantage

    ∘ There is danger that PROTOCOL_ERROR error 
      is sent with GOAWAY. This means closing
      of connection (expensive).

      If PROTOCOL_ERROR error is sent with
      RST_STREAM, then this is good.

      ∷  SETTINGS_WEBSOCKET_CAPABLE  from http/2
	 server to http/2 client avoids
	 possible PROTOCOL_ERROR

    ∘ ":scheme" and ":path" may be ignored. 
      { Ignoring violates MUST clause on RFC 7540. }
      ( Forward proxy may try open tcp connection
        to ":authority". )

• Assuming

  ( This may need: RFC of Websocket2 registers new method )

  ‣ benefit

    ∘ if http/2 (or http/1.1 behind reverse proxy) server 
      does not support that method it can give
      http status code 405 (Method Not Allowed)

      Also http status 501 (Not Implemented) is possible.

    ∘ If http/2 reverse proxy does not buffer DATA,
      it may work without Websocket2 support

    ∘ response header (was it sec-ws2-error? or sec-ws2-ack?)
      may indicate that origin server understand
      ":scheme" and ":path".

  ‣ disadvantage

    ∘ If http/2 component does not support Websocket2,
      it may try buffer assumed http request body
      (all request DATA frames) and response body
      (all response DATA frames). Because some 
      responses can be cached, it make sense to buffer
      them all (to some upper limit of data).

    ∘ ":scheme" may be ignored (or dropped when
      converted to HTTP/1.1 request).

      ∷  SETTINGS_WEBSOCKET_CAPABLE  from http/2
         server to http/2 client help quarantee
	 that DATA frames are not buffered
         excessively and promises that
         ":scheme" is not ignored.

    ∘ If response header (was it sec-ws2-error? or 
      sec-ws2-ack?) is not received, 
      http/2 client may need RST_STREAM 
      opened stream.

If middle box does not support Websocket2,
then it can be assumed that it does not
send SETTINGS_WEBSOCKET_CAPABLE to http/2
client.

It may be possible to send

to http/2 origin server and then look if
SETTINGS_WEBSOCKET_CAPABLE = 1 is set
by http/2 server. If SETTINGS_WEBSOCKET_CAPABLE
is not set (not received via SETTINGS)
then http/2 client may need RST_STREAM 
opened stream.

That RST_STREAM happens when http/2 server
is finished handshake.  May be expensive.

It is also possible that http/2 server
is set SETTINGS_WEBSOCKET_CAPABLE = 1 on
initial SETTINGS frame. 

> tackling the problem to make WebSocket2 work across different transport

Obviously you then need omit SETTINGS_WEBSOCKET_CAPABLE
when QUIC is used. I do not know if buffering of DATA
frames is concern on here.

/ Kari Hurtta

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 13 Oct 2016 16:20:57 +0900

Hi Kari,

Sorry for belated reply.

On Wed, Oct 5, 2016 at 4:03 PM, Kari Hurtta <[email-masked]>
wrote:

<snip>

> ( same apply for ":method" = "CONNECT" )

Yes, it's great if the HTTP2 layer guarantee such characteristics. It's
trade-off.

<snip>

> WebSocket

I think you meant the SETTINGS sent following the connection preface
(following the Finished message of TLS from the server).

Yeah, that's another option with some disadvantage as you described.

<snip>

> :scheme" = "wss" and "https".

Oh, good point. I wasn't aware of that.

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 13 Oct 2016 16:59:07 +0900

On Thu, Oct 6, 2016 at 1:29 AM, Van Catha <[email-masked]> wrote:

<snip>

> > Do you have any strong opinion about how WS2 should be exposed on the
> web platform?

Got it.

As I explained privately a bit, we're developing the Fetch API
<https://fetch.spec.whatwg.org/> and the Streams API
<https://streams.spec.whatwg.org/> for the web platform to allow for using
bidirectional byte stream over HTTP (over TCP, HTTP2, QUIC, ...). The
transform stream <https://streams.spec.whatwg.org/#ts-model> allows for
implementing e.g. JSON stream parser/serializer, deflate codec (like RFC7692
<https://tools.ietf.org/html/rfc7692>), etc. either in JavaScript or as
standardized web API implemented in C++, etc., and connecting them to flow
data efficiently (transfer offloading). This is the story behind the WiSH
proposal for web browsers. I'm going to explain this more in the
introduction section of the I-D. Not only Chrome but some other vendors are
also showing support, experimenting and shipping some of them. This might
work for your needs.

<snip>

> The minimum size of a frame is 1 byte.  Which would be a text or binary

ok

> max payload size I think is too much, will we ever want to send any
> message larger
> then 4.3~ gigabyte? I think a MAX_SIZE should be imposed, and one way to
> do this
> without imposing a MAX_SIZE spec is to use a defined amount of bits for
> the lengths.
>

Yeah, MASK is gone as long as there's no risk of confusing non-WS endpoints.

We can choose to just use the extra bit there though it's separated from
the other RSV bits which are in in the first octet.

What did you mean by "to use a defined amount of bits for the lengths"?
Imposing the limit in the form of the value in the field which specifies
the length of the payload length field?

> if the mask bit can be freed.
Yes. We have enough space for the features.

>
> > > Second last 2 RSV bits are for compression.   0 is no compression, 1
> is lz4, 2 is
> deflate.  3 is reversed.
>
> occasionally get a less than 20 byte message (that gets framed),
> specifying to not

Did you mean turning on / off LZ4 using the 2 RSV bits?

Just turning on/off a selected compression could be realized by flipping
the single bit indicating whether the message is compressed or not.

<snip>

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 13 Oct 2016 10:16:24 +0000

On 05 October 2016, Van Catha wrote: 17:29

Standardized protocol or not is irrelevant.  I was simply considering the options
and what would work.  QUIC uses different negotiation and transport frames then HTTP/2
does (also QUIC does not have HTTP/2 SETTINGS frames). I was proposing ideas at
tackling the problem to make WebSocket2 work across different transport layers.

The current QUIC-H2 mapping document (https://tools.ietf.org/html/draft-shade-quic-http2-mapping-00) discusses the use of HTTP/2 SETTINGS frames in section 3. Some settings are superseded by QUIC transport parameters but I would think that extension negotiation would work fine in QUIC. The charter explicitly calls out accommodation for H2 extension mechanisms.

Lucas

----------------------------

http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

---------------------

---

From: Van Catha <[email-masked]>
Date: Thu, 13 Oct 2016 11:20:22 -0400

> As I explained privately a bit, we're developing the Fetch API and the
> Streams API for the web platform to allow for using bidirectional byte
> stream over HTTP (over TCP, HTTP2, QUIC, ...). The transform stream
> allows for implementing e.g. JSON stream parser/serializer, deflate
> codec (like RFC7692), etc. either in JavaScript or as standardized
> web API implemented in C++, etc., and connecting them to flow data
> efficiently (transfer offloading). This is the story behind the
> WiSH proposal for web browsers. I'm going to explain this more in
> the introduction section of the I-D. Not only Chrome but some other
> vendors are also showing support, experimenting and shipping some of
> them. This might work for your needs.

I am very interested in the Streams API and would even like to contribute,
but I see it as a long road ahead with many questions.  First there is no
framing so how do you handle compression? Ideally with something like
Streams API, you want javascript inbrowser to expose the C++ level RFC7692
deflate/inflate, then you can have full control of your framing +
compression.
I like this scenario the most.  Otherwise Streams API will just turn into
another mutated WebSockets, which sacrificed correctness for simplicity to
developers that know little about writing network services.  I hope the
future
will not follow in these footsteps.

Also LZ4 with levels 1 and 9 I think should be highly considered to be
added as
a compression method alongside of RFC7692.

Google Chrome solved a lot of performance problems with NaCl and hopefully
that will be merged into Chromium soon.  I can see NaCl doing a lot
of the heavy lifting related to Streams API like decoding/encoding protobuf
or even a bitserialized protocol. The possibilities with simple 2 ways
streams
will be endless.

A lot of enterprise rely on NaCl for their B2B and B2C apps.

This NaCl bit is to mention how the future is moving towards C/C++ in the
browser,
so Javascript performance for things like encryption/compression should be
downplayed when considering whether to bake things into the API.

> the length of the payload length field?

I mean that if you only have 32 bits for length the maximum value you can
represent is 4.3~ billion.  Cut that down to 24 bits and its 16~ million.
So you can limit the max framesize from 4.3~ gigabyte to 16~ megabyte by
simply
reducing the amount of bits used for the frame length.

> Did you mean turning on / off LZ4 using the 2 RSV bits?

Yea exactly. It adds a little flexibilty and removes the need to keep a
state
on the server side (not in the case of RFC7692 deflate). If there would be
a
use case for that.  The main benefit I see is if you do not want to
compress small messages say under 20 octets in length.

I updated the RFC for WebSocket2 over HTTP/2 if anyone wants to take another
look, it now has a more solid handshake and a few tweaks to the wire format.

I still want to push for this WebSocket2 because already the internet runs
on WebSocket, and this will be a smooth transition.

I do believe thought that a Steams API with pure 2 way binary streaming (no
compression, no framing) with C/C++ compressor exposed
through Javascript browser API is the distant future.

---

From: Kari Hurtta <[email-masked]>
Date: Thu, 13 Oct 2016 19:52:32 +0300 (EEST)

Takeshi Yoshino <[email-masked]>: (Thu Oct 13 10:20:57 2016)
> (following the Finished message of TLS from the server).

Yes, that SETTINGS which is connection preface.

3.5.  HTTP/2 Connection Preface
https://tools.ietf.org/html/rfc7540#section-3.5

|   The server connection preface consists of a potentially empty
|   SETTINGS frame (Section 6.5) that MUST be the first frame the server
|   sends in the HTTP/2 connection.

/ Kari Hurtta