Subject: Re: [hybi] HTTP Upgrade - A Modest Proposal

From: Andy Green <[email-masked]>
Date: Fri, 01 Apr 2011 10:06:52 +0100

On 04/01/2011 03:54 AM, Somebody in the thread at some point said:

Hi -

> 2. When WebSocket is provided as part of a HTTP server stack, having to
> instantiate the WebSocket serving process (thread/handler/etc.) just to
> resolve the handshake and determine if a WebSocket connection will
> actually proceed is awkward, potentially expensive
> and possibly insecure.    The responsibility to determine if the HTTP
> server will support a HTTP Upgrade to any given protocol for a given URL
> is properly the responsibility of the HTTP server, not WebSocket.   The
> HTTP server must be able, via the Upgrade request to consider only (a)
> the URL and (b) the Upgrade protocol and decide if it will support it.
> If it will, then all further negotiation is the responsibility of the
> protocol being upgraded to.

I dunno that seems clean to me.  Somebody connects claiming to want 
websocket service then the websocket plugin / library / code is 
obviously the place to form an opinion about the connection request.  If 
that's expensive or insecure then that's an implementation issue?

> 4. When WebSocket is provided as a stand-alone process, using a HTTP
> Upgrade as the handshake presents at least two problems.  (a) changing
> from line-oriented I/O (in the HTTP handshake) to a byte-oriented I/O

libwebsockets uses a bytewise parser for both HTTP and websocket 
protocol traffic.  It could have bugs like anything else but it's not 
inherently more fragile or dodgy or 'line orientated', it's the same 
bytewise state machine.  So again this is implementation and there are 
reasonable and secure implementations possible.

> (for WebSocket frames) is awkward and potentially error prone, and (b)
> any HTTP client-agent can potentially make a request to a WebSocket
> server and get back a HTTP response, this exposes the availability and
> capabilities of the WebSocket server to a client-agent that is not
> actually a WebSocket client.   Neither of these concerns is really all
> that severe, but it is something to be aware of.

Well, if it doesn't like what is told, typically the server will just 
hang upon the client.  To get anything long-lived you have to have 
correct magic in the keys in the client request.

-Andy

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 07:20:07 -0400

>
> > 1. Further, WebSocket protocol servers may not be part of a HTTP serving
> environment and having it answer HTTP (however limited) could confuse
> client
> agents.
>
> I'm not sure I understand the 'confuse client agents' concept.  If a client
> agent includes the header 'Upgrade: websocket' in an HTTP request, it
> should
> be expecting the response as defined in the spec.
>

What I was thinking of was a HTTP client-agent that mistakenly connected to
a WebSocket server.  It would send a normal HTTP GET.  The WebSocket server
would, of course, reject that connection.  That rejection response (or
dropped connection) might befuddle a client-agent expecting a typical HTTP
response.   Or, perhaps worse, the client-agent sends a reasonably legit but
mistaken HTTP Upgrade and gets the Upgrade accepted response and is then
completely confused by the non-http traffic that follows.

I admit these are unlikely scenarios and it isn't really our problem to deal
with client-agents pointed at the wrong endpoint.

> resolve the handshake
> I'm not sure why 'when A' implies 'B.'  Somewhere, the stack needs know
> what
> will be considered valid 'Request-URI' and possibly
> 'Sec-WebSocket-Protocol'
> information.  That is a concern of whomever administers the stack.
>
> Agreed.  The stack does need to know.  I argue that the proper place in the
stack (when WebSocket is deployed as part of it) is the HTTP server.  It's
the HTTP servers job to deal with URLs in it's context.   WebSockets is a
transport layer into/with the resource(s) and should participate in a
unified URL scheme.  It should not side-step the established URL scheme and
introduce its own - which if it's examining the given URL I would argue it
is.

WebSocket is a transport protocol, not an application protocol - URL mapping
& access control is the job of the HTTP server.   WebSocket is a transport
of frames for a URL defined resource, much like TCP is the transport for
HTTP packaged URL defined resources.

> > 4 (a) changing from line-oriented I/O (in the HTTP handshake) to a
> byte-oriented I/O (for WebSocket frames) is awkward and potentially error
> prone
>
> Considering WS servers will be designed for byte-oriented I/O, the
> line-oriented I/O coding requirements are minimal, and not necessarily
> error
> prone.
>

As I mentioned, it's not big deal.   But there is a conceptual shift.   For
me, that suggests a change in protocols and is further evidence that perhaps
HTTP Upgrade is outside the domain of WebSocket.

> 4 (b) any HTTP client-agent can potentially make a request to a WebSocket
> capabilities of the WebSocket server
>
> Without the correct Request-URI and possibly 'Sec-WebSocket-Protocol'
> header, what can the client find out?  If the HTTP client request is
> invalid, the spec states (somewhat vaguely) in Section 7.2.2. the following
> 'Certain algorithms require or recommend that the server _abort the
> WebSocket connection_ during the opening handshake.'  Hence, 'get back a
> HTTP response' may not occur with the client agent's request.
>
> If the client-agent request is valid, the availability of that URL as
WebSocket, and some capabilities of that connection are provided in the
handshake response.   This may, or may not be a concern.   Requiring the
WebSocket server to evaluate any incoming HTTP request increases the
possibility of denial-of-service as any HTTP client-agent can send any HTTP
request (Upgrade or not) and make the WebSocket server evaluate it before
terminating connection.   It need not even be a malicious client-agent, just
a confused one (perhaps doing XMLHttpRequest polling)  If the HTTP server
could make that initial determination we get to leverage that infrastructure
to reduce exposure.   If it's a stand-alone WebSocket server, incoming HTTP
requests can be dropped much sooner in the communication (like on the first
few bytes).

I think everyone is a little sensitive and spent from all the reading.

Indeed.  Protocol design is *hard*.

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 07:26:02 -0400

On Fri, Apr 1, 2011 at 1:21 AM, Willy Tarreau <[email-masked]> wrote:

> On Thu, Mar 31, 2011 at 11:07:19PM -0500, Greg Longtin wrote:
> I would add that any HTTP server already deals with that. They accept
> POST requests which are byte-oriented, they can send any file (eg: HTML,
> GIF, etc...) which are byte-oriented too and browsers can correctly
> retrieve these files. So in fact it's something which is already
> implemented and works very well in every client and server.
>
>
Agreed.  It's not onerous.   However, WebSocket is not HTTP.    And simpler
implementations are generally better.    If we don't really need to, why
should we?   As I mentioned to Greg, it's not that having to switch trains
in mid stream is a big problem, it's that it signals (to me at least) that
we switching protocols.  And if we're switching protocols (which we are),
then the protocols should keep themselves to their own domains.

> There were discussions in the past about exposing some capabilities as
> control frames, but that did not bring any major advantage, and that
> required an additional round-trip before communication could start,
> which is really not acceptable in some environments (eg: smartphones).
>

I would argue that one extra exchange of frames when connecting via HTTP
Upgrade is not an undue burden.   Further, if you establish a WebSocket
connection directly (ie. not via a HTTP server), the actual data transport
cost would be smaller.

Again, this is nothing more than my belief that WebSocket should be defined
as the transport protocol it is, and that HTTP Upgrade is yet another
mechanism to boot-strap a WebSocket connection.   If WebSocket only spoke
websocket frames, then I could implement any number of ways for a
client-agent to establish a connection, not just via HTTP Upgrade.

> Willy
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 07:40:14 -0400

> websocket service then the websocket plugin / library / code is obviously
> the place to form an opinion about the connection request.  If that's
> expensive or insecure then that's an implementation issue?

True, in some implementations the cost of passing the HTTP Upgrade request
over to be evaluated won't be much.  However, in the work I've been doing,
spawning the WebSocket server as a seperate process has been a good & valid
method (for many unrelated reasons), this can be expensive.    It seems to
me that when deployed in a HTTP stack, letting the webserver decided based
on URL & Upgrade: protocol if it will even consider supporting it is a
useful arrangement.  Also, I believe it more properly reflects the real
conceptual domains.

For example, if the HTTP server can decide, "Yes, I'm going to do WebSocket
for that URL", it can chroot to that URLs place in the filesystem,
instantiate the WebSocket server/handler and pass the TCP connection off
with a clean conscience.

Evaluating the validity of a URL is not the job of a transport protocol.
WebSocket is a transport protocol.

> libwebsockets uses a bytewise parser for both HTTP and websocket protocol
> traffic.  It could have bugs like anything else but it's not inherently more
> fragile or dodgy or 'line orientated', it's the same bytewise state machine.
>  So again this is implementation and there are reasonable and secure
> implementations possible.

As previously mentioned, it's not the doing I have a problem with.  It's
what the doing implies.

> Well, if it doesn't like what is told, typically the server will just hang
> upon the client.  To get anything long-lived you have to have correct magic
> in the keys in the client request.
>

Possibly causing client-agent timeout delays.  Perhaps not.   That's the
client-agent's problem.    Again, I don't this is all that serious of a
concern, just that it's another ramification of the mixing of HTTP and
WebSockets.

And don't get me started on the magic keys (Sec-WebSocket-Key).   Personally
I believe this part of the handshake to be useless hand-waving that does not
provide what it's supposedly intended to do.    The client can generate any
value and ignore the server's response, it proves nothing.      I have not
expressed a concern about it as it is harmless and not onerous to implement.
 And I'm ok with implementing hand-waving if it'll get asynchronous traffic
with a browser client out the door into the real world sooner.

---

From: "Greg Longtin" <[email-masked]>
Date: Fri, 1 Apr 2011 07:25:39 -0500

David,

First of all, my interest in WS is in using it for LAN based real-time
control.  Simply put, sockets in a browser.  I know very little about DOS
attacks.

> Requiring the WebSocket server to evaluate any incoming HTTP request
increases the possibility of denial-of-service as any HTTP client-agent can
send any HTTP request (Upgrade or not) and make the WebSocket server
evaluate it before terminating connection.

I would assume that most DOS attacks are not using traditional client-agents
(browsers), and hence, can adapt to whatever 'packet layout' suits their
needs.  Also, I suspect most current DOS prevention technology is based on
http filtering, which would imply that it could filter DOS WS upgrade
requests with minimal modification.

If the handshake request was byte oriented, I suspect a small percentage of
current 'firewall/protection' equipment could correctly filter it against WS
DOS attacks.

Greg Longtin

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 08:53:40 -0400

Hey Greg,

Your use case sounds interesting.   In my case, asynchronous communication
with a browser addresses a *very* pressing (ie. critical) need in my
industry.  We work with "real-time" data  (financial market data, if you
want to be specific) and we've been unable to take real advantage of web
technologies because of the lack of server-push type transports.   I also
consider this to be sockets-in-a-browser - but the framed nature works
nicely with the browser side Javascript object.   My intention is to be
tossing around JSON strings that represent messages over WebSocket.

Aside: why do we differentiate between binary and text frames?  WebSockets
is a transport protocol with a length field.  The payload is opaque.   Why
does it care what the content-type is?   Wouldn't interpretation of
content-type be up to the application layer?  (like how an application
decides if it'll open a file in text or binary mode).    Not that I really
care, just curious.

While most of our stuff will be on private networks (ie. not via public web
presence), dealing with confused client-agents or badly implemented customer
code, we do have to consider DOS issues, even unintended ones.

When a HTTP server has to instantiate a WebSocket server handler to evaluate
the handshake, the time & cost is greater than if it can just reject it out
of hand.  It gets worse if the WebSocket server needs to spin up databases
or open a back-end connection to other servers (authentication, data
providers, etc.).    Yes, a good implementation won't do that until the
handshake completes successfully, but allowing the HTTP server to decide on
it's own if it will accept WebSocket (HTTP Upgrade: WebSocket) on the
requested URL saves several steps.   And, of course, if the WebSocket server
is operating stand-alone, it can drop connection within the first few bytes,
assuming we're not using a HTTP Upgrade handshake.

I don't believe the Sec-WebSocket-Protocol is the concern of the HTTP
server.  That is an aspect of WebSockets and I believe should properly be
part of that layer.  The HTTP server can define, "I support WebSockets on
that URL" and then WebSockets can specify which sub-protocols (ie.
application protocols) it is going to carry.   These are separate domains
of responsibility.

---

From: "Greg Longtin" <[email-masked]>
Date: Fri, 1 Apr 2011 08:07:59 -0500

David,

> Aside: why do we differentiate between binary and text frames?

So the browser/user-agent knows what to do with it?  AFAIK, that API at the
W3C/WhatWG is TBD.

> allowing the HTTP server to decide on its own if it will accept WebSocket
(HTTP Upgrade: WebSocket) on the requested URL saves several steps.

I believe that was the reason the handshake is HTTP, so (at some point) HTTP
servers could be configured to accept/reject the WS upgrade.  In the mean
time...

Confused.  I thought you wanted a non HTTP handshake.  

> My intention is to be tossing around JSON strings

That's what I'm using.  Certainly helpful that JavaScript (ECMA5?) natively
supports JSON.

Thanks,

Greg Longtin

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 09:38:20 -0400

> So the browser/user-agent knows what to do with it?  AFAIK, that API at the

Yes, but...     If we're going to tell the user-agent what the content type
is, we should tell them the 'content-type', not a half-arsed binary/text.
If we're going to leave the content type knowledge to an understanding
between the server & user-agent at a higher level, then an untyped opaque
payload would be appropriate.  Personally I like to view WebSockets as
"framed TCP", so opaque payloads would be my choice.

 In my case, when as a user-agent, I connect with WebSocket sub-protocol
"foo", I know that traffic via "foo" is going to be in some format.  That's
a property of the specification of sub-protocol "foo".   If "foo" is going
to support multiple content-types, I will need to include that ability as
part of sub-protocol "foo".   In neither case should WebSocket, the
transport, care what the payload is.

> Confused.  I thought you wanted a non HTTP handshake.

Oh I do - I was probably confusing things there, jumping between what-is and
what-I-would.

I would like to see the HTTP Upgrade include only "Upgrade: WebSocket" and
the URL.   ie. HTTP Upgrade is not part of WebSockets, but just another
mechanism for establishing it.

Then, the WebSocket negotiation/handshake (using websocket frames) would
specify the application protocol (aka: Sec-WebSocket-Protocol) - and any
other parameters of the connection (compression,
masking, palindromic invariance, etc.)

Thus, once a "WebSocket" connection is established (by whatever means), the
two ends can determine the context of their connection (sub-protocol,
capabilities, etc.).

When deployed in a HTTP stack and using HTTP Upgrade, the webserver can
decide if it will establish the connection based on it's domain
of responsibility (Upgrade & URL) and WebSockets can decide if it will
accept the connection based on it's domain of responsibility (application
protocol, etc.)   The URL is a parameter that identifies content resource
(and might have been re-mapped/translated/redirected by the HTTP server).

When deployed as a stand-alone, the WebSocket can decide if it will accept
the connection based on .... hey, exactly like the previous paragraph.

> That's what I'm using.  Certainly helpful that JavaScript (ECMA5?) natively

Thank the Goddess for fortuitous happenstance.    Ends up making a really
clean application protocol framework, doesn't it?

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 01 Apr 2011 10:00:00 -0700

On 04/01/2011 06:38 AM, David Endicott wrote:
> Yes, but...     If we're going to tell the user-agent what the content 
> type is, we should tell them the 'content-type', not a half-arsed 
> binary/text.   If we're going to leave the content type knowledge to 
> an understanding between the server & user-agent at a higher level, 
> then an untyped opaque payload would be appropriate.  Personally I 
> like to view WebSockets as "framed TCP", so opaque payloads would be 
> my choice.

As I remember, the text/binary is a pragmatic decision primarily based 
on balancing browser API needs with low-level specialized protocols.

Tossing around JSON, for example, is really text. And since no one wants 
to make the ASCII/ISO-8859-1 assumptions/mistakes again, it's good to 
enforce UTF-8 encoding at the framing layer so it's correct from the 
beginning.

For more specialized clients (like some I'm working on), having the 
direct binary is what's needed.

We did discuss a full mime-type extension, but that very quickly became 
complicated for a simple framing protocol.

So while text/binary is perhaps not the most minimal solution possible, 
it's still simple and solves the main use cases.

-- Scott

>
>  In my case, when as a user-agent, I connect with WebSocket 
> sub-protocol "foo", I know that traffic via "foo" is going to be in 
> some format.  That's a property of the specification of sub-protocol 
> "foo".   If "foo" is going to support multiple content-types, I will 
> need to include that ability as part of sub-protocol "foo".   In 
> neither case should WebSocket, the transport, care what the payload is.
>
>
> Oh I do - I was probably confusing things there, jumping between 
> what-is and what-I-would.
>
> I would like to see the HTTP Upgrade include only "Upgrade: WebSocket" 
> and the URL.   ie. HTTP Upgrade is not part of WebSockets, but just 
> another mechanism for establishing it.
>
> Then, the WebSocket negotiation/handshake (using websocket frames) 
> would specify the application protocol (aka: Sec-WebSocket-Protocol) - 
> and any other parameters of the connection (compression, 
> Thus, once a "WebSocket" connection is established (by whatever 
> means), the two ends can determine the context of their connection 
> (sub-protocol, capabilities, etc.).
>
> When deployed in a HTTP stack and using HTTP Upgrade, the webserver 
> can decide if it will establish the connection based on it's domain 
> accept the connection based on it's domain of responsibility 
> (application protocol, etc.)   The URL is a parameter that identifies 
> content resource (and might have been re-mapped/translated/redirected 
> by the HTTP server).
>
> When deployed as a stand-alone, the WebSocket can decide if it will 
> accept the connection based on .... hey, exactly like the previous 
> paragraph.
>
>
> Thank the Goddess for fortuitous happenstance.    Ends up making a 
> really clean application protocol framework, doesn't it?
>
>
>

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 13:17:03 -0400

On Fri, Apr 1, 2011 at 1:00 PM, Scott Ferguson <[email-masked]> wrote:

> binary/text.   If we're going to leave the content type knowledge to an
> understanding between the server & user-agent at a higher level, then an
> untyped opaque payload would be appropriate.  Personally I like to view
> WebSockets as "framed TCP", so opaque payloads would be my choice.
>
>
> As I remember, the text/binary is a pragmatic decision primarily based on
> balancing browser API needs with low-level specialized protocols.
>
> Tossing around JSON, for example, is really text. And since no one wants to
> make the ASCII/ISO-8859-1 assumptions/mistakes again, it's good to enforce
> UTF-8 encoding at the framing layer so it's correct from the beginning.
>
> For more specialized clients (like some I'm working on), having the direct
> binary is what's needed.
>
Yes, I would agree the benefits / convenience for the client-agent outweigh
any "purity of design" considerations.   I was not complaining, just curious
if there were reasons beyond the obvious real-world practicality that I
wasn't aware of.

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 01 Apr 2011 11:41:30 -0700

[I hesitate to reply to a thread started so close to April 1st. However...]

On 31/03/2011 7:54 PM, David Endicott wrote:
>
> 1. HTTP Upgrade request (and it's response) are properly part of a 
> HTTP exchange, not a WebSocket one.  The use of HTTP Upgrade as a 
> mechanism to support WebSockets via a HTTP server is a good and valid 
> use of that HTTP capability, but it is not unique or specific to 
> WebSocket.

If I understand you right, you are suggesting that WebSockets run as a 
separate protocol, but with an option to use the HTTP upgrade mechanism 
if desired. This has been suggested on the list about 9 months ago, when 
it got a little bit of support but not enough to push it forward. See 
http://www.ietf.org/mail-archive/web/hybi/current/msg02175.html and 
http://www.ietf.org/mail-archive/web/hybi/current/msg02203.html. I seem 
to recall that some of the browser guys pushed back saying that they 
really didn't want to support 4 different schemes for handshaking for 
such a minor benefit.

It would also have some security implications for browsers, as follows.

> Specifically I propose that the HTTP Upgrade based handshake be 
> removed from the WebSocket specification and replaced with a Control 
> frame handshake or negotiation.

Some would argue that this would break the security of WebSockets. Right 
now, an XmlHTTPRequest can control all the bits after an HTTP handshake 
has completed, but it cannot influence any header that starts with 
"Sec-". This means that an attacker cannot provide a malicious WebSocket 
client written in Javascript that is automatically downloaded by the 
user just by visiting a web page.

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 15:19:59 -0400

> [I hesitate to reply to a thread started so close to April 1st. However...]

I hesitated to write it.

1. HTTP Upgrade request (and it's response) are properly part of a HTTP
>> exchange, not a WebSocket one.  The use of HTTP Upgrade as a mechanism to
>> support WebSockets via a HTTP server is a good and valid use of that HTTP
>> capability, but it is not unique or specific to WebSocket.
>>
>
> separate protocol, but with an option to use the HTTP upgrade mechanism if
> desired. This has been suggested on the list about 9 months ago, when it got
> a little bit of support but not enough to push it forward. See
> http://www.ietf.org/mail-archive/web/hybi/current/msg02203.html. I seem to
> recall that some of the browser guys pushed back saying that they really
> didn't want to support 4 different schemes for handshaking for such a minor
> benefit.
>

Yes, that is what I am suggesting.   I have not read the archived discussion
yet, but I will review it.

The reason I decided to comment was I kept seeing other people stumble over
the handshake process.   I already knew from my implementations that it
smelled wrong and when I was able to articulate clearly why I thought that,
I tossed in my 0.02.    As I said, I do not want to derail the
standardization process, and maintaining the status-quo is (grudgingly)
acceptable to me.   I've dealt with quirks in other protocols, and this
isn't really that big of a deal.

To reiterate - imho, WS is a transport protocol.   It should be considered
entirely separate from HTTP.   WS rides on TCP like HTTP rides on TCP - they
are peers.   HTTP Upgrade as a mechanism to establish WS is a Good Idea, but
it seems inappropriately co-mingled with the WS specification.   They are
two different things and I believe should be clearly delineated as such.   I
don't view this as a minor benefit, I see it as true reflection of the
protocol's roles.

We need to stop thinking that we're slipping WS quietly in the back door
like a nervous teenager breaking curfew.   WS is a full fledged transport
protocol on the same level as HTTP, SSH, SMTP and their ilk and should be
treated as such.    The significance of WS in the family of internet
protocols is along side HTTP, not as it's black-sheep offspring.   WS is
intended to do what HTTP does but asynchronously.    If that means more work
for the browser implementers (and I would argue it doesn't) or that it
affects how proxies and firewalls behave, then so be it - that is the real
consequence of what we're trying to do and the sooner we face it the better.

> now, an XmlHTTPRequest can control all the bits after an HTTP handshake has
> completed, but it cannot influence any header that starts with "Sec-". This
> means that an attacker cannot provide a malicious WebSocket client written
> in Javascript that is automatically downloaded by the user just by visiting
> a web page.
>

Yes, I'm sure some would.  I would argue that that most of those concerns
are properly the responsibility of the client-agent implementers.   In much
that same way that javascript has the same-origin restriction (enforced by
the client-agent), WS in the browser would have the same restriction.   A
malicious WS using script could only talk to the server it was downloaded
from.   I believe this exposes no more vulnerability than ajax does.  If
anyone has any specific examples to the contrary I would be open to hearing
them.

If a foul tempered client crafts a XmlHTTPRequest and attempts to access a
WS server (assuming we're not using HTTP Upgrade) - it will fail.
 XmlHTTPRequest is a HTTP request, not a WS frame.   Javascript clients
cannot create an XmlHTTPRequest that is a proper WS frame so any attempt to
perform a WS frame based negotiation/handshake is impossible.

If the WS server is being provided via a HTTP Upgrade mechanism, then yes,
some care may be needed - but that is part of "Using HTTP Upgrade to
Establish WebSocket"

And finally, if no other transport protocol takes extreme measures to
prevent tomfoolery with the content (other than what is necessary to get it
delivered, or reject it out of hand), I see no reason why WS should either.
   WS properly wraps content opaquely.     IP doesn't care what's in the TCP
it carries, TCP doesn't care what's in the HTTP it carries, and HTTP doesn't
care (it escapes!) what's in the content it carries.

Protecting the end user from malicious or incompetent implementations is the
responsibility of the application layer (ie. the browser's javascript's WS
object).   Protecting the server from evil & stupid clients is the
responsibility of the server implementation, not the transport protocol it
uses.

If I turn out to be wrong about my position, I pledge to consume the
requisite number of crows.

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 01 Apr 2011 13:20:31 -0700

On 01/04/2011 12:19 PM, David Endicott wrote:
>
>
>     Some would argue that this would break the security of WebSockets.
>     Right now, an XmlHTTPRequest can control all the bits after an
>     HTTP handshake has completed, but it cannot influence any header
>     that starts with "Sec-". This means that an attacker cannot
>     provide a malicious WebSocket client written in Javascript that is
>     automatically downloaded by the user just by visiting a web page.
>
>
> Yes, I'm sure some would.  I would argue that that most of those 
> concerns are properly the responsibility of the client-agent 
> implementers.   In much that same way that javascript has the 
> same-origin restriction (enforced by the client-agent), WS in the 
> browser would have the same restriction.   A malicious WS using script 
> could only talk to the server it was downloaded from.   I believe this 
> exposes no more vulnerability than ajax does.  If anyone has any 
> specific examples to the contrary I would be open to hearing them.

The reason I referred to others who would argue the point is that I am 
not an expert on browser security, so I bow to the knowledge of those 
who are. They have said that the security is not the same with 
WebSockets. See 
http://www.ietf.org/mail-archive/web/hybi/current/msg06134.html for an 
example. I'll leave it to others to argue more fully the specifics.

---

From: David Endicott <[email-masked]>
Date: Fri, 1 Apr 2011 16:42:54 -0400

On Fri, Apr 1, 2011 at 4:20 PM, Bruce Atherton <[email-masked]> wrote:

> ...   I believe this exposes no more vulnerability than ajax does.  If
> The reason I referred to others who would argue the point is that I am not
> an expert on browser security, so I bow to the knowledge of those who are.
> They have said that the security is not the same with WebSockets. See
Nor am I (hence the corvid consumption caveat)   It is very possible I have
overlooked or am unaware of something.    That notwithstanding, I do feel
fairly confident that doing WebSocket without a HTTP Upgrade handshake is no
less secure than with it - and may actually be safer.   Much of the jumping
around and Sec-Websocket headers seem to be  simply to deal with the
behavior of HTTP & XmlHTTPRequest.  Eliminate it from the WebSocket core
specification and it seems to me that several problems / design warts just
go away.

I look forward to input from those more knowledgeable on this topic.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 17:13:40 +0200

2011/4/1 David Endicott <[email-masked]>:
> Specifically I propose that the HTTP Upgrade based handshake be removed from
> the WebSocket specification and replaced with a Control frame handshake or
> negotiation.    This would mean that all WebSocket communication is done in
> websocket frames.

This makes lot of sense. In this way WebSocket would be a real
protocol rather than a mix between pseudo-HTTP and other protocol.

By looking like HTTP (during handshake process) WS gets into varios
issues this working group has been dealing with for long time. It also
doesn't take real advantages from HTTP world as response codes
different than 101 are not defined by the draft (it's like an amateur
PHP developer that just assumes things always work and never reacts on
possible errors).

Making WS a complete separate protocol would also involve not reusing
HTTP default ports (80 and 443). Why to reuse them? just because the
handshake mechanism (a hacked HTTP request/response) seems like HTTP?

Unfortunatelly I think it's too late for a new approach of the
WebSocket protocol, even if it would be really a much better design
(IMHO).

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Sun, 3 Apr 2011 11:30:29 -0400

> handshake mechanism (a hacked HTTP request/response) seems like HTTP?

Please read the many discussions about this topic.

Basically, if it uses dedicated ports and doesn't make use of existing HTTP
infrastructure, it was felt that it would not be sufficiently deployable to
displace existing mechanisms.

It is expected that the current approach over SSL will work in the majority
of cases (there are issues, particularly with proxies that don't support
CONNECT outside of SSL for non-SSL WebSockets), so that is a big advantage.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Andy Green <[email-masked]>
Date: Sun, 03 Apr 2011 16:32:20 +0100

> handshake mechanism (a hacked HTTP request/response) seems like HTTP?

The reason for reusing them is they can be the only way out in 
restrictively firewalled sites.  But it's not like it's defined to rely 
on being on 80 or 443, you can do what you like.

> WebSocket protocol, even if it would be really a much better design

It's not any new approach, you're just suggesting splitting off the 
handshake and the websocket transport definition into separate 
documents.  The end result is the same, who cares.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 17:44:16 +0200

2011/4/3 Andy Green <[email-masked]>:
> handshake and the websocket transport definition into separate documents.
>  The end result is the same, who cares.

Instead of a HTTP handshake, I just suggest (as the first mail in this
thread) the usage of a Websocket control frame to establish the WS
"connection".

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 17:54:47 +0200

> Please read the many discussions about this topic.

I've read them and understand motivations, but I cannot agree (just my opinion).

> CONNECT outside of SSL for non-SSL WebSockets), so that is a big advantage.

It "seems" that any new Internet protocol must be built on top of HTTP
or, at least, reuse ports 80 or 443. If not, it seems that "it could
not work".

Some day Internet providers (specially mobile data connection
providers) will close all the ports but HTTP ones and nobody will
complain because Facebook still works. IMHO it sad to assume that end
users are just able to contact port 80, and a new protocol should not
be built on top of such assumption.

Again, this is just my opinion, an opinion from somebody who doesn't
come from HTTP world.

Cheers.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Andy Green <[email-masked]>
Date: Sun, 03 Apr 2011 17:26:45 +0100

On 04/03/2011 04:44 PM, Somebody in the thread at some point said:
> 2011/4/3 Andy Green<[email-masked]>:

I assume this means running websocket on a different port altogether then.

Given your user can get your html + scripts on port 80 or 443 via http, 
it sounds like a really good bet that he can do more connections on port 
80 for the websocket ones too, leveraging specifically http semantics as 
the spec has it now.

It's a much poorer bet that because he was able to get your html scripts 
from 80 or 443, that he can do websocket on 1234.

I played with websockets being behind a really restrictive firewall and 
understood the value of re-using those magic ports first-hand.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 18:38:01 +0200

> understood the value of re-using those magic ports first-hand.

If WebSocket would use a separate port (i.e: 1234) and the protocol is
widely implemented, then those really restrictive firewalls would also
open port 1234, but using 80/443 from the beginning is like surronding
to bad admins.

If not, let use port 80 for everything (SMTP, SIP, IRC...). Oh wait,
the purpose of a port is to identify a service, is WS breaking this
sacred rule? IMHO yes.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Sun, 3 Apr 2011 12:43:04 -0400

> the purpose of a port is to identify a service, is WS breaking this

Those services aren't accessed by code downloaded over HTTP.  While
certainly there are non-HTTP uses for WebSockets, they aren't the primary
one as they could just use TCP directly.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 18:58:06 +0200

> one as they could just use TCP directly.

With the inclusion of Websocket I expect that many services could
offer an API so clients could use them directly via Websocket
protocol, which could be useful not just for web browsers, but also
for desktop application (those which today must poll data from a HTTP
server each N seconds/minutes). I would like to imagine a realtime RSS
service (with no periodical polling) for my RSS desktop client.

So, you are assuming that:

1) Just web browsers will use WebSocket protocol.

2) Web browsers just should speak with a port 80/443.

About point 2, how do you expect web browsers will be able -in a
future- to open a realtime communication (voice/video) with a server
(not just streaming media from server to client)? Will web browsers
also send client's audio/video to a port 80/443?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Sun, 3 Apr 2011 13:10:22 -0400

> 1) Just web browsers will use WebSocket protocol.

No, just that they are the primary use-case we care about.  Many of the
restrictions come about because we are giving the ability for third-party
code to make network connections from a user's browser.  If we didn't have
to support this use case, many things could be much simpler.

> also send client's audio/video to a port 80/443?

Since most audio/video streaming to a browser currently takes place over
HTTP despite alternatives, yes that seems likely.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sun, 3 Apr 2011 19:24:38 +0200

> HTTP despite alternatives, yes that seems likely.

So, in case I open a audio/video session with other person in my same
city via web, then the media should be sent over TCP to the WebServer
(which would be in other country) and come back to the other user. Are
you assuming it? that would lead to several seconds of latency!
Or maybe you are asuming that interactive media communication in a web
page just means downloading or uploading videos to Youtube?

Have you heard about RTP protocol integration into web browsers? there
are already efforts in standarizing it:

http://rtc-web.alvestrand.com/ietf-activity

-------------------------------------------------------------------------------------------
Name: Real-Time Communication in WEB-browsers (RTCWeb)

There are a number of proprietary implementations that provide direct
interactive rich communication using audio, video, collaboration,
games, etc. between two peers' web-browsers. These are not
interoperable, as they require non-standard extensions or plugins to
work.  There is a desire to standardize the basis for such
communication so that interoperable communication can be established
between any compatible browsers. The goal is to enable innovation on
top of a set of basic components.   One core component is to enable
real-time media like audio and video, a second is to enable datagram
and byte stream data transfer directly between clients.
-------------------------------------------------------------------------------------------

There is life out of HTTP world, and several protocol much better than
the old and limited HTTP. Please open your minds.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 4 Apr 2011 15:51:17 +1000

> thread) the usage of a Websocket control frame to establish the WS

I think we need to consider this proposal in two parts:

 1) Designing the handshake/protocol so that it could work well over a
dedicated port
 2) Actually deciding we want a dedicated port.

Note that this was one of the reasons that I'm an advocate of Hello
frames.  If we move much of the protocol parameterisation (eg sub
protocol and extension negotiation) into hello frames rather than HTTP
headers, then this is more portable and would work  WHEN AND IF we
ever decided to have a dedicated port for ws.

Hello frames can be done server->client->server to avoid a RTT's but
maybe it is better to have them client->server->client so that the
upgrade over HTTP would take one more RTT than a dedicated port and
thus give an incentive to push for opening a port.

So if there was interest at this late stage of moving some headers to
hello frames  - I'd be supportive of that.   But I think we are too
far down the upgrade path to change now and should keep HTTP upgrade
for at least the short term.

cheers

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 4 Apr 2011 02:01:22 -0400

Is there some reason you can't specify the field invovled in the handshake
independent of the medium carrying them? Then use this to initialize the
defaults of the dedicated WS framework handshake?

For instance, what's to prevent an XMPP or other transport from being used
if the protocol is designed with the proper degree of abstraction.

Timothy Meade
Real-time web developer
>>> handshake and the websocket transport definition into separate
documents.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 4 Apr 2011 08:07:29 +0200

> for at least the short term.

Agreed, and unfortunately, mandatory masking makes it harder to use
hello frames to negociate anything. Or we should decide that the masking
method can be negociated and defaults to XOR32 for client->server stream
over HTTP.

Willy

---

From: Andy Green <[email-masked]>
Date: Mon, 04 Apr 2011 09:26:18 +0100

> for at least the short term.

I'm scratching my head at the value of this compared to the existing 
http handshake on a random port.

Even if you use the websocket framing to do so, you will still have to 
do the dances like agree extensions, confirm it really talks websocket, 
look at protocol versioning, all the stuff that's already being done. 
The overhead of doing it like http is just the token names being given 
in ASCII?

-Andy

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 4 Apr 2011 11:27:25 +0300

(as a co-chair)

back in late December 2010 we run the Straw Poll:
"/is Upgrade with client->sever XOR mask acceptable/"

some references:
http://www.ietf.org/mail-archive/web/hybi/current/msg05378.html
http://www.ietf.org/mail-archive/web/hybi/current/msg05389.html
http://www.ietf.org/mail-archive/web/hybi/current/msg05426.html

we reached rough consensus on the proposal,
since then all the people involved in HyBi/WebSocket are working to 
standardize
and improve (especially from a security point of view) that approach

just to be clear
the wg was not able to reach any form of consensus on any other 
alternative proposals
(we had also one similar to this one), for a reason or another
(e.g. NPN has been put on hold until TLS wg will make some progress)
and *nothing let me think that something is changed since then*.

So we have to live with Upgrade.

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

On 4/1/11 5:54 AM, David Endicott wrote:
> While I am not interested in disrupting or derailing the progress 
> being made on finalizing the Websocket specification, the recent 
> discussions about the initial HTTP Upgrade handshake leads me to offer 
> up the following proposal that I believe may resolve the situation.
>
> I propose we drop the HTTP Upgrade element from the WebSocket 
> specification entirely.  I believe that the co-mingling of HTTP 
> Upgrade with WebSocket is a confusion of protocols and the two should 
> be more firmly separated.
>
> My justification is:
>
> WebSocket.   Further, WebSocket protocol servers may not be part of a 
> HTTP serving environment and having it answer HTTP (however limited) 
> could confuse client agents.
>
> 2. When WebSocket is provided as part of a HTTP server stack, having 
> to instantiate the WebSocket serving process (thread/handler/etc.) 
> just to resolve the handshake and determine if a WebSocket connection 
> will actually proceed is awkward, potentially expensive 
> server will support a HTTP Upgrade to any given protocol for a given 
> URL is properly the responsibility of the HTTP server, not WebSocket. 
>   The HTTP server must be able, via the Upgrade request to consider 
> only (a) the URL and (b) the Upgrade protocol and decide if it will 
> support it.   If it will, then all further negotiation is the 
> responsibility of the protocol being upgraded to.
>
> 3. If a HTTP server decides that it will allow an HTTP Upgrade to 
> WebSocket on the requested URL, the HTTP server can restrict the 
> WebSocket serving process to the docroot of that URL, thereby 
> maintaining consistency of the HTTP server's URL structure and 
> preventing the WebSocket server process from affecting any other part 
> of the filesystem as well as establishing a relative filesystem 
> context for the WebSocket serving process.   The WebSocket server 
> process should not be allowed to determine the validity, 
> accessibility, remapping, etc. of the requested URL.  That is the HTTP 
> server's responsibility.
>
> frame handshake or negotiation.    This would mean that all WebSocket 
> communication is done in websocket frames.   This simplifies the 
> implementation and keeps each part in its own protocol context.   It 
> strengthens the error checking between client-agents and the WebSocket 
> server.  For example, if a HTTP client-agent incorrectly makes a 
> request of a stand-alone WebSocket server, the client's HTTP request 
> will fail as a WebSocket frame and the server can drop the connection. 
>  The WebSocket server can further be assured that any establishing 
> connection that can provide valid frames is a client-agent that is 
> likely a conforming implementation.
>
> Personally I would prefer a negotiation control frame to be used as 
> the initial exchange.  It defines that capability setup is determined 
> at connection establishment but also allows the possibility of later 
> re-negotiation should the situation change.    Failing that, a simple 
> websocket based handshake would be fine.
>
> A note could be included on the use of the HTTP Upgrade by HTTP 
> servers as a mechanism to bootstrap a WebSocket connection, but this 
> is no different than it's use to boot-strap any other protocol. 
>   Separating the two would allow the implementer to decide if and how 
> they wish to deal the ramifications of using that mechanism.
>
>
> Again, let me reiterate that I have no interest in derailing this 
> working group, I have strong motivation to get a real 
> bidirectional asynchronous protocol into browser agents as soon as 
> possible, no matter how mangled the implementation.   I will take good 
> over perfect, and soon over good.
>
>
> tl;dr: HTTP is HTTP, WebSocket is WebSocket, do not mix.  Leverage.
>

---

From: David Endicott <[email-masked]>
Date: Mon, 4 Apr 2011 09:44:06 -0400

> I assume this means running websocket on a different port altogether then.

To clarify:  No.   I was only suggesting that WebSocket be divorced from
HTTP Upgrade as part of the specification.

Yes, I think in many many cases people will use HTTP Upgrade as the gateway
to establishing a WS connection.  That makes good sense for
many environments.   There are many good reasons to provide it over standard
HTTP ports.

But, there are also many good reasons to not *insist* that it is done that
way.    I don't believe WS is a sub-set of HTTP.   WS is a different
transport than HTTP and there are (should be) numerous other ways to get it
connected.    Many of us have deployment situations where the firewall
situation isn't a concern and we already have customers opening specific
ports for us.

I perceive HTTP and WS as apples and oranges^H^H^H^H pork chops...they work
well together but are different entities.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 01:10:38 +0100

> method can be negociated and defaults to XOR32 for client->server stream

The ostensible reason for masking is to mitigate security concerns
from browsers running downloaded scripts, taking advantage of
proxies/servers that treat the script-controlled bytes as HTTP.

That concern is valid for any byte stream following an HTTP UPGRADE
request, not just WebSocket, if the stream is controlled by a
downloaded script.

So it would make sense for the *initial* masking procedure, whatever
it is, to be at least mentioned in any generic "safe HTTP UPGRADE"
specification - irrespective of whether the following data is
WebSocket or something else.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 01:25:36 +0100

> thus give an incentive to push for opening a port.

To reduce RTTs, I would have the protocol following HTTP Upgrade be
allowed to _inherit_ initial bytes encoded in one or more designated
HTTP headers from the HTTP Upgrade request.  Both request and response
could contain such "pre-transmitted" bytes.

For simplicity, they would be a standard encoding of bytes to follow,
such that they have exactly the same meaning as if those bytes were
transmitted in both directions following a successful HTTP Upgrade, or
as if they were transmitted from the start, when used on a dedicated
port without a HTTP Upgrade handshake.

There'd be no need to define them beyond that - the protocol spec
would already define what they mean, as normal protocol or hello frames.

The RTT-removing property is analogous to TCP SYN+data (a subset of
T/TCP, which didn't work out due to security and interop problems),
and to something I'm only vaguely familiar with that may happen in TLS.

If the HTTP Upgrade part were separated into its own document, the
encoding of "pre-transmitted" bytes would be specified in that,
generically, with the proviso that the feature is only used by
protocols which say they use those bytes, and specify which headers
they get them from.

> for at least the short term.

I agree, it is too late for this round.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 01:46:44 +0100

David Endicott wrote:
>    Nor am I (hence the corvid consumption caveat)   It is very possible I
>    have overlooked or am unaware of something.    That notwithstanding, I
>    do feel fairly confident that doing WebSocket without a HTTP Upgrade
>    handshake is no less secure than with it - and may actually be safer.
>      Much of the jumping around and Sec-Websocket headers seem to be
>    simply to deal with the behavior of HTTP & XmlHTTPRequest.  Eliminate
>    it from the WebSocket core specification and it seems to me that
>    several problems / design warts just go away.

The HTTP part comes from the desire to run over port 80, due to
firewall restrictions on all other ports.

Then that means a WebSocket connection may be connecting over an
transparent/intercepting HTTP proxy - that cannot be avoided, because
they intercept port 80 traffic whether you want it or not.

Some of those intercepting proxies are HTTP caches - whether you want
it or not, you can't avoid it.

Downloaded Javascript, in the browser, is not written by the user
themselves, and in general trust must be carefully limited.  It must
be restricted in the effects it can have, especially effects on web
domains different from where the script came from, or was linked from.

Many security concerns aren't about absolutely preventing a possible
attack from a single, specially written client, or from a specially
written server that someone visits.  They are about limiting the scope
of damage on a population of users, particularly the damage caused by
a malicious script that is downloaded and run by large numbers of
unsuspecting users through cross-site scripting attacks, misleading
sites and similar, and doubly especially damage that allows one site
to spoof what users get when downloading from a different (innocent)
site, which allows web virus/worm behaviour, and breaks the
assumptions of existing security mechanisms.

So it's important that downloaded Javascript can't use WebSocket to
send controlled bytes that look enough like HTTP that they can
poison a HTTP proxy's cache or HTTP framing, especially not that
could make the browser show spoofed pages on _other_ web sites
than where the script came from.

There is some debate about whether this attack is realistic or not.

I'm not sure if the reverse matters (mentioned earlier in this
thread): downloaded Javascript using XmlHttpRequest to send controlled
bytes (after the HTTP request headers) that look enough like WebSocket
to make a server respond to that.

None of this would apply if port 80 were not used, but it's regarded
as an important use.  (Though one test suggested port 80 is not
actually much better for WebSockets than another port, and may be more
prone to problems.)

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 02:03:03 +0100

David Endicott wrote:
>    [...]

Javascript doesn't handle binary (at the moment), only text and
integer arrays.  A later version will support byte arrays.

So the Javascript WebSocket API needs to know what type to give your
application when a frame is received - turn it into a text string, or
something else.

Without the distinction on the wire, it would have to give you integer
arrays (in current Javascript) every time.  Because it wouldn't know
when your byte sequences are to be interpreted as text.  That would be
very inconvenient.

They could have designed the API so that _you_ ask for the message in
whatever format you require, _after_ getting the message event.

Like how XmlHttpRequest does it...

(Aside: Or auto-formatted the payload to a string or array, when it is
used in the script - but that would need a non-standard Javascript
extension.)

That would have saved a bit on the wire, reused an existing pattern, and
been more extensible for future frame interpretations.

But that's not how the API was designed, so the protocol must
assist the API as it is.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 02:16:39 +0100

David Endicott wrote:
>    And don't get me started on the magic keys (Sec-WebSocket-Key).
>    Personally I believe this part of the handshake to be useless
>    hand-waving that does not provide what it's supposedly intended to do.
>       The client can generate any value and ignore the server's response,
>    it proves nothing.

On the contrary, as that mechanism is intended to limit the effect of
attacks via browsers running downloaded malicious scripts, and all
browsers will check the server's response, it cannot be subverted in
the way you describe.

The attack vector is entirely due to a very large population of users
who download and run scripts which can be malicious and whose effects
have to be carefully limited to provide a more secure web.

It is not there to protect non-browser clients from anything, or
servers from non-browser clients doing whatever thay want.

If someone writes a client which downloads scripts and runs them, or
does the same attacks itself, yes that could attack the same web
infrastructure, but not at the same scale because you can't muster
millions of your custom clients so easily, and not necessarily
affecting the places (esp. HTTP caches used by your targets) that you
can reach with a browser attack.

(Like you I'm not entirely convinced that it's all that great a
mechanism, but let's at least understand what it is for.)

-- Jamie

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Apr 2011 03:20:11 +0200

> actually much better for WebSockets than another port, and may be more

A TCP/UDP port identifies a service, so reusing port 80 for another
different protocol is a hack (IMHO). Thinking that operators and
firewalls just allow outoing traffic to port 80/443 is giving them
reasons to continue with such absurd policy.

Designing a new protocol from scratch, with any kind of framing or
codification (so a JavaScript code could not generate "valid HTTP
requests" to poison a server) would solve most of the artificial
issues appearing due to the usage of port 80 and the "like-HTTP"
handshake. No HTTP proxies having to deal with WebSocket protocol,
neither firewalls observing strange TCP traffic through the well known
port 80 (which could seem an anomaly). Just a clean new protocol
designed from scratch and running in a separate port, as any other
network protocol.

But I expect it's too late to change it or discuss about it. Just my opinion.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 7 Apr 2011 02:59:32 +0100

> reasons to continue with such absurd policy.

You shouldn't assume they are *deliberately* blocking all other ports.

In some environments, ports 80/443 work because special forwarding has
been *installed* for those ports, and there is no normal route to the
internet.  For example in some environments, clients have no
connection to the internet at all, and port 80 works because an HTTP
proxy gets all port 80 traffic and has its own, separate access to the
internet, with either a browser-proxy-configuration script distributed
to all browsers, or DNS pointing every non-local domain to that proxy,
or an intercepting proxy as the default route.

In those environments, "enabling" other ports is not a simple matter
of removing port restrictions.

Other than that: one of WebSocket's goals is to work in environments
where HTTP works, to displace existing HTTP long-polling hacks with
something a bit better.

Also, to be fair, WebSocket does participate in the "web security
model" (= restrictions on what scripts/HTML can do), so it has to some
extent earned the right to be allowed in environments that trust "web
browsing" (because they know browsers implement a security model and
are heavily audited), and don't trust "arbitrary clients".

It doesn't make that much sense, because many clients use port 80 and
443 for all sorts of random protocols these days.  For better or
worse, the "absurd policy" is likely to continue for some time.

-- Jamie

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 7 Apr 2011 07:47:03 +0200

> could contain such "pre-transmitted" bytes.

+1, I like this idea a lot.

We could even exchange small messages within a single RTT :
   HTTP request + small request + closing handshake
   HTTP response + small response + closing handshake

That would be similar to HTTP GET+200 with short-lived connections
but adapted to some services offered over WS.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 7 Apr 2011 07:50:59 +0200

> designed from scratch and running in a separate port, as any other

And conversely, it can work *because* some forwarding service, URL
filtering etc... is installed on these ports only.

Why do you think that previous attempts to make HTTP bidirectional were
also running on port 80, why didn't they simply pick any random port
instead ? Simply because this one only works.

If we defined a specific port *and* decided to abandon the HTTP upgrade,
I predict that WS would not replace the other technologies that already
work.

Regards,
Willy

---

From: David Endicott <[email-masked]>
Date: Thu, 7 Apr 2011 09:48:35 -0400

> browsers will check the server's response, it cannot be subverted in

Yes, but...all it provides is a method for a browser to check if the server
is doing WS.   Which, imho, is functionally useless, especially when paired
with HTTP Upgrade: WebSocket in the opening handshake.   The server accepts
the upgrade to WS or it doesn't.  A secondary mechanism to confirm that
is...redundantly useless.

A malicious script trying to...

a) attack WS - is going to be happy the browser confirmed it
b) attack HTTP using WS - won't care,
c) attack WS using crafted HTTP- will create bogus Key and ignore server's
response
d) attack HTTP using crafted WS - it's a HTTP Upgrade and Key again doesn't
matter.

A non-browser client-agent can, of course, generate anything it wants and
all bets are off.

> have to be carefully limited to provide a more secure web.

The same-origin policy is a much better mechanism.   Scripts should only be
trusted to talk with the servers they came from.

> Ya, I understood the motivation.   I didn't understand why anyone thought
it actually helped.

But, it's a non-impediment and harmless, so I'm going to let it slide.
Sometimes it's nice to leave a little WTF for future generations.

---

From: David Endicott <[email-masked]>
Date: Thu, 7 Apr 2011 09:56:24 -0400

On Wed, Apr 6, 2011 at 9:03 PM, Jamie Lokier <[email-masked]> wrote:

> >      So the browser/user-agent knows what to do with it?  AFAIK, that
>

> application when a frame is received - turn it into a text string, or

Javascript strings can hold any byte values, yes?    We could have delivered
the WS frame payload as a JS string and be done with it.   It would be up to
the application to know if that's UTF-8 or ASCII or raw binary or whatever.

>  Like how XmlHttpRequest does it...

Ya, that might have been nice.

> But that's not how the API was designed, so the protocol must

I don't have a significant concern about the frame data typing.

It makes excellent practical sense to have a TEXT frame for the majority of
use-cases where it's passing text around (HTML,JSON, etc.).    And a BINARY
frame for the odd folks who need to do something different.   It is a
practically simpler mechanism than incorporating a whole Content-Type
system.

I suppose you could specify Content-Type on the HTTP Upgrade, but I'd rather
have a per-message content specification, not a entire-stream one.

---

From: John Tamplin <[email-masked]>
Date: Thu, 7 Apr 2011 10:45:15 -0400

On Thu, Apr 7, 2011 at 9:56 AM, David Endicott <[email-masked]> wrote:

> Javascript strings can hold any byte values, yes?    We could have
> delivered the WS frame payload as a JS string and be done with it.   It
> would be up to the application to know if that's UTF-8 or ASCII or raw
> binary or whatever.
>

No, Javascript strings are sequences of UTF16 characters.  Even subject to
that restriction, several browsers munge the data you give it if it includes
combining marks or unpaired surrogates, so you can't just pass arbitrary
data in a JS string and expect it to be unmodified.

---

From: David Endicott <[email-masked]>
Date: Thu, 7 Apr 2011 11:17:54 -0400

On Thu, Apr 7, 2011 at 10:45 AM, John Tamplin <[email-masked]> wrote:

> On Thu, Apr 7, 2011 at 9:56 AM, David Endicott <[email-masked]>wrote:
>

That's what I thought.    I had noticed that any pseudo-binary that I had
turned into a JS string was making it thru ok.    It's very likely I just
hadn't generated any combination that triggered munging.  Nonetheless,
stuffing arbitrary binary into a UTF-16, and expecting the other side to
properly decode it is a Bad Idea.

What the Text/Binary frames lack in conceptual purity are *more* than made
up for in real-world practicality.

---

From: Brian <[email-masked]>
Date: Thu, 7 Apr 2011 10:04:28 -0700

While manipulating binary data in JavaScript is inconvenient, it is not true
to say that it is not possible to represent binary data cleanly as a string.
 See
https://developer.mozilla.org/En/Using_XMLHttpRequest#Handling_binary_datafor
example.  I've successfully used that interface to write a full ABC
(ActionScript 3 Byte Code) disassembler in JavaScript.  I can tell you that
it does work.

var aByte = inputString.charCodeAt(2) & 0xFF;

It may be reasonable to hold out for a better native mechanism for handling
binary data to be added to JavaScript, but I'm not yet aware of any such
thing in the ES5 spec (if there is something on the horizon please let me
know!)  In any case, JavaScript *can* correctly and reliably interpret
binary messages as a string -- no question about it.

Brian

On Apr 7, 2011, at 8:17 AM, David Endicott <[email-masked]> wrote:

On Thu, Apr 7, 2011 at 10:45 AM, John Tamplin < <[email-masked]>
[email-masked]> wrote:

> On Thu, Apr 7, 2011 at 9:56 AM, David Endicott < <[email-masked]>
> [email-masked]> wrote:
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 7 Apr 2011 13:20:17 -0400

On Thu, Apr 7, 2011 at 1:04 PM, Brian <[email-masked]> wrote:

> While manipulating binary data in JavaScript is inconvenient, it is not
> true to say that it is not possible to represent binary data cleanly as a
> string.  See
> https://developer.mozilla.org/En/Using_XMLHttpRequest#Handling_binary_datafor example.  I've successfully used that interface to write a full ABC

Sure, but that is encoding bytes 0x00-0xFF as characters 0-255, which means
the upper half of these bytes actually takes two bytes on the wire using
UTF8.  GWT quake does the same thing to send Quake's binary message formats
(involving lots of floating point numbers) via WebSocket using the text API.

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 7 Apr 2011 13:23:04 -0400

There are mechanisms in modern browsers or coming soon for binary data in
javascript, such as TypedArray (from WebGL). The question I think is not if
binary support is coming, but in what form.

Quoted:
Compatibility Typed arrays are available in WebKit as well. Chrome 7
includes support for ArrayBuffer,Float32Array,Int16Array, andUint8Array.
Chrome 9 adds support forDataView objects.

https://developer.mozilla.org/en/JavaScript_typed_arrays
Uses Uint8Array

--
Timothy Meade
real -time web developer
> While manipulating binary data in JavaScript is inconvenient, it is not
true

---

From: Brian McKelvey <[email-masked]>
Date: Thu, 7 Apr 2011 11:45:16 -0700

With UTF-8 it only takes two bytes for values over 127.  And it doesn't take two bytes on the wire at all if you use a binary message frame type, which I thought was what we were talking about.  The browser would be responsible for converting it to an appropriately encoded string for binary processing.

Brian

Sent from my iPhone

On Apr 7, 2011, at 10:20 AM, John Tamplin <[email-masked]> wrote:

> While manipulating binary data in JavaScript is inconvenient, it is not true to say that it is not possible to represent binary data cleanly as a string.  See https://developer.mozilla.org/En/Using_XMLHttpRequest#Handling_binary_data for example.  I've successfully used that interface to write a full ABC (ActionScript 3 Byte Code) disassembler in JavaScript.  I can tell you that it does work.
> 
> Sure, but that is encoding bytes 0x00-0xFF as characters 0-255, which means the upper half of these bytes actually takes two bytes on the wire using UTF8.  GWT quake does the same thing to send Quake's binary message formats (involving lots of floating point numbers) via WebSocket using the text API.
> 
> -- 
> John A. Tamplin

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Apr 2011 22:59:44 +0200

2011/4/7 David Endicott <[email-masked]>:
> Scripts should only be trusted to talk with the servers they came from.

Please not, that assumes that the HTTP server must also speak
websocket (and the subprotocol over it). Let's design scalable systems
please.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: David Endicott <[email-masked]>
Date: Thu, 7 Apr 2011 17:58:11 -0400

Right, sorry.   I misspoke.

 I meant to say, "scripts should only be trusted to talk with the *domains*
they came from".

(or subdomains, of course)

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 8 Apr 2011 08:51:01 +1000

> (or subdomains, of course)

or with domains that have been checked with a CORS preflight check?

http://www.w3.org/TR/cors/

---

From: Brian McKelvey <[email-masked]>
Date: Thu, 7 Apr 2011 17:38:34 -0700

I see nothing wrong with the current model.  The server declares whether or not connections should be accepted from a given origin.  If you want to clamp that down to one domain for your server, that's very easy to do.  But for others that want to enable a broadly used API (Facebook for example) it's a huge boon to be able to optionally allow connections from any origin.  The current websocket model follows from the widely deployed and very successful socket policy file approach pioneered by flash, but without the extra round trip.

Brian

Sent from my iPhone

---

From: David Endicott <[email-masked]>
Date: Fri, 8 Apr 2011 00:33:25 -0400

Alright, now I'm confused.

(ignoring CORS et. al. for the moment)

If I had a website on foo.com and it served a browser client some Javascript
and that JS attempted a XmlHttp (or whatever) to facebook.com, it fails,
yes?

But, if the browser hits facebook.com and is served the Javascript from
there, then it would succeed, yes?

Doesn't/Shouldn't WS work within similar restrictions?

On Thu, Apr 7, 2011 at 8:38 PM, Brian McKelvey <[email-masked]>wrote:

> I see nothing wrong with the current model.  The server declares whether or
> not connections should be accepted from a given origin.  If you want to
> clamp that down to one domain for your server, that's very easy to do.  But
> for others that want to enable a broadly used API (Facebook for example)
> it's a huge boon to be able to optionally allow connections from any origin.
>  The current websocket model follows from the widely deployed and very
> successful socket policy file approach pioneered by flash, but without the
> extra round trip.
>
> Brian
>
> Sent from my iPhone
>

---

From: Patrick McManus <[email-masked]>
Date: Thu, 07 Apr 2011 22:08:37 -0700

David,

> (ignoring CORS et. al. for the moment)

you simply cannot ignore CORS - it is what is relevant here.

>
> If I had a website on foo.com <http://foo.com> and it served a browser 
> client some Javascript and that JS attempted a XmlHttp (or whatever) 
> to facebook.com <http://facebook.com>, it fails, yes?
>
it fails the same-origin policy for javascript, but that xhr can still 
succeed if the cors requirements are met. cors requires the server to 
explicitly opt-in based on the origin information in the request. in 
your example the request to facebook.com would have origin information 
indicating foo.com and the server can decide whether or not that is a 
relationship it expects to see.
> Doesn't/Shouldn't WS work within similar restrictions?
The WS handshake uses a mechanism like CORS - it requires the server to 
opt in to the websocket connection given the origin information before 
any WS data is exchanged. That is the purpose of the client presenting a 
random token along with the origin information and the server performing 
a hash on that token and returning the result. This shows that the 
server has opted into connections from that origin. We know this because 
the websockets specification defines it and performing the esoteric hash 
proves the server knows websockets in particular, and not just http in 
general.

To the working group: Let's stop going in circles and publish this darn 
protocol already and get some real experience with it.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 8 Apr 2011 15:50:04 +1000

> protocol already and get some real experience with it.

There is nothing preventing us shipping implementations that support
the current draft and getting real experience with it.

The currently released jetty server supports -76, -00, -01 and -06
drafts.   The cometd framework uses that was was getting some real
usage until such time as the browsers turned WS off. When -07 is
shipped, we'll support that as well.

I'd much rather we widely deployed -07,-08,... and then with
experience  obtained we can finalise the document, rather than the
other way around.

cheers

---

From: David Endicott <[email-masked]>
Date: Fri, 8 Apr 2011 09:18:43 -0400

> you simply cannot ignore CORS - it is what is relevant here.

What I meant by "ignoring CORS" was only that I was considering the case
where it wasn't specified (it still being a working draft after all).
 However, you're entirely right that it does provide a mechanism to deal
with cross-origin resources.

The WS handshake uses a mechanism like CORS - it requires the server to opt
> in to the websocket connection given the origin information before any WS
> data is exchanged.
>

"like CORS"  - hence my confusion.   I was under the impression that the
Javascript on the browser would enforce same-origin and WS would never get a
chance to "opt-in".  It would be prevented from establishing the connection
in the first place.

That is the purpose of the client presenting a random token along with the
> origin information and the server performing a hash on that token and
> returning the result. This shows that the server has opted into connections
> from that origin.
> We know this because the websockets specification defines it and performing
> the esoteric hash proves the server knows websockets in particular, and not
> just http in general.
>

Not really.   Responding "101 Switching Protocols" along with the websocket
headers shows that it has accepted the connection and it's using WS.    The
esoteric hash is irrelevant redundant hand-wavery.

> protocol already and get some real experience with it.

Totally agreed.   More people need to be seeing what happens in real-world
implementations so we can have a better understanding & discussion of the
implications.   But a caveat: sometimes going in circles means you haven't
found a good solution.  Perhaps there is still debate because some of WS is
still questionable.   The longer we leave it to "fix" something, the more
painful it's going to be.

---

From: David Endicott <[email-masked]>
Date: Fri, 8 Apr 2011 09:24:49 -0400

> experience  obtained we can finalise the document, rather than the
Strongly agreed.   Actually having an implementation in context with
intended use-cases has been....enlightening.

Note: my implementations are currently supporting -75, -00, -04, & -05.
 I'm going to add the most recent "real-soon-now".    And once I've cleared
some bureaucracy at work, I'm putting it up for public consumption.   They
are python based if anyone is interested.