Subject: [hybi] Websocket: two protocols into one, and Internet rules broken

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 16 Jun 2011 15:06:30 +0200

Hi, WebSocket is, most probably, the first protocol in the History of
Internet which is, in fact, two protocols over the same stream
(swtiching from HTTP to WebSocket "pure" binary protocol). As an
author of HTTP commented in this maillist, that is not the aim of HTTP
101 response.

So first, WebSocket uses HTTP for the handshake. This is an extension
to HTTP, of course, and as an extension to HTTP it MUST define HTTP
response codes for certain cases not covered by HTTP itself, but just
by this specification (just talking about HTTP for now). The draft
seems not to want to define such codes:

----------------
   The |Sec-WebSocket-Origin| header is used to protect against
   unauthorized cross-origin use of a WebSocket server by scripts using
   the |WebSocket| API in a Web browser.  The server is informed of the
   script origin generating the WebSocket connection request.  If the
   server does not wish to accept connections from this origin, it can
   choose to reject the connection by sending an appropriate HTTP error
   code.
----------------

Which "appropriate HTTP error code"? Please, define it.

What about if the HTTP handshake request contains an invalid
Sec-WebSocket-Key header? what about if the HTTP request misses some
required header? which HTTP status code should be returned by the
server? Please define it.

In the other side, as I said above WebSocket is the first protocol
that uses two different application protocols (HTTP and WebSocket
binary protocol) within the same transport stream (a TCP connection).
Please don't confuse it with SOAP over HTTP, JSONRPC over HTTP,
anything over HTTP or Flash protocol (RTMP) which is a *single*
protocol over TCP and such protocol contains different payloads for
different application streams (signalling, audio, video...).

The reason this WG gives is that "we want WebSocket to work in any
environment in which HTTP just works". So it assumes that Internet is
broken and HTTP is the only working protocol (or just the only valid
protocol). So, the only way to make a webbrowser to speak a protocol
different than HTTP is by opening first a TCP connection, send a HTTP
request and then "switch" to other protocol. Ugly assumption IMHO.

Have you taken a look to rtc-web? http://rtc-web.alvestrand.com/

  ---------------------
  The RTC-Web effort (Real Time Collaboration on the World Wide Web)
is an effort
  to achieve a standardized infrastructure in Web browsers on which real-time
  interactive communication between users of the World Wide Web can be achieved.
  -----------------------

RTC-Web's aim is to provide realtime communication capabilities to web
browsers, so they could speak other protocols as RTP for audio/video
sessions. Of course I mean pure RTP and not "RTP over HTTP" neither
"first HTTP and then RTP" (following WebSocket fashion).

Now let's imagine we want web browsers to become real SIP or XMPP
clients (I don't mean SIP or XMPP over WebSocket). Do you expect that
the browser will first open a TCP connection, then send an HTTP
"handshake" and then switch same TCP stream to speak SIP or XMPP? and
all this stuff just because "HTTP is the only protocol in the world"?

So, from my point of view, WebSocket breaks Internet model and their
authors should ask themself "who are us to make such aberration in the
History of Internet protocols"?

Please, don't take me wrong. I also assume that there is no way to
rebuild WebSocket with a better design as it would require designing
it from the scratch. So at least, please complete the HTTP part of the
protocol by defining specific HTTP response codes for certain
situations that WebSocket creates (as it is also an extension to HTTP
protocol).

For example, in SIP protocol there are even more SIP response codes
than in HTTP. If a new extension for SIP appears, it may need to
create new response codes or, at least, specify which exact SIP
response codes the server should reply in certain cases (specific to
the given extension). IMHO WebSocket should do the same instead of
trying to avoid responsabilities at HTTP level (since it *does* speak
HTTP).

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Anthony Catel <[email-masked]>
Date: Thu, 16 Jun 2011 15:44:05 +0200

"101 Switching Protocols
The server understands and is willing to comply with the client's 
request, via the Upgrade message header field (section 14.42), for a 
change in the application protocol being used on this connection. The 
server will switch protocols to those defined by the response's Upgrade 
header field immediately after the empty line which terminates the 101 
response.
The protocol SHOULD be switched only when it is advantageous to do so. 
For example, switching to a newer version of HTTP is advantageous over 
older versions, and switching to a real-time, synchronous protocol might 
be advantageous when delivering resources that use such features."

HTTP doesn't define that a binary protocol should not be used. It also 
doesn't define that "swithing protocols" means "must switch between 
different HTTP version".

IMHO, WebSocket *needs* to be deployed in the wild as soon as possible 
and understood by a maximum of proxy/firewall.

Ok, but how do you handle origin security? By writting a SIP extension? 
And thus for every new protocol you want to handle?

Browsers were designed to speak HTTP. IMHO, if you break this rule, it 
leaves the doors wide open to any kind of crazy stuff.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 16 Jun 2011 16:09:39 +0200

2011/6/16 Anthony Catel <[email-masked]>:
> "101 Switching Protocols
> The server understands and is willing to comply with the client's request,
> via the Upgrade message header field (section 14.42), for a change in the
> application protocol being used on this connection. The server will switch
> protocols to those defined by the response's Upgrade header field
> immediately after the empty line which terminates the 101 response.
> The protocol SHOULD be switched only when it is advantageous to do so. For
> example, switching to a newer version of HTTP is advantageous over older
> versions, and switching to a real-time, synchronous protocol might be
> advantageous when delivering resources that use such features."
>

Ok, RFC 2616 said that 21 years ago. That does not mean that it's
suitable. For me is just a vague specification that nobody cared too
much in 1999.

> Ok, but how do you handle origin security? By writting a SIP extension? And
> thus for every new protocol you want to handle?

Security would be handled at SIP or XMPP level. I dont' understand the
problem. If a web browser becomes a real SIP or XMPP client it could
handle authentication as stated for SIP/XMPP and prompt the human user
when required (as when a web server replies 401).

> leaves the doors wide open to any kind of crazy stuff.

Web-browsers must also speak DNS protocol (directly or indirectly, it
does not matter). If a new protocol is standarized for web browsers,
which is the problem? what about Flash or Java applets? they can open
TCP/UDP connections from the webbrowser to make usage of any other
protocol. Is it a risk? of course, so let's do standards and mandate
security mechanims. But that has nothing to do with "just use HTTP".

Note that web browsers were originally designed to speak HTTP and
render HTML pages. But now everybody wants IM, video, audio and
whatever through a web browser. There is no need to implement all
these new requeriments on top of HTTP. Maybe HTTP is not a good
protocol for that! there are other protocols.

Also, take into account that, after the handshake WebSocket is no
longer HTTP, so it *is* a different protocol. It doesn't matter what
browsers were originally designed to.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Anthony Catel <[email-masked]>
Date: Thu, 16 Jun 2011 16:21:39 +0200

> when required (as when a web server replies 401).

I mean, it the browser can open a "raw" TCP connection and implement any 
kind of protocol.
This must lead to a prompt "Do you allow your browser to open a 
connection to xxxx:xxx?" which I think it's not suitable for user 
experience.

> security mechanims. But that has nothing to do with "just use HTTP".
No that's not true. For instance flash use HTTP to ask for a 
"crossdomain.xml"

> browsers were originally designed to.

Maybe but intermediate think it's HTTP :)

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 16 Jun 2011 16:28:24 +0200

2011/6/16 Anthony Catel <[email-masked]>:
> This must lead to a prompt "Do you allow your browser to open a connection
> to xxxx:xxx?" which I think it's not suitable for user experience.

I never said that a web browser should allow opening any kind of
communication with a remote server :)

I just said that, if it's standarized, why not to allow a web browser
to directly speak other protocols as SIP or XMPP? I don't mean raw
speaking such protocol from JavaScript or whatever, that should be not
allowed as per security reasons. I mean that web browsers could
implement a SIP and/or XMPP client and provide an API (i.e. for
JavaScript) to use it (the very same as WebSocket proposes). Security
would be built-in within the browser implementation.

PS: I don't want web browsers implementing SIP or XMPP, it was just an
example ;)

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 16 Jun 2011 16:40:20 +0200

> PS: I don't want web browsers implementing SIP or XMPP, it was just an

Those subjects were discussed to great extent last year. One of the points
was to reuse existing infrastructure (filtering, proxying, load balancing).
Another one was that a protocol relying on existing infrastructure and
policy rules will get faster adoption than one which requires new ports
to be qualified then opened at many places for the protocol to work. Many
of us are used to work at places where only 80 is opened via proxies, 443
is on a white-list and nothing else is allowed. In such environments, XHR
has been working well precisely because it did not require revisiting
established policies.

Regards,
Willy

---

From: Anthony Catel <[email-masked]>
Date: Thu, 16 Jun 2011 16:40:41 +0200

Le 16/06/2011 16:28, Iñaki Baz Castillo a écrit :
> 2011/6/16 Anthony Catel<[email-masked]>:

Nobody said that one day a vendor isn't going to implement a wide 
deployed protocol.
But for now, I think that WebSocket is a good choice to stop the cancer 
of "long polling"
and other ugly hacks (which is what I do).

Also, WebSocket is more or less a generic protocol where you can 
encapsulate other protocol
without having to deal boring things like Origin security, 
proxy/firewall traversal, etc...

---

From: Joel Martin <[email-masked]>
Date: Thu, 16 Jun 2011 10:54:57 -0500

I started to try and answer this StackOverflow question about the protocol
text and realized that I didn't understand it as well as I thought:
http://stackoverflow.com/questions/6372252/why-does-the-server-in-a-websocket-request-have-to-answer-a-challenge

The question is related to this paragraph (which has been in the drafts for
a while unchanged):

    Finally, the server has to prove to the client that it received the
    client's WebSocket handshake, so that the server doesn't accept
    connections that are not WebSocket connections.  This prevents an
    attacker from tricking a WebSocket server by sending it carefully-
    crafted packets using |XMLHttpRequest| or a |form| submission.

I can see how the Sec-WebSocket-Accept would prevent a WebSocket client from
being tricked into connecting to a non-WebSocket server. However, I'm having
difficulty understanding how the Sec-WebSocket-Accept header would prevent a
XMLHttpRequest from succeeding to a WebSockets server since it is sent from
the server. I am aware of the prohibition against "Sec-" headers in the
client to server direction. Is there a requirement that XMLHttpRequest
responses with unrecognized "Sec-" headers be rejected by user agents (and
if so does this only apply in the browser case)?

It's likely I just don't have enough context to understand the paragraph,
but perhaps it could be clarified a bit.

Regards,

Joel Martin

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 16 Jun 2011 09:49:56 -0700

Sec-WebSocket-Accept uses a GUID defined in the protocol and the key from
the handshake to prove that the remote end is indeed a websocket server as
you point out. The other part is that Sec-WebSocket-Accept takes the key
from the Sec-WebSocket-Key header, which XHR cannot send and thus the server
doesn't have the information necessary to generate the Accept value as there
is no key from the client.

The original question you link to is basically "why does the server have to
answer a challenge", and it breaks down into two parts:

1. Force server to validate that the client is a WebSocket client (by making
it use a value from Sec-WebSocket-Key along with the GUID, we ensure that
the server actually checks for the presence of Sec-WebSocket-Key.)
2. Let the client validate that the server is actually a WebSocket server
(by checking that Sec-WebSocket-Accept is properly computed).

#1 lets the server be sure it's not responding to some random XHR request,
#2 lets the browser know that it's safe to connect and that it's
communicating to a WebSocket server and not some non-WS target.

On Thu, Jun 16, 2011 at 8:54 AM, Joel Martin <[email-masked]> wrote:

> I can see how the Sec-WebSocket-Accept would prevent a WebSocket client
> from being tricked into connecting to a non-WebSocket server. However, I'm
> having difficulty understanding how the Sec-WebSocket-Accept header would
> prevent a XMLHttpRequest from succeeding to a WebSockets server since it is
> sent from the server. I am aware of the prohibition against "Sec-" headers
> in the client to server direction. Is there a requirement that
> XMLHttpRequest responses with unrecognized "Sec-" headers be rejected by
> user agents (and if so does this only apply in the browser case)?
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Joel Martin <[email-masked]>
Date: Thu, 16 Jun 2011 15:22:41 -0500

Ian,

I should have been more clear. I understand why the client and server have
to validate each other, it's the text of the paragraph in the draft that I'm
confused about and I posted here because I think the draft text at least
needs clarification.

    "Finally, the server has to prove to the client that it received the
    client's WebSocket handshake, ..."

- Good so far

    "...so that the server doesn't accept connections that are not
    WebSocket connections."

- What does proving to the client have to do with the _server_ refusing
non-websockets connections? Perhaps this is supposed to be "...so that the
client will not procede with connections to non-WebSocket servers."

    "This prevents an attacker from tricking a WebSocket server by sending
    it carefully-crafted packets using |XMLHttpRequest| or a |form|
submission."

- Again, what does proving to the client have to do with tricking the
WebSocket server? Maybe this should really be "This prevents an attacker
from tricking a non-WebSocket server by sending it carefully-crafted
WebSocket packets because a non-WebSocket server will not return a correct
Sec-WebSocket-Accept value during the handshake".

Regards,

Joel Martin

2011/6/16 Ian Fette (イアンフェッティ) <[email-masked]>

> 1. Force server to validate that the client is a WebSocket client (by
> making it use a value from Sec-WebSocket-Key along with the GUID, we ensure
> that the server actually checks for the presence of Sec-WebSocket-Key.)
>> The question is related to this paragraph (which has been in the drafts
>> for a while unchanged):
>>

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 16 Jun 2011 13:39:35 -0700

Historically, many servers are lazy. They will not bother validating
whatever the client sends, and will just return some value and then get
exploited. By forcing the server to prove something to the client, we
essentially also force the server to validate at least part of the client's
handshake (rather than just hardcoding in some 101 Upgrade WS response). So,
by making the server prove something to the client, it's a way of forcing
the server to actually take some steps that have a side effect of protecting
it (e.g. this /forces/ the server to parse the Sec-WebSocket-Key header.)

2011/6/16 Joel Martin <[email-masked]>

>> The original question you link to is basically "why does the server have
>> to answer a challenge", and it breaks down into two parts:
>>
>> On Thu, Jun 16, 2011 at 8:54 AM, Joel Martin <[email-masked]>wrote:
>>
>>> I started to try and answer this StackOverflow question about the
>>> protocol text and realized that I didn't understand it as well as I thought:
>>>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 16 Jun 2011 22:51:47 +0200

> it (e.g. this /forces/ the server to parse the Sec-WebSocket-Key header.)

And also to ensure that the response was not delivered by a lazy cache.

Willy

---

From: Philipp Serafin <[email-masked]>
Date: Thu, 16 Jun 2011 22:56:09 +0200

I think it would be useful to explicitly state the kinds of exploits
that could happen though.

So far, it looks that the only situation where a XMLHTTP-treated-as-WS
request could do harm were when it could access a normally
origin-protected resource from a foreign origin. (From what I
understand, the handshake doesn't protect against attacks on
intermediaries, that's the job of the masking).

However, to properly origin-protect a resource, the server already needs
to check the sec-websocket-origin header - so it is already forced to do
validation.
On the other hand, if the operators of the server don't bother with
origin checks and allow the resource to be accessed cross-origin I don't
see any additional security risks in (trying to) access the resource via
a very kludgy XMLHTTP request than via a WS connection.

I know this has probably been the most-discussed topic on this list and
it's not my intention to cause any additional eye rolling. I just think
it should be taken care to clarify the reasoning behind the handshake,
since this will probably a lot of discussion in web developer forums
once WS is up and running.

Regards,
Philipp Serafin

Am 16.06.2011 22:39, schrieb Ian Fette (イアンフェッティ):
> whatever the client sends, and will just return some value and then
> get exploited. By forcing the server to prove something to the client,
> we essentially also force the server to validate at least part of the
> client's handshake (rather than just hardcoding in some 101 Upgrade WS
> response). So, by making the server prove something to the client,
> it's a way of forcing the server to actually take some steps that have
> a side effect of protecting it (e.g. this /forces/ the server to parse
> the Sec-WebSocket-Key header.)

---

From: Joel Martin <[email-masked]>
Date: Thu, 16 Jun 2011 16:22:15 -0500

I must be missing some key concept here because your answers have been what
I already know (and agree with), but they still seem out of sync with the
text of the paragraph.

The draft paragraph is talking about protecting WebSocket servers from
NON-WebSocket clients (i.e. XMLHttpRequest or forms) and implying that the
Sec-WebSocket-Accept somehow helps in that situation. I understand how a
XMLHttpRequest/form submission will not have Sec-WebSocket-Key so this
prevents a WebSockets server from answering a non-WebSocket request. But
what does sending a proper Sec-WebSocket-Accept have to do with this case?
The Sec-WebSocket-Accept value is so that a WebSockets client knows the
server is a real WebSocket server. Won't a non-WebSockets request just
ignore the Sec-WebSocket-Accept?

Regards,

Joel Martin

>>>> It's likely I just don't have enough context to understand the
>>>> paragraph, but perhaps it could be clarified a bit.
>>>>
>>>> Regards,
>>>>
>>>> Joel Martin
>>>>

---

From: Joel Martin <[email-masked]>
Date: Thu, 16 Jun 2011 16:24:04 -0500

> And also to ensure that the response was not delivered by a lazy cache.

Sure, that makes sense... for the WebSocket client connection case. But what
how does it help in the non-WebSocket client case (which the paragraph is
talking about).

Regards,

Joel Martin

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 16 Jun 2011 23:26:04 +0200

> how does it help in the non-WebSocket client case (which the paragraph is

Since there is already open a new thread for this topic, could you
please discuss there about it instead of using this thread whose topic
is not that? :)

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 16 Jun 2011 14:27:16 -0700

On Thu, Jun 16, 2011 at 2:24 PM, Joel Martin <[email-masked]> wrote:

> Sure, that makes sense... for the WebSocket client connection case. But
> what how does it help in the non-WebSocket client case (which the paragraph
> is talking about).
>

Sorry, I feel like we're talking past each other :) You ask how
Sec-WebSocket-Accept helps in the non-WS client case. Sec-WebSocket-Accept
serves two purposes -- one is to (as we seem to both agree on) let the
client validate that it's a WS server. The second purpose is to force the
server to validate that it's responding to a WebSocket client. (To compute
Accept the server has to read Key, Key can not be sent via XHR, therefore we
are ensuring that the server actually checks for -Key and thus protects
itself from non WS client.)

---

From: Dave Cridland <[email-masked]>
Date: Thu, 16 Jun 2011 22:32:31 +0100

On Thu Jun 16 22:27:16 2011, Ian Fette (イアンフェッティ)  
> itself from non WS client.)

If I can rephrase, by requiring the Sec-WS-Key, you prevent a server  
being fooled by an XHR.

By requiring Sec-WS-Accept, you prevent a WS client being fooled by a  
lazy cache, etc.

In combination, though this means a server has to do Sec-WS-Accept  
magic in order to work with WS clients, and is thus forced into  
preventing XHR from working, even though the XHR won't be looking for  
Sec-WS-Accept.

Joel, does this make sense?

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade