Subject: Re: [hybi] TSV-Directorate review of draft-ietf-hybi-thewebsocketprotocol-07

From: Magnus Westerlund <[email-masked]>
Date: Mon, 2 May 2011 10:14:31 +0200

Willy Tarreau skrev 2011-04-30 17:21:
> On Sat, Apr 30, 2011 at 05:18:43PM +0200, Iñaki Baz Castillo wrote:
>> 2011/4/29 Magnus Westerlund <[email-masked]>:
>>> 41. Section 9.
>>>
>>> The security consideration section does not discuss client
>>> authentication. Is that because this is supposed to happen in the HTTP
>>> GET with the upgrade headers. there is some reference to cookies etc.
>>> But shouldn't there be more on this? So that a server can ensure the
>>> client identity?
>>
>> Cookies are retrieved from a web page, so if we assume that the
>> websocket server will be able to validate the client provided cookie
>> then we are wrong (the websocket server could be a separate server).
> 
> Not only that, but cookies are used by infrastructure components too
> in order to maintain persistence (association between a client and a
> server).

This is far from my area of real expertise, however I think the
specification are going to need to point to the client authentication
mechanism that can be used for websocket. This as authentication is one
of these security mechanisms that one very commonly need.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 2 May 2011 10:55:34 +0200

Hi,

On Mon, May 02, 2011 at 10:09:39AM +0200, Magnus Westerlund wrote:
> >>From memory, it's because XHR can't set a Sec- header so we're certain
> > that evil code running in browsers will not fake WS handshakes. We don't
> > want to prevent any non-browser from establishing a connection (that is
> > neither possible nor desired), but to ensure that owned browsers won't
> > do that.
> 
> I agree that the web-application running in the browser can't fake this
> if you trust the browser. However, a server has no way of knowing if
> this header comes from a browser doing all correct or if it the
> websocket establishment is from do-no-good implementation. Thus, I fail
> to see the security value of it.

The idea was that the risk comes from browsers. Those are the only ones
that can remotely be massively controlled by visiting an evil site. Other
tools require an access on the system. Now that said, I'm not certain
there is more risk in letting javascript fake a WS connection and letting
javascript send a valid POST request somewhere.

> >> 22. Section 4.3:
> >> If I understand the masking correctly it for one particular frame it
> >> uses the same mask over and over again over each 32-bits. As the frame
> >> format supports message of size up to 2^63 I think I can perform a cache
> >> poisining attack anyway.
> > 
> > There was a very lengthy debate on this several months ago and it was hard
> > to reach a rough consensus. I invite you to consult the archives. In short,
> > we're certain products exist with a risk for the first bytes, we're not
> > aware of any product which present this risk past whatever bytes break
> > HTTP parsing, and the way to write such a vulnerable product appears very
> > difficult to several developers, though it can't be demonstrated that none
> > exists. The resulting proposal was accepted as a trade-off between a very
> > low risk and some serious performance hits in some specific usages, and
> > still provides good protection against the known issues.
> 
> Ok, long discussion has happened and you have a rough consensus. From my
> perspective I think you are going to need to defend the trade-off in two
> ways. First, make it clear in the security consideration that the attack
> is still possible to some degree. Secondly include some of the reasoning
> behind it. Otherwise people will continue to comment on it. And it will
> be discussed on IESG level I am pretty certain.

One of the points was that the 32-bit "GET\n" is already a valid request
for Apache, so anyway whatever the size of the key you use, it will still
appear there once every 4 GB. However, Apache, just as any other known
HTTP server supporting keep-alive, does not randomly look into a stream
looking for what looks like a request.

If you want my personal opinion, if the IESG could rely on deeper HTTP
knowledge and conclude that the masking would need to protect better the
first few bytes and does not need to be applied later, we'd be able to
make it optional and simplify the spec. I mean that in my opinion, the
masking is too much for an imagined risk, and possibly limited if the
risky component was carefully designed to fail on this.

> >From personal view I do think the WG is taking the wrong trade-off to
> leave a evident security vulnerability in the protocol. By not securing
> this sufficiently the effect is going to be that people write firewall
> rules that block websocket. I do understand that there will be a
> performance hit. But from the perspective of a somewhat slower protocol
> that can be used and a protocol that can't I know what I pick.

Well, as I explained above, the risk has been "imagined" from scratch.
Whoever has already had to deal with developing a server-side HTTP stack
knows that it's useless and almost impossible to scan a stream trying to
parse an HTTP request where there is none. Basically we're talking about
looking for "GET\n" in /dev/random. So any such method always comes down
to statistics in the end.

Considering the performance hit, we showed that the only way to safely
try to crypt the traffic was by putting the key at the end of a frame,
which forces every intermediary to buffer everything until they got the
key to decode the frame. And that still does not stop the 32-bit pattern
above from appearing.

There were valid alternative suggestions such as prepending a fake request
between the payload to force the non-compliant intermediaries to process
this fake request instead of parsing the payload. Other ones consisted in
using a masking with the 7th bit always set so that a valid request could
never be encoded in the stream.

> >> 29. Section 4.5.2:
> >>    An endpoint MAY send a Ping message any time after the connection is
> >>    established and before the connection is closed.  NOTE: A ping
> >>    message may serve either as a keepalive, or to verify that the remote
> >>    endpoint is still responsive.
> >>
> >> So one allows to have multiple outstanding PING messages? That sounds a
> >> bit strange over a reliable in-order delivery channel such as TCP. And
> >> if one does allow it should there be any rules for answering pings in
> >> order one receives them?
> > 
> > Indeed, we should possibly remind implementors that it's useless to send
> > a new ping as long as one is still pending.
> 
> I think there is two possible ways. Allow it and simply include a note
> that it is mostly useless. Or make it clear that one is not allowed to
> send new pings until one have received the pong.

Good point.

> >> 36. Section 7.1.1.
> >> This section should recommend that the client is the one that performs
> >> the TCP close so that it holds the TIME_WAIT state and not the server.
> > 
> > If we recommend something, we should recommend that the server does it, as
> > only the client is impacted by holding a time_wait, as it prevents it from
> > re-opening the connection for 2 MSL, while on the server it has no impact
> > as reopining a time_wait connection is immediate upon a new SYN with a
> > higher seq number. Do not forget that there will be quite a number of
> > intermediaries in the whole chain and we don't want to cause basic networking
> > issues between server-side proxies and the servers themselves.
> 
> Ok, I haven't thought of it from this perspective. That makes sense as
> long as you have no real issues with potential build up of TIME_WAIT
> states on the server. I guess the main issue would be the sheer number
> of connection states

TIME_WAIT states are always very cheap. My record was 35 million on a 4-GB
core2duo machine :-)  And internet-exposed HTTP servers are already tuned
to support large numbers. BTW, there will likely be many more TW from HTTP
than from WS because the former leads to far many open/close events.

Cheers,
Willy

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 2 May 2011 10:09:39 +0200

Hi,

Please see inline for comments on your answers. I have dropped the ones
that are fine.

Willy Tarreau skrev 2011-04-29 20:34:
> Hi Magnus,
> 
> thank you for this deep review. I can't spend enough time to comment on
> many of your points (in part because many are the results of very long
> discussions), but there are a few short ones that can already be easily
> addressed.
> 
> On Fri, Apr 29, 2011 at 07:00:48PM +0200, Magnus Westerlund wrote:
>> 2. Section 1.3 and 9. I really don't see how the Sec-WebSocket-Origin
>> header provides any security without additional mechanisms. Yes, a
>> trusted browser implementation will set it correctly. However, as non
>> browser or hacked browser can fake this header the server has no direct
>> trust in this value. To me it appears that one would need to make
>> additional bindings to some other contexts to be able to trust the value
>> of this header. Or am I missing some important part here?
> 

> 
>> 27. Section 4.5.1:
>>    After both sending and receiving a close message, an endpoint
>>    considers the websocket connection closed, and SHOULD close the
>>    underlying TCP connection.
>>
>> What is the alternative to actually closing the TCP connection? Can one
>> actually start a new websocket over the existing TCP connection? If
>> there is no alternative, why not change SHOULD to MUST or at least will.
> 
> Agreed (same as point 5 above BTW).

Yes, but in a different place.

> 
>> 32. Section 5.1, page 29:
>>   Once a connection to the server has been established (including a
>>    connection via a proxy or over a TLS-encrypted tunnel), the client
>>    MUST send a handshake to the server.  The handshake consists of an
>>    HTTP upgrade request, along with a list of required and optional
>>    headers.  The requirements for this handshake are as follows.
>>
>>
>> Considering the cache poising and the security issue, why isn't
>> websocket specified to always run in a TLS tunnel? Why is the non-secure
>> mode existing at all?
> 
> TLS + NPN was evocated as a possible later upgrade. Still there is a real
> need for cleartext traffic so that communications can be analysed and
> filtered in schools or other places where it is mandatory to filter
> contents and/or to protect people.

Ok, explanation provided. I have my view of that design. I do prefer
that one would find a solution allowing for per leg protection.

Cheers

Magnus Westerlund

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 12 May 2011 21:18:51 +0900

> > send new pings until one have received the pong.
I'm posting my question about Ping/Pong -07 here since it's related to this
topic.

Now an endpoint may ignore some Pings (according to the sentence "In the
case multiple Pings have been received ...") in -07. What is this for? It's
very strange that we have this and the requirement about body "The
message bodies (i.e. both ..." together.

I thought the purpose of echoing back the same body is to allow endpoints to
check if a certain Ping is really acked by Pong with the same body (maybe
sequence number or something). But if an endpoint sends Ping "ABC" and then
sends "DEF" without waiting for Pong for "ABC", Pong for "ABC" may not be
sent by the other peer. This cannot be realized. We need to disallow two or
more pending Ping at the same time as Willy says to address this point, too.

Even with this fix, we still have problem. Now an endpoint can send
unsolicited Pongs (according to the text "A Pong message MAY be sent
unsolicited."). We cannot distinguish unsolicited one from one in reply to
Ping. Here, it's not possible to check if Ping "ABC" is pending or not since
the spec doesn't prohibit the other peer sending unsolicited Pong "ABC". How
should we deal with this?

Takeshi

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 12 May 2011 23:25:52 +1000

> should we deal with this?

This is all over a reliable transport, so if we send Ping "ABC", then
receive Pong "XYZ", we know that the pong was unsolicited and not a
corrupted "ABC".  Thus we have to keep waiting for the Pong "ABC".
   Obviously there is a chance that a unsolicited pong matches a ping,
but some careful design of payloads can easily fix that.

cheers

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 13 May 2011 03:36:58 +0900

> but some careful design of payloads can easily fix that.

I see. Coordinating body value assignment so that unsolicited one and reply
one don't conflict each other (e.g. "ABC" and "XYZ" as you said) is one
possible solution.

This should be discussed well when we add some keep alive feature to browser
that sends Ping to the server. If we prepare some API for JavaScript to tell
the browser what string to use for Ping, service designers may coordinate
assignment by themselves. If we don't, we should reserve some string to be
used for Ping/unsolicited Pong by browsers to avoid collision.

----

Isn't it simpler to reserve some string, say "NOOP" (or "\x00" to save
bytes), for Ping body and require endpoints to ignore such Ping rather than
diverting Pong for heartbeat?

Thanks,
Takeshi

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 13 May 2011 08:29:24 +1000

Takeshi,

what I have seen before is the use of sequence numbers where the
client side uses even and the server side uses odd, so can tell by the
seq number in a pong where it was generated and thus tell if it is a
response or unsolicited.

cheers