Subject: [hybi] About Autobahn test server: why must the server close TCP connection without sending close frame first???

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 15:04:35 +0200

Hi, Autobahn WS server test says:

---------------------------
Test Case 3.1

Description
  Send small text message with RSV = 1.

Expectation
  The connection is failed immediately, since RSV must be 0, when
   no extension defining RSV meaning has been negoiated.
-----------------------------

If my server replies a close control frame with status 1002, then the
test marks it as "failed", it just marks it as "passed" in case my
server *directly* terminates the TCP connection. Why? is it the
expected behaviour? IMHO there has been a WS protocol level error, so
a WS protocol message must take place rather than terminating the TCP
connection. Am I wrong?

Thanks.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 15:10:57 +0200

> Hi, Autobahn WS server test says:

IMHO it's clear in the spec:

-----------------------------------------------------
7.1.7.  Fail the WebSocket Connection

   [...]
   If _The WebSocket Connection is Established_ prior to the point where
   the endpoint is required to _Fail the WebSocket Connection_, the
   endpoint SHOULD send a Close frame with an appropriate status code
   Section 7.4 before proceeding to _Close the WebSocket Connection_.
-------------------------------------------------------

So if the test suite is wrong.... good news.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 16:10:06 +0200

> IMHO it's clear in the spec:

If my WS server closes the TCP connection without sending a close
frame, then Autobahn client detects it and marks as "valid" the test
(since the remote has closed).

But if my WS server sends a close frame and after it closes the TCP
connection ("close_connection_after_writting") then Autobahn client
does not detect the TCP close and closes it after 1 second (this is
what the network trace says in the results web). Must investigate a
bit.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 17:39:17 +0200

> what the network trace says in the results web). Must investigate a

Apart from the above minor issue (which is more related to TCP
disconnection issues) my WS server passes all the Autobahn tests :)

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 26 Aug 2011 08:39:40 -0700

> If my WS server closes the TCP connection without sending a close frame,
> then Autobahn client detects it and marks as "valid" the test (since the
> remote has closed).
> 
> connection ("close_connection_after_writting") then Autobahn client does
> not detect the TCP close and closes it after 1 second (this is what the network
> trace says in the results web). Must investigate a bit.

These test cases are currently constructed so that they expect
"fail connection" = "peer closes TCP without performing WS close handshake"

Comments:

- This fits the behavior of Chrome/FF and all other libs
- It does not fit the SHOULD in the spec
- I will expand the cases/test suite so that it can detect both behaviors
- Note that this is somewhat announced on the web site: the suite still lacks in 2 areas: UTF-8 and Open/Close Handshake handling
- feel free to submit patches ..

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 26 Aug 2011 08:42:53 -0700

> disconnection issues) my WS server passes all the Autobahn tests :)

nice;)

Would be helpful if you could publish a link to the generated reports
(for me to look into the wirelogs to see what happens in the cases
you mentioned)

Also:
http://en.wikipedia.org/wiki/Comparison_of_WebSocket_implementations

with regard to "fail connection" SHOULD send Close .. search postings
here .. I've asked Takeshi why Chrome behaves so .. answer: it's by
design and allowed since spec only says SHOULD. see the arguments there.
then decide how you want your lib to behave ... I'm neutral in this point.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 18:17:54 +0200

> - This fits the behavior of Chrome/FF and all other libs

Buth nothing wrong occurs if Chrome/FF receives a close frame, they
just behave correctly and send a close reply.

> - It does not fit the SHOULD in the spec

OK :)

> - I will expand the cases/test suite so that it can detect both behaviors

Great.

> - Note that this is somewhat announced on the web site: the suite still lacks in 2 areas: UTF-8 and Open/Close Handshake handling

And IPv6 support ;)

> - feel free to submit patches ..

Unfortunately I'm not a Python guy, but Ruby one :)

Thanks.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 18:22:31 +0200

> (for me to look into the wirelogs to see what happens in the cases

Sure, I'm running the test again with the "hack" dissabled (the "hack"
means not sending valid close frame before closing the TCP
connection). Let me a minutes and I will upload it to my server.

> http://en.wikipedia.org/wiki/Comparison_of_WebSocket_implementations

The problem is that my WS server is not a generic one, but it's
integrated within a SIP server I'm building, so it's not useful to
include it (neither it's released yet) in such list.

> then decide how you want your lib to behave ... I'm neutral in this point.

If my server sends a close frame to Chrome-15/Firefox-8, they reply a
close frame correctly. And then both sides close the TCP connection
(not sure which one first), so I still don't see the problem.

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 26 Aug 2011 09:26:55 -0700

> > These test cases are currently constructed so that they expect "fail
> > connection" = "peer closes TCP without performing WS close handshake"
> >
> > Comments:
> >
> Buth nothing wrong occurs if Chrome/FF receives a close frame, they just
> behave correctly and send a close reply.

Yes, when they passively receive a close.
When they actively "fail the connection", I have never seen them to do a WS close handshake.

> > - Note that this is somewhat announced on the web site: the suite
> > still lacks in 2 areas: UTF-8 and Open/Close Handshake handling
> 
> And IPv6 support ;)

Unfortunately, that is an "ongoing Twisted issue":

http://twistedmatrix.com/trac/ticket/3014
http://twistedmatrix.com/trac/wiki/IPv6

When Twisted supports IPv6, the test suite will. Until then, I can't do anything, I'm totally out of queue space for anything new ..

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 19:23:46 +0200

> connection). Let me a minutes and I will upload it to my server.

Ok, the issue is "fixed". I explain the "problem":

When my WS servers detects a WS protocol error it sends a close frame
and waits for 1.5 seconds before clossing the TCP connection (in order
to let the WS client to reply its close frame).

But AutoBahn test client does not wait too much and expects that the
server must close the TCP connection before 1 second. So I've
decreased such waiting time to 0.8 seconds and now there are no test
failures.

BTW the results of my server are available here:

http://oversip.net/public/oversip/websocket-autobahn-results/

The WS server is a built on Ruby using EventMachine (similar to
Python's Twister), running in a laptop Pentium(R) Dual-Core CPU T4400
@ 2.20GHz.

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 19:30:45 +0200

> When they actively "fail the connection", I have never seen them to do a WS close handshake.

Aha, ok, so they must receive something wrong at WS level and then
they would just close TCP instead of sending a WS close frame.
Understood, I will test it.

> When Twisted supports IPv6, the test suite will. Until then, I can't do anything, I'm totally out of queue space for anything new ..

Don't worry, the very same occurs to me in Ruby EventMachine :)

-- 
Iñaki Baz Castillo
<[email-masked]>