Subject: [hybi] Test suite for WebSocket servers?

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 17 Aug 2011 10:10:41 +0200

Hi, there is a powerful test suite for WebSocket clients but, is there
any WebSocket client to test a WS server? This is, I'm building a WS
server and would like a WS client performing several "standard" tests
(I mean more than sending a Ping and expecting a Pong).

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 17 Aug 2011 09:18:00 +0100

> (I mean more than sending a Ping and expecting a Pong).

libwebsockets has a "fraggle" test that tests both sides can cope with 
random messages composed of random number of random sized frames.

http://git.warmcat.com/cgi-bin/cgit/libwebsockets/tree/test-server/test-fraggle.c

I dunno what a standard test would look like without it being bound to a 
particular protocol.  And then you need to be careful you're not testing 
the code just in that protocol and can expect the same good results on 
another non-test protocol.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 17 Aug 2011 10:25:15 +0200

> http://git.warmcat.com/cgi-bin/cgit/libwebsockets/tree/test-server/test-fraggle.c

Great, thanks a lot.

> particular protocol.  And then you need to be careful you're not testing the
> code just in that protocol and can expect the same good results on another
> non-test protocol.

Well, I mean application agnostic tests, like sending invalid
FIN/opcode and expecting a remote connection close, or sending
opcode=1 (UTF-8 text) and sending invalid UTF-8 in the application
payload.

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 17 Aug 2011 09:41:30 +0100

> opcode=1 (UTF-8 text) and sending invalid UTF-8 in the application

That's not a bad idea at all.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 17 Aug 2011 10:49:50 +0200

> That's not a bad idea at all.

We need reference WS servers for testing WS clients, and reference WS
clients for testing WS servers. The chicken or the egg? :)

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Joe Mason <[email-masked]>
Date: Wed, 17 Aug 2011 17:26:35 -0400

http://www.tavendo.de/autobahn/testsuite.html includes a server test suite that includes FIN/opcode tests, but not tests for valid UTF-8.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> "Andy Green (???)"
> Sent: Wednesday, August 17, 2011 4:42 AM
> To: Iñaki Baz Castillo
> Cc: [email-masked]
> Subject: Re: [hybi] Test suite for WebSocket servers?
> 
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

---

From: Tobias Oberstein <[email-masked]>
Date: Wed, 17 Aug 2011 16:44:28 -0700

Yep, there are only 3 test cases in UTF-8 handling currently.
One of those caught 1 (fixed) bug in Chrome regarding UTF-8 handling.
There should be more of course .. what exactly should be tested?

i.e. here is a list a post of Greg some time ago:

* UTF-8 Text 0000-007f: Send/receive text message with characters in
code point range  U+0000 to U+007F
* UTF-8 Text 0080-07ff: Send/receive text message with characters in
code point range  U+0080 to U+07FF
* UTF-8 Text 0800-ffff: Send/receive text message with characters in
code point range  U+0800 to U+FFFF
* UTF-8 Text 010000-10ffff: Send/receive text message with characters
in code point range  U+010000 to U+10FFFF
* Illegal bytes UTF-8: Send message with invalid UTF-8 byte sequence.
Verify message is not received.
* Illegal codes UTF-8: Send message with invalid UTF-8 code points.
Verify message is not received.

http://www.ietf.org/mail-archive/web/hybi/current/msg07975.html
http://www.ietf.org/mail-archive/web/hybi/current/msg07995.html

Is it possible to generate invalid code points / byte sequences
_exhaustively_? How?

On 17.08.11 23:26, "Joe Mason" <[email-masked]> wrote: