Subject: [hybi] JavaScript WebSockets Dependencies

From: Bob Gezelter <[email-masked]>
Date: Sat, 09 Apr 2011 18:24:01 -0500

There have been several references recently on the dichotomy between text
and binary. A recurring theme seems to be the effect that this is helpful to
the JavaScript library.

As a matter of protocol design, IMHO, this is a misjudgment (which is the
underlying reason for the similar comments in my recent blog posting). Just
because there may be implementation issues in JavaScript is (emphatically)
not a reason to burden the Websocket protocol with the resulting baggage.
JavaScript is not the only language in which the protocol will be
implemented. The history of language-favoring APIs is not a pleasant one to
contemplate.

On a similar note, I would not recommend the use of Base64 encoding within
the wire protocol (outside of the initial handshake, which I concur with the
IETF recommendation of UTF-8). Once communications have been established,
character set negotiation is properly a function of the protocol levels
above the WebSocket protocol, and the WebSocket protocol should be
completely agnostic in this regard.

- Bob
+--------------------------------------------------------------------------+
| Robert "Bob" Gezelter                       E-Mail:  [email-masked]  |
| Robert Gezelter Software Consultant                                      |
| 35-20 167th Street, Suite 215               Fax:       (on Request)      |
| Flushing, New York  11358-1731                                           |
| United States of America                    http://www.rlgsc.com         |
+--------------------------------------------------------------------------+

---

From: Micheil Smith <[email-masked]>
Date: Sun, 10 Apr 2011 10:16:26 +1000

[Bob, apologies for the double delivery of this response.]

Despite being an author of one of the javascript websocket-server implementations 
out there, I agree that WebSocket's shouldn't be changed just to make them easier 
to implement in specific languages.

If there are things that your language can't do, chances are you can drop down to the 
interpreter or VM or compiler, which is more then likely capable of doing what needs 
to be done. (eg, going from javascript down to C or C++)

– Micheil

On 10/04/2011, at 9:24 AM, Bob Gezelter wrote:

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 11 Apr 2011 08:19:44 +1000

On 10 April 2011 09:24, Bob Gezelter <[email-masked]> wrote:
> Once communications have been established,

I certainly don't disagree with you on this.

However, this group did have long and hard fought discussions about
being character encoding agnostic and about having meta data such as
content-encoding explicitly supported in the protocol.  Eventually the
consensus was to not have a framing that was based on UTF-8 encoding,
but to have a frame type that indicated a UTF-8 encoded payload.
Binary payloads are unspecified and I'm sure(hope) subprotocols will
be developed to carry arbitrary content with meta data.

cheers

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 11 Apr 2011 12:34:28 +0900

Hello Bob,

> and binary. A recurring theme seems to be the effect that this is

That was one of the reasons to do it, but not the only one.

> completely agnostic in this regard.

First, I'd just say that this is for WebSockets to decide (and 
essentially already has been decided). Because ultimately, characters 
are encoded as binary, it's certainly possible to claim that at a 
certain level, everything is, or should be, just binary.

But the idea of binary with character *encoding* negotiation on top is 
just one way to deal with the binary/textual division. (Caution, what's 
negotiated are not character *sets*, although the parameter name for it 
is often "charset".) This negotiation didn't exist a long time ago, and 
it may fall out of favor in the future. As an example of the past, ftp 
has a binary/textual distinction, but no character encoding negotiation.

Character encoding has been a big mess for a long time. Character 
labeling (and negotiation, although there's rather few negotiation going 
on even in HTTP, where it's available in the protocol) was an attempt 
deal with this. It was only partially successful. But with Unicode/ISO 
10646 and UTF-8, we may be slowly but steadily see the light at the end 
of the tunnel. The IETF essentially knew that the tunnel would end at 
UTF-8 in 1998 (see http://tools.ietf.org/html/rfc2277).

The fact that UTF-8 is preferred for text interchange, but that many 
important frameworks (Java, JavaScript, .Net,...) store Unicode 
internally as UTF-16 increases the benefit of distinguishing text and 
binary, because the application programmer gets the conversion for free 
and has one less thing to worry about or mess up. Also, programming 
languages that don't have something like a 'string' datatype are very 
few and very far between. It's not at all that we are adapting to a 
peculiar API in a single programming language.

The way the WebSocket protocol does away with character encoding 
negotiation may be seen as a big problem when looking backwards, but 
when looking forwards, it should turn out to be something that will be 
followed by other protocols. There are already quite a few XML formats 
for example that just say "always UTF-8", although XML itself would 
allow more variation.

And once we are rid of character encoding negotiation, the distinction 
between binary and text makes quite a bit more sense. It actually exists 
in many protocols. I have already mentioned ftp. In email, there are 
textual headers, and a textual body if it's ASCII.

In WebSockets, I imagine textual data being used for things such as 
commands (e.g. as in SMTP), short textual data (e.g. something like irc 
messages) and longer textual blobs (e.g. HTML, XML, JSON,... in UTF-8). 
Binary will be used for everything else, mostly binary blobs (e.g. 
images) and binary bits and pieces of continuous media (audio, video).

Regards,   Martin.

---

From: Scott Ferguson <[email-masked]>
Date: Mon, 11 Apr 2011 08:58:59 -0700

On 04/10/2011 08:34 PM, "Martin J. Dürst" wrote:
> certain level, everything is, or should be, just binary....
>
> labeling (and negotiation, although there's rather few negotiation 
> going on even in HTTP, where it's available in the protocol) was an 
> attempt deal with this. It was only partially successful. But with 
> Unicode/ISO 10646 and UTF-8, we may be slowly but steadily see the 
> light at the end of the tunnel. The IETF essentially knew that the 
> tunnel would end at UTF-8 in 1998 (see 
> http://tools.ietf.org/html/rfc2277).

Solving the encoding negotiation problem is a big win for the WebSocket 
text encoding.

I've seen (and been forced to implement) far too many failures of 
character encoding negotiation to want to ever deal with it again.

For example, the HTTP URL encoding (or rather lack of encoding) has been 
a recurring painful experience.

The full-blown character negotiation of IIOP/CORBA was a genuine horror. 
That IIOP example shows, in full gory detail, that adding full 
flexibility for encoding choice is a far worse solution than mandating 
UTF-8.

Also, PHP's encoding is in such a difficult state because it initially 
used binary strings.

In contrast, the Java decision to always use UTF-8 for its strings 
encoding in .class files, serialization, etc. has been very successful 
and easy to work with.

Those aren't theoretical concerns; they're examples of other protocols 
and systems that choose to start with binary and put off text encoding 
for later. I would strongly discourage following those mistakes.

-- Scott