Subject: [hybi] Fwd: The MessageBroker WebSocket Subprotocol

From: Salvatore Loreto <[email-masked]>
Date: Sun, 2 Oct 2011 11:40:14 +0300

for some reason the mail below didn't go through the HyBi mailing list,
so I am forwarding it (and of course working on try to figure out the 
reason of why it didn't go)

cheers
/Sal

-------- Original Message --------
Subject: 	The MessageBroker WebSocket Subprotocol
Date: 	Sun, 2 Oct 2011 00:33:52 +0200
From: 	Hapner mark <[email-masked]>
To: 	[email-masked] <[email-masked]>
CC: 	[email-masked] <[email-masked]>, Salvatore Loreto 
<[email-masked]>

Hi,

Let me introduce myself since I have not posted to the list before.I've 
been involved with message brokers for some time. I was the engineer 
behind the creation of the Java Message Service API. I current work for 
Huawei's US Software Lab in Santa Clara, CA.

Clebert is the JBoss HornetQ lead.

Clebert and I are excited about the potential to bring the functionality 
of message brokers to the internet and we see WebSocket as the best 
message transport to enable this. We believe the WebSocket subprotocol 
described in the attached document provides the minimal extension to 
WebSocket needed for this purpose. We hope this subprotocol reflects 
well on the expectations the WG has for the use of WebSocket extensibility.

We are submitting this document informally to the WG for its comment. It 
is our hope that it will mature into a subprotocol that can be submitted 
for either inclusion into the WebSocket specification; or, as a separate 
RFC. We would like to get the WG's thoughts on how best to proceed in 
this direction.

http://www.ietf.org/id/draft-hapner-hybi-messagebroker-subprotocol-00.txt

Mark Hapner
Clebert Suconic

---

From: Martin Sustrik <[email-masked]>
Date: Sun, 02 Oct 2011 13:54:23 +0200

Hi Mark, Clebert,

What I *really* like about the proposal is that unlike other messaging 
protocols it takes care to cleanly separate the reliability from the 
broker behaviour. The latter is explicitly defined as out-of-scope.

Some comments follow:

1. Given the explicit focus on reliability, it's strange that the 
proposal deals with message metadata, which have to do with broker or 
even application behaviour and have absolutely nothing to do with 
reliability. What ensues is a spec where metadata are defined as opaque 
syntactic placeholders with no associated semantics. The semantics are 
meant to be defined on the broker or application level. The question is 
whether it doesn't follow that the syntax should be defined on those 
layers as well.

2. AFAICS there's nothing in the spec that presumes existence of the 
broker. It can be as well used for direct communication between 
applications. Thus, I would suggest replacing messaging broker/messaging 
client terminology with WebSocket server and WebSocket client wording.

3. As for reliability itself, it should be clear what kind of error 
conditions is the protocol meant to handle. Possible options:

a. Network failure. If so, how does it differ from simply turning off 
TCP keepalives?

b. Client failure and restart?

c. Server failure and restart?

Martin

On 10/02/2011 10:40 AM, Salvatore Loreto wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: John Tamplin <[email-masked]>
Date: Sun, 2 Oct 2011 12:51:27 -0400

On Sun, Oct 2, 2011 at 4:40 AM, Salvatore Loreto <
[email-masked]> wrote:

> -------- Original Message --------  Subject: The MessageBroker WebSocket
> Subprotocol  Date: Sun, 2 Oct 2011 00:33:52 +0200  From: Hapner mark
> <[email-masked]> <[email-masked]>  To: [email-masked]
> <[email-masked]> <[email-masked]>  CC: [email-masked]
> <[email-masked]> <[email-masked]>, Salvatore Loreto
> <[email-masked]> <[email-masked]>
>
>  Hi,
>
> been involved with message brokers for some time. I was the engineer behind
> the creation of the Java Message Service API. I current work for Huawei's US
> Software Lab in Santa Clara, CA.
>
> of message brokers to the internet and we see WebSocket as the best message
> transport to enable this. We believe the WebSocket subprotocol described in
> the attached document provides the minimal extension to WebSocket needed for
> this purpose. We hope this subprotocol reflects well on the expectations the
> WG has for the use of WebSocket extensibility.
>
> is our hope that it will mature into a subprotocol that can be submitted for
> either inclusion into the WebSocket specification; or, as a separate RFC. We
> would like to get the WG's thoughts on how best to proceed in this
> direction.
>

It isn't entirely clear to me whether this is defining a subprotocol or a
WebSocket extension.  3.3 clearly makes it seem to be a subprotocol, but
then 2.1.2 defines two opcodes, which is something that can only be done by
an extension.  Also, if it is an extension, there should probably be more
care to reduce the number of opcodes used, as they are in very limited
supply.  Finally, since new opcodes can only be allocated by IANA, it is
hard to experiment with before it is standardized - an alternative would be
to define binary frames exchanged over the subprotocol as having a 1-byte
extended opcode in the first byte, which would not require a WebSocket
extension or allocation of scarce opcodes.  After gaining some experience
with the protocol, then we will have better information about whether
allocating scarce opcodes to this use is warranted.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Bruce Atherton <[email-masked]>
Date: Sun, 02 Oct 2011 11:21:01 -0700

On 02/10/2011 9:51 AM, John Tamplin wrote:
>
> It isn't entirely clear to me whether this is defining a subprotocol 
> or a WebSocket extension.  3.3 clearly makes it seem to be a 
> subprotocol, but then 2.1.2 defines two opcodes, which is something 
> that can only be done by an extension.

As I pointed out in my example with HTTP (which required Request and 
Response message types), it is going to be very common for subprotocols 
to want to define new opcodes. They are the right fit for defining 
message types in a subprotocol.

I did try to suggest that subprotocols should be able to define new 
opcodes, but got push back when I tried. Eventually I realized that 
requiring there be an extension to change opcodes for use with a 
subprotocol was a better solution because then it doesn't put any 
requirements for understanding subprotocols on to intermediaries, which 
would have caused complications for some usecases for the subprotocol 
field such as versioning.

So I foresee a good number of subprotocols will also want to define 
extensions to identify opcodes that they want to use. I think this is 
fine, but in drafts that require this (such as this one)  the definition 
of the extension and the opcodes it defines should probably be in a 
separate section.

>  Also, if it is an extension, there should probably be more care to 
> reduce the number of opcodes used, as they are in very limited supply.

I don't believe that is the case. Every Websocket connection (which may 
be a socket or a channel inside a MUX connection) is going to speak only 
one subprotocol, so they are only going to need one extension defining 
new opcodes for message types.  There is no reason these opcodes cannot 
be reused between different subprotocols. That logic does not apply to 
control opcodes, of course, since they are a Websocket layer concern.

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 3 Oct 2011 11:45:37 +1100

On 3 October 2011 05:21, Bruce Atherton <[email-masked]> wrote:
> Response message types), it is going to be very common for subprotocols to
> want to define new opcodes. They are the right fit for defining message
> types in a subprotocol.

Bruce,

I do not think  opcodes are the right fit for defining messages in a
subprotocol (at least not in the ws protocol as we have designed it).

While I can imagine some subprotocols would desire to have opcode like
metadata associated with a message, I  can also imagine many
subprotocols will use JSON/XML as their message exchange format and
thus will better use fields/elements/attributes to convey message
type.

The way we have defined the protocol, opcodes are a constrained and
regulated resource that must be allocated by IANA. They may be used by
extensions, which themselves have a high barrier to deployment as they
need to be implemented in the browsers and servers, since the
application API is insufficient (and inappropriate) for implementing
extensions.

Subprotocols are exposed to the application in the browser API, so I
believe that sub protocol designs need to be restricted to what can be
achieved without API changes or IANA allocations.   I think this is a
good thing and keeps the barrier low for sub protocol development.

regards

---

From: Brian <[email-masked]>
Date: Sun, 2 Oct 2011 22:37:28 -0700

Absolutely correct.  Opcodes are not meant to be usable by subprotocols,
only extensions.  Any subprotocol that wants its own opcode must encode it
somewhere in the application data area of the WebSocket message.

Brian

---

From: Frank Salim <[email-masked]>
Date: Mon, 3 Oct 2011 08:24:01 -0700

I agree that this should be a protocol rather than an extension and
that new opcodes are not required.

Subprotocols should be defined in terms of their message payload only.
To give an analogy, WebSocket subprotocols should not require opcodes
just as protocols on top of TCP should not new require control bits.

Other notes so far:

3.4 "WebSocket uses the HTTP origin model to identify clients. MBWS
uses the same client identity model."

Origin cannot be used to identify clients.

4.2/5 "Client or Broker may initiate/respond to Ping/Pong during
session as defined by WebSocket."

Subprotocols shouldn't be aware of ping/pong, since they aren't
exposed in the API.

-Frank Salim

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 03 Oct 2011 09:50:31 -0700

> thus will better use fields/elements/attributes to convey message

Exactly. Many subprotocols will not need anything more than the 
text/binary division we have. These subprotocols will not need any 
extension because they will not need new opcodes. Other subprotocols 
will contain the message type within the message themselves. HTTP 
requests are an example, as are SIP messages.

But then there are the ones where the type is not carried within the 
message. While you could define an extra header on the payload to carry 
that information, it seems like what the opcode is designed for. In 
fact, I can't think of any other use for the opcode than that. I could 
be wrong, perhaps it will turn out there are other uses for it. Time 
will tell once there are lots of implementations and deployments.

I am working on a Websocket-SIP bridge that uses different opcodes for 
SIP messages versus media channel messages. Now by happenstance I can 
send the SIP messages as text messages and media channel messages as 
binary (even if the media channel is a text one) and get the behaviour I 
want, but I would be lying to myself if I thought the opcodes still 
represented text and binary. I am overloading them to mean SIP and 
media. If I had needed more than 2 types of messages, or if I needed two 
types that were both binary, I would have had to define my own opcodes 
in an extension or prepend a header to get the behaviour I wanted.

MUX will be an example of a combination of subprotocol and extension 
assuming it ends up requiring new opcodes or flags. It defines a 
specific way for information to be passed over the Websocket transport 
making it a subprotocol that carries multiple channels, each potentially 
with its own subprotocol.

> good thing and keeps the barrier low for sub protocol development.

I expect that IANA registrations of opcodes will end up being specific 
to a particular subprotocol, but I could be wrong. We'll have to see how 
things turn out. But I would be surprised if this draft turned out to be 
the last one that wanted to use opcodes to define message types.

As for the barrier to entry for subprotocols, I am only suggesting that 
a few subprotocols with clear needs will want to define new opcodes. And 
I am hopeful that eventually browsers will provide APIs for people to 
plug in their own extensions.

---

From: Frank Salim <[email-masked]>
Date: Mon, 3 Oct 2011 10:21:02 -0700

If you needed more than two types of messages, you would simply put
the message type in the payload. The fact that there are typed
messages at all is an artifact of the API and protocol evolution, not
something to design your protocols around.

Extensions should be used to transform the message stream (for example
by compressing payload bytes). A protocol, on the other hand, defines
an interaction. Protocols should not depend on the presence or absence
of particular extensions.

-Frank Salim

On Mon, Oct 3, 2011 at 9:50 AM, Bruce Atherton <[email-masked]> wrote:
> Exactly. Many subprotocols will not need anything more than the text/binary
> division we have. These subprotocols will not need any extension because
> they will not need new opcodes. Other subprotocols will contain the message
> type within the message themselves. HTTP requests are an example, as are SIP
> messages.
>
> message. While you could define an extra header on the payload to carry that
> information, it seems like what the opcode is designed for. In fact, I can't
> think of any other use for the opcode than that. I could be wrong, perhaps
> it will turn out there are other uses for it. Time will tell once there are
> lots of implementations and deployments.
>
> I am working on a Websocket-SIP bridge that uses different opcodes for SIP
> messages versus media channel messages. Now by happenstance I can send the
> SIP messages as text messages and media channel messages as binary (even if
> the media channel is a text one) and get the behaviour I want, but I would
> be lying to myself if I thought the opcodes still represented text and
> binary. I am overloading them to mean SIP and media. If I had needed more
> than 2 types of messages, or if I needed two types that were both binary, I
> would have had to define my own opcodes in an extension or prepend a header
> to get the behaviour I wanted.
>
> assuming it ends up requiring new opcodes or flags. It defines a specific
> way for information to be passed over the Websocket transport making it a
> subprotocol that carries multiple channels, each potentially with its own
> subprotocol.
>
> I expect that IANA registrations of opcodes will end up being specific to a
> particular subprotocol, but I could be wrong. We'll have to see how things
> turn out. But I would be surprised if this draft turned out to be the last
> one that wanted to use opcodes to define message types.
>
> As for the barrier to entry for subprotocols, I am only suggesting that a
> few subprotocols with clear needs will want to define new opcodes. And I am
> hopeful that eventually browsers will provide APIs for people to plug in
> their own extensions.
>

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 4 Oct 2011 09:21:50 +1100

On 4 October 2011 03:50, Bruce Atherton <[email-masked]> wrote:
> information, it seems like what the opcode is designed for.

Many months ago I would have strenuously agreed with you about the
need to pass application meta-data along with each frame (with opcode
being just one example of meta-data). But the protocol evolved to have
a very limited and restricted view of what an opcode is, and having
accepted that, I now I'll strenuously argue that supprotocol and
application specific meta data must be carried in-payload.

If you consider the opcodes currently defined: control codes, text,
binary - they all relate to different behaviours that the WS
implementation is required to perform.  In the case of text vs binary,
this is utf-8 validation and decoding.  I think future opcodes should
follow that pattern and only be used to define new capabilities to be
implemented within the WS layer (and it's extensions) and not to
define meta-data only relevant to a subprotocol or other application
layer logic.  Indeed new opcodes should be like existing opcodes and
essentially be invisible to applications and subprotocols. So for
example, a MUX extension could use op-codes to identify frames that
need to be routed by the MUX extension to specific application end
points.  The use of MUX and it's opcodes will be transparent to the
application layers above (as are all opcodes).

> a few subprotocols with clear needs will want to define new opcodes.
> And I am hopeful that eventually browsers will provide APIs for people
> to plug in their own extensions.

They may do, but if so then they are adding API to allow user defined
extensions and this is separate to the use of any subprotocols.
Specifically the usage of any new opcodes would need to be negotiated
in the hand shake via the extensions header on not via the protocol
header.       Remember that the use of new opcodes (and extensions)
may have an impact on intermediaries as they are obliged to understand
all negotiated extensions before performing some actions.  the use of
new subprotocols impose no such restrictions on intermediaries.

regards