Subject: [hybi] Flat MUX

From: Tobias Oberstein <[email-masked]>
Date: Mon, 15 Aug 2011 03:52:52 -0700

Here is a variant of muxing that 

- does not wrap/nest complete WS connections (recursively) inside WS or
reframe the payload of MUXed messages

- does not require intermediaries to understand message interleaving or
adjust it's fragmentation/coalescing logic

- uses 1 reserved Opcode and 2 reserved bits

The approach is to pack a MUXed message specific FIN bit into reserved bit 2:

RSV2 = MUX FIN

0 = non-final MUX message frame
1 = final MUX message frame

On the base WS layer, all data is sent (by endpoints) using a new Opcode 0x8 and FIN = 1.

The extension data of a Opcode 0x8 frame has the channel ID, which
is an integer encoded the same as the payload length in base WS.

The payload type of a muxed message (text or binary) is signaled in
the first frame of the MUXed message

RSV3 = MUX Payload Type (0 = binary, 1 = text)

For a MUX continuation frame (MUX FIN = 0 after first frame) the RSV3 bit is 0.

An example of 2 MUXed messages interleaved (a text message on chn 23
and a binary message on chn 70):

chn23, text: "data23_a-data23_b-data23_c-data23_d"
chn70, binary: "data70_a-data70_b-data70_c"

WS base layer frames:

FIN, Opcode, RSV2 (MUX FIN), RSV3 (MUX Payload Type), Channel ID, Payload

1, 0x8, 0, 1, 23, "data23_a-"
1, 0x8, 0, 0, 70, "data70_a-"
1, 0x8, 0, 0, 70, "data70_b-"
1, 0x8, 0, 0, 23, "data23_b-"
1, 0x8, 1, 0, 70, "data70_c"
1, 0x8, 0, 0, 23, "data23_c-"
1, 0x8, 1, 0, 23, "data23_d"

The reason to choose RSV 2 and 3 is to allow compatibility with the per-frame
compression extension proposed by Takeshi which uses RSV 1 to signal whether a frame
is compressed or not.

Since there is no interleaving on the base WS layer to understand, intermediaries
don't get confused.

Moreoever, since endpoints are required to send Mux messages unfragmented,
intermediaries can fragment those (and subsequent intermediaries could coalesce again).

The receiving endpoint can just reassemble the fragments that make up a MUX frame.

Mux message frame sent by endpoint:

1, 0x8, 0, 0, 23, "data23_b-"

Fragmented frames received by endpoint:

0, 0x8, 0, 0, 23, "dat"
0, 0x8, ?, ?, "a2"
1, 0x8, ?, ?, "3_b-"

The intermediary can set the RSV bits of the fragments following the first arbitrary.
The receiver only cares about RSV for the first base WS frame of a Mux frame.

The question of channel setup/teardown and channel credit replenishment
signaling is somewhat orthogonal .. tbd.

Is there a catch I missed?

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 16 Aug 2011 09:51:41 +1000

Tobias,

This appears workable.  However, I would prefer a MUX system that
actually uses websocket frames as designed rather than inventing new
framing that is based on a layer above WS messages.

We sweated virtual blood to agree on the framing mechanism that would
be flexible enough for all/most concerns - and it would be a shame
that if on the first hurdle we decided that intermediaries could not
respect the requirements for the spec and discarded the flexibility of
the framing system we have.

WS framing allows extension data per frame.  This is sufficient to tag
each frame with a channel.
WS framing allows interleaving if there is an extension to sort out
the interleaving.
WS framing does not allow intermediaries that do not understand the
extensions to fragment or aggregate frames.

Now, there may be real technical reasons to go for MUX as a layer on
top of ws messages - but allowing poor intermediary implementations
should not be one of them.

cheers

On 15 August 2011 20:52, Tobias Oberstein <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 16 Aug 2011 01:18:04 -0700

> This appears workable.  However, I would prefer a MUX system that actually
> uses websocket frames as designed rather than inventing new framing that
> is based on a layer above WS messages.

I agree, it would prefer that also, but the price is that intermediaries needed
to understand (or at least don't touch) the interleaving. This is required by
the spec, but will intermediaries obey?

Maximum compatibility with intermediaries for me is an even bigger concern
than design cleanness ..

> 
> We sweated virtual blood to agree on the framing mechanism that would be
> flexible enough for all/most concerns - and it would be a shame that if on the
> first hurdle we decided that intermediaries could not respect the
> requirements for the spec and discarded the flexibility of the framing system
> we have.
> 
> WS framing allows extension data per frame.  This is sufficient to tag each
> frame with a channel.
> WS framing allows interleaving if there is an extension to sort out the
> interleaving.

Yes, formally, this is sufficient. However I fear that intermediaries that
do not understand an extension that modifies the rules for interleaving
message fragments could then choose to either block the extension
or don't play to the rules and get confused by the interleaving.

> 
> Now, there may be real technical reasons to go for MUX as a layer on top of

Layer on top (with some flat muxed interleaving .. be it baked into standard
framing or with a new MUX FIN bit) is something I prefer in any case to
wrapping/tunneling whole WS connection within WS like x-google-mux,
which introduces a lot complexity.

> ws messages - but allowing poor intermediary implementations should not
> be one of them.

Ideally, yes. But practically?

> > - does not wrap/nest complete WS connections (recursively) inside WS
> > or reframe the payload of MUXed messages
> >
> > - does not require intermediaries to understand message interleaving
> > or adjust it's fragmentation/coalescing logic
> >
> > The extension data of a Opcode 0x8 frame has the channel ID, which is
> > an integer encoded the same as the payload length in base WS.
> >
> > The reason to choose RSV 2 and 3 is to allow compatibility with the
> > per-frame compression extension proposed by Takeshi which uses RSV 1
> > to signal whether a frame is compressed or not.
> >
> > Since there is no interleaving on the base WS layer to understand,
> > intermediaries don't get confused.
> >
> > Moreoever, since endpoints are required to send Mux messages
> > unfragmented, intermediaries can fragment those (and subsequent
> intermediaries could coalesce again).
> >
> > The question of channel setup/teardown and channel credit
> > replenishment signaling is somewhat orthogonal .. tbd.
> >