Subject: Re: [hybi] Multiplexing extension spec draft 02

From: "Arman Djusupov" <[email-masked]>
Date: Mon, 6 Feb 2012 11:26:31 +0200

Hello,

The peer is able to refuse creating a new channel in real-time. Asking the
user to specify a maximum number of channels will impose a static limit
which is not flexible when adaptive scaling is required. Each side should be
able limit the number of sub-channels depending on the current load (idle
threads in thread pool etc.) 

When the peer rejects creating a new sub-channel there are still 2 options
(a) stop creating new channels or (b) create a new connection and try to
allocate new channels within that one.

With best regards,
Arman

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Martin Sustrik
Sent: Monday, January 30, 2012 8:42 AM
To: Willy Tarreau
Cc: [email-masked]
Subject: Re: [hybi] Multiplexing extension spec draft 02

On 01/30/2012 03:10 PM, Willy Tarreau wrote:

> This means the endpoint should be smart enough not to announce the 
> whole buffer for a single channel, and always announce less than what 
> remains divided by the number of channels, in order to be able to 
> infinitely divide it. That's suboptimal but offers equal space to
everyone.

This way one overloaded channel can starve out other channels, although
it'll happen in multiple iterations.

In case of 2 channels, one idle, the other one overloaded the overloaded
channel would first advertise 1/2 of the rx buffer, then 1/2 of the
remaining part of the buffer (1/4 of the whole buffer) then 1/2 of that etc.

The whole algorithm asymptotically approaches the entire buffer size. 
Once it is full, both channels are stuck, even the innocent one that was
sending no data.

The only way to ensure that one channel won't starve another one is to
allocate separate buffer for each channel.

That in turn means that you can DoS your peer by simply asking for
inordinate number of channels and running it out of memory that way.

SCTP which is another protocol with multiplexing built in solves this
problem by asking user to specify the max number of channels when the
endpoint is established.

Martin
_______________________________________________
hybi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/hybi

---

From: "Arman Djusupov" <[email-masked]>
Date: Mon, 6 Feb 2012 11:28:36 +0200

Hello Takeshi,

"Any compression extensions negotiated apply only to the channel they are
negotiated on -- therefore, any compression extension in the initial
handshake applies only to logical channel 1.  If WebSocket payload data is
masked by a per-frame key, such masking is applied to frames for each
logical channel separately."

Providing the ability to specify a compression extension per sub-channel
greatly increases the complexity of the implementation. I believe it would
be simpler if the compressed logical channels were grouped in a compressed
connection, while the logical channels that do not need compression were
grouped in a separate non-compressed connection. Maintaining a DEFLATE
window per sub-channel is very expensive, doing so could eliminate all the
benefits of multiplexing.

But if you believe that there is a strong use case for per channel
compression, then we could support both options (common compression and per
channel) by having logical channel 1 perform its own handshake rather than
using the initial handshake for it.

With best regards,

Arman

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Takeshi Yoshino
Sent: Friday, January 27, 2012 6:55 AM
To: [email-masked]
Subject: [hybi] Multiplexing extension spec draft 02

http://tools.ietf.org/html/draft-tamplin-hybi-google-mux-02

Diff http://tools.ietf.org/rfcdiff?url2=draft-tamplin-hybi-google-mux-02.txt

No semantic change.

- some typo fixes

- named "control channel" explicitly

- updated reference to point RFC

- added some vertical space for readability

---

From: Martin Sustrik <[email-masked]>
Date: Mon, 06 Feb 2012 22:06:18 +0100

Hi Arman,

> threads in thread pool etc.)

Agreed. The max-channel parameter is an API concern, not the protocol 
concern.

Martin

---

From: Martin Sustrik <[email-masked]>
Date: Mon, 06 Feb 2012 22:09:53 +0100

On 30/01/12 08:00, Takeshi Yoshino wrote:
> On Mon, Jan 30, 2012 at 15:05, Martin Sustrik <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     The scenario that I was referring to looks like this:
>
>     1. Receiver on channel 1 has 100 bytes in rx buffer free.
>     2. Despite of that it announces quota of 200 bytes.
>
> ...snip...
>
>     My original proposition was meant to explicitly prevent this kind of
>     situation.
>
>
> Yes, I imagined similar example from your original post, and proposed
> the generalized text.

Ok. Sorry for not getting it straight away.

Still, even if we don't want to use MUST, so that the spec allows for 
exotic mutliplexing algorithms that may be ok with one channel blocking 
other channels for arbitrary amount of time, it still may be a good idea 
to use SHOULD.

IIRC "SHOULD" is defined as "do so unless you have a very good reason 
not to" which fits the bill here.

Martin

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 7 Feb 2012 16:33:54 +0900

Hello Arman,

On Mon, Feb 6, 2012 at 18:28, Arman Djusupov <[email-masked]> wrote:

> window per sub-channel is very expensive,
>
Good point. It's not so good idea to have compression contexts for each
logical channel from the perspective of memory consumption.

>  doing so could eliminate all the benefits of multiplexing.
>
We still can reduce TCP's portion of memory pressure.

WS + compression
  memory pressure = big
  flexibility = big
WS + mux + channel-wise compression
  memory pressure = middle
  flexibility = big
WS + mux + grouped compression
  memory pressure = small
  flexibility = small

> channel)
The original assumption because of which we chose this approach is that
each stream might have very different characteristics (one is text, another
is compressed binary, ... different compressibility/redundancy pattern) and
so it would be better to make it able to switch compression on/off for each
stream. I still think it's useful.

> by having logical channel 1 perform its own handshake rather than using
> the initial handshake for it.
>
We shouldn't add any more latency for opening handshake. So, we cannot take
the way to do handshake for the logical channel 1 using a mux control
message (since we need to wait until we receive server's opening handshake).

I'd suggest
a) just give "Sec-WebSocket-Extensions: mux, deflate-frame" and
"Sec-WebSocket-Extensions: deflate-frame, mux" different meanings as RFC
6455 says order is significant. I.e. "mux, deflate-frame" means
deflate-frame is applied to logical channel 1 while "deflate-frame, mux"
means deflate-frame is applied to physical channel. When received an
AddChannel mux control with R=1 (delta-encoded), Sec-WebSocket-Extensions
header for the new logical channel is computed from extension entries after
mux.
b) change mux extension parameter spec to be able to carry compression
settings for physical channel
c) change mux extension parameter spec so that it holds compression
settings for the logical channel 1 in it

I prefer a).

Problem here is that we need to set client's default behavior. If a browser
sends "deflate-frame, mux, deflate-frame", the server can choose
physical/channel-wise by rejecting either of deflate-frame tokens to show
its preference. The browser can still insist on use of physical channel (or
logical channel-wise) compression by sending "deflate-frame, mux" (or "mux,
deflate-frame").

Thanks

---

From: "Arman Djusupov" <[email-masked]>
Date: Tue, 7 Feb 2012 12:05:43 +0200

Hello Takeshi,

I also prefer a), unless we find a better solution in the future. I think
that the specification can prohibit or at least discourage using both
physical channel compression and logical channel compression. This simply
doesn't make sense and does not give any benefits. Even if the specification
does not prohibit this than the server still can refuse such a mixture of
extensions. Normally when a client sends "deflate-frame, mux,
deflate-frame", the server can always reply with "deflate-frame, mux " or
"mux, deflate-frame" depending what it supports or what is preferable to
this particular server. Using the Sec-WebSocket-Extensions header in such a
manner also looks logically consistent since the order in which the
extensions are applied in this case indeed does make a meaningful
difference.

With best regards,

Arman

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 8 Feb 2012 14:22:43 +0900

On Tue, Feb 7, 2012 at 06:09, Martin Sustrik <[email-masked]> wrote:

> other channels for arbitrary amount of time, it still may be a good idea to
> use SHOULD.
>
> IIRC "SHOULD" is defined as "do so unless you have a very good reason not
> to" which fits the bill here.

OK. I'll try to take in your suggestion.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 8 Feb 2012 14:26:43 +0900

On Tue, Feb 7, 2012 at 19:05, Arman Djusupov <[email-masked]> wrote:

> physical channel compression and logical channel compression.
>
Agreed

> This simply doesn't make sense and does not give any benefits. Even if the
> specification does not prohibit this than the server still can refuse such
> a mixture of extensions. Normally when a client sends "deflate-frame, mux,
> this particular server.
Yes.

> Using the Sec-WebSocket-Extensions header in such a manner also looks
> logically consistent since the order in which the extensions are applied in
> this case indeed does make a meaningful difference.
>
Yes.

Thanks. I'll reflect this idea in the next version.

---

From: "Arman Djusupov" <[email-masked]>
Date: Wed, 8 Feb 2012 09:50:00 +0200

Great! Thank you.

From: Takeshi Yoshino [mailto:[email-masked]] 
Sent: Wednesday, February 08, 2012 7:27 AM
To: Arman Djusupov
Cc: [email-masked]
Subject: Re: [hybi] Multiplexing extension spec draft 02

Agreed

This simply doesn't make sense and does not give any benefits. Even if the
specification does not prohibit this than the server still can refuse such a
mixture of extensions. Normally when a client sends "deflate-frame, mux,
deflate-frame", the server can always reply with "deflate-frame, mux " or
"mux, deflate-frame" depending what it supports or what is preferable to
this particular server.

Yes. 

Yes.

---

From: John Tamplin <[email-masked]>
Date: Wed, 8 Feb 2012 13:34:53 -0500

On Mon, Feb 6, 2012 at 4:28 AM, Arman Djusupov <[email-masked]> wrote:

> “Any compression extensions negotiated apply only to the channel they are
> logical channel separately.”
>
> **
>
> ** **
>
> **
The rationale was that an intermediate MUX would otherwise have to
decompress inbound channels and recompress the combined channel, which
seems a waste.  I agree that in the browser case, where the individual
channels never exist separately, it would be better to do compression only
on the physical channel.

So, it seems like we need to have options for either behavior.

-- 
John A. Tamplin
Software Engineer (GWT), Google