Subject: Re: Mnot's Pub/Sub for the Web

From: Jeffrey Barber <[email-masked]>
Date: Sun, 20 Feb 2022 09:58:56 -0600

I want to second the notion of state synchronization over pub/sub.

Publish Subscribe has notoriously difficult edge cases to contend with. For
example, what happens when you lose connection? How do you resubscribe and
pick up messages missed while disconnected? What if the time was too long,
how do you skip ahead? Do you accept message loss? Or do you need an
external store for picking up a snapshot? What if the link between the
subscriber and publisher is congested?

A key problem with WebSocket is that it has all this great potential, but
it requires you to know a great deal of the trade to be useful. WebSocket
is too low level, and I've written my opinions over at
http://www.adama-lang.org/blog/woe-of-websocket/ with an errata from HN
comments http://www.adama-lang.org/blog/more-websocket-woe/

I don't believe Pub/Sub is much better than WebSocket except that you can
multiplex and have acks, so it does some of the heavy lifting.
https://rsocket.io/ is better in many ways, and it's wonderful until you
use it and realize you must contend with the flow control mechanism as a
first class citizen. In other words, you will have to deal with loss
up-front which is way better than dealing with loss during an event.

The best you can get with Pub/Sub is a measurement of honesty on the
connection, and I regretfully have the details spelled out in this patent:
https://dl.acm.org/doi/10.1145/3477132.3483572 This work emerged from
decades of man-hours being thrown away because MQTT was unable to be made
reliably honest. Most of the problems emerge at scale when you have various
proxies between the client and the data sources, and
https://dl.acm.org/doi/10.1145/3477132.3483572 is my former team's paper
which illustrates the protocol in action.

At the core of this is what is the medium being transmitted? Is it a queue
full of commands, or is it state synchronization. A queue full of commands
is bound to have issues as it will require a snapshot store while state
synchronization is an algebraic game of the server and client playing catch
up. I'm of the opinion that JSON is a pretty cool medium for state
synchronization: http://www.adama-lang.org/blog/json-on-the-brain

Personally, I'm abandoning pub/sub (
http://www.adama-lang.org/blog/pubsub-sucks ) with a bias towards
downstream state synchronization and upstream request response (simple
message delivery).