Rendered at 13:10:21 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
giovannibonetti 1 days ago [-]
I find it fascinating when different people independently arrive at the same architecture when working on a hard problem like this. In my company we built our offline-first apps with PowerSync, which has the same idea of optimistic local changes while waiting for the central server to acknowledge the definitive changes. In PowerSync's case, the sync engine reads Postgres replication logs directly.
tyleo 24 hours ago [-]
Yeah, I'm surprised this pattern doesn't have a more general name.
I've also used it to synchronize content for local multi-window editing applications: each window is its own process but routes through one "coordination" process to ensure consistent ordering.
Easily solves some cross-process concurrency issues.
It's not quite the same since we aren't doing any sort of interpolation. I would acknowledge it's a similar idea though. Perhaps the similarity is one reason we came to this solution as a game studio.
Rohansi 23 hours ago [-]
In this case it's just a (transaction/commit) log. Very common in distributed systems, databases, and filesystems.
Rohansi 15 hours ago [-]
I misread the question but it could be a read-your-writes consistency of the transaction log!
wpietri 24 hours ago [-]
For those interested in a keep-everything-hot approach like this, it's worth checking out the 25-year-old library Prevayler. Full ACID guarantees, and radically faster than a database. I happily used it for a project forever ago and was disappointed to see it so thoroughly ignored.
h4ch1 16 hours ago [-]
I did a bit of digging after reading this comment and came across its' currently maintained implementation in Clojure by author Klaus Wuestefeld[0] :)
Oh, this is interesting. What sort of project did you use it for?
davidanekstein 23 hours ago [-]
I’m surprised there was no mention of operational CRDT’s, or CRDT’s generally.
tyleo 23 hours ago [-]
Yeah, I glossed over a bunch of the things we tried near the bottom of the "In-Memory Database" section. One of the things was CRDTs.
We did a few days of CRDT investigation along with the other techniques and decided the juice wasn't worth the squeeze when we found a simpler solution to our problem.
That being said, we've actually retained a CRDT-like implementation in our codebase and have been considering moving shapes (which have a slightly different set of tradeoffs) to that model.
Rohansi 23 hours ago [-]
There's no need for CRDTs here because there is always a central authority.
forrestthewoods 22 hours ago [-]
Cool post!
I don’t quite understand the “funnel” section. Users see some change local immediately (S1->S2). And all users send all commands to the host. The host then effectively chooses the sort order and broadcasts back.
So does the initial user effectively rewind and replay state? If action applied to S1 was actually from another player they rewind to S1 and apply the sequence?
How many state snapshots do they need to persist? There was no mention of invertible steps.
I feel like I’m missing a step.
tyleo 21 hours ago [-]
Whether users see the action locally or not is decided on a per-action basis. For more visual actions like movement, we use a local imposter that gets overwritten on completion of the action (success or failure). These revert to the most recent “committed” state which already passed through the action funnel.
We have some actions which happen when saving config UI which don’t really need realtime sync when “save” is pressed and we treat them more like a normal POST request (e.g. we wait for full commit before rerendering). The form fields are kinds a built-in imposter.
For state snapshots we only do big ones. The “full save” that happens every 25 actions or so. These aren’t generally used for reverting people.
So it kinda works like:
1. Do I have an outstanding imposter? If so read from it.
2. Otherwise read from the committed state.
forrestthewoods 12 hours ago [-]
Ahh that’s interesting.
So instead of “rollback and replay” it’s more like “have a committed state and transient preview on top that may or may not commit”.
Implementation could vary between “commit the preview” vs “delete preview and create commit”. But that’s just an implementation detail.
I've also used it to synchronize content for local multi-window editing applications: each window is its own process but routes through one "coordination" process to ensure consistent ordering.
Easily solves some cross-process concurrency issues.
https://en.wikipedia.org/wiki/Client-side_prediction
[0] https://github.com/klauswuestefeld/prevayler-clj
We did a few days of CRDT investigation along with the other techniques and decided the juice wasn't worth the squeeze when we found a simpler solution to our problem.
That being said, we've actually retained a CRDT-like implementation in our codebase and have been considering moving shapes (which have a slightly different set of tradeoffs) to that model.
I don’t quite understand the “funnel” section. Users see some change local immediately (S1->S2). And all users send all commands to the host. The host then effectively chooses the sort order and broadcasts back.
So does the initial user effectively rewind and replay state? If action applied to S1 was actually from another player they rewind to S1 and apply the sequence?
How many state snapshots do they need to persist? There was no mention of invertible steps.
I feel like I’m missing a step.
We have some actions which happen when saving config UI which don’t really need realtime sync when “save” is pressed and we treat them more like a normal POST request (e.g. we wait for full commit before rerendering). The form fields are kinds a built-in imposter.
For state snapshots we only do big ones. The “full save” that happens every 25 actions or so. These aren’t generally used for reverting people.
So it kinda works like:
1. Do I have an outstanding imposter? If so read from it.
2. Otherwise read from the committed state.
So instead of “rollback and replay” it’s more like “have a committed state and transient preview on top that may or may not commit”.
Implementation could vary between “commit the preview” vs “delete preview and create commit”. But that’s just an implementation detail.
Anyhow. That’s neat and clever. Thanks!