AutoMerge for sync
filipesilva opened this issue · comments
Heya, really cool project you made!
I was reading the bit about the data persistence and how it's hard to sync between clients without incurring costs right now, and https://github.com/automerge/automerge came to mind. It uses CRDTs to sync between clients and get to a consistent state, working as offline-first.
I think using that approach for mapus could work if sync was between clients with something like web rtc, and clients kept local state. Then whenever two users that wanted to do stuff together connected to each other, they'd propagate the changes they have to the map. A new user coming to see that map without no other client online would be out of luck though.
Maybe that's enough for ephemeral collaborative sessions though?
AutoMerge has a webrtc lib in https://github.com/automerge/mpl, but for the life of me I can't understand how that example connects.
An alternative for AutoMerge is https://github.com/yjs/yjs, that seems to have a first party webrtc connector in https://github.com/yjs/y-webrtc. I've heard good things about yjs so maybe it's a better starting point right now than AutoMerge.
That's pretty cool yeah. That said, the way Mapus is set up it doesn't really have any way of going "out of sync" or causing any sort of inconsistencies simply due to the fact that users can't edit or make changes to annotations once placed, only when deleting. So each user is simply creating new "objects" and pushing to that independently, if a user goes offline and they keep working, and they go online again they will see the new annotations the other users made, and their own changes will also be sent to the database and shown to the rest. It won't cause conflicts in that sense (and even then, Firebase itself does handle those scenarios AFAIK)
I suppose it could be optimized a variety of ways though in terms of cost. For example, go "offline" if only 1 user is working on a map until someone else joins, don't update cursor coordinates if you're out of view from any other user, add a bigger delay between cursor coordinates and simply smooth out the movement between positions, same with freedrawing (just save the actual path once complete & update for all users), etc. Still I have to admit I'm not 100% certain it's all too expensive, I just didn't want to take the risk myself :P Firebase gives 1GB of free storage and 10GB of free download (with uploads not being metered), + $5/GB extra stored and $1/GB extra downloaded. Cursor coordinates and whatnot take just a few bytes, it would take many users and heavy usage to start getting costly.
Yeah I think you could go pretty far with a public firebase. I've seen it get expensive (20k+/month) but with stuff that was obviously going to be expensive. Bit of a pity that firebase doesn't let you just set a limit and turn off the service if it's hit...
If this issue is rather about self-hosting options of Mapus, and indeed this comes to mind having seen the request from @rosano on Twitter, their might be other "rehoming" possibilities in the air, to only replace Firebase, if not with a CRDT.
- Supabase, a dedicated Firebase replacement, have a recent beta release that could be worth checking out.
From that conversation, other alternatives also appear worth having a look:
