I recently stumbled upon Roshi, a CRDT set server written on to of Redis. Roshi looked like an obvious good usecase for Riak. Roshiak is a simple demonstration of how this could be implemented on top of Riak.
Because Roshiak is written on top of Riak it offers some functionality that Roshi lacks including:
- Elasticity - nodes can be added and removed and Riak will rebalance the data automatically.
- Read repair - Riak handles ensuring that all nodes contain the same values for a key on read.
- Active Anti-Entropy - Riak will occasionally make sure that the keys on various nodes match even without being read. Roshi-walker implements a similar mechanism. AAE in Riak is implemented using merkle trees, minimizing the amount of data moved around.
- Multi datacenter - If you don’t mind paying, Riak gives multi datacenter support.
- Hinted handoff - If a node goes down, the lost data is shuffled to other nodes, ensuring the replication factor is maintained.
Roshiak is implemented in ~140 lines of code, about 1/10th of that of Roshi as of this writing and supports more operational features.
Roshi is actually used in production. Roshiak has not even been tested in production. Currently Roshiak is probably slower than Roshi. Roshiak only supports getting the entire dataset on read, Roshi supports more sophisticated ‘select’ calls. There are a few possible solutions for this in the Roshiak model depending on requirements.
The Roshi set implementation is also incomplete in that it does not garbage collect aggressively.
Part of doing this project was realizing how behind my help libraries were relative to their dependencies. Right now I am working on bringing them up to date then publishing them through OPAM. You will also need a local Riak running. Setting up a devrel is straight forward and documented at the Basho website.
This won’t work yet, but soon!
opam install riakc cd roshiak make examples
You will now have examples/roshiak_cli.native.
roshiak_cli.native add localhost 8081 snoopdogg foobar roshiak_cli.native sel localhost 8081 snoopdogg roshiak_cli.native del localhost 8081 snoopdogg foobar roshiak_cli.native sel localhost 8081 snoopdogg