lucid-kv / lucid

High performance and distributed KV store w/ REST API. 🦀

Home Page:https://clintnetwork.gitbook.io/lucid/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Proposition] Dump / Export

imclint21 opened this issue · comments

Hi,

While I was eating a cake I think about making exporting/dump function.

What do you think @CephalonRho?

Best regards

commented

Shouldn't be too hard to do. We would have to use dashmap instead of chashmap and maybe put an RwLock around it to make sure that it doesn't change during iteration. We might also want to wait until there's progress on persistence since it could impact this significantly.

There's also the question of how we want to expose this feature. We could provide a list of all key names over http and make the client retrieve each value independently (but values could get deleted while that's happening). Once persistence is done there could just be another subcommand instead that dumps all elements to a single file in a binary format and another subcommand to load this file.

Shouldn't be too hard to do. We would have to use dashmap instead of chashmap and maybe put an RwLock around it to make sure that it doesn't change during iteration. We might also want to wait until there's progress on persistence since it could impact this significantly.

About dashmap, I already spoken with @xacrimon, but I think it was only promotion, not real benefits, but apparently it's not the case.

There's also the question of how we want to expose this feature. We could provide a list of all key names over http and make the client retrieve each value independently (but values could get deleted while that's happening). Once persistence is done there could just be another subcommand instead that dumps all elements to a single file in a binary format and another subcommand to load this file.

For me there are two things, for an endpoint go get all keys, it's surely a good idea, I don't really know how we can do that and be REST compliant, but anyway, a backup/restore command should to be implemented, for DevOps usage!