Custom data directory paths for Sled and Rocksdb

[HyperAlch]

Currently, the data directory for Sled and RocksDB are decided by this code:

        let dirs = ProjectDirs::from(
            config.qualifier.as_deref().unwrap_or(""),
            &config.organization,
            &config.application,
        );
        let parent_dir = match dirs.as_ref() {
            Some(dirs) => dirs.data_dir(),
            None => Path::new("."), // todo: maybe warn?
        };

This makes sense for most people since they are most likely using bevy_pkv for saving a users game state. However, there are edge cases where selecting the data path manually may be necessary.

For example, in my case I need to ship my RocksDB files with my game as I'm using bevy_pkv to allow my game designers to change the state of the game and it's entities to build levels and areas. My idea is we could rewrite the new method for SledStore and RocksDBStore to take 2 parameters like this:

// Old
pub(crate) fn new(config: &StoreConfig) -> Self {

// New
pub(crate) fn new(config: &StoreConfig, custom_path: Option<&Path>) -> Self {

And since we don't want existing users to have to change their code, we could change PkvStore to always pass in None into custom_path and make a second version called PkvStoreWithPath that allows for a path to be passed in like so:

impl PkvStoreWithPath {
    pub fn new(organization: &str, application: &str, custom_path: &Path) -> Self {

---

Once again, I would be happy to implement this myself, and it should be much less complicated than when we added RocksDB, because we don't have to deal with the build.rs stuff.

[HyperAlch]

Update

I just realized you have different types of new like new_with_qualifier.

So instead of making PkvStoreWithPath we can just keep PkvStore and do something like this:

impl PkvStore {
    pub fn new_with_path(organization: &str, application: &str, custom_path: &Path) -> Self {

[Johan Klokkhammer Helsing]

I think a builder or a config struct would be better. Haven't quite made up my mind which would be the best of the two though

[Johan Klokkhammer Helsing]

I'm also slightly worried about scope creep. Depends how much it complicates things

[HyperAlch]

I mean, giving the users the ability to choose where their data is saved isn't scope creep, it's a basic feature.

[HyperAlch]

You are essentially deciding where the user can and can not save their data, and if they don't want to save it where you choose they just can't use the crate unless they fork and modify it.

If your worried about scope creep don't use a builder or config struct to implement adding an Optional &Path value to the ecosystem.

That's just my 2 cents.

[HyperAlch]

The only reason I have been so active with this project is because the crate is in an unusable state for me.

1. I needed a back end that isn't dead and is maintained, which is now done.
2. I want to be able to save my data where I choose.

And of course I'm willing to do the work to implement both. If we were adding a bunch of new features, then I would agree that a builder or config struct would be the best option. But adding a single extra parameter? That seems unnecessary to me, personally.

Ultimately, I'm not trying to add a bunch of random features to your project, I just need those 2 things and we are already half way there. Of course, it's your project and if you don't want to add the ability for people to choose where they save data, that is 100% okay.

[Johan Klokkhammer Helsing]

You are essentially deciding where the user can and can not save their data, and if they don't want to save it where you choose they just can't use the crate unless they fork and modify it.

Correct. The mission of this crate is to solve the problem of "I have some serde-serializable data, I wish i could retrieve it the next time my game is launched, I don't want to have to learn how it's done, but I want it to be well-behaved on all platforms."

Note, I'm also not saying it's a definite no, just that it's bordering on scope creep. I'm just telling you up front that I need to think about it, so you don't waste your time on a PR I might reject.

If your worried about scope creep don't use a builder or config struct to implement adding an Optional &Path value

The reason I want a builder, is that if we have more than two optional parameters, it leads to a combinatorial explosion of parameters. e.g. new_with_qualifier_and_path_and_something and all the various permutations. A builder or config struct is a relatively simple construct, and I will not remove the simple constructor just because i add a builder or config.

That said, I like making things extensible as long as it:

1. Doesn't complicate the simple usage
2. Doesn't make the implementation hairy

Some of my concerns with paths, is that it's probably going to be irrelevant to most users and doesn't make sense on wasm.

If you know where you want to store your db, and what db you want to use and only target one platform, I don't really understand what kind of value of the abstraction layer this crate provides to you. Maybe just use the crates directly?

[Johan Klokkhammer Helsing]

So path would be mutually exclusive with qualifier/name/etc/org. I agree a builder/config doesn't make sense, at least not yet.

[HyperAlch]

Alright, well let me know when you decide what you want to do.

[HyperAlch]

If you know where you want to store your db, and what db you want to use and only target one platform, I don't really understand what kind of value of the abstraction layer this crate provides to you. Maybe just use the crates directly?

I want to use bevy_pkv and not RocksDB directly because even though my game engine will need custom paths, the game itself will not. This means the game itself will use the default, cross platform, features of this crate. This includes WASM.

I would rather use bevy_pkv with a completely optional &Path value then have to worry about 2 different code bases that would do 99% the same thing.

[Johan Klokkhammer Helsing]

Alright, well let me know when you decide what you want to do.

After realizing the mutual exclusiveness, it doesn't seem too bad to just another simple constructor, go for it :) Something like this?

#[cfg(any(sled, rocksdb))]
fn new_with_path<P: AsRef<Path>>(path: P) -> PkvStore

What does a path mean in rocks db, btw? Is it a single file, or a folder?

[HyperAlch]

This constructor you wrote looks good to me. I'll go ahead and use it.

#[cfg(any(sled, rocksdb))]
fn new_with_path<P: AsRef<Path>>(path: P) -> PkvStore

---

Both RocksDB and Sled are creating folders with multiple files inside when I look at the paths for the bevy_pkv tests.

What does a path mean in rocks db, btw? Is it a single file, or a folder?

test_set root

Sled

RocksDB