Context/State for Job Workers?

[Matthew DuCharme]

To preface, I'm a fairly amateur rust developer and what I'm suggesting is no doubt a breaking change for the worker API, and I'm probably not skilled enough to implement a completely sound library change with this sort of scope.

I'm attempting to use workers written in rust, but I quickly ran into issues with managing state that exists outside of the worker itself. While reading over previously closed issues (particularly #13 and the PR that closed it), I'm wondering if there is a roadmap or want for something akin to rocket's request guards. My primary use case for this is managing DB connections with diesel. Something utilizing https://docs.rs/state/0.4.1/state/ might be possible?

I suppose with the current API I could write a closure that wraps a Fn and grabs a connection, but that seems a bit awkward and I could easily see the function stringing together the workers becoming harder to reason about.

This isn't an absolutely critical or show-stopping goal, since I'm writing my workers in python in the interim but it would be a "nice to have" so that my workers and main codebase can share the exact same model structs and logic without duplication.

[Jon Gjengset]

I'm not sure I follow exactly what it is you are proposing? If you wanted access to a database connection in your handler, then I think what you'd do is construct a connection pool (something like r2d2::Pool), and then call get on that pool (which only requires &self, so is fine from a Fn closure) to get a connection. Will that not work for the use-case you have in mind?

I think ultimately this crate should probably be re-written to use async/await, I just haven't gotten around to it yet, as it would be a fairly substantial change. Now that I think about it though, that probably wouldn't actually make a difference for your use-case 😅

[Matthew DuCharme]

My original thought was something like:

fn do_db_work(job: Job, connection: State<Connection>) -> ioResult {
    ...
}

And since my connection pool currently isn't a static global I didn't have a way to grab a connection inside of my worker class. Now I could do something like:

fn do_db_work(job: Job, connection: &Connection) -> ioResult {
    ...
}

consumer.register("db_work", |job: Job| do_db_work(job, &connection_pool.get()))

But it's a bit awkward. All that being said, I honestly hadn't considered the fact that R2D2's connection pool is Send + Sync, so it could technically be lazy_static initialized. I'll have to play around with my architecture some more to see if I can get a cleaner representation of it. I'll close out this issue since it's not really an issue with your library, it's an issue with my structure. Thanks!

[Jon Gjengset]

Let me know if you come up with some specific change that would make your life easier after fiddling with it a bit yourself :D