question: are async calls into the vm Send?
wez opened this issue · comments
I'm using mlua quite heavily in a couple of applications.
One pain point that I have in one of them is that that futures that result from async calls into the VM are !Send, making it rather awkward to integrate in the regular multi threaded tokio runtime.
Is piccolo different?
Also: one thing I love about mlua is how easy it is to define custom embedded modules and types and map them into lua.
How's that look in piccolo? I'm happy to read code examples, but I didn't spot anything obviously like that on a very very quick glance. Can you point me to stuff to look at?
Thanks! This project sounds very interesting to me!
Just to note, next mlua version will have send feature flag replaced with sync to easily work with multithreaded apps. Under the hood it will use ReentrantMutex to support multi lock within a single thread.
Also: one thing I love about mlua is how easy it is to define custom embedded modules and types and map them into lua.
How's that look in piccolo? I'm happy to read code examples, but I didn't spot anything obviously like that on a very very quick glance. Can you point me to stuff to look at?
There's not much to look at, this project is still in its early stages. Usually most things work but they're not easy. You can check out the util crate and the [Static]UserMethods helpers: https://github.com/kyren/piccolo/blob/master/util/src/user_methods.rs if you want to see an example.
Is piccolo different?
Yes, in piccolo the entire VM isn't even Send, so the question is easier to answer.
Making the VM Send safely involves either something really complicated like kyren/gc-arena#61, which I will probably never do, or something better like https://doc.rust-lang.org/beta/unstable-book/language-features/auto-traits.html which would enable a much cleaner way to do the same thing.
I could take piccolo and do the same thing that mlua does, which is to sort of guard all of the entry points to user defined objects and require that they be Send, therefore being able to soundly assert that the whole VM itself is Send.
However, gc-arena allows deep GC integration from safe Rust, which is the whole big selling point, so I can't universally require Send on custom GC objects without doing so from gc-arena, which is the whole idea behind kyren/gc-arena#61 which is too complicated to do without custom auto traits.
But you the user could do the same assertion yourself, and then unsafely (but soundly, if you don't break the rules) impl Send on something holding a Lua, but the assertion would need to cover any custom GC object you insert into the VM.
This is all irrelevant anyway because piccolo does not provide automatic async integration or return any futures, so you would have to design such a thing first. This is always going to be fundamentally different in piccolo because there are at least two separate ways to integrate async code into piccolo in the first place, depending on whether you want to use Lua coroutines or not.
piccolo does not compete with mlua, or it at least it shouldn't be considered to. piccolo competes with PUC-Rio Lua or Luau, mlua could even wrap piccolo and provide exactly the same API, but it would take away a bunch of the usefulness of piccolo so it's probably not worth it.
If you're looking for something ready to go right now, use mlua. If you want to safely sandbox scripts, use mlua with Luau. If you need to use piccolo, you'll probably already understand why and won't need this explanation.
This issue made me think about it again, and I'm going to look with fresh eyes on a way to make gc-arena conditionally Send in a different way than kyren/gc-arena#61, because being able to make a Lua instance Send without infectious unsafety is also very important to me.
I believe there is now a path forward with gc-arena towards safe "send jails" that can reasonably make the piccolo VM send.
I'm going to close this issue though because the discussion of mlua is unrelated.