Using Async through Lift effect?
rl-king opened this issue · comments
What would be the recommended way of using async in combination with fused-effects?
It looked like I need something like Distribute. But the comment at the top
-- TODO: We should kill this entirely, because with fused-effects 1.0 we can unlift the various runConcurrently operations.
made me curious to look for (simpler) alternatives so I tried to get it working with liftWith but got stuck....
Situation
I'm fetching some data with an http effect and insert that data into the database with another effect. I'd like to call mapConcurrently on a list and fetch and insert using those effects.
Let me know if more info/context is required, thanks!
@rl-king: I’m doing something vaguely similar with forkIO:
liftWith (\ ctx run -> (<$ ctx) <$> CC.forkIO (void (run (m <$ ctx))))Here I take m :: m a, and squeeze it through the liftWith. Note that since forkIO takes IO (), we have to discard the action’s result, which means this actually discards effects of things like Control.Carrier.Strict.StateC. So this is to be used with caution.
I think something similar could work for async, but I haven’t actually gone through the exercise. The trick is to remember that ctx is a functor holding the state for each effect, so threading it through the list should mean that we use the initial value only once and then thread the results through the next, and so on, left-to-right. We might want to relax that restriction if it would prevent us from actually usefully making this work concurrent; but again, do be careful with that. (It’s rather like the thing with e.g. MonadBaseControl and finally where state changes can be forgotten, if you’re familiar with that.)
Looking for an example I actually came across your forkIO example in starlight :). Which does solve my problem as I'm not interested in the result at the location I'm calling the function, and it doesn't use any state.
However, I'm still wondering if it's actually possible to use liftWith to do for example: async :: Has (Lift IO) sig m => m a -> m (Async a) like async
I always end up having the ctx in the wrong position, which makes me think it's impossible, but I'm just not sure.
For example:
async' :: Has (Lift IO) sig m => m a -> m (Async a)
async' m =
liftWith $ \ctx run -> async (run (m <$ ctx))Errors with:
...
Expected type: IO (ctx (Async a))
Actual type: IO (Async (ctx a))
...
Which makes sense, but I don't think there is a way around that
You can only have m a -> m (Async a) with this context threading trick if you know that CTX is isomorphic to Identity, as far as I'm aware. fused-effect's liftWith is too polymorphic for this. I know @robrix once added support for context constraints in the past but abandoned it
I think we could also do this if we had a Traversable constraint for the context.
cf #194
Ah, thanks for the clarification!
I wrote a wrapper for the async library here: https://github.com/patrickt/fused-effects-async/blob/master/src/Control/Effect/Concurrent/Async.hs
Great, thanks for sharing
Nice! I still think it's a shame that you can't have just Async without letting your programs do arbitrary IO, but this is a good step forward.
@ocharles FWIW I agree. The space around concurrency is pretty interesting; for example, if I have a Thread effect with a fork operation, and I run Writer operations on both the forking and forked thread, I presumably want to serialize appends—but there isn’t a single suitable one-size-fits-all serialization strategy. For example, do I want to serialize background tells as they occur? Or all of them at the point at which I run fork?
Async’s semantics actually complicate things, since you could also make an argument for synchronizing into waiting threads. (And what if there are multiple such…?)
All of which suggests to me that concurrency is a mess (not exactly a novel observation, I know), and that perhaps I’d prefer exploring the use of STM instead.
I'm currently in the process of wiring some of my fused-effects types into the lifted-async package. Everyone has to contend with how effects distribute over concurrency at some point or another if they ever use effect systems that intend to leverage concurrency.
I respect the hesitance to put this in the base package as you don't want people to eschew effect systems altogether because they "silently do the wrong thing" and destroy the intended semantics. At the same time, I think it would be nice to have a fusednomicon style package (to steal some rust lore) that puts the burden of semantics verification on the user, without forcing them to redefine all of the async machinery and standard interpreters.
Would there be support for this sort of thing? Or is the "correct" way to do it, through the use of the lifted-async package?
I definitely think it’d be nice to have a straightforward, idiomatic answer for this in the fused-effects ecosystem, I just haven’t really got the time or attention span to devote to that myself at the moment. I’m likely to explore the design space more in the future, but don’t let that stop you from taking a look!
For those interested, there are MonadUnliftIO instances for the carriers that support them now, so you can bring in unliftio for fairly fluent access to async stuff.