What to do about calls to get_event_loop() in Trio context when no loop is active?
oremanj opened this issue · comments
Many asyncio objects (such as asyncio.Lock) call asyncio.get_event_loop() in their constructor, if no loop was passed. Much asyncio-using code seems to assume that such objects can be created when no loop is running, and that they'll attach themselves to the loop that winds up running later.
This pattern works fine (at least post #66) outside of Trio context. get_event_loop() will delegate to the installed event loop policy; if no custom policy has been installed, it will return the event loop for the current thread, creating it as a SyncTrioEventLoop if necessary.
Inside Trio context, though, we currently expect every event loop to be a TrioEventLoop bound to an async with open_loop() block somewhere. If no TrioEventLoop has yet been created, what can we do?
- Old behavior (pre #66): throw an exception from
get_event_loop(). This caused problems for a user on gitter today: https://gitter.im/python-trio/general?at=5e1a40f2b720fa5b3cf9dbc4 - New behavior (post #66): return None from
get_event_loop(). This would have fixed the user's issue, but is confusing because it raises an error later (when someone tries to use the loop) without indicating why they got None.get_event_loop()for the default asyncio event loop policy never returns None. - Possible change: return a dummy event loop that raises when you try to use it. This would also fix the user's issue, and is better than returning None, but doesn't allow for patterns like
lock = asyncio.Lock()
async with trio_asyncio.open_loop():
[use lock here]
which seems like it might still break some use cases.
- Possible change: create a new TrioEventLoop, set the current_loop cvar to use it, and teach open_loop() to manage the existing ambient loop instead of creating a new one if an existing ambient loop exists and isn't running. (It should still be possible to nest open_loop() calls to get two different loops with different scoping behavior if the outer loop is running.) This seems like it may have the closest behavior to non-trio-enabled asyncio: you can still set things up against a loop when it's not running, but your callbacks won't, like, run.
- Possible change: create a new TrioEventLoop and start running it in a system task. I think this is strictly inferior to the previous bullet but it was the first thing I thought of so I mention it for completeness, in case there's some reason I'm missing why it would be better. The hazard is that in the above example,
Lockwould be in the global loop which is not the same loop as everything inside theasync withblock. And while I can't immediately think of a reason this wouldn't work, if it breaks it's going to be really confusing to understand.
Third possible solution: create a dummy event loop object with a __getattr__ that delegates everything to the currently-running TrioEventLoop, or raises an exception if that doesn't exist.
That being said I'd sort this possibility between your two, in order of decreasing desirability.
Would it be fair to say that in this library, this line too falls in this trap:
Mostly, yes. Caching the current event loop is a bad idea these days.
Thank you @smurfix
I am currently using python-trio to interact with asyncio based grpclib (the details of it really is that I use betterproto and that in turn uses grpclib https://github.com/danielgtaylor/python-betterproto/blob/02e41afd09f0050a10fea764b15279b82cdd6e6b/src/betterproto/__init__.py#L31)
Since there's a lot of asyncio code out there - I was curious about those asyncio code that call get_event_loop() from __init__
Is there a pattern we can come up with to handle this case reliably (short of putting a patch to the project that works with trio)?
The way to handle this pattern reliably is to convince upstream not to use it in the first place.
Yes, trio-asyncio could possibly add a workaround (or two or three …) for it, but (a) none of them reliably work in more complicated scenarios, (b) there are other use cases where this pattern goes splat.
Example for (a): consider two libraries where both A and B set up a trio-asyncio loop. A starts its loop and calls B, which creates one of those objects (which stores a ref to A's loop), then starts its own separate trio-asyncio loop, making things go haywire because freely mixing calls to two different asyncio loops simply does not work.
Example for (b): you might want to set up your objects in your main thread's sync code, but then you run the actual asyncio loop in a subthread while the main thread handles your GUI.
@oremanj shared some more thoughts here: https://gitter.im/python-trio/general?at=609bea20012fc62dd5b44c97