mis-detects curio as asyncio when running in curio in asyncio.to_thread (py3.9)
graingert opened this issue · comments
import curio
import sniffio
import asyncio
async def current_framework():
return sniffio.current_async_library()
async def amain():
sniffio.current_async_library()
return await asyncio.to_thread(curio.run, current_framework)
print(asyncio.run(amain())) # prints asyncio - should print curio
this also happens when running curio inside an asyncio coro
import curio
import asyncio
import sniffio
async def run_in_curio():
return sniffio.current_async_library()
async def run_curio():
sniffio.current_async_library()
return curio.run(run_in_curio)
print(asyncio.run(run_curio())) # prints asyncio - should print curio
and the other way around, running asyncio inside a curio coro:
import curio
import asyncio
import sniffio
async def run_in_asyncio():
return sniffio.current_async_library()
async def run_asyncio():
return asyncio.run(run_in_asyncio())
print(curio.run(run_asyncio)) # prints asyncio - should print curio
Calling curio.run
inside an asyncio task and calling asyncio.run
inside a curio task are probably not fixable? At least I can't think of any way to do it without some explicit assistance from curio+asyncio. I'm not super bothered about that though, because doing this doesn't really make any sense anyway.
The top-level problem with to_thread
does seem like a real issue though. It's happening because sniffio uses a ContextVar
to track the current loop, so (1) sniffio is detecting asyncio and caching that in the contextvar, (2) asyncio is blindly copying the entire Context
into the thread, (3) sniffio in the thread is re-reading its cached contextvar, even though it's no longer appropriate.
As we discussed a bit on gitter, I'm no longer convinced that ContextVar
s are really the right approach here, for this and other reasons (in particular, the cvar approach originally assumed that trio/asyncio interop would be through trio-asyncio where we emulate the asyncio operations and have a chance to fix up the cvars; but now trio/asyncio via guest mode is also on the table, and I don't see how the cvar approach can work there, because it uses native asyncio ops and there's no chance for us to intercept trio<->asyncio transitions and fixup the cvar.)
Probably we should just use a thread-local instead of a cvar. It'll make sniffio slightly slower on non-cooperating libraries like asyncio/curio, but I don't see any way to avoid that. Something like:
class CurrentLibHolder(threading.local):
value = None
_current_lib_holder = CurrentLibHolder()
def set_current_lib(new_lib):
old_lib = _current_lib_holder.value
def restore():
_current_lib_holder.value = old_lib
_current_lib_holder.value = new_lib
return restore
def get_current_lib():
lib = _current_lib_holder.value
if lib is not None:
return lib
if asyncio.current_task() is not None:
return "asyncio"
...
Calling curio.run inside an asyncio task and calling asyncio.run inside a curio task are probably not fixable?
You have to walk the stack and see which library is calling g.send
We're not walking the stack just to fix an issue with code that no-one should even be writing in the first place :-)
Calling curio.run inside an asyncio task and calling asyncio.run inside a curio task are probably not fixable?
Turns out it's totally fixable because asyncio and curio both communicate via sys.set_asyncgen_hooks