can't pickle Loop objects with uvloop

Question

can't pickle Loop objects with uvloop

samuelcolvin opened this issue 7 years ago · comments

Samuel Colvin commented 7 years ago

TypeError: can't pickle Loop objects, see #149 for more details

Alexander Chernetsky · Answer 1 · Fri Oct 20 2017 03:28:15 GMT+0800 (China Standard Time)

@samuelcolvin,
Also, server not responding and not reloading if code changed.

Samuel Colvin · Answer 2 · Fri Oct 20 2017 03:30:29 GMT+0800 (China Standard Time)

That should be the same problem which is only showed when on shutdown because task.result() is called.

Two minutes and I'll have a PR to fix this.

Samuel Colvin · Answer 3 · Fri Oct 20 2017 03:35:33 GMT+0800 (China Standard Time)

should be fixed in #154, would be great if you could check this @achernetsky and if it's ok i'll release again.

Alexander Chernetsky · Answer 4 · Fri Oct 20 2017 03:41:08 GMT+0800 (China Standard Time)

I checked and got other problem/issue

$ PYTHONASYNCIODEBUG= adev runserver --root test
[22:38:39] pre-check enabled, checking app factory
[22:38:39] Starting aux server at http://localhost:8001 ◆
[22:38:39] Starting dev server at http://localhost:8000 ●
[22:38:40] ● GET / 200 163B
^C[22:38:42] shutting down server...
Process Process-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "../venv/lib/python3.6/site-packages/aiohttp_devtools/runserver/serve.py", line 138, in serve_main_app
    loop.close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/unix_events.py", line 63, in close
    super().close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 110, in close
    self._close_self_pipe()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 121, in _close_self_pipe
    self._ssock.close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 417, in close
    self._real_close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 411, in _real_close
    _ss.close(self)
KeyboardInterrupt
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=False debug=False>>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 511, in __del__
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/unix_events.py", line 63, in close
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 110, in close
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 120, in _close_self_pipe
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 279, in _remove_reader
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/selectors.py", line 189, in get_key
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/selectors.py", line 70, in __getitem__
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/selectors.py", line 224, in _fileobj_lookup
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/selectors.py", line 41, in _fileobj_to_fd
ValueError: Invalid file descriptor: -1

If I did not send GET request everything alright

$ PYTHONASYNCIODEBUG= adev runserver --root test
[22:43:07] pre-check enabled, checking app factory
[22:43:07] Starting aux server at http://localhost:8001 ◆
[22:43:07] Starting dev server at http://localhost:8000 ●
^C[22:43:08] shutting down server...
$

Samuel Colvin · Answer 5 · Fri Oct 20 2017 03:43:03 GMT+0800 (China Standard Time)

Does the code reload properly?

Alexander Chernetsky · Answer 6 · Fri Oct 20 2017 03:44:54 GMT+0800 (China Standard Time)

@samuelcolvin, yes, and server reloaded if code changed.

$ PYTHONASYNCIODEBUG= adev runserver --root test
[22:43:39] pre-check enabled, checking app factory
[22:43:39] Starting aux server at http://localhost:8001 ◆
[22:43:39] Starting dev server at http://localhost:8000 ●
[22:43:44] Restarting dev server at http://localhost:8000 ●
^C[22:43:46] shutting down server...
$

Alexander Chernetsky · Answer 7 · Fri Oct 20 2017 03:53:15 GMT+0800 (China Standard Time)

@samuelcolvin, I think, as I wrote above, problem reproduced if I sent GET request.

Samuel Colvin · Answer 8 · Fri Oct 20 2017 03:53:15 GMT+0800 (China Standard Time)

ok, try this. I've committed again but annoyingly it still seems to be in github's queue and not showing here.

This works for me most of the time but occationally I get an error on shutdown, I think that's just how it is with uvloop, if you want to be sure of no errors on termination, don't use uvloop with aiohttp-devtools.

Alexander Chernetsky · Answer 9 · Fri Oct 20 2017 04:02:00 GMT+0800 (China Standard Time)

@samuelcolvin,
I run with using uvloop and got same issue.

...
ValueError: Invalid file descriptor: -1

After I run the following code without uvloop

PYTHONASYNCIODEBUG= adev runserver --root test

where test/main.py

from aiohttp import web
import asyncio
# import uvloop

async def handle(request):
    return web.Response(text='testing more')

# asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

app = web.Application()
app.router.add_get('/', handle)

and got

$ PYTHONASYNCIODEBUG= adev runserver --root test
[22:58:51] pre-check enabled, checking app factory
[22:58:51] Starting aux server at http://localhost:8001 ◆
[22:58:51] Starting dev server at http://localhost:8000 ●
[22:58:52] ● GET / 200 163B
^C[22:58:56] shutting down server...
Process Process-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "../venv/lib/python3.6/site-packages/aiohttp_devtools/runserver/serve.py", line 138, in serve_main_app
    loop.close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/unix_events.py", line 63, in close
    super().close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 110, in close
    self._close_self_pipe()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 121, in _close_self_pipe
    self._ssock.close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 417, in close
    self._real_close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 411, in _real_close
    _ss.close(self)
KeyboardInterrupt
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=False debug=False>>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 508, in __del__
AttributeError: 'NoneType' object has no attribute 'warn'
$

Samuel Colvin · Answer 10 · Fri Oct 20 2017 04:33:25 GMT+0800 (China Standard Time)

see my comments on #154 I'm going to merge than and release..