"RuntimeError: This event loop is already running" When attempting REPL example
JacobCallahan opened this issue · comments
Each time I attempt the REPl example, I get the traceback below. Sync context-manager example worked though.
playwright==0.8.0
REPL Example: https://github.com/microsoft/playwright-python#repl-support-without-context-managers
IPython session
jake (e) venvplaywright ~ ipython
/home/jake/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py:935: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
warn("Attempting to work in a virtualenv. If you encounter problems, please "
Python 3.8.3 (default, May 29 2020, 00:00:00)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from playwright import sync_playwright
In [2]: playwright = sync_playwright().start()
Traceback (most recent call last):
File "/usr/local/bin/ipython", line 11, in <module>
sys.exit(start_ipython())
File "/home/jake/.local/lib/python3.8/site-packages/IPython/__init__.py", line 126, in start_ipython
return launch_new_instance(argv=argv, **kwargs)
File "/home/jake/.local/lib/python3.8/site-packages/traitlets/config/application.py", line 664, in launch_instance
app.start()
File "/home/jake/.local/lib/python3.8/site-packages/IPython/terminal/ipapp.py", line 356, in start
self.shell.mainloop()
File "/home/jake/.local/lib/python3.8/site-packages/IPython/terminal/interactiveshell.py", line 558, in mainloop
self.interact()
File "/home/jake/.local/lib/python3.8/site-packages/IPython/terminal/interactiveshell.py", line 541, in interact
code = self.prompt_for_code()
File "/home/jake/.local/lib/python3.8/site-packages/IPython/terminal/interactiveshell.py", line 467, in prompt_for_code
text = self.pt_app.prompt(
File "/home/jake/.local/lib/python3.8/site-packages/prompt_toolkit/shortcuts/prompt.py", line 994, in prompt
return self.app.run(set_exception_handler=set_exception_handler)
File "/home/jake/.local/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 811, in run
return loop.run_until_complete(
File "/usr/lib64/python3.8/asyncio/base_events.py", line 592, in run_until_complete
self._check_running()
File "/usr/lib64/python3.8/asyncio/base_events.py", line 552, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
If you suspect this is an IPython 7.13.0 bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
sys:1: RuntimeWarning: coroutine 'Application.run_async' was never awaited
for iPython / Jupiter notebooks it seems like the current workaround would be to use that before you run your shell:
import nest_asyncio
nest_asyncio.apply()
Out of curiosity to understand more the user case. You are trying out Playwright in a REPL or are you using it only with with the REPL in the end?
@mxschmitt thanks for the quick response!
nest_asyncio isn't a standard package or one installed as a requirement for playwright. As such, I'm guessing most users wouldn't have it installed. However, that did solve the problem:
In [1]: import nest_asyncio
In [2]: nest_asyncio.apply()
In [3]: from playwright import sync_playwright
In [4]: playwright = sync_playwright().start()
In [5]: My use-case within IPython is largely API exploration and interactive process development before writing a script or integration with an application.
Additional note, your example works perfectly in the base python REPL.
@mxschmitt - FYI, the nested async doesn't work for jupyter notebooks for at least the sync REPL example:
import nest_asyncio
nest_asyncio.apply()
from playwright import sync_playwright
playwright = sync_playwright().start()
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
<ipython-input-3-85d403adca24> in <module>
----> 1 playwright = sync_playwright().start()
c:\users\bubth\development\playwright\venv\lib\site-packages\playwright\__init__.py in sync_playwright()
32
33 def sync_playwright() -> SyncPlaywrightContextManager:
---> 34 return SyncPlaywrightContextManager()
35
36
c:\users\bubth\development\playwright\venv\lib\site-packages\playwright\main.py in __init__(self)
81 class SyncPlaywrightContextManager:
82 def __init__(self) -> None:
---> 83 self._connection = run_driver()
84 self._playwright: SyncPlaywright
85
c:\users\bubth\development\playwright\venv\lib\site-packages\playwright\main.py in run_driver()
76 # if loop.is_running():
77 # raise Error("Can only run one Playwright at a time.")
---> 78 return loop.run_until_complete(run_driver_async())
79
80
c:\users\bubth\development\playwright\venv\lib\site-packages\nest_asyncio.py in run_until_complete(self, future)
93 raise RuntimeError(
94 'Event loop stopped before Future completed.')
---> 95 return f.result()
96 finally:
97 events._set_running_loop(old_running_loop)
C:\Program Files\Python37\lib\asyncio\futures.py in result(self)
176 self.__log_traceback = False
177 if self._exception is not None:
--> 178 raise self._exception
179 return self._result
180
C:\Program Files\Python37\lib\asyncio\tasks.py in __step(***failed resolving arguments***)
221 # We use the `send` method directly, because coroutines
222 # don't have `__iter__` and `__next__` methods.
--> 223 result = coro.send(None)
224 else:
225 result = coro.throw(exc)
c:\users\bubth\development\playwright\venv\lib\site-packages\playwright\main.py in run_driver_async()
62 stdout=asyncio.subprocess.PIPE,
63 stderr=_get_stderr_fileno(),
---> 64 limit=32768,
65 )
66 assert proc.stdout
C:\Program Files\Python37\lib\asyncio\subprocess.py in create_subprocess_exec(program, stdin, stdout, stderr, loop, limit, *args, **kwds)
215 program, *args,
216 stdin=stdin, stdout=stdout,
--> 217 stderr=stderr, **kwds)
218 return Process(transport, protocol, loop)
C:\Program Files\Python37\lib\asyncio\base_events.py in subprocess_exec(self, protocol_factory, program, stdin, stdout, stderr, universal_newlines, shell, bufsize, *args, **kwargs)
1531 transport = await self._make_subprocess_transport(
1532 protocol, popen_args, False, stdin, stdout, stderr,
-> 1533 bufsize, **kwargs)
1534 if self._debug and debug_log is not None:
1535 logger.info('%s: %r', debug_log, transport)
C:\Program Files\Python37\lib\asyncio\base_events.py in _make_subprocess_transport(self, protocol, args, shell, stdin, stdout, stderr, bufsize, extra, **kwargs)
461 extra=None, **kwargs):
462 """Create subprocess transport."""
--> 463 raise NotImplementedError
464
465 def _write_to_self(self):
NotImplementedError:
@bubthegreat: This looks like a SelectorEventLoop's limitation on Python 3.7 on Windows. Starting with 3.8, a more complete implementation of the loop is used on Windows and it should support running subprocesses. See the platform support section for more details.
We do install this more capable ProactorEventLoop manually on 3.7 see here, but it might be that nest_asyncio does not respect our selection. So it is a combination of Windows + Python 3.7 + nest_asyncio that triggers it.
I assume upgrading to 3.9 would improve this, still saw issues, and downgraded to 3.8.3 - still having issues with sync_playwright:
import nest_asyncio
nest_asyncio.apply()
from playwright import sync_playwright
playwright = sync_playwright().start()
---------------------------------------------------------------------------
Error Traceback (most recent call last)
<ipython-input-2-dfa883f2869c> in <module>
1 from playwright import sync_playwright
----> 2 playwright = sync_playwright().start()
c:\users\bubth\development\playwright-test\venv\lib\site-packages\playwright\__init__.py in sync_playwright()
32
33 def sync_playwright() -> SyncPlaywrightContextManager:
---> 34 return SyncPlaywrightContextManager()
35
36
c:\users\bubth\development\playwright-test\venv\lib\site-packages\playwright\main.py in __init__(self)
86 class SyncPlaywrightContextManager:
87 def __init__(self) -> None:
---> 88 self._connection = run_driver()
89 self._playwright: SyncPlaywright
90
c:\users\bubth\development\playwright-test\venv\lib\site-packages\playwright\main.py in run_driver()
80 loop = asyncio.get_event_loop()
81 if loop.is_running():
---> 82 raise Error("Can only run one Playwright at a time.")
83 return loop.run_until_complete(run_driver_async())
84
Error: Can only run one Playwright at a time.
Currently, playwright disallows attaching to a running loop:
playwright-python/playwright/main.py
Lines 79 to 83 in d98d38b
This should be fine in plain IPython (does not launch an asyncio event loop in the main thread), but will fail in Jupyter that uses tornado>=5 (launches asyncio event loop on startup under the hood).
As calling run_until_complete is disallowed on running loops, this will fail in Jupyter.
Minimal reproducible example that will run in latest IPython, but will break on Jupyter environments that use tornado>=5 under the hood:
import asyncio
print(asyncio.run(asyncio.sleep(0)))So for Jupyter, additionally a pip install 'tornado<5' 'notebook<5.7.9' should fix loop issues introduced in tornado>=5 (and if you need jupyterhub, use jupyterhub<0.9.0). refs jupyter/notebook#3397 (comment) ipython/ipykernel#548
My understanding is that resolving this requires cooperation between the embedder that controls the loop and sync Playwright that also wants to dispatch its messages forever. And I don't think such cooperation is the right way to go. Sync API is designed as self-contained, assumes it controls execution. Is there a way to use async in Jupyter?
Executing a cell in Jupyter will actually require a lock on the main loop from ipykernel, so you could schedule execution of a Future in one cell (which will start execution once that cell has finished), and get the result in the next cell. Doing it all in one cell is certainly not trivial without nested loops.
for iPython / Jupiter notebooks it seems like the current workaround would be to use that before you run your shell:
import nest_asyncio nest_asyncio.apply()Out of curiosity to understand more the user case. You are trying out Playwright in a REPL or are you using it only with with the REPL in the end?
Was just bitten by this.
It is very convenient to use Jupyter for explorative programming, especially when the thing you're working with is heavy (like a browser in a particular state).
For my use case, I wanted to spin up a few browser instances and use them to test out a complicated feature that involves webrtc etc.
Also ran into this cute tutorial demonstrating why this is nice (he's using Selenium): https://medium.com/shakuro/adopting-ipython-jupyter-for-selenium-testing-d02309dd00b8
The stated workaround sadly didn't work for me, neither on sync nor async versions.
Hope this helps!
Found a workaround for getting Playwright working within Jupyter.
It leverages an awesome python library called RPyc to run Playwright outside of the kernel while keeping the API the same.
!pip install rpyc
import rpyc
import subprocess
subprocess.Popen('rpyc_classic.py -m forking'.split()) # start RPyc worker
conn = rpyc.classic.connect('localhost')
rplaywright = conn.modules['playwright']
# rplaywright has the same API as playwright
pr = rplaywright.sync_playwright().start()
browser = pr.chromium.launch()
page = browser.newPage()
page.goto('http://whatsmyuseragent.org/')
page.screenshot(path='example.png')
browser.close()# Grab screen
from PIL import Image
from io import BytesIO
browser = pr.chromium.launch()
page = browser.newPage()
page.goto('http://localhost:4000/v/tf3nhh')
img=page.screenshot(path='example.png')
Image.open(BytesIO(img))
Found a workaround for getting Playwright working within Jupyter.
It leverages an awesome python library called RPyc to run Playwright outside of the kernel while keeping the API the same.
!pip install rpyc import rpyc import subprocess subprocess.Popen('rpyc_classic.py -m forking'.split()) # start RPyc worker conn = rpyc.classic.connect('localhost') rplaywright = conn.modules['playwright'] # rplaywright has the same API as playwright pr = rplaywright.sync_playwright().start() browser = pr.chromium.launch() page = browser.newPage() page.goto('http://whatsmyuseragent.org/') page.screenshot(path='example.png') browser.close()# Grab screen from PIL import Image from io import BytesIO browser = pr.chromium.launch() page = browser.newPage() page.goto('http://localhost:4000/v/tf3nhh') img=page.screenshot(path='example.png') Image.open(BytesIO(img))
Hi tals, I follow the wolkaround that you mentioned, however, seems it can't help in win10. The error info shows "OSError: [WinError 193] %1 is not a valid Win32 application"
The complete error is as follows. Thanks for any further info.
OSError Traceback (most recent call last)
in
9 import rpyc
10 import subprocess
---> 11 subprocess.Popen('rpyc_classic.py -m forking'.split()) # start RPyc worker
12
13 conn = rpyc.classic.connect('localhost')
c:\program files\python38\lib\subprocess.py in init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)
852 encoding=encoding, errors=errors)
853
--> 854 self._execute_child(args, executable, preexec_fn, close_fds,
855 pass_fds, cwd, env,
856 startupinfo, creationflags, shell,
c:\program files\python38\lib\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_start_new_session)
1305 # Start the process
1306 try:
-> 1307 hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
1308 # no special security
1309 None, None,
OSError: [WinError 193] %1 is not a valid Win32 application
@wohenniubi sorry don't have a windows machine to test this out, but try figuring out how to run the rpyc_classic.py program on command line first
@tals Thanks for the response. After several trials without success, I might switch back to use selenium by now.
Anyway, I'll keep on watching this jupyter issue and this promising tool, playwright.
@wohenniubi sorry don't have a windows machine to test this out, but try figuring out how to run the rpyc_classic.py program on command line first
Closed for now as part of the triage. Please reopen if it's still persistent.
@mxschmitt not sure why its closed - this problem still exists. You can reproduce this on the latest versions of jupyter+playwright
The bug is upstream: ipython/ipykernel#548
And there is a workaround: #178 (comment)
Maybe could warn the jupyter user with a verbose re-raising try-except?
@ddelange the proposed workaround sadly doesn't work on jupyter: #178 (comment)
The only thing I could do to get it working was running it outside of the process with RPyc
It's been almost two years and this still isn't fixed that I can find - playwright is explicitly stopping me from using nested loops:
import nest_asyncio
nest_asyncio.apply()
from playwright.sync_api import sync_playwright
playwright = sync_playwright().start()
---------------------------------------------------------------------------
Error Traceback (most recent call last)
Input In [3], in <cell line: 2>()
1 from playwright.sync_api import sync_playwright
----> 2 playwright = sync_playwright().start()
File c:\users\bubth\development\soc2\venv\lib\site-packages\playwright\sync_api\_context_manager.py:76, in PlaywrightContextManager.start(self)
75 def start(self) -> SyncPlaywright:
---> 76 return self.__enter__()
File c:\users\bubth\development\soc2\venv\lib\site-packages\playwright\sync_api\_context_manager.py:42, in PlaywrightContextManager.__enter__(self)
40 own_loop = loop
41 if loop.is_running():
---> 42 raise Error(
43 """It looks like you are using Playwright Sync API inside the asyncio loop.
44 Please use the Async API instead."""
45 )
47 def greenlet_main() -> None:
48 loop.run_until_complete(self._connection.run_as_sync())
Error: It looks like you are using Playwright Sync API inside the asyncio loop.
Please use the Async API instead.
The workarounds don't work for those of us using windows, and they don't appear to work in WSL either - is there a reason we can't get more information on why this is still closed two years later with no traction and no functional workarounds for presumably a fairly normal use case of using a jupyter notebook for exploratory programming?
It seems hard to believe that we just assume everyone will use raw iPython for everything so I'm confused by the reluctance to actually fix this.
If I'm understanding the asyncio base_events.py error, it's because there's no implementation of the _make_subprocess_transport function - is that not something that would be on the playwright package to implement in it's async implementation?
@tals Rpyc will only works if we will not call any playwright sub-modules directly, tomerfiliba-org/rpyc#496.
In my case, it not works well, still finding a way to let playwright sync api work with jupyter now...
still waiting i want to leave selenium but this stop me
If you want to use an async framework (playwright-python is natively async) inside Jupyter Notebooks, the ipykernel maintainer suggests to use the async api directly (await & async def syntax), instead of the sync wrappers. They don't support sync wrappers around async libs...
still does not work in jupyter (vscode interactive).
Sync API is not allowed!
import asyncio
from playwright.async_api import async_playwright
from playwright.sync_api import sync_playwright
# https://github.com/microsoft/playwright-python/issues/178#issuecomment-680249269
import nest_asyncio
nest_asyncio.apply()
# Error: It looks like you are using Playwright Sync API inside the asyncio loop.
# Please use the Async API instead.
p = sync_playwright().start()Async API is not worked!
import asyncio
from playwright.async_api import async_playwright
from playwright.sync_api import sync_playwright
# https://github.com/microsoft/playwright-python/issues/178#issuecomment-680249269
import nest_asyncio
nest_asyncio.apply()
async def main():
async with async_playwright() as p:
browser = await p.chromium.launch(channel="chrome",headless=False, slow_mo=100)
page = await browser.new_page()
await page.goto("https://cnki.net/")
await browser.close()
# NotImplementedError
asyncio.run(main())Maybe I should switch to selenium, see run-selenium-in-jupyter-notebook-on-wsl2-or-ubuntu。
can you try the async snippet without nest_asyncio, and post the full error traceback here?
can you try the async snippet without
nest_asyncio, and post the full error traceback here?
Here is my snapshot and error logs.
NotImplementedError Traceback (most recent call last)
Cell In [33], line 9
7 await browser.close()
8 # NotImplementedError
----> 9 asyncio.run(main())
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\nest_asyncio.py:35, in _patch_asyncio.<locals>.run(main, debug)
[33](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=32) task = asyncio.ensure_future(main)
[34](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=33) try:
---> [35](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=34) return loop.run_until_complete(task)
[36](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=35) finally:
[37](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=36) if not task.done():
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\nest_asyncio.py:89, in _patch_loop.<locals>.run_until_complete(self, future)
[86](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=85) if not f.done():
[87](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=86) raise RuntimeError(
[88](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=87) 'Event loop stopped before Future completed.')
---> [89](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/nest_asyncio.py?line=88) return f.result()
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\futures.py:201, in Future.result(self)
[199](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/futures.py?line=198) self.__log_traceback = False
[200](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/futures.py?line=199) if self._exception is not None:
--> [201](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/futures.py?line=200) raise self._exception.with_traceback(self._exception_tb)
[202](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/futures.py?line=201) return self._result
...
[496](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=495) extra=None, **kwargs):
[497](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=496) """Create subprocess transport."""
--> [498](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=497) raise NotImplementedError
NotImplementedError:
this error is coming from nest_asyncio ^, can you run it in a fresh kernel without nest_asyncio?
It seems sync API is not work in plain python script.
The following code just launch browser, open page then exit. I did not call browser.close().
from playwright.sync_api import sync_playwright
p = sync_playwright().start()
browser = p.chromium.launch(channel="chrome",headless=False, slow_mo=100)
page = browser.new_page()
page.goto('https://cnki.net/')Here is ipython error logs.
C:\Users\Liu.D.H>ipython
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from playwright.sync_api import sync_playwright
...:
...: p = sync_playwright().start()
...: browser = p.chromium.launch(channel="chrome",headless=False, slow_mo=100)
...: page = browser.new_page()
...: page.goto('https://cnki.net/')
Out[1]: <Response url='https://cnki.net/' request=<Request url='https://cnki.net/' method='GET'>>
Traceback (most recent call last):
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\Scripts\ipython.exe\__main__.py", line 7, in <module>
sys.exit(start_ipython())
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\__init__.py", line 124, in start_ipython
return launch_new_instance(argv=argv, **kwargs)
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\traitlets\config\application.py", line 976, in launch_instance
app.start()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\terminal\ipapp.py", line 318, in start
self.shell.mainloop()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\terminal\interactiveshell.py", line 685, in mainloop
self.interact()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\terminal\interactiveshell.py", line 670, in interact
code = self.prompt_for_code()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\terminal\interactiveshell.py", line 609, in prompt_for_code
text = self.pt_app.prompt(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\prompt_toolkit\shortcuts\prompt.py", line 1034, in prompt
return self.app.run(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\prompt_toolkit\application\application.py", line 937, in run
return loop.run_until_complete(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 622, in run_until_complete
self._check_running()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 582, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
If you suspect this is an IPython 8.5.0 bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
sys:1: RuntimeWarning: coroutine 'Application.run_async' was never awaited
C:\Users\Liu.D.H>this error is coming from
nest_asyncio^, can you run it in a fresh kernel withoutnest_asyncio?
I restarted the kernel, the errors changed.
I run the following code block.
async def main():
async with async_playwright() as p:
browser = await p.chromium.launch(channel="chrome",headless=False, slow_mo=100)
page = await browser.new_page()
await page.goto("https://cnki.net/")
await browser.close()
asyncio.run(main())RuntimeError Traceback (most recent call last)
d:\code\python\cnki_crawler_playwright\main.py in line 9
[35](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=34) await browser.close()
[36](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=35) # NotImplementedError
----> [37](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=36) asyncio.run(main())
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py:33, in run(main, debug)
[9](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=8) """Execute the coroutine and return the result.
[10](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=9)
[11](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=10) This function runs the passed coroutine, taking care of
(...)
[30](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=29) asyncio.run(main())
[31](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=30) """
[32](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=31) if events._get_running_loop() is not None:
---> [33](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=32) raise RuntimeError(
[34](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=33) "asyncio.run() cannot be called from a running event loop")
[36](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=35) if not coroutines.iscoroutine(main):
[37](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/runners.py?line=36) raise ValueError("a coroutine was expected, got {!r}".format(main))
RuntimeError: asyncio.run() cannot be called from a running event loop
instead of asyncio.run(main()), try await main()
instead of
asyncio.run(main()), tryawait main()
I tried, but still not work.
NotImplementedError Traceback (most recent call last)
d:\code\python\cnki_crawler_playwright\main.py in line 10
[34](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=33) await page.goto("https://cnki.net/")
[35](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=34) await browser.close()
---> [38](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=37) await main()
d:\code\python\cnki_crawler_playwright\main.py in line 3, in main()
[30](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=29) async def main():
----> [31](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=30) async with async_playwright() as p:
[32](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=31) browser = await p.chromium.launch(channel="chrome", headless=False, slow_mo=100)
[33](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=32) page = await browser.new_page()
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\async_api\_context_manager.py:46, in PlaywrightContextManager.__aenter__(self)
[44](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/async_api/_context_manager.py?line=43) if not playwright_future.done():
[45](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/async_api/_context_manager.py?line=44) playwright_future.cancel()
---> [46](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/async_api/_context_manager.py?line=45) playwright = AsyncPlaywright(next(iter(done)).result())
[47](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/async_api/_context_manager.py?line=46) playwright.stop = self.__aexit__ # type: ignore
[48](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/async_api/_context_manager.py?line=47) return playwright
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_transport.py:121, in PipeTransport.connect(self)
[118](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=117) if getattr(sys, "frozen", False):
[119](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=118) env.setdefault("PLAYWRIGHT_BROWSERS_PATH", "0")
--> [121](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=120) self._proc = await asyncio.create_subprocess_exec(
[122](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=121) str(self._driver_executable),
...
[496](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=495) extra=None, **kwargs):
[497](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=496) """Create subprocess transport."""
--> [498](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=497) raise NotImplementedError
NotImplementedError:
I also tried asyncio.get_event_loop().run_until_complete(main()), however, I got RuntimeError: This event loop is already running.
logs
RuntimeError Traceback (most recent call last)
d:\code\python\cnki_crawler_playwright\main.py in line 10
[34](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=33) await page.goto("https://cnki.net/")
[35](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=34) await browser.close()
---> [38](file:///d%3A/code/python/cnki_crawler_playwright/main.py?line=37) asyncio.get_event_loop().run_until_complete(main())
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:622, in BaseEventLoop.run_until_complete(self, future)
[611](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=610) """Run until the Future is done.
[612](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=611)
[613](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=612) If the argument is a coroutine, it is wrapped in a Task.
(...)
[619](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=618) Return the Future's result, or raise its exception.
[620](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=619) """
[621](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=620) self._check_closed()
--> [622](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=621) self._check_running()
[624](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=623) new_task = not futures.isfuture(future)
[625](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=624) future = tasks.ensure_future(future, loop=self)
File c:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:582, in BaseEventLoop._check_running(self)
[580](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=579) def _check_running(self):
[581](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=580) if self.is_running():
--> [582](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=581) raise RuntimeError('This event loop is already running')
[583](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=582) if events._get_running_loop() is not None:
[584](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=583) raise RuntimeError(
[585](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/asyncio/base_events.py?line=584) 'Cannot run the event loop while another loop is running')
RuntimeError: This event loop is already running
I have no problem running the example on a fresh jupyter server:
The fact that your traceback hangs here:
--> [121](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=120) self._proc = await asyncio.create_subprocess_exec(
Makes me think you might not be running in a clean jupyter environment. Maybe vscode does some magic?
I have no problem running the example on a fresh jupyter server:
The fact that your traceback hangs here:
--> [121](file:///c%3A/Users/Liu.D.H/AppData/Local/Programs/Python/Python310/lib/site-packages/playwright/_impl/_transport.py?line=120) self._proc = await asyncio.create_subprocess_exec(Makes me think you might not be running in a clean jupyter environment. Maybe vscode does some magic?
Hi, I tried to update jupyter via pip install -U jupyterlab notebook voila. Then tried the following code in jupyterlab. But still not works.
import asyncio
from playwright.async_api import async_playwright
async def run(playwright):
chromium = playwright.chromium
browser = await chromium.launch(channel="chrome",headless=False, slow_mo=100).launch()
page = await browser.new_page()
await page.goto("https://example.com")
await browser.close()
async def main():
async with async_playwright() as playwright:
await run(playwright)
await main()
logs
C:\Users\Liu.D.H>pip install -U jupyterlab notebook voila
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: jupyterlab in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (3.4.7)
Collecting jupyterlab
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2b/d4/e0627f216bfb451e807cb8c2c8a0fc27e47cc76e483e539e8e213b95518e/jupyterlab-3.5.0-py3-none-any.whl (8.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 9.2 MB/s eta 0:00:00
Requirement already satisfied: notebook in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (6.4.12)
Collecting notebook
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/db/40/2d321ba572dc9a94a090d92c9826291a1dcee1e05bc6c1d641ce419b701d/notebook-6.5.2-py3-none-any.whl (439 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 439.1/439.1 kB 9.1 MB/s eta 0:00:00
Requirement already satisfied: voila in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (0.3.6)
Collecting voila
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ce/63/55971b7c40ab1c4d87beec34775811b8aeea9a123b2d0abc41febc58526c/voila-0.4.0-py3-none-any.whl (5.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 11.0 MB/s eta 0:00:00
Requirement already satisfied: nbclassic in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (0.4.3)
Requirement already satisfied: packaging in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (21.3)
Requirement already satisfied: jupyterlab-server~=2.10 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (2.15.0)
Requirement already satisfied: jupyter-core in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (4.11.1)
Requirement already satisfied: jupyter-server<3,>=1.16.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (1.18.1)
Requirement already satisfied: tomli in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (2.0.1)
Requirement already satisfied: jinja2>=2.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (3.1.2)
Requirement already satisfied: ipython in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (8.5.0)
Requirement already satisfied: tornado>=6.1.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab) (6.2)
Requirement already satisfied: terminado>=0.8.3 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (0.15.0)
Requirement already satisfied: ipykernel in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (6.15.1)
Requirement already satisfied: argon2-cffi in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (21.3.0)
Requirement already satisfied: traitlets>=4.2.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (5.3.0)
Requirement already satisfied: nest-asyncio>=1.5 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (1.5.5)
Requirement already satisfied: nbconvert>=5 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (6.5.3)
Requirement already satisfied: ipython-genutils in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (0.2.0)
Requirement already satisfied: jupyter-client>=5.3.4 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (7.3.4)
Collecting nbclassic
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a6/85/2a240df7326b7311ebd926c12d7df5394aef2f9f76ffbb294079cc43960e/nbclassic-0.4.8-py3-none-any.whl (9.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.8/9.8 MB 11.2 MB/s eta 0:00:00
Requirement already satisfied: prometheus-client in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (0.14.1)
Requirement already satisfied: pyzmq>=17 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (23.2.1)
Requirement already satisfied: nbformat in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (5.4.0)
Requirement already satisfied: Send2Trash>=1.8.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from notebook) (1.8.0)
Requirement already satisfied: websockets>=9.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from voila) (10.3)
Requirement already satisfied: nbclient<0.8,>=0.4.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from voila) (0.5.13)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jinja2>=2.1->jupyterlab) (2.1.1)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-client>=5.3.4->notebook) (2.8.2)
Requirement already satisfied: entrypoints in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-client>=5.3.4->notebook) (0.4)
Requirement already satisfied: pywin32>=1.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-core->jupyterlab) (304)
Requirement already satisfied: anyio<4,>=3.1.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-server<3,>=1.16.0->jupyterlab) (3.6.1)
Requirement already satisfied: websocket-client in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-server<3,>=1.16.0->jupyterlab) (1.3.3)
Requirement already satisfied: pywinpty in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyter-server<3,>=1.16.0->jupyterlab) (2.0.7)
Requirement already satisfied: babel in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab-server~=2.10->jupyterlab) (2.10.3)
Requirement already satisfied: json5 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab-server~=2.10->jupyterlab) (0.9.9)
Requirement already satisfied: requests in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab-server~=2.10->jupyterlab) (2.28.1)
Requirement already satisfied: jsonschema>=3.0.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jupyterlab-server~=2.10->jupyterlab) (4.10.2)
Requirement already satisfied: notebook-shim>=0.1.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbclassic->jupyterlab) (0.1.0)
Requirement already satisfied: pygments>=2.4.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (2.13.0)
Requirement already satisfied: beautifulsoup4 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (4.11.1)
Requirement already satisfied: pandocfilters>=1.4.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (1.5.0)
Requirement already satisfied: mistune<2,>=0.8.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (0.8.4)
Requirement already satisfied: lxml in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (4.9.1)
Requirement already satisfied: defusedxml in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (0.7.1)
Requirement already satisfied: bleach in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (5.0.1)
Requirement already satisfied: jupyterlab-pygments in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (0.2.2)
Requirement already satisfied: tinycss2 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbconvert>=5->notebook) (1.1.1)
Requirement already satisfied: fastjsonschema in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from nbformat->notebook) (2.16.1)
Requirement already satisfied: argon2-cffi-bindings in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from argon2-cffi->notebook) (21.2.0)
Requirement already satisfied: matplotlib-inline>=0.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipykernel->notebook) (0.1.5)
Requirement already satisfied: psutil in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipykernel->notebook) (5.9.1)
Requirement already satisfied: debugpy>=1.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipykernel->notebook) (1.6.3)
Requirement already satisfied: backcall in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (0.2.0)
Requirement already satisfied: prompt-toolkit<3.1.0,>3.0.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (3.0.30)
Requirement already satisfied: decorator in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (5.1.1)
Requirement already satisfied: pickleshare in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (0.7.5)
Requirement already satisfied: jedi>=0.16 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (0.18.1)
Requirement already satisfied: colorama in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (0.4.5)
Requirement already satisfied: stack-data in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from ipython->jupyterlab) (0.4.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from packaging->jupyterlab) (3.0.9)
Requirement already satisfied: idna>=2.8 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from anyio<4,>=3.1.0->jupyter-server<3,>=1.16.0->jupyterlab) (3.3)
Requirement already satisfied: sniffio>=1.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from anyio<4,>=3.1.0->jupyter-server<3,>=1.16.0->jupyterlab) (1.2.0)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jedi>=0.16->ipython->jupyterlab) (0.8.3)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jsonschema>=3.0.1->jupyterlab-server~=2.10->jupyterlab) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from jsonschema>=3.0.1->jupyterlab-server~=2.10->jupyterlab) (22.1.0)
Requirement already satisfied: wcwidth in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from prompt-toolkit<3.1.0,>3.0.1->ipython->jupyterlab) (0.2.5)
Requirement already satisfied: six>=1.5 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from python-dateutil>=2.8.2->jupyter-client>=5.3.4->notebook) (1.16.0)
Requirement already satisfied: cffi>=1.0.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from argon2-cffi-bindings->argon2-cffi->notebook) (1.15.1)
Requirement already satisfied: pytz>=2015.7 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from babel->jupyterlab-server~=2.10->jupyterlab) (2022.2.1)
Requirement already satisfied: soupsieve>1.2 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from beautifulsoup4->nbconvert>=5->notebook) (2.3.2.post1)
Requirement already satisfied: webencodings in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from bleach->nbconvert>=5->notebook) (0.5.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from requests->jupyterlab-server~=2.10->jupyterlab) (2022.6.15)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from requests->jupyterlab-server~=2.10->jupyterlab) (1.26.11)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from requests->jupyterlab-server~=2.10->jupyterlab) (2.1.0)
Requirement already satisfied: executing in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from stack-data->ipython->jupyterlab) (0.10.0)
Requirement already satisfied: asttokens in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from stack-data->ipython->jupyterlab) (2.0.8)
Requirement already satisfied: pure-eval in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from stack-data->ipython->jupyterlab) (0.2.2)
Requirement already satisfied: pycparser in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook) (2.21)
Installing collected packages: voila, nbclassic, notebook, jupyterlab
Attempting uninstall: voila
Found existing installation: voila 0.3.6
Uninstalling voila-0.3.6:
Successfully uninstalled voila-0.3.6
Attempting uninstall: nbclassic
Found existing installation: nbclassic 0.4.3
Uninstalling nbclassic-0.4.3:
Successfully uninstalled nbclassic-0.4.3
Attempting uninstall: notebook
Found existing installation: notebook 6.4.12
Uninstalling notebook-6.4.12:
Successfully uninstalled notebook-6.4.12
Attempting uninstall: jupyterlab
Found existing installation: jupyterlab 3.4.7
Uninstalling jupyterlab-3.4.7:
Successfully uninstalled jupyterlab-3.4.7
Successfully installed jupyterlab-3.5.0 nbclassic-0.4.8 notebook-6.5.2 voila-0.4.0
C:\Users\Liu.D.H>jupyter --version
Selected Jupyter core packages...
IPython : 8.5.0
ipykernel : 6.15.1
ipywidgets : not installed
jupyter_client : 7.3.4
jupyter_core : 4.11.1
jupyter_server : 1.18.1
jupyterlab : 3.5.0
nbclient : 0.5.13
nbconvert : 6.5.3
nbformat : 5.4.0
notebook : 6.5.2
qtconsole : not installed
traitlets : 5.3.0
C:\Users\Liu.D.H>Task exception was never retrieved
future: <Task finished name='Task-5' coro=<Connection.run() done, defined at C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py:217> exception=NotImplementedError()>
Traceback (most recent call last):
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 224, in run
await self._transport.connect()
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_transport.py", line 133, in connect
raise exc
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_transport.py", line 121, in connect
self._proc = await asyncio.create_subprocess_exec(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\subprocess.py", line 218, in create_subprocess_exec
transport, protocol = await loop.subprocess_exec(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1667, in subprocess_exec
transport = await self._make_subprocess_transport(
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
Cell In [1], line 15
12 async with async_playwright() as playwright:
13 await run(playwright)
---> 15 await main()
Cell In [1], line 12, in main()
11 async def main():
---> 12 async with async_playwright() as playwright:
13 await run(playwright)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\async_api\_context_manager.py:46, in PlaywrightContextManager.__aenter__(self)
44 if not playwright_future.done():
45 playwright_future.cancel()
---> 46 playwright = AsyncPlaywright(next(iter(done)).result())
47 playwright.stop = self.__aexit__ # type: ignore
48 return playwright
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_transport.py:121, in PipeTransport.connect(self)
118 if getattr(sys, "frozen", False):
119 env.setdefault("PLAYWRIGHT_BROWSERS_PATH", "0")
--> 121 self._proc = await asyncio.create_subprocess_exec(
122 str(self._driver_executable),
123 "run-driver",
124 stdin=asyncio.subprocess.PIPE,
125 stdout=asyncio.subprocess.PIPE,
126 stderr=_get_stderr_fileno(),
127 limit=32768,
128 creationflags=creationflags,
129 env=env,
130 )
131 except Exception as exc:
132 self.on_error_future.set_exception(exc)
File ~\AppData\Local\Programs\Python\Python310\lib\asyncio\subprocess.py:218, in create_subprocess_exec(program, stdin, stdout, stderr, limit, *args, **kwds)
215 loop = events.get_running_loop()
216 protocol_factory = lambda: SubprocessStreamProtocol(limit=limit,
217 loop=loop)
--> 218 transport, protocol = await loop.subprocess_exec(
219 protocol_factory,
220 program, *args,
221 stdin=stdin, stdout=stdout,
222 stderr=stderr, **kwds)
223 return Process(transport, protocol, loop)
File ~\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:1667, in BaseEventLoop.subprocess_exec(self, protocol_factory, program, stdin, stdout, stderr, universal_newlines, shell, bufsize, encoding, errors, text, *args, **kwargs)
1665 debug_log = f'execute program {program!r}'
1666 self._log_subprocess(debug_log, stdin, stdout, stderr)
-> 1667 transport = await self._make_subprocess_transport(
1668 protocol, popen_args, False, stdin, stdout, stderr,
1669 bufsize, **kwargs)
1670 if self._debug and debug_log is not None:
1671 logger.info('%s: %r', debug_log, transport)
File ~\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:498, in BaseEventLoop._make_subprocess_transport(self, protocol, args, shell, stdin, stdout, stderr, bufsize, extra, **kwargs)
494 async def _make_subprocess_transport(self, protocol, args, shell,
495 stdin, stdout, stderr, bufsize,
496 extra=None, **kwargs):
497 """Create subprocess transport."""
--> 498 raise NotImplementedError
NotImplementedError:
do you get the same traceback when you run the snippet in ipython instead of jupyter? if yes, we eliminated the ipykernel loop issue, and my next guess would be the fact that you're on windows...
potentially indeed your OS ref https://stackoverflow.com/q/70349876
@mxschmitt would this be caught by the Windows CI?
edit: more specifically https://stackoverflow.com/a/44639711/5511061
@ddelange Thanks. I found a workaround on windows now. NOT RECOMMEND!
From jupyter/notebook#5916, minrk/ipykernel@079f072, https://stackoverflow.com/questions/44633458/why-am-i-getting-notimplementederror-with-async-and-await-on-windows/74311290#74311290, https://github.com/ipython/ipykernel/blob/99a1becaa958b33d80fe337fdbc41305030fdb6d/ipykernel/kernelapp.py#L633-L637. I found a way to make it work hackly (not recommend) on windows.
Comment asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy()) in %LOCALAPPDATA%\Programs\Python\Python310\Lib\site-packages\ipykernel\kernelapp.py (change to your path).
Then the following code will work in jupyter notebook.
import asyncio
from playwright.async_api import async_playwright
async def main():
async with async_playwright() as p:
browser = await p.chromium.launch(channel="chrome", headless=False, slow_mo=100)
page = await browser.new_page()
await page.goto("https://cnki.net/")
await browser. Close()
await main()still does not work in jupyter (vscode interactive).
Sync API is not allowed!
import asyncio from playwright.async_api import async_playwright from playwright.sync_api import sync_playwright # https://github.com/microsoft/playwright-python/issues/178#issuecomment-680249269 import nest_asyncio nest_asyncio.apply() # Error: It looks like you are using Playwright Sync API inside the asyncio loop. # Please use the Async API instead. p = sync_playwright().start()Async API is not worked!
import asyncio from playwright.async_api import async_playwright from playwright.sync_api import sync_playwright # https://github.com/microsoft/playwright-python/issues/178#issuecomment-680249269 import nest_asyncio nest_asyncio.apply() async def main(): async with async_playwright() as p: browser = await p.chromium.launch(channel="chrome",headless=False, slow_mo=100) page = await browser.new_page() await page.goto("https://cnki.net/") await browser.close() # NotImplementedError asyncio.run(main())Maybe I should switch to selenium, see run-selenium-in-jupyter-notebook-on-wsl2-or-ubuntu。
I have another question, why can't I use sync API in jupyter? I have to write a lot await in async API. The sync API is simple and straight, performance or jupyter does not allowed?
If you want to use an async framework (playwright-python is natively async) inside Jupyter Notebooks, the ipykernel maintainer suggests to use the async api directly (
await&async defsyntax), instead of the sync wrappers. They don't support sync wrappers around async libs...
jup it's not allowed
If you want to use an async framework (playwright-python is natively async) inside Jupyter Notebooks, the ipykernel maintainer suggests to use the async api directly (
await&async defsyntax), instead of the sync wrappers. They don't support sync wrappers around async libs...jup it's not allowed
Not allowed? However, I used a lot of packages like numpy, pandas, matplotlib, tensorflow, pytorch, and so on are sync style of API, most of them are async only.
Is there any ways to make sync API of playwright-python work on jupyter?
I didn't double check, but I'm pretty sure those libraries don't use asyncio under the hood (they may still release the GIL however). For more information on the topic, see ipython/ipykernel#548 (comment).