:JupyniumStartSynch hangs and then closes notebook window
ethanmsl opened this issue · comments
Describe the bug
Neovim Hangs on :JupyniumStartSynch (with command selection area open) and then eventually closes the firefox notebook window.
Priort to that: It successfully opens a Firefox window with Jupiter notebook with :JupyniumStartSyncAndAttachToServer command. :J...StartSynch also does open up one of the files -- it just is never able to do anything with it.
The hang and close happens regardless of input from the user. (I can not touch board nor mouse or switch windows -- either way the same result.)
Final output in nvim command line:
Jupynium: RPC channel closed. Stop sending all notifications.
[Jupynium]: Cancelling sync...
Instal versions:
GeckoDriver: 0.33.0
NVIM: 0.9.1
Jupyter Notebook: 6.5.4
Python: 3.11.3
macOS: 13.4
To Reproduce
Run the exact descriptions in the QuickStart section
Expected behavior
A clear and concise description of what you expected to happen.
Output when using jupynium command
Unable to open jupynium via command line, despite it being an installed Python module
jupynium 0.1.dev1+fg5faa4b
(SideNote: this plugin sounds amazing. It would solve a huge gap -- making notebooks civilized coding environments.)
Hi, thanks for reporting. I'll need to check with the new neovim version.
You can try :JupyniumStartAndAttachToServerInTerminal to see a more detailed error message.
Hmm.
Uninstalled all python files and reinstalled these. (So I no longer have access to the original error.)
Manually installed jupynium with pip install jupynium
It now can start and synch. Occasionally dys-synching and requiring manual click. (actually seems to still be synched, just that the pop up needs to be dismissed)
It can't connect to any kernel however. Even when manually running a kernel select command.
I may no longer have a specific bug to share. I'm not sure what to look at to debug this as I go.
I'll bang at things and see if I can get things running.
Okay. once thing to tell you is that, it's better to install the master branch than using pip install jupynium because there are some updates since the last release that may have fixed some errors.
Remember to check the actual error message using :JupyniumStartAndAttachToServerInTerminal!
Interesting.
The update does seem to have fixed the problem *if* I'm running :JupyniumStartAndAttachToServerInTerminal followed by :JupyniumStartAndSynch.
It synchs and code is executable.
However, if I just run :JupyniumStartAndAttachToServer(not in terminal) followed by :JupyniumStartAndSynchit still hangs and then closes.
With the only message being:
Jupynium: RPC channel closed. Stop sending all notifications.
[Jupynium]: Cancelling sync..
To be clear: both of the 'Start&Attach' commands open up firefox and jupyter. They only observably diverge when I run 'Start&Synch' -- at which point if I ran '...InTerminal' before everything works and if I didn't then neovim hangs and eventually the webpage (and presumably server) just close out.
Going to start playing around with it more, but this is still super exciting for now.
I can work around there being an unnecessary terminal open. Will try doing some dev work in this mode and see if I run into anything else unusual.
I'm glad it works somehow. I'm not sure why it only works using terminal. I should add some logging messages in the future version.
The terminal feature is supposed to work as a debugging interface, but I didn't experience that it can only work in terminal and not without. So now I should log in a file so we can see what's going on.
Thanks. Just a quick update, playing with various versions:
jupynium 0.1.dev2+gaaa0e59 -- works, but only with '...InTerminal'
jupynium 0.2.0. -- doesn't work due to service_log_path issue
jupynium 0.2.1.dev18+gaaa0e59. -- works, but only with '...InTerminal'
This is true in both WezTerm and default Mac terminal.
That said, installing from github works as long as I use the terminal start&attach (and I can just close the terminal nvim windo wafter).
Super happy to have this working.
More debug logs would be amazong. (I've only just recently started getting serious about adding logging in my own projects as I go; I feel you there :)
By the way, the first and the third version are the same version. Notice they have the same commit number. I don't know what went wrong with the versioning, and maybe something unusual is going on with the installation. The third version is the right version.
In #83 there is a new file logging feature in a temp directory. Can you update it to the recent commit and check the output of the run and paste it here so we know how it's failing?
Hmm.
I'm sporting version:
jupynium --version : Jupynium v0.2.1.dev19+gff0ffd8
but using both fd jupynium -u -t d and fd /tmp/jupynium -u -t d I'm not finding a /tmp/jupynium/ folder even after attaching, synching, and saving <...>.ju.py files
Mac probably has a different temp dir.
You can use this python code to figure out
import tempfile
tempfile.gettempdir()
My browser also closes a few moments after :JupyniumStartSync. Unfortunately this happens for the pypi and git version and regardless of the InTerminal.
I uninstalled the vim plugin and just used the jupynium cli.
Upon opening I already had an nvim instance listening to localhost:18898.
When calling
jupynium --notebook_URL localhost:8889 --nvim_listen_addr localhost:18898
It outputs
jupynium.pynvim_helpers: 11 - INFO - nvim addr: localhost:18898
jupynium.pynvim_helpers: 30 - INFO - nvim attached
jupynium.pynvim_helpers: 34 - INFO - Initialising..
jupynium.pynvim_helpers: 35 - INFO - Communicating with channel_id 7
jupynium.cmds.jupynium: 93 - INFO - Using firefox profile: None
jupynium.cmds.jupynium: 388 - INFO - Writing Jupyter Notebook server log to: /tmp/tmp9_zb9q5n
[I 17:53:29.498 NotebookApp] Serving notebooks from local directory: /home/user
[I 17:53:29.499 NotebookApp] Jupyter Notebook 6.5.4 is running at:
[I 17:53:29.499 NotebookApp] http://localhost:8889/?token=...
[I 17:53:29.499 NotebookApp] or http://127.0.0.1:8889/?token=...
[I 17:53:29.499 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
The browser starts up, showing the jupyter root directory in the only open tab.
This pauses until in nvim I call JupyniumStartSync
That opens a new tab with an untittled file for a while. Nvim keeps the cursor on the :JupyniumStartSync while the original command outputs:
jupynium.events_control: 195 - INFO - Event from nvim: ['request', 'start_sync', [1, '', True, ['#! /usr/bin/env python3', '# vim:fenc=utf-8', '', '# %% python', 'print("hello world")'], 'python', '/home/user/temp/.venv'], <pynvim.msgpack_rpc.async_session.Response object at 0x7f5646b3da50>]
[I 17:53:46.448 NotebookApp] Creating new notebook in /temp
[W 17:53:48.364 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js (127.0.0.1) 21.540000ms referer=http://localhost:8889/notebooks/temp/Untitled1.ipynb?kernel_name=python3
[I 17:53:48.512 NotebookApp] Kernel started: 1700576c-c7d3-491f-841d-29a53ae54569, name: python3
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[IPKernelApp] WARNING | Unknown error in handling startup files:
jupynium.selenium_helpers: 38 - ERROR - Timed out waiting for kernel to load
Traceback (most recent call last):
File "/home/user/temp/.venv/lib/python3.11/site-packages/jupynium/selenium_helpers.py", line 32, in wait_until_notebook_loaded
WebDriverWait(driver, timeout).until(
File "/home/user/temp/.venv/lib/python3.11/site-packages/selenium/webdriver/support/wait.py", line 95, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
[I 17:53:59.945 NotebookApp] Starting buffering for 1700576c-c7d3-491f-841d-29a53ae54569:30e13caebdf647acaea32f286a4d137b
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b38b10>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b38b50>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b37e90>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window
jupynium.cmds.jupynium: 537 - ERROR - Uncaught exception occurred while processing events. Detaching nvim.
Traceback (most recent call last):
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 200, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 388, in request
self.endheaders()
File "/usr/lib/python3.11/http/client.py", line 1278, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1038, in _send_output
self.send(msg)
File "/usr/lib/python3.11/http/client.py", line 976, in send
self.connect()
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 236, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 215, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5646b36e10>: Failed to establish a new connection: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/temp/.venv/lib/python3.11/site-packages/jupynium/cmds/jupynium.py", line 532, in main
status, rpcrequest_event = process_events(nvim_info, driver)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/jupynium/events_control.py", line 205, in process_events
status, request_event = process_request_event(nvim_info, driver, event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/jupynium/events_control.py", line 429, in process_request_event
start_sync_with_filename(
File "/home/user/temp/.venv/lib/python3.11/site-packages/jupynium/events_control.py", line 324, in start_sync_with_filename
nvim_info.attach_buffer(bufnr, content, driver.current_window_handle)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 482, in current_window_handle
return self.execute(Command.W3C_GET_CURRENT_WINDOW_HANDLE)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
response = self.command_executor.execute(driver_command, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 290, in execute
return self._request(command_info[0], url, body=data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 311, in _request
response = self._conn.request(method, url, body=body, headers=headers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/_request_methods.py", line 110, in request
return self.request_encode_url(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/_request_methods.py", line 143, in request_encode_url
return self.urlopen(method, url, **extra_kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/poolmanager.py", line 443, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 874, in urlopen
return self.urlopen(
^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 874, in urlopen
return self.urlopen(
^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 874, in urlopen
return self.urlopen(
^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/home/user/temp/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=49881): Max retries exceeded with url: /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b36e10>: Failed to establish a new connection: [Errno 111] Connection refused'))
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b35f90>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window/handles
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b5bb90>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window/handles
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b5aad0>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769/window/handles
jupynium.cmds.jupynium: 566 - INFO - Browser disconnected. Quitting Jupynium.
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b35810>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b95490>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769
urllib3.connectionpool: 871 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5646b95bd0>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/99a904fd-52e0-40fc-97db-e6367fd0e769
jupynium.cmds.jupynium: 161 - SUCCESS - Piecefully closed as the browser is closed.
[I 17:54:00.498 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
jupynium.cmds.jupynium: 339 - INFO - Jupyter Notebook server (pid=75568) has been killed.
~/temp 0.00s - Debugger warning: It seems that frozen modules are being used, which may temp
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[IPKernelApp] WARNING | Unknown error in handling startup files:
[IPKernelApp] WARNING | Parent appears to have exited, shutting down.
At this point the browser has exited and nvim returns to normal mode (the cursor is no longer stuck in command mode).
I'm running this in a venv.
The installation I ran was
mkdir temp && cd temp && python -m venv .venv && . .venv/bin/activate && pip install --upgrade pip && pip install notebook git+https://github.com/kiyoon/jupynium.nvim
Here's the output of my pip list in that venv.
Package Version
------------------------- --------------------
anyio 3.7.1
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
asttokens 2.2.1
attrs 23.1.0
backcall 0.2.0
beautifulsoup4 4.12.2
bleach 6.0.0
certifi 2023.5.7
cffi 1.15.1
coloredlogs 15.0.1
comm 0.1.3
debugpy 1.6.7
decorator 5.1.1
defusedxml 0.7.1
exceptiongroup 1.1.2
executing 1.2.0
fastjsonschema 2.17.1
fqdn 1.5.1
gitdb 4.0.10
GitPython 3.1.31
greenlet 2.0.2
h11 0.14.0
humanfriendly 10.0
idna 3.4
ipykernel 6.24.0
ipython 8.14.0
ipython-genutils 0.2.0
isoduration 20.11.0
jedi 0.18.2
Jinja2 3.1.2
jsonpointer 2.4
jsonschema 4.18.0
jsonschema-specifications 2023.6.1
jupynium 0.2.1.dev20+g4f2fe81
jupyter_client 8.3.0
jupyter_core 5.3.1
jupyter-events 0.6.3
jupyter_server 2.7.0
jupyter_server_terminals 0.4.4
jupyterlab-pygments 0.2.2
MarkupSafe 2.1.3
matplotlib-inline 0.1.6
mistune 3.0.1
msgpack 1.0.5
nbclassic 1.0.0
nbclient 0.8.0
nbconvert 7.6.0
nbformat 5.9.0
nest-asyncio 1.5.6
notebook 6.5.4
notebook_shim 0.2.3
outcome 1.2.0
overrides 7.3.1
packaging 23.1
pandocfilters 1.5.0
parso 0.8.3
persist-queue 0.8.1
pexpect 4.8.0
pickleshare 0.7.5
pip 23.1.2
platformdirs 3.8.1
prometheus-client 0.17.0
prompt-toolkit 3.0.39
psutil 5.9.5
ptyprocess 0.7.0
pure-eval 0.2.2
pycparser 2.21
Pygments 2.15.1
pynvim 0.4.3
PySocks 1.7.1
python-dateutil 2.8.2
python-json-logger 2.0.7
PyYAML 6.0
pyzmq 25.1.0
referencing 0.29.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.8.10
selenium 4.10.0
Send2Trash 1.8.2
setuptools 65.5.0
six 1.16.0
smmap 5.0.0
sniffio 1.3.0
sortedcontainers 2.4.0
soupsieve 2.4.1
stack-data 0.6.2
terminado 0.17.1
tinycss2 1.2.1
tornado 6.3.2
traitlets 5.9.0
trio 0.22.1
trio-websocket 0.10.3
uri-template 1.3.0
urllib3 2.0.3
verboselogs 1.7
wcwidth 0.2.6
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.1
wsproto 1.2.0
geckodriver: 0.32.2
python: 3.11.3
nvim: 0.9.1
Linux: 6.4.1-arch2-1 x86_64 GNU/Linux
Hi,
I had the same issue with my setup where the kernel takes a long time to init. I fixed it by replacing:
https://github.com/kiyoon/jupynium.nvim/blob/master/src/jupynium/selenium_helpers.py#L11
by :
def wait_until_notebook_loaded(driver, timeout=30):
This increases the timeout by 20 secs and it should solve your issue, (or maybe you need to bump it even more)
Thank you, that did it.
@MathisClautrier Thank you. If you want to create a PR that changes the value, feel free to! Otherwise I'll fix it myself sometime.
Fixed in #86