Giters

executablebooks / jupyter-book

Create beautiful, publication-quality books and documents from computational content.

Home Page:http://jupyterbook.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

MyST Evaluation fails: no running kernel found

aritmos opened this issue · comments

commented

Describe the bug

context
Unsure if this is an error within my environment or an actual bug. But JupyterBook fails to evaluate MyST markdown using the {eval} syntax.

expectation
Running jb build book evaluates any necessary MyST markdown.

bug
MyST cannot find the kernel for notebooks with MyST evaluations. In all cases the notebook is open in a Jupyter Lab instance with the kernel connected and idling.

jupyter lab clean did not change the outcome.

$ jupyter-book build book
...
<filepath>:40002: WARNING: This document does not have a running kernel [mystnb.eval]
...

problem
This is a problem for people that want to include MyST's evaluation in their notebooks.

Reproduce the bug

$ py -m venv .venv
$ source .venv/bin/activate
$ pip install jupyter-book
$ jb create book
$ pip install jupyterlab
  • open up notebook in jupyterlab
  • install myst extension
  • delete all cells in notebooks.ipynb, add a title to evade compilation warning
  • add a code cell with var = 3
  • add a subsequent markdown cell with {eval}'a' (with actual backticks)
  • restart the kernel and the evaluation works
$ jb build book

...
<filepath>:40002: WARNING: This document does not have a running kernel [mystnb.eval]
...

All while the notebook is open in jupyterlab with the kernel connected and idling.

List your environment

$ jb --version
Jupyter Book      : 0.15.1
External ToC      : 0.3.1
MyST-Parser       : 0.18.1
MyST-NB           : 0.17.2
Sphinx Book Theme : 1.0.1
Jupyter-Cache     : 0.6.1
NbClient          : 0.7.4
commented

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

commented

Upon further inspection upon opening the Jupyter Lab link I get 404 errors stating that the kernel doesn't exist:

[W 2023-12-10 17:35:31.248 ServerApp] 404 GET /api/kernels/dd565959-9cea-454e-bd2d-8459cecf60d7?1702226130934 (127.0.0.1): Kernel does not exist: dd565959-9cea-454e-bd2d-8459cecf60d7
[W 2023-12-10 17:35:31.248 ServerApp] wrote error: 'Kernel does not exist: dd565959-9cea-454e-bd2d-8459cecf60d7

The id that it refers to does not match any of the ids in the notebooks. I assume it is giving the notebooks new kernels upon failing to fetch the quoted kernel or something like that. The notebooks run fine but clearly MyST is probably trying to connect to the quoted kernel and failing.
Unsure how to fix this but clearly not a Jupyter Book issue.

commented

As I was playing around with identifying the kernel error I came across an instance where jupyter-lab raised no kernel problems, jupyter-book still showcased the same warning.

Reopening the issue, here are the full outputs:

$ jupyter lab --no-browser
[I 2023-12-10 17:49:48.086 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-12-10 17:49:48.105 ServerApp] Package jupysql_plugin took 0.0176s to import
[I 2023-12-10 17:49:48.114 ServerApp] Package jupyter_lsp took 0.0092s to import
[W 2023-12-10 17:49:48.115 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was f
[I 2023-12-10 17:49:48.119 ServerApp] Package jupyter_server_terminals took 0.0044s to import
[I 2023-12-10 17:49:48.177 ServerApp] Package jupyterlab_jupytext took 0.0576s to import
[I 2023-12-10 17:49:48.179 ServerApp] Package jupyterlab_myst took 0.0008s to import
[I 2023-12-10 17:49:48.179 ServerApp] Package notebook took 0.0000s to import
[I 2023-12-10 17:49:48.182 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-12-10 17:49:48.182 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was
[I 2023-12-10 17:49:48.182 ServerApp] jupysql_plugin | extension was successfully linked.
[I 2023-12-10 17:49:48.183 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-12-10 17:49:48.187 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-12-10 17:49:48.193 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-12-10 17:49:48.193 ServerApp] jupyterlab_jupytext | extension was successfully linked.
[I 2023-12-10 17:49:48.193 ServerApp] jupyterlab_myst | extension was successfully linked.
[I 2023-12-10 17:49:48.197 ServerApp] notebook | extension was successfully linked.
[I 2023-12-10 17:49:48.385 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-12-10 17:49:48.405 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-12-10 17:49:48.405 ServerApp] Registered jupysql-plugin server extension
[I 2023-12-10 17:49:48.405 ServerApp] jupysql_plugin | extension was successfully loaded.
[I 2023-12-10 17:49:48.407 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-12-10 17:49:48.408 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-12-10 17:49:48.410 LabApp] JupyterLab extension loaded from /home/aritmos/dev/github/steam-insights/.venv/lib/python3.12/site-packages/jupyterlab
[I 2023-12-10 17:49:48.411 LabApp] JupyterLab application directory is /home/aritmos/dev/github/steam-insights/.venv/share/jupyter/lab
[I 2023-12-10 17:49:48.411 LabApp] Extension Manager is 'pypi'.
[I 2023-12-10 17:49:48.413 ServerApp] jupyterlab | extension was successfully loaded.
[W 2023-12-10 17:49:48.414 ServerApp] [Jupytext Server Extension] Async contents managers like AsyncLargeFileManager are not supported at the moment (https://github.com/mwou
[I 2023-12-10 17:49:48.414 ServerApp] [Jupytext Server Extension] Deriving a JupytextContentsManager from LargeFileManager
[I 2023-12-10 17:49:48.415 ServerApp] jupyterlab_jupytext | extension was successfully loaded.
[I 2023-12-10 17:49:48.417 ServerApp] jupyterlab_myst | extension was successfully loaded.
[I 2023-12-10 17:49:48.420 ServerApp] notebook | extension was successfully loaded.
[I 2023-12-10 17:49:48.421 ServerApp] Serving notebooks from local directory: /home/aritmos/dev/github/steam-insights/src
[I 2023-12-10 17:49:48.421 ServerApp] Jupyter Server 2.10.1 is running at:
[I 2023-12-10 17:49:48.421 ServerApp] http://localhost:8888/lab?token=cada420f5b4a25739d7d2aba0910e05fa3f5edf8ed258a67
[I 2023-12-10 17:49:48.421 ServerApp]     http://127.0.0.1:8888/lab?token=cada420f5b4a25739d7d2aba0910e05fa3f5edf8ed258a67
[I 2023-12-10 17:49:48.421 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2023-12-10 17:49:48.424 ServerApp]

    To access the server, open this file in a browser:
        file:///home/aritmos/.local/share/jupyter/runtime/jpserver-10323-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=cada420f5b4a25739d7d2aba0910e05fa3f5edf8ed258a67
        http://127.0.0.1:8888/lab?token=cada420f5b4a25739d7d2aba0910e05fa3f5edf8ed258a67
[I 2023-12-10 17:49:48.757 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[W 2023-12-10 17:50:11.568 LabApp] Could not determine jupyterlab build status without nodejs
[I 2023-12-10 17:50:23.996 ServerApp] Kernel started: ca478665-80d2-4a58-9673-211d98919b0a
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-12-10 17:50:24.710 ServerApp] Connecting to kernel ca478665-80d2-4a58-9673-211d98919b0a.
[I 2023-12-10 17:50:24.720 ServerApp] Connecting to kernel ca478665-80d2-4a58-9673-211d98919b0a.
[I 2023-12-10 17:50:24.743 ServerApp] Connecting to kernel ca478665-80d2-4a58-9673-211d98919b0a.
[I 2023-12-10 17:50:37.412 ServerApp] Kernel started: d153665c-ef13-471c-aa5f-3d35696b68ef
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-12-10 17:50:38.048 ServerApp] Connecting to kernel d153665c-ef13-471c-aa5f-3d35696b68ef.
[I 2023-12-10 17:51:05.938 ServerApp] Kernel restarted: ca478665-80d2-4a58-9673-211d98919b0a
[I 2023-12-10 17:51:05.943 ServerApp] Starting buffering for ca478665-80d2-4a58-9673-211d98919b0a:7e3487dc-556e-4cbc-a340-9d0f665951d5
[I 2023-12-10 17:51:05.950 ServerApp] Connecting to kernel ca478665-80d2-4a58-9673-211d98919b0a.
[I 2023-12-10 17:51:05.950 ServerApp] Restoring connection for ca478665-80d2-4a58-9673-211d98919b0a:7e3487dc-556e-4cbc-a340-9d0f665951d5
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-12-10 17:52:23.606 ServerApp] Saving file at /book/dlc.ipynb
[I 2023-12-10 17:52:23.607 ServerApp] Saving dlc.ipynb

$ jb clean src/book
...
$ jb build src/book
Running Jupyter-Book v0.15.1
Source Folder: /home/aritmos/dev/github/steam-insights/src/book
Config Path: /home/aritmos/dev/github/steam-insights/src/book/_config.yml
Output Path: /home/aritmos/dev/github/steam-insights/src/book/_build/html
[sphinxcontrib-bibtex] Beware that docutils versions 0.18 and 0.19 (you are running 0.18.1) are known to generate invalid html for citations. If this issue affects you, plea
se use docutils<0.18 (or >=0.20 once released) instead. For more details, see https://sourceforge.net/p/docutils/patches/195/
Running Sphinx v5.0.2
making output directory... done
[etoc] Changing master_doc to 'intro'
checking bibtex cache... out of date
parsing bibtex file /home/aritmos/dev/github/steam-insights/src/book/references.bib... parsed 5 entries
myst v0.18.1: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['colon_fence', 'dollarmath', 'linkify', 'substitution', 'tasklist'], disable_syntax=[]
, all_links_external=False, url_schemes=['mailto', 'http', 'https'], ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anch
ors=None, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_s
pace=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area')
myst-nb v0.17.2: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='cache', execution_cache_path='',
 execution_excludepatterns=[], execution_timeout=-1, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_st
reams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}'
, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_m
arkdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/aritmos/dev/github/steam-insights/src/book/_build/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 6 source files that are out of date
updating environment: [new config] 6 added, 0 changed, 0 removed
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb: Using cached notebook: ID=6 [mystnb]
/home/aritmos/dev/github/steam-insights/src/book/markdown-notebooks.md: Using cached notebook: ID=1 [mystnb]
/home/aritmos/dev/github/steam-insights/src/book/notebooks.ipynb: Using cached notebook: ID=2 [mystnb]
/home/aritmos/dev/github/steam-insights/src/book/overview.ipynb: Using cached notebook: ID=4 [mystnb]

/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:10002: WARNING: Document headings start at H2, not H1 [myst.header]
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:100002: WARNING: Document headings start at H2, not H1 [myst.header]
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:110002: WARNING: Non-consecutive header level increase; H2 to H4 [myst.header]
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:200002: WARNING: Non-consecutive header level increase; H2 to H4 [myst.header]
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:300002: WARNING: Non-consecutive header level increase; H2 to H4 [myst.header]
/home/aritmos/dev/github/steam-insights/src/book/dlc.ipynb:370002: WARNING: This document does not have a running kernel [mystnb.eval]
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] overview
generating indices... genindex done
writing additional pages... search done
copying images... [100%] _build/jupyter_execute/a8664b9d567f7c4c4cd1924ee773ebb071160aed08f1f192169489c090f39639.png
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
[etoc] missing index.html written as redirect to 'intro.html'
build succeeded, 6 warnings.

The HTML pages are in src/book/_build/html.

===============================================================================

Finished generating HTML for book.
Your book's HTML pages are here:
    src/book/_build/html/
You can look at your book by opening this file in a browser:
    src/book/_build/html/index.html
Or paste this line directly into your browser bar:
    file:///home/aritmos/dev/github/steam-insights/src/book/_build/html/index.html

===============================================================================
commented

You need to enable inline execution for notebooks that use eval:

execute:
  execute_notebooks: inline

If you set that in your _config.yml, what do you see?

commented

I see a warning telling me that 'inline' is technically not one of the allowed values for execute_notebooks, but it works! no errors!

I double checked the documentation as I read it yesterday in case it was a config fault and I still can't find it. Is this stated anywhere in the documentation?

commented

@aritmos yes, it looks like we don't yet expose that option-value through Jupyter Book's configuration 🙃. You can set the myst-nb config value directly with nb_execution_mode in your Sphinx configuration (e.g. in the config block), but we're soon to release jupyter-book==1 that will fix this!

Thus, let's leave this open till that's merged!