Installed python packages cannot be used in notebooks / which python interpreter is being used
robert-elles opened this issue · comments
Hi,
How to control which python interpreter is being used to execute the notebooks?
I am using a virtual python environment with several libraries installed in it. Running a notebook with mercury I get import errors for those libraries. How can I configure mercury to use the python environment that has the libraries installed?
I started mercury from a shell that has the correct python interpreter in its path.
Hi @robert-elles,
My workflow is as follow:
- Create a virtual env and activate it.
- Add virtual env as a kernel to Jupyter. I'm doing it with command
python -m ipykernel install --user --name my_env - Start mercury server in the current terminal, mercury should use the current virtual env.
Please let me know if it works for you.
Hi,
thanks but it didn't help. Somehow it keeps using the jupyter from my global system python installation and not the local one.
Hi @robert-elles,
Could you please provide me a screenshot from your terminal when you run mercury? Please also try to run the command python -c "import sys; print(sys.path)" to check your environment paths.
I found out the import error is only about packages that are installed in editable mode. So it actually is using the local jupyter install but it cannot import packages from the same project that are editable packages.
If I run jupyter nbconvert --execute ./notebook.ipynb --to html --no-input it works and everything can be imported. But running merucry run from the same terminal and folder leads to the import error.
When I run python -c "import sys; print(sys.path)" the path of the library I want to import is there.
Hi,
I was able to find the cause of this. In mercury.py celery is started using subprocess.popen. This somehow starts celery with some other python interpreter, not the one that had my editable package installed.
I also saw that the notebook cells are executed using the execnb package. I am not sure if execnb will take the interpreter from the kernel.json to execute the code cells.
Pull request 395 solves my issue.