Run Jupyter notebooks on clusters.
If there is an anaconda module available on the cluster, then load it and create a conda environment:
conda create --name <env> ipykernel numpy matplotlib --channel conda-forgeThe newly created environment can be activated and deactivated with conda <activate|deactivate> <env>.
All the conda environments can be listed with conda info --envs.
Conda environments can take between 1G to 10G bytes, make sure there is enough disk quota available with
quota -u <username> or if there may be machine specific way of disk space distribution between users,
e.g. WORK or SCRATCH space limitations on a specific machine. If the conda environment can't be created
on the home directory, it can be created elsewhere, e.g. SCRATCH but then must be soft linked to ~/.conda:
ln -s /path/to/conda ~/.condaTo install any other packages later on, e.g.jupyter_contrib_nbextensions, after activating the environment,
just install normally any package from any channel:
conda activate <env>
conda install jupyter_contrib_nbextensions --channel conda-forgeTo enable an extensions, e.g. collapsible_headings:
jupyter nbextension enable collapsible_headings/mainNow to access the juyter notebook from a local machine, first initialize the jupyter-notebook server on the cluster:
jupyter-notebook --no-browser --port=8893 --ip=127.0.0.1mind the url with the token. Now on the local machine forward the specified port to the cluster:
ssh -N -f -L localhost:8893:localhost:8893 <username>@<hostname> -i /path/to/parivate/keyNow the notebook server can be accessed locally with browser and the url generated at the last step.