-
nix-shell
is a package manager that lets you:-
Use a package without installing it globally.
-
Create isolated, reproducible virtual environments with any packages you require without causing bunch of version conflicts - think python
venv
but for system packages (but it's so much more).
-
-
This blog by Mattia Gheda has a good introduction. to
nix-shell
. -
I have switched to using
nix-shell
for all my development needs. This repository contains nix files I use quite frequently when working with Python and CUDA, as well as other development languages and tools.
-
nix-shell
can be installed on both linux and macOS using the following command (follow on screen instructions):sh <(curl -L https://nixos.org/nix/install) --daemon
-
Assuming you have
nix-shell
installed, you can create environments with python and CUDA by invokingnix-shell
with the appropriate.nix
file. -
There are several in the repo with different combinations of python and CUDA versions.
-
For example, for
python 3.8
withCUDA 11.3
and cuDNN usepy38cuda113.nix
like so:# The location of this repository. REPO=/home/$USER/nixshells # Python 3.8 with CUDA 11.3 and CuDNN. # # After this command completes, you will be prompted with a new shell # with python v3.8.x and CUDA installed. When used the for the first time, # it will download python and CUDA files from the Nix package manager to your # local nixstore. Subsequent invocations will use downloaded files. # # CUDA and cuDNN is a big download (~4 GB), so the first invocation will take # a while depending on your internet speed. nix-shell ${REPO}/py38cuda113.nix
-
This environment comes with
python 3.8
,virtualenvwrapper
andCUDA 11.3
installed. You can now create a python virtual environment the usual way of usingvirtualenvwrapper
by running:# Will create a virutal environment called `tf28` under ~/.nixshells/py38-cuda113 # and enable it. Different .nix files will have a different directory for storing # python virtual environments. mkvirutalenv tf28
-
Install any packages you want in the python environment (which is now active inside the nix-shell):
# Installing tensorflow. pip install tensorflow==2.8.0 # Check if CUDA is available. python -c 'import tensorflow as tf; print(tf.test.is_gpu_available())'
-
To deactivate the
virtualenv
and nix shell, pressCtrl + D
. Your python environments will be preserved and be accessible you re-activate the nix shell again.
-
To use different python or CUDA versions, you can pretty much duplicate one of the
py.*cuda.*.nix
files and update the build parameters. -
The nix files are relatively straight forward, and you can add / remove packages you want by updating the
buildInputs
argument in the common.nix file. The package list here is shared with all other.nix
files in the repository. -
To search through the the 80,000+ available packages, check https://search.nixos.org/packages.
-
Of course you can create your own nix files too :D
-
This blog by Mattia Gheda has a good introduction.
-
The official wiki can be a bit intimidating for newbies but is an excellent resource nonetheless.
-
More about Nix and NixOS here.
-