dask-mini-tutorial

Mini tutorial on Dask for CMS skills week.

Setup

Clone the github repo

git clone https://github.com/stefgrs/dask-mini-tutorial

Create a virtual environment or a conda environment.

With virtual env do:

python3 -m venv /path/to/new/virtual/environment

source /path/to/new/virtual/environment/bin/activate

With conda do:

conda create --name dask-tutorial

conda activate dask-tutorial

With virtualenvwrapper do:

mkvirtualenv dask-tutorial

workon dask-tutorial

Install the requirements

From within the folder dask-mini-tutorial, type:

pip install -r requirements.txt

We also need to install the graphviz system library

On Ubuntu this is done by typing sudo apt-get install graphviz (note: this is the only one I actually tested).
On MAC, this should be done by typing brew install graphviz.
On Windows this should be done with the following steps:
- Install windows package from: https://graphviz.gitlab.io/_pages/Download/Download_windows.html
- Add C:\Program Files (x86)\Graphviz2.38\bin to User path
- Add C:\Program Files (x86)\Graphviz2.38\bin\dot.exe to System Path

More info here on how to install graphviz at this stackoverflow thread. For more details on the Windows installation also see this link.

Open jupyter notebook

Finally, open jupyter notebook by typing:

jupyter notebook

Easier setup

There is also a Colab version of this two-parts tutorial, but some functionalities will not work as well. Make a copy of the notebooks before starting (click on 'File', then on 'save a copy in Drive').

The dask.delayed notebook.
The dask.dataframe notebook

stefgrs / dask-mini-tutorial