An introductory tutorial about leveraging Ray core features for distributed patterns.
These examples have been tested in the following environments:
- Ubuntu 18.04 LTS
- macOS 10.15
Using:
- Ray versions 1.2, 1.3, 1.4 (not the early release candidates)
- Python versions: 3.6, 3.7, 3.8
Currently, Python 3.9 is not supported by Ray.
See the slides.pdf file for the presentation slide deck that
accompanies this tutorial.
To get started use git to clone this public repository:
git clone https://github.com/DerwenAI/ray_tutorial.git
cd ray_tutorial
Set up a local virtual environment and activate it:
python3 -m venv venv
source venv/bin/activate
Then use pip to install the required dependencies:
python3 -m pip install -U pip
python3 -m pip install -r requirements.txt
Alternatively, if you use conda for installing Python packages:
conda create -n ray_tutorial python=3.7
conda activate ray_tutorial
python3 -m pip install -r requirements.txt
Note: if you run into any problems on Python 3.8 with "wheels"
during a pip installation, you may need to use the conda
approach instead.
Then launch the JupyterLab environment to run examples in this repo:
jupyter-lab
Browse to http://localhost:8888/lab to continue.
A Guided Tour of Ray Core covers an introductory, hands-on coding tour through the core features of Ray, which provide powerful yet easy-to-use design patterns for implementing distributed systems in Python. This training includes a brief talk to provide overview of concepts, then coding for remote functions, actors, parallel iterators, and so on. Then we'll follow with Q&A. All code is available in notebooks in the GitHub repo.
- Python developers who want to learn how to parallelize their application code
Note: this material is not intended as an introduction to the higher level components in Ray, such as RLlib and Ray Tune.
- Some prior experience developing code in Python
- Basic understanding of distributed systems
- What are the Ray core features and how to use them?
- In which contexts are the different approaches indicated?
- Profiling methods, to decide when to make trade-offs (compute cost, memory, I/O, etc.) ?
- Introduction to Ray core features as a pattern language for distributed systems
- Overview of the main Ray core features and their intended usage
- Background, primary sources, and closely related resources about distributed systems
- Code samples:
- Remote Functions:
ex_01_remo_func.ipynb - Remote Objects:
ex_02_remo_objs.ipynb - Remote Methods:
ex_03_remo_meth.ipynb - Multiprocessing Pool:
ex_04_mult_pool.ipynb - JobLib:
ex_05_job_lib.ipynb - Parallel Iterators:
ex_06_para_iter.ipynb
- Remote Functions:
- Profiling: comparing trade-offs and overhead
- Estimate Pi:
pi.ipynb
- Estimate Pi:
- Ray Summit, Anyscale Connect, developer forums, and other resources
- Q&A