jeasoft / caribbeanspark

Conference Demonstration of in-memmory distributed data processing with Apache Spark

Caribbean Spark

Demonstration of in-memory distributed data processing with Apache Spark

PyCaribbean 2019

How to use

Create Virtual Environmet and install dependencies

python -m venv caribbeanspark
source caribbeanspark/bin/activate
pip install pip --upgrade
pip install -r requirements.txt 

Install Jupyter

Create a new Kernel for Jupyter with dependencies

ipython kernel install --user --name=caribbeanspark

Start Jupyter Notebook

jupyter notebook

In Jupyter Notebook, select the new kernel:

Kernel > Change Kernel > caribbeanspark

Run the cells in the notebook and enjoy!