AMPcamp5 Hands-On Exercise
I'm currently taking MOOC Introduction to Bit Data with Apach Spark. As an exercise to gain more experience, I'm also going through AMP camp 5 hands-on exercise using IPython notebook and posting it here. To run this note book, you need to download USB stick's content from here. Then, extract the content to your machine and copy the notebook to the root directory of the USB file content.
I chose to use Anaconda python distribution 2.2.0 and Spark 1.4.0 on Ubuntu 14.04 LTS. This blog post has instruction on how to install Spark locally.
Once everything is setup, just type
IPYTHON_OPTS="notebook" pyspark
from the root directory of the USB file content and select the notebook you want to run.
For the second lab, I used tachyon 0.6.4 prebuilt bindary from here.
For the third lab, I used spark 1.4.1. I ran into problems when building and had to modify sbt-launch-lib.bash as instructed here, last comment.