wesenu / cscie88c-final-project

CSCIE88c - Programming in Scala for Big Data Systems (Final Project)

Usage

make start-kafka
make stop-kafka

make start-spark
make stop-spark

Resources

• Creating a Spark Standalone Cluster with Docker and docker-compose

Limitations

• It is not possible to do more than one aggregation in a given streaming application. Spark Streaming appears to still be relatively experimental or not fully feature-complete.

Notes

• Reset vm.max_map_count

sudo sysctl -w vm.mac_map_count=65530