TODO add wercker pipeline + icon
This is a pipeline to familiarize with the reactive platform and do do something fun.
docker-compose up -d
activator run
Tweets are ingested to Kafka. Spark is used to analyse them. The results are streamed via a web-socket to a play application to visualize the results. This project uses two microservices: one for the frontend & one for the backend
- this application runs at
http://localhost:9000 - spark can be accessed at
http://dockerHost:4040 - zookeeper can be accessed at
http://dockerHost:4040 - kafka can be accessed at
http://dockerHost:4040
TODO
Execute activator docker:publishLocaland run the docker use: docker run --name sparkPlay -p 8080:9000 -p 4040:4040 sparkplay:1.0-SNAPSHOT
Todo provide deploy script
- Future plans: use h2o-sparkling-water `"ai.h2o" %% "sparkling-water-core" % "1.5.6", However Scala 2.11 not yet supported
- Setup Spark Streaming with Kafka for Twitter sentiment analysis
- productionize the whole ting to try scaling it using mesos
- create vagrant / docker + ansible scripts
- use own zookeeper / kafka versions
- wercker
- scala-style http://www.scalastyle.org/
- http://devcenter.wercker.com/quickstarts/deployment/mesosphere.html
- http://devcenter.wercker.com/docs/using-the-cli/local-development.html
- https://hub.docker.com/r/ingensi/play-framework/