paperTrail
TweetSpout => ParseTweetBolt => CountBolt => IntermediateRankingsBolt => TotalRankingsBolt => ReportBolt
Experiments with Apache Storm writing analytics to Redis
Data visualized in D3.js via Flask
storm install
https://gist.github.com/bigsnarfdude/93561cf1c4181758ede0
run project
Build jar file for storm:
mvn package
Submit jar to storm:
storm jar target/storm-basic-analytics-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.example.TopNTweetTopology
run frontend
Start flask:
$ paperTrail/frontend> python app.py
Open Browser:
http://localhost:5000
Potential Algos or data structures
- Average
- Cardinality
- Moments
- Mean
- Quantiles
- Geo Bounds?
- Max
- Min
- Top K
- Percentiles
- Percentile Ranks
- Sum
- Top hits
- Counts
- Increments
- T-Digest
- MinHash
Questions to answer
- Activity Tracking on Strings and Things (Did user 'x' log into System 'y'?)
- Event Tracking on Things (Tell me about all the 500 errors in the past week on System 'y'?)
- Leaderboards
.