TwitStreamer is a application used to help demonstrate Apache Spark's Streaming & Structured Streaming APIs
It uses Twitter4J to connect to Twitter's Firehose of public, filtered, tweets and convert those messages into a TCP-IP stream.
- Got https://apps.twitter.com/ and setup a new application
- In setting up your app, you need four peices of information.
- Consumer Key (API Key)
- Consumer Secret (API Secret)
- Access Token
- Access Token Secret
- To build, use the gradle targets clean, build and aws. The aws target creates the Buildfile and Procfile necessary to run on Elastic Beanstalk.
- The zip file located in /streaming-tweets-server/build/distributions/streaming-tweets-server-x.x.x.zip can be uploaded directly to Elastic Beanstalk or unziped and ran locally.
- Before running you need to setup four environment variables (either on your local PC or in the ElasticBeanstalk configuration). The four variables are listed below and are specified by your Twitter Application settings (step #2 above).
- To start the application (locally only), extract the zip file and run either
- /bin/streaming-tweets-server (mac/linux)
- /bin/streaming-tweets-server.bat (windows)