matpalm / gardenhose-microslurp

bootstrap scripts for getting a micro ec2 instance piping gardenhose to s3

Home Page:matpalm.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

simple set of scripts for slurping twitter gardenhose and uploading to s3 daily

restarts curl from gardenhose hourly to ensure dropped connections are reestablished from time to time

requires a s3 bucket to be created beforehand (should really roll into the script)

start a micro instance and ssh on and run...
$ yum install git
$ git clone git://github.com/matpalm/gardenhose-microslurp.git
$ cp gardenhose-microslurp/* .
$ ./bootstrap.sh

bootstrap sets some env vars (in ~/env_vars) for twitter uid, 
s3 creds and sets up some cron jobs

About

bootstrap scripts for getting a micro ec2 instance piping gardenhose to s3

matpalm.com


Languages

Language:Shell 100.0%