pfarrell / dooper

helper library for interacting with S3, HDFS, and WebHCat

Dooper

Exposes some objects that allow simple interaction with S3, WebHcat, HDFS, and YARN

Installation

gem 'dooper'
gem install dooper

Environment vars

We let this gem pull in credentials and metadata from environment variables

• AMAZON_SECRET_ACCESS_KEY
• AMAZON_ACCESS_KEY_ID
• AWS_REGION
• WEBHCAT_URL (defaults to localhost:50111)
• WEBHCAT_USER
• YARN_URL (defaults to localhost:8088)
• HDFS_HOST (defaults to localhost)
• HDFS_PORT (defaults to 50070)

Usage

TODO: Write usage instructions here

References

• S3 v2 API
• WebHDFS REST API
• WebHCat REST API
• YARN REST API