kvz/airbud

Retrieving stuff from the web is unreliable. Airbud adds retries for production, and fixture support for test.

Airbud is a wrapper around request with support for for handling JSON, retries with exponential backoff & injecting fixtures. This will save you some boilerplate and allow you to easier test your applications.

Install

Inside your project, type

npm install --save airbud

Use

To use Airbud, first require it

In JavaScript

var Airbud = require('airbud');

Or CoffeeScript:

Airbud = require "airbud"

Airbud doesn't care.

Example: simple

A common usecase is getting remote JSON. By default Airbud.json will already:

Timeout each single operation in 30 seconds
Retry 5 times over 10 minutes
Return parsed JSON
Return err if
- A non-2xx HTTP code is returned (3xx redirects are followed first)
- The json could not be parsed

In CoffeeScript:

Airbud.json "https://api.github.com/events", (err, events, meta) ->
  if err
    throw err
  console.log events[0].created_at

Example: local JSON fixtures

Say you're writing an app that among things, retrieves public events from the GitHub API.

Using environment variables, your production environment will have a GITHUB_EVENTS_ENDPOINT of "https://api.github.com/events", but when you source envs/test.sh, it can be "file://./fixtures/github-events.json".

Now just let Airbud.retrieve the process.env.GITHUB_EVENTS_ENDPOINT, and it will either retrieve the fixture, or the real thing, depending which environment you are in.

This makes it easy to test your app's depending functions, without having to worry about GitHub ratelimiting, downtime, or sloth when running your tests. All of this without making your app aware, or changing it's flow. In JavaScript:

var opts   = {
  url: process.env.GITHUB_EVENTS_ENDPOINT,
};

Airbud.json(opts, function (err, events, meta) {
  if (err) {
    throw err;
  }

  console.log('Number of attempts: '+ meta.attempts);
  console.log('Time it took to complete all attempts: ' + meta.totalDuration);
  console.log('Some auto-parsed JSON: ' + events[0].created_at);
});

Example: customize

You don't have to use environment vars or the local fixture feature. You can also use Airbud as a wrapper around request to profit from retries with exponential backoffs. Here's how to customize the retry flow in CoffeeScript:

opts =
  retries         : 3
  randomize       : true
  factor          : 3
  minInterval     : 3  * 1000
  maxInterval     : 30 * 1000
  operationTimeout: 10 * 1000
  expectedStatus  : /^[2345]\d{2}$/
  expectedKey     : "status"
  url             : "https://api.github.com/events"

Airbud.retrieve opts, (err, events, meta) ->
  if err
    throw err

  console.log events

Example: 3 retries in one minute, retry after 3s timeout for each operation

opts =
  url             : "https://api2.transloadit.com/instances"
  retries         : 2
  factor          : 1.73414
  expectedKey     : "instances"
  operationTimeout: 3000

Some other tricks up Airbud's sleeves are expectedKey and expectedStatus, to make it error out when you get invalid data, without you writing all the extra if and maybes.

Options

Here are all of Airbud's options and their default values.

# Timeout of a single operation
operationTimeout: 30000

# Retry 5 times over 10 minutes
# http://www.wolframalpha.com/input/?i=Sum%5Bx%5Ek+*+5%2C+%7Bk%2C+0%2C+4%7D%5D+%3D+10+*+60+%26%26+x+%3E+0
# The maximum amount of times to retry the operation
retries: 4

# The exponential factor to use
factor: 2.99294

# The number of milliseconds before starting the first retry
minInterval: 5 * 1000

# The maximum number of milliseconds between two retries
maxInterval: Infinity

# Randomizes the intervals by multiplying with a factor between 1 to 2
randomize: true

# Automatically parse json
parseJson: null

# A key to find in the rootlevel of the parsed json.
# If not found, Airbud will error out
expectedKey: null

# An array of allowed HTTP Status codes. If specified,
# Airbud will error out if the actual status doesn't match.
# 30x redirect codes are followed automatically.
expectedStatus: "20x"

# Custom headers to submit in the request
headers: []

Contribute

I'd be happy to accept pull requests. If you plan on working on something big, please first give a shout!

Compile

This project is written in CoffeeScript, but the JavaScript it generates is commited back into the repository so people can use this module without a CoffeeScript dependency. If you want to work on the source, please do so in ./src and type: make build or make test (also builds first). Please don't edit generated JavaScript in ./lib!

Test

Run tests via make test.

To single out a test use make test GREP=30x

Release

Releasing a new version to npmjs.org can be done via make release-patch (or minor / major, depending on the semantic versioning impact of your changes). This:

updates the package.json
saves a release commit with the updated version in Git
pushes to GitHub
publishes to npmjs.org

Authors

Kevin van Zonneveld

got has a similar purpose but a much larger community to maintain it. Consider using it. Airbud however does provide meta information (in addition to err and data which are similar to got), that passes you the number of retries involved as well as the time it took for the first successful operation to complete. Airbud also supports file:// URLs meaning you could substitute URLs with fixtures easily for your project's testing purposes.

License

MIT Licensed.

kvz / airbud