jandjshi / vanity

Experiment Driven Development for Ruby

Vanity is an Experiment Driven Development framework for Rails.

• All about Vanity: vanity.labnotes.org

• On github: github.com/assaf/vanity

A/B Testing With Rails (In 5 Easy Steps)¶ ↑

Step 1: Start using Vanity in your Rails application:

== Rails 2.x configuration

Rails::Initializer.run do |config|
  gem.config "vanity"

  config.after_initialize do
    require "vanity"
  end
end

== Rails 3 configuration

gem 'vanity' # in Gemfile, either outside of a group or inside of a bundler group

And:

class ApplicationController < ActionController::Base
  use_vanity :current_user
end

Step 2: Define your first A/B test. This experiment goes in the file experiments/price_options.rb:

ab_test "Price options" do
  description "Mirror, mirror on the wall, who's the better price of all?"
  alternatives 19, 25, 29
  metrics :signups
end

NOTE: If using a metric as above ("signups"), there needs to be a corresponding ruby file for that metric. Inside the "experiments" directory create a "metrics" directory with a file called "signup.rb". The contents of the file can describe the signup metric, refer to the "Metrics" Vanity documentation page for an example.

Step 3: Present the different options to your users:

<h2>Get started for only $<%= ab_test :price_options %> a month!</h2>

Step 4: Measure conversion:

class SignupController < ApplicationController
  def signup
    @account = Account.new(params[:account])
    if @account.save
      track! :signups
      redirect_to @acccount
    else
      render action: :offer
    end
  end
end

Step 5: Check the report:

vanity report --output vanity.html

== Rails 3

There is currently an issue with report generation. The vanity-talk Google Group has a couple posts that outline the issue for now. This is one of the posts: http://groups.google.com/group/vanity-talk/browse_thread/thread/343081a72a0cefb6

If you are collecting data (in development you need to opt-in to this by setting Vanity.playground.collecting = true in environments/development.rb) you can view experiment results with the vanity dashboard instead of the report. 

The vanity dashboard setup instructions with Vanity work for Rails 3.x except the route is different. A Rails 3.x-style route would look like this:

  `match '/vanity(/:action(/:id(.:format)))', :controller=>:vanity`

Contributing¶ ↑

• Fork the project

• Please use a topic branch to make your changes, it’s easier to test them that way

• Fix, patch, enhance, document, improve, sprinkle pixie dust

• At minimum run rake test, if possible, please run rake test:all

• Tests. Please. Run rake test, of if you can, rake test:all

• Send a pull request on GitHub

Credits/License¶ ↑

Original code, copyright of Assaf Arkin, released under the MIT license.

Documentation available under the Creative Commons Attribution license.

For full list of credits and licenses: vanity.labnotes.org/credits.html.