faktory_worker_ruby

Faktory_worker_ruby is the official Ruby client and worker process for the Faktory background job server. It is similar to Sidekiq.

                       +--------------------+
                       |                    |
                       |     Faktory        |
                       |     Server         |
        +---------->>>>|                    +>>>>--------+
        |              |                    |            |
        |              |                    |            |
        |              +--------------------+            |
+-----------------+                            +-------------------+
|                 |                            |                   |
|    Client       |                            |     Worker        |
|    pushes       |                            |     pulls         |
|     jobs        |                            |      jobs         |
|                 |                            |                   |
|                 |                            |                   |
+-----------------+                            +-------------------+

Client - an API any process can use to push jobs to the Faktory server.
Worker - a process that pulls jobs from Faktory and executes them.
Server - the Faktory daemon which stores background jobs in queues to be processed by Workers.

This gem contains only the client and worker parts. The server part is here

Installation

First, make sure you have the Faktory server installed.

Next, install this gem:

gem install faktory_worker_ruby

Documentation

See the wiki for more details.

Your First Job

Your Jobs should include the Faktory::Job module and have a perform method.

class SomeJob
  include Faktory::Job

  def perform(...)
  end
end

then just call SomeJob.perform_async(...) to create a job.

Arguments to the perform method must be simple types supported by JSON, exactly like Sidekiq.

Start the worker

Once you've created a job, you need to start a Worker process to execute those jobs.

bundle exec faktory-worker

Why not Sidekiq?

Sidekiq is awesome; it's stable and useful. It suffers from two design limitations:

Sidekiq uses Redis and Redis is dumb datastore, all Sidekiq features are implemented in Ruby and have to travel over the network to access data.
Because of (1), Sidekiq is limited to Ruby. You can't execute jobs with, e.g., Python and get the same Sidekiq features.

Instead Faktory is a "smart" server that implements all major features within the server itself; the worker process can be dumb and rely on the server for job retries, reservation, Web UI, etc. This also means we can create workers in any programming language.

If your organization is 100% Ruby, Sidekiq will serve you well. If your organization is polyglot, Faktory will be a better fit.

faktory_worker_ruby tries to be Sidekiq API compatible where possible (and PRs to improve this are very welcome).

Author

Mike Perham, @mperham, mike @ contribsys.com

thebadmonkeydev / faktory_worker_ruby