Papierkorb / diplomat

A HTTP Ruby API for Consul

Diplomat

A HTTP Ruby API for Consul

FAQ

What's Diplomat for?

Diplomat allows any ruby application to interact with Consul's distributed key value store, and also receive information about services currently available in the Consul cluster.

Does it work in rails?

Yup! In fact, we're using it in all of our rails production apps instead of any previous case where it'd be right to use environment variables according to 12Factor configuration principals. This gives us the ability to scale up without making any changes to the actual project codebase, and to move applications around the cluster with ease.

Here's what a production database.yml file might look like:

<% if Rails.env.production? %>
production:
  adapter:            postgresql
  encoding:           unicode
  host:               <%= Diplomat::Service.get('postgres').Address %>
  database:           <%= Diplomat::Kv.get('project/db/name') %>
  pool:               5
  username:           <%= Diplomat::Kv.get('project/db/user') %>
  password:           <%= Diplomat::Kv.get('project/db/pass') %>
  port:               <%= Diplomat::Service.get('postgres').ServicePort %>
<% end %>

Why would I use Consul over ZooKeeper, Doozerd, etcd, Nagios, Sensu, SmartStack, SkyDNS, Chef, Puppet, Ansible, etc?

Read up what makes Consul different here

How do I install Consul?

See here. I managed to roll it out on my production machines with the help of Ansible in one working day.

Usage

The most up to date place to read about the API is here.

Here's a few examples of how diplomat works:

Key Values

Setting

Setting the value of a key is easy as pie:

foo = Diplomat::Kv.put('foo', 'bar')
# => "bar"

Getting

Getting the value of a key is just as simple:

foo = Diplomat::Kv.get('foo')
# => "bar"

Or retrieve a value from another datacenter:

foo = Diplomat::Kv.get('foo', :dc => 'dc-west')
# => "baz"

You can also retrieve values recursively:

Diplomat::Kv.put('foo/a', 'lorem')
Diplomat::Kv.put('foo/b', 'ipsum')
Diplomat::Kv.put('foo/c', 'dolor')

Diplomat::Kv.get('foo/', recurse: true)
# => [{:key=>"foo/a", :value=>"lorem"}, {:key=>"foo/b", :value=>"ipsum"}, {:key=>"foo/c", :value=>"dolor"}]

Nodes

Getting

Look up a node:

foo_service = Diplomat::Node.get('foo')
# => {"Node"=>{"Node"=>"foobar", "Address"=>"10.1.10.12"}, "Services"=>{"consul"=>{"ID"=>"consul", "Service"=>"consul", "Tags"=>nil, "Port"=>8300}, "redis"=>{"ID"=>"redis", "Service"=>"redis", "Tags"=>["v1"], "Port"=>8000}}}

Get all nodes:

nodes = Diplomat::Node.get_all
# => [#<OpenStruct Address="10.1.10.12", Node="foo">, #<OpenStruct Address="10.1.10.13", Node="bar">]

Services

Getting

Looking up a service is easy as pie:

foo_service = Diplomat::Service.get('foo')
# => #<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>

Or if you have multiple nodes per service:

foo_service = Diplomat::Service.get('foo', :all)
# => [#<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>,#<OpenStruct Node="indigo", Address="1.2.3.5", ServiceID="indigo_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>]

Or if you want to find services for a particular datacenter

foo_service = Diplomat::Service.get('foo', :all, { :dc => 'My_Datacenter'})
# => [#<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>,#<OpenStruct Node="indigo", Address="1.2.3.5", ServiceID="indigo_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>]

If you wish to list all the services on consul:

services = Diplomat::Service.get_all
# => #<OpenStruct consul=[], foo=[], bar=[]>

If you wish to list all the services for a specific datacenter:

services = Diplomat::Service.get_all({ :dc => 'My_Datacenter' })
# => #<OpenStruct consul=[], foo=[], bar=[]>

Datacenters

Getting a list of datacenters is quite simple and gives you the option to extract all services out of all accessible datacenters if you need to.

datacenters = Diplomat::Datacenter.get()
# => ["DC1", "DC2"]

Sessions

Creating a session:

sessionid = Diplomat::Session.create({:Node => "server1", :Name => "my-lock"})
# => "fc5ca01a-c317-39ea-05e8-221da00d3a12"

Or destroying a session:

Diplomat::Session.destroy("fc5ca01a-c317-39ea-05e8-221da00d3a12")

Renew a session:

Diplomat::Session.renew(sessionid)

List sessions:

Diplomat::Session.list.each {|session| puts "#{session["ID"]} #{session["Name"]}"}

Locks

Acquire a lock:

sessionid = Diplomat::Session.create({:Node => "server1", :Name => "my-lock"})
lock_acquired = Diplomat::Lock.acquire("/key/to/lock", sessionid)
# => true

Or wait for a lock to be acquired:

sessionid = Diplomat::Session.create({:hostname => "server1", :ipaddress => "4.4.4.4"})
lock_acquired = Diplomat::Lock.wait_to_acquire("/key/to/lock", sessionid)

Release a lock:

Diplomat::Lock.release("/key/to/lock", sessionid )

Events

Fire an event:

Diplomat::Event.fire('do_something', 'payload')

List all events with a certain name received by the local agent:

Diplomat::Event.get_all('do_something')

Get the latest event with a certain name received by the local agent:

Diplomat::Event.get('do_something')

Iterate through the events with a certain name received by the local agent:

events = Enumerator.new do |y|
  ret = {token: :first}
  while ret = begin Diplomat::Event.get('do_something', ret[:token], :reject) rescue nil end
    y.yield(ret[:value])
  end
end

events.each{ |e| puts e }

Custom configuration

You can create a custom configuration using the following syntax:

Diplomat.configure do |config|
  # Set up a custom Consul URL
  config.url = "http://localhost:8888"
  # Set up a custom Faraday Middleware
  config.middleware = MyCustomMiddleware
  # Connect into consul with custom access token (ACL)
  config.acl_token =  "xxxxxxxx-yyyy-zzzz-1111-222222222222"
  # Set extra Faraday configuration options
  config.options = {ssl: { version: :TLSv1_2 }}
end

This is traditionally kept inside the config/initializers directory if you're using rails. The middleware allows you to customise what happens when faraday sends and receives data. This can be useful if you want to instrument your use of diplomat, for example. You can read more about Faraday's custom middleware here.

Todo

• Updating Docs with latest changes
• Using custom objects for response objects (instead of openStruct)
• PUTing and DELETEing services
• Custom SSL Cert Middleware for faraday
• Allowing the custom configuration of the consul url to connect to
• Deleting Keys
• Listing available services
• Health
• Members
• Status
• Datacenter support for services
• Ruby 1.8 support
• Events

Enjoy!

![Photo Copyright "merlinmann". All rights reserved.](http://i.imgur.com/3mBwzR9.jpg Photo Copyright "merlinmann" https://www.flickr.com/photos/merlin/. All rights reserved.)