MikaAK / libcluster_ec2_tag_strategy

LibCluster EC2 Tag Strategy to help nodes cluster together with different topologies

LibCluster Cluster.Strategy.EC2Tag

This clustering strategy relies on Amazon EC2 Tags as well as EPMD to find hosts, and then uses the :net_adm module to connect to nodes on those hosts.

It also supports setting multiple toplogies, and will choose which one to use based off the current host_name and tags found. For example if your current node has the tags that match one topology, they will be considered to be part of that topology and attempt to connect into that mesh

Note: This module requires ExAws to be configured

Installation

Available in Hex, the package can be installed by adding libcluster_ec2_tag_strategy to your list of dependencies in mix.exs:

def deps do
  [
    {:libcluster_ec2_tag_strategy, "~> 0.1.0"}
  ]
end

The docs can be found at https://hexdocs.pm/libcluster_ec2_tag_strategy.

Usage

You can have LibCluster automatically connect to nodes that match tags and setup multiple topologies:

config :libcluster, :topologies, [
  frontend_nodes: [
    strategy: Cluster.Strategy.EC2Tag,
    config: [
      tag_name: "Backend Group",
      tag_value: ~r/(user|account) Frontend/i,
      filter_fn: {MyHelper, :filter_instances}
    ]
  ],

  data_nodes: [
    strategy: Cluster.Strategy.EC2Tag,
    config: [
      tag_name: "Backend Group",
      tag_value: "Data Nodes",
      region: "us-east-2",
      filter_node_names: {MyHelper, :filter_node_names}
    ]
  ],

  some_other_nodes: [
    strategy: Cluster.Strategy.EC2Tag,
    config: [
      tag_name: "Backend Group",
      tag_value: "Extra Nodes",
      region: "us-east-2",
      host_name_fn: {MyHelper, :host_name}
    ]
  ]
]

defmodule MyHelper do
  def filter_instances(%{"tagSet" => %{"item" => tags}}) do
    case Enum.find(tags, &(&1["name"] === "Name")) do
      nil -> false
      %{"value" => value} -> value === "Learn Elixir Lander"
    end
  end

  def filter_node_names(node) do
    node =~ "my_node@host-00.&"
  end

  # This comes from ExAws.EC2 describe_instances
  # By default we use the `instanceId`
  def host_name(ec2_instance) do
    ec2_instance["privateDns"]
  end
end