edgurgel / verk

How to restore queues with data from redis after the application restart, which have been added dynamically Verk.add_queue(:new, 10)
At the initial startup, the supervisor only reads the data from the config

verk/lib/verk/supervisor.ex

Line 19 in 062811e

queues = Confex.get_map(:verk, :queues, [])

Verk does not saves the queues that were started dynamically. The initial configuration should have them or when Verk starts, add the queues back again dynamically.

One common use case is to start a supervision tree with the Verk.Supervisor then a process that adds the dynamic queues when it starts. So if the supervision tree restarts it will guarantee that the queues were added. I can add more information and examples around this if needed.

I can add more information and examples around this if needed.

I would be very grateful to you)

You could for example have a supervisor that has the following supervision tree:

tree = [supervisor(Verk.Supervisor, []), worker(QueuesLoader, [])]
opts = [name: MySupervisor, strategy: :rest_for_one]
Supervisor.start_link(tree, opts)

The QueuesLoader can be a GenServer that looks like this:

defmodule QueuesLoader do
  use GenServer
  require Logger

  @doc false
  def start_link do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @doc false
  def init(_args) do
    for queues <- MyQueues.all do
      Verk.add_queue(...)
    end
    { :ok, nil }
  end
end

Notice that this supervisor is a rest_for_one so that if Verk.Supervisor crashes, it will restart the QueuesLoader worker as it's defined after the Verk.Supervisor.

How to restore dynamically added queues