Table of Contents
HoneydewEctoNotifyQueue is a queue built for Honeydew that uses postgres notifications instead of polling. It was originally built before honeydew offered the ecto polling solution.
Contrary to the honeydew ecto polling solution, this package sets up two independent tables for managing the queue of jobs, and managing configuration over multiple instances;
A jobs
table tracks jobs and job_configs
table tracks configurations.
Honeydew.yield
is not supported by this adapter.
Jobs are reserved using postgres' FOR UPDATE NOWAIT
locking.
It is possible to suspend all job processing across instances by updating the suspended
job config;
{:ok, _config} = HoneydewEctoNotifyQueue.Config.update_config(MyApp.Repo, "suspended", "true")
See more about configuration handling here
The package is available on hex.pm here.
You can add it to your mix.exs,
defp deps do
[
# ..,
{:honeydew, "~> 1.1.5"},
{:honeydew_ecto_notify_queue, "~> 0.1"}
]
end
You can generate a migration to set up the required db tables with
$ mix honeydew_ecto_notify_queue.db.gen.migration
Note: You should read how to install honeydew here first
This queue takes some additional options. An example below,
def background_job_processes do
[
notifier_process(),
{
Honeydew.Queues,
[
:process, # queue name
queue: {
HoneydewEctoNotifyQueue, [
repo: YourApp.Repo,
max_job_time: 3_600, # seconds
retry_seconds: 15, # seconds,
notifier: YourApp.Notifier # this should match the `name` in `notifier_process` below
]
},
failure_mode: {Honeydew.FailureMode.Retry, times: 3}
]
},
{
Honeydew.Workers,
[:process, YourApp.Workers.ProcessWorker, num: 1]
}
]
end
def notifier_process do
%{
id: Postgrex.Notifications,
start: {Postgrex.Notifications, :start_link, [YourApp.Repo.config() ++ [name: YourApp.Notifier]]}
}
end
def start(_type, _args) do
children = [
# ... The rest of your app's supervision tree
] ++ background_job_processes
Supervisor.start_link(children, opts)
end
$ MIX_ENV=test mix do ecto.create, ecto.migrate
$ mix test
This queue also adds support for persisting job configuration state via postgres. By default, this is how queue suspension is managed across multiple instances.
You can leverage the existing notification setup to synchronise other configurations across instances.
An example of this may be the disabling of automatic queuing of a job when an API is hit.
You can see an example of how to listen for configuration changes in
examples/configuration_listener.ex