This Sidekiq plugin allows Render dynos to be started, stopped, and scaled based on job workload. Why? Because running non-stop Sidekiq dynos on Render can rack up unnecessary costs for apps with modest background processing needs.
This is a adaptation of the sidekiq heroku autoscale project to work with render.
Tested with Sidekiq 6, but should be compatible with other recent Sidekiq versions.
This plugin operates by tapping into Sidekiq startup hooks and middleware.
-
Whenever a server is started or a job is queued, the appropriate process manager is called on to adjust its scale. Adjustments are throttled across process instances (dynos) so that the Render API is only called once every N seconds – 10 by default.
-
When workload demands more dynos, scale will adjust directly upward to target capacity.
-
As workload diminishes, scale will adjust downward one dyno at a time. When downscaling a process, the highest numbered dyno (ex:
worker.1
,worker.2
, etc...) will be quieted and then removed from the formation. This combines Render's autoscaling logic with Sidekiq's quieting strategy.
gem 'sidekiq'
gem 'sidekiq-render-autoscale'
If you're not using Rails, you'll need to require sidekiq-render-autoscale
after sidekiq
.
You'll need to generate a Render platform API token that enables your app to adjust its own dyno formation. This can be done through the Render CLI with:
render authorizations:create
Copy the Token
value and add it along with your app's name as environment variables in your app:
SIDEKIQ_RENDER_AUTOSCALE_API_TOKEN=<token>
SIDEKIQ_RENDER_AUTOSCALE_APP=<app-name>
The Render Autoscaler plugin will automatically check for these two environment variables. You'll also find some setup suggestions in Sidekiq's Render deployment docs. Specifically, you'll want to include the -t 25
option in your Procfile's Sidekiq command to maximize process quietdown time:
web: bundle exec rails start
worker: bundle exec sidekiq -t 25
Add a configuration file for the Render Autoscale plugin. YAML works well. A simple configuration with one worker
process (named in your Procfile) that monitors all Sidekiq queues and starts/stops in the presence of work looks like this:
config/sidekiq_render_autoscale.yml
app_name: test-app
processes:
worker:
system:
watch_queues: *
include_retrying: true
include_scheduled: false
scale:
mode: binary
max_dynos: 1
Then, add an initializer to hand your configuration off to the plugin:
config/initializers/sidekiq.rb
config = YAML.load_file('<path/to/config.yml>')
Sidekiq::RenderAutoscale.init(config)
A more advanced configuration with multiple process types that watch specific queues would look like this – where first
and second
are two Render process types:
api_token: <optional - for dynamic insertion only!>
app_name: test-app
throttle: 20
history: 7200
processes:
first:
system:
watch_queues:
- default
- low
include_retrying: false
include_scheduled: false
scale:
mode: binary
max_dynos: 2
quiet_buffer: 15
second:
system:
watch_queues:
- high
include_retrying: false
include_scheduled: false
scale:
mode: linear
max_dynos: 5
workers_per_dyno: 20
min_factor: 1
Config Options
api_token:
optional, same asSIDEKIQ_RENDER_AUTOSCALE_API_TOKEN
. Always prefer the ENV variable, or dynamically insert this.app_name:
optional, same asSIDEKIQ_RENDER_AUTOSCALE_APP
.throttle:
number of seconds to throttle between scale adjustments. The default is 10, so the Render API will only be hit once every ten seconds regardless of how many time the process is pinged during that timeframe. This value also dictates the tick frequency on the web UI history graph.history:
number of seconds to track history in the web UI. The default is 3600, or 1 hour. The history graph renders ticks using the history duration divided by throttle time – so 3600 seconds of history on a 10 second throttle produce 360 data points. Therefore, it's best to keep these settings in modest proportions to one another. You'll probably be sad if you try to display days or weeks of history.processes:
a list of Render process types named in your Procfile. For example,worker
orsidekiq
.process.system.watch_queues:
a list of Sidekiq queues to watch for work, or*
for all queues. Queue names must be mutually exclusive to avoid collisions. That means a queue name may only appear once across all processes, and that*
(all) may not be combined with other queue names.process.system.include_retrying:
specifies if the Sidekiq retry set should be included while assessing workload. Watching retries may cause undesirable levels of uptime.process.system.include_scheduled:
specifies if the Sidekiq scheduled set should be included while assessing workload. Watching scheduled jobs may cause undesirable levels of idle uptime. Also, no new jobs will be scheduled unless Sidekiq is running.process.scale.mode:
accepts "binary" (on/off) or "linear" (scaled to workload).process.scale.max_dynos:
maximum allowed concurrent dynos. In binary mode, this will be the fixed operating capacity. In linear mode, this will be the maximum extent that dynos may scale up to.process.scale.workers_per_dyno:
Linear mode only. This specifies the anticipated workforce per dyno to calculate scale around. This should generally align with Sidekiq'sconcurrency
setting.process.quiet_buffer:
number of seconds to quiet a dyno (stopping it from taking on new work) before downscaling its process. This buffer occurs before reducing the number of dynos for a given process type. After downscale, you may configure an additional quietdown threshold. Note: during this quiet buffer your formation includes a decomissioned dyno, which is awkward – thus no other scale adjustments (up or down) are allowed until the quieted dyno has been dropped. Be accordingly judicious with this buffer.
The web UI is an optional extension of Sidekiq's web UI. To activate it, just require sidekiq/render_autoscale/web
after the base sidekiq/web
, and then mount Sidekiq::Web
as normal:
require 'sidekiq/web'
require 'sidekiq/render_autoscale/web'
Rails.application.routes.draw do
mount Sidekiq::Web, at: '/sidekiq'
end
Nothing fancy...
# start a redis server
redis-server test/redis_test.conf
# then run tests in another terminal window
bundle exec rake test
- Justin Love @wondible, https://github.com/JustinLove
- Benjamin Kudria https://github.com/bkudria
- claudiofullscreen https://github.com/claudiofullscreen
- Fix Peña https://github.com/fixr
- Gabriel Givigier Guimarães https://github.com/givigier
- Matt Anderson https://github.com/tonkapark
- Thibaud Guillaume-Gentil https://github.com/jilion
Sidekiq Render Autoscale plugin is released under the MIT license.