Whitespace in `queues` configuration can cause issues.
TAGraves opened this issue · comments
We had the following queues
config for a worker:
"queues": "+workload_5s: 3; +workload_15s,workload_5s:3; +workload_1m,workload_15s,workload_5s:2; +workload_2m,workload_1m,workload_15s,workload_5s:2; +workload_5m,workload_2m,workload_1m,workload_15s,workload_5s:2",
We observed that with this configuration, workload_5m
jobs were never being worked. I removed all the spaces from the above configuration:
"queues": "+workload_5s:3;+workload_15s,workload_5s:3;+workload_1m,workload_15s,workload_5s:2;+workload_2m,workload_1m,workload_15s,workload_5s:2;+workload_5m,workload_2m,workload_1m,workload_15s,workload_5s:2",
Then the workload_5m
jobs started getting picked up. So it seems like the space between the ;
and the +workload_5m,
was causing an issue. I don't see it documented anywhere that whitespace is relevant, and indeed the readme even has several examples of whitespace being used in the queue configuration, so I suspect this is a bug.
GoodJob should be whitespace tolerant. I'm probably missing some strip
's 🤔 Here's where the string gets broken down into a configuration hash:
-
Where the configuration constant is broken down:
good_job/lib/good_job/multi_scheduler.rb
Lines 12 to 15 in 2586313
-
Where the comma-separated list of queues gets broken down:
good_job/app/models/good_job/execution.rb
Lines 42 to 68 in 2586313
Also, I love how you've set up your queues 😍
Let me make a test case, but I think it needs more strip
ing
Also, I love how you've set up your queues 😍
Unrelated: my one piece of feedback would be not to use the +
-queue ordering. That has performance downsides and you likely want to clear faster-SLO jobs first rather than the slower-SLO ones.