Change in behaviour of DatabaseScheduler.max_interval causing high CPU load for Celery beat
Tirzono opened this issue · comments
Change in behaviour is introduced here:
Since self.app.conf.CELERYBEAT_MAX_LOOP_INTERVAL
defaults to 0
(https://github.com/celery/celery/blob/c1171da3da4547a8a5048e52551d2c764def48f4/celery/app/defaults.py#L91), self.max_interval
only uses DEFAULT_MAX_INTERVAL
if CELERYBEAT_MAX_LOOP_INTERVAL
is explicitly set to None
. This change in behaviour caused our Celery beat to query the DB thousand times per second causing a high CPU load for Celery beat.
Celery's Scheduler
is using how it was before:
https://github.com/celery/celery/blob/28e0e30d51fc2666a71918bb9d95de4c15dbe7a6/celery/beat.py#L247
I would opt for changing it back to how it was and how it currently is in Celery's Scheduler
. Workaround for now is to explicitly set CELERYBEAT_MAX_LOOP_INTERVAL
to for example the default value 5
.