celery / django-celery

Old Celery integration project for Django

Home Page:http://celery.github.com/django-celery

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Change in behaviour of DatabaseScheduler.max_interval causing high CPU load for Celery beat

Tirzono opened this issue · comments

Change in behaviour is introduced here:

9866b6b

Since self.app.conf.CELERYBEAT_MAX_LOOP_INTERVAL defaults to 0 (https://github.com/celery/celery/blob/c1171da3da4547a8a5048e52551d2c764def48f4/celery/app/defaults.py#L91), self.max_interval only uses DEFAULT_MAX_INTERVAL if CELERYBEAT_MAX_LOOP_INTERVAL is explicitly set to None. This change in behaviour caused our Celery beat to query the DB thousand times per second causing a high CPU load for Celery beat.

Celery's Scheduler is using how it was before:

https://github.com/celery/celery/blob/28e0e30d51fc2666a71918bb9d95de4c15dbe7a6/celery/beat.py#L247

I would opt for changing it back to how it was and how it currently is in Celery's Scheduler. Workaround for now is to explicitly set CELERYBEAT_MAX_LOOP_INTERVAL to for example the default value 5.