tklx/celery - Distributed task queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready). Celery is used in production systems to process millions of tasks a day.

Features

Based on the super slim tklx/base (Debian GNU/Linux).
Celery installed directly from Debian.
Uses tini for zombie reaping and signal forwarding.
Includes ENV CELERY_BROKER_URL amqp://guest@rabbit so linking with a RabbitMQ container works by default.

Usage

Start a celery worker utilizing RabbitMQ as broker

$ docker run --name some-rabbitmq -d tklx/rabbitmq
$ docker run --link some-rabbitmq:rabbit --name some-celery -d tklx/celery

Check the status of the cluster (RabbitMQ)

$ docker run --link some-rabbitmq:rabbit --rm celery python -m celery status

Start a celery worker utilizing Redis as broker

$ docker run --name some-redis -d tklx/redis
$ docker run --link some-redis:redis -e CELERY_BROKER_URL=redis://redis --name some-celery -d tklx/celery

Check the status of the cluster (Redis)

$ docker run --link some-rabbitmq:rabbit -e CELERY_BROKER_URL=redis://redis --rm tklx/celery python -m celery status

Automated builds

The Docker image is built, tested and pushed by CircleCI from source hosted on GitHub.

Tag: x.y.z refers to a release (recommended).
Tag: latest refers to the master branch.

Status

Currently on major version zero (0.y.z). Per Semantic Versioning, major version zero is for initial development, and should not be considered stable. Anything may change at any time.

Issue Tracker

TKLX uses a central issue tracker on GitHub for reporting and tracking of bugs, issues and feature requests.

tklx / celery