Multiple consumer process job and its subsequent jobs which ever is free, It all should by processed by one only.
iflairgit opened this issue · comments
First of all Thank you for providing this wonderfull package.
My question / case is I wanted to run multiple workers (consumers) in docker container with same code and they all will wait for the jobs for the same redis connection with same queue name.
The issue is I have a task that have sequential jobs like Task 1 (job 1, job 2, job 3), Task 2 (job 1, job 2, job 3) and I would like to process this Task 1 jobs in the same worker only at the moment what is happening job 1 process by worker 1 then it adds the another job to the queue that processed by worker 2 and so on...
Please note the subsequent jobs (child) jobs added to queue once parent finishes.
What I want is All the jobs of the particular task must be processed by the single worker.
I want multiple worker (consumer) instance with concurrency 2 for each for the load balancing purpose.
Any help will be appreciated.
Bull version
bull : 4.10.1
@nestjs/bull : 0.6.3
I'am using https://github.com/nestjs/bull as a wrapper of bull inside my nestjs project.
In BullMQ we have support for child jobs, however binding a worker to a given job is not supported. Not sure we will add support for this as it breaks a bit the current philosophy of having a bunch of dummy workers that just process jobs as fast as they can. More can be read here: https://docs.bullmq.io/guide/flows