Question: Semantics on cancel of workerpool
bjartek opened this issue · comments
I am trying to implement logic where when i cancel the context a workerPool run in the active jobs are allowed to finish, but not accept new jobs. Is this a pattern that gue supports?
I tried to experiment a little but i did not find an obvious way to do this.
This is exactly how worker loop works - it checks if the context was cancelled before trying to handle the next job.
But the context is propagated to the handler as well, so if it is used in the handler logic then it is up to a handler to handle context cancellation. E.g. when the context is propagated further to the DB calls - normally drivers are handling cancellations as well, so you need to avoid this and use your own context for handler logic.
Very interesting, thanks Vladimir.
This is the log of my current experiement
2022/10/19 12:45:39 Performing work id=55 using worker=1
2022/10/19 12:45:39 Performing work id=57 using worker=0
^C2022/10/19 12:45:42 Received interrupt (interrupt), exiting app
2022/10/19 12:45:49 Hello name=name-55, index=1! jobId=55
2022/10/19 12:45:49 Got an error on deleting a job level=error worker-pool-id=d276a4 worker-id=d276a4/worker-1 job-id=55 job-type=printer error=timeout: context already done: context canceled
2022/10/19 12:45:49 Job finished level=debug worker-id=d276a4/worker-1 job-id=55 job-type=printer worker-pool-id=d276a4
2022/10/19 12:45:49 Failed to mark job as done level=error worker-pool-id=d276a4 worker-id=d276a4/worker-1 job-id=55 job-type=printer error=timeout: context already done: context canceled
2022/10/19 12:45:49 Worker finished level=info worker-pool-id=d276a4 worker-id=d276a4/worker-1
2022/10/19 12:45:49 Hello name=name-57, index=0! jobId=57
2022/10/19 12:45:49 Got an error on deleting a job level=error worker-pool-id=d276a4 worker-id=d276a4/worker-0 job-id=57 job-type=printer error=timeout: context already done: context canceled
2022/10/19 12:45:49 Job finished level=debug worker-id=d276a4/worker-0 job-id=57 job-type=printer worker-pool-id=d276a4
2022/10/19 12:45:49 Failed to mark job as done level=error worker-pool-id=d276a4 worker-id=d276a4/worker-0 job-id=57 job-type=printer error=timeout: context already done: context canceled
2022/10/19 12:45:49 Worker finished level=info worker-pool-id=d276a4 worker-id=d276a4/worker-0
2022/10/19 12:45:49 Worker pool finished level=info worker-pool-id=d276a4
2022/10/19 12:45:49 Worker finished
2022/10/19 12:45:52 done
Can i make job 55 and 57 be marked as successfull here? They where started before the cancellation so I want them to finish.
That's an interesting case, this seems solvable by adding an option to use different contexts for worker loop and worker tasks as well. Will think about it.
Thank you :)
Yeah i think in my usecase i want each worker to be allowed to run with deadline of x but on sigterm i want to stop enquing new jobs
Just merged #117 - it needs a little bit of testing before releasing. And want add transaction exposing for the job to enable more atomicity to the same release.