vgarvardt / gue

Golang queue on top of PostgreSQL

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Jobs are not being precessed immediately

kotyara85 opened this issue · comments

Hi there,
I have this workers pool set

	workers, err := gue.NewWorkerPool(
		gc,
		wm,
		10,
		gue.WithPoolQueue(printerQueue),
		gue.WithPoolPollInterval(time.Minute*5),
		gue.WithPoolLogger(azap.New(zap.NewExample())),
		gue.WithPoolPollStrategy(gue.PriorityPollStrategy),
		gue.WithPoolHooksJobDone(finishedJobsLog),
	)

My impression was that gue will process any added messages right away, based on priority, but it looks like it's waiting for 5 minutes poll interval

Is this an expected behavior?

My use case is that I want to process newly added jobs right away and make them retry in 5 min if they fail.

Thanks

If you need near-real-time jobs handling you need to set poll interval to a very small value - e.g. 500ms. This parameter is responsible for polling pause in case there are no jobs currently to be handled.

For retry in 5min you need to either set Backoff strategy when you instantiate a client using WithClientBackoff() option, for your case NewConstantBackoff(5*time.Minute) should work, or you can also return ErrRescheduleJobIn(5*time.Minute, ...) error explicitly for the failed jobs in the handler.

If you need near-real-time jobs handling you need to set poll interval to a very small value - e.g. 500ms. This parameter is responsible for polling pause in case there are no jobs currently to be handled.

For retry in 5min you need to either set Backoff strategy when you instantiate a client using WithClientBackoff() option, for your case NewConstantBackoff(5*time.Minute) should work, or you can also return ErrRescheduleJobIn(5*time.Minute, ...) error explicitly for the failed jobs in the handler.

I appreciate.

One more question,
Is it explicitly possible to mark job as done? Besides removing its record from the db.

Thanks!

You can return ErrDiscardJob(...) in the handler to discard a job explicitly https://pkg.go.dev/github.com/vgarvardt/gue/v5#ErrDiscardJob

You can return ErrDiscardJob(...) in the handler to discard a job explicitly https://pkg.go.dev/github.com/vgarvardt/gue/v5#ErrDiscardJob

I was mostly interested on calling it outside of any gue handlers.

Then only direct DB intervention.

Thank you!