On complete is called while there are pending jobs

[Alexander S.]

We do add more jobs to the batch from an existing job. Something like this:

batch = Sidekiq::Batch.new

batch.on(:complete, MyJob, 'arg' => 123)

batch.jobs do
  MyJob.perform_async
end

And then in MyJob we do:

def perform
  batch.jobs do
    AnotherJob.perform_async
  end
end

Jobs are being added to the queue and are reflected in batch #pending but #complete? returns true and on_complete callback is called in MyJob once the first job is finished.

As a workaround we have to schedule another job like this:

batch.jobs do
  MyJob.perform_async
  ExtraJob.perform_in(5.minutes)
end

Batch details:

(byebug) Site.find(options['site_id']).batch_status("pages").complete?
true
(byebug) Site.find(options['site_id']).batch_status("pages").pending
25
(byebug) Site.find(options['site_id']).batch_status("pages").data
{:bid=>"AsffjPpN7BwOKA", :total=>29, :failures=>0, :pending=>25, :created_at=>"1631141513.511842", :complete=>true, :failure_info=>[], :parent_bid=>nil, :child_count=>0}

[Oduah Tobi]

@heaven could you add a reproducible demo for this so it's easier to investigate?

[github-actions]

Stale issue message