breamware / sidekiq-batch

Sidekiq Batch Jobs Implementation

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

on_success callback is fired when some workers are still ongoing

schodevio opened this issue · comments

For a few days, I'm facing a problem with success callback:

class ParentWorker
  include Sidekiq::Worker

  def perform
    batch = Sidekiq::Batch.new

    batch.on(:success, ParentWorker::Success)

    batch.jobs do
      ChildOneWorker.perform_async

      ChildTwoWorker.perform_async
    end
  end

  class Success
    def on_success(_status, _options)
      Rails.logger.info "On Success Callback"
    end
  end
end

class ChildOneWorker
  include Sidekiq::Worker

  def perform
    Rails.logger.info "ChildOne Start"

    a, b = 2, 4
    Rails.logger.info "a + b = #{a + b}" # simple and quick action

    Rails.logger.info "ChildOne Finish"
  end
end

class ChildTwoWorker
  include Sidekiq::Worker

  def perform(page = 1)
    Rails.logger.info "ChildTwo Start"

    # make API request and process data
    5.times do |n|
      Rails.logger.info "Page: #{page} - Second #{n}"
      sleep 1
    end

    # re-call when seconds page of data needs to be fetched
    self.class.perform_async(page + 1) if page < 2

    Rails.logger.info "ChildTwo Finish"
  end
end

Current behavior

ChildTwo Start
Page: 1 - Second 0
ChildOne Start
a + b = 6
ChildOne Finish
Page: 1 - Second 1
Page: 1 - Second 2
Page: 1 - Second 3
Page: 1 - Second 4
ChildTwo Finish
ChildTwo Start
Page: 2 - Second 0
On Success Callback
Page: 2 - Second 1
Page: 2 - Second 2
Page: 2 - Second 3
Page: 2 - Second 4
ChildTwo Finish

Expected behavior

ChildTwo Start
Page: 1 - Second 0
ChildOne Start
a + b = 6
ChildOne Finish
Page: 1 - Second 1
Page: 1 - Second 2
Page: 1 - Second 3
Page: 1 - Second 4
ChildTwo Finish
ChildTwo Start
Page: 2 - Second 0
Page: 2 - Second 1
Page: 2 - Second 2
Page: 2 - Second 3
Page: 2 - Second 4
ChildTwo Finish
On Success Callback

Is there any workaround for this?

Stale issue message