yammer / circuitbox

Circuit breaker built with large Ruby apps in mind.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Once circuit is open, it never closes

luizkowalski opened this issue · comments

I have the following configuration

require 'faraday'
require 'circuitbox/faraday_middleware'

module CircuitBreaker
  extend ActiveSupport::Concern
  def circuitbox
    @circuit ||= Circuitbox.circuit(:device_api_call,
                                    exceptions: [Faraday::Error::TimeoutError, Faraday::Error::ConnectionFailed],
                                    sleep_window:     10,
                                    time_window:      10,
                                    volume_threshold: 10,
                                    cache: Moneta.new(:Memory),
                                    error_threshold:  50,
                                    timeout_seconds:  1,
                                    logger: Logger.new(STDOUT))
  end

  def http_client(adddress = ZENGUARD_API)
    @client ||= Faraday.new(url: adddress)
  end
end

and in the controller:

  # GET /devices/:id
  def show
    response = Rails.cache.fetch(params[:id]) do
      circuitbox.run! do
        request = http_client.get do |req|
          req.url "/devices/#{params[:id]}"
          req.options.timeout = 5
        end

        if request.success?
          Oj.parse request.body
        else
          { device: 'error', message: request.body }
        end
      end
    end

    render json: response, status: 200
  rescue Circuitbox::OpenCircuitError
    render json: { error: 'api is not reachable' }, status: 503
  end
end

once the circuit is open, no matter how long I wait, it never closes again.
I'm using Rails 5.1.2 with Ruby 2.4.1

note: when using Redis as backend, it works. I'm seeing this using on memory backend

What server are you using? Be aware that the Memory backend is not threadsafe.

I am seeing this with File backend

Ah you need to use a Moneta backend which supports expiry or explicitly set the expires: true option in Moneta to add in a middleware to emulate key expiry (which worked for me with the :File backend, which I use in dev/test)

Yes expire relies on the moneta functionality, not well documented I, sorry about that.