Elasticsearch ruby errors in rails boot when more than 1 http client available

[William Flanagan]

I have both HTTPClient and Typhoeus in my app. I cannot use Typhoeus for ElasticSearch (and Searchkick) because I use the caching capabilities of Typhoeus in other parts of my application. When this is enabled in Typhoeus, ElasticSearch does NOT work.

When I attempt to boot my app, the transport adapter attempts to use the Typhoeus even though Faraday is NOT configured to use Typhoeus as the default adapter. Faraday is configured to use HTTPClient as the adapter.

This configuration is currently Rails Unbootable, the app fails on load.

The error message is:

...3.2.0/gems/faraday-2.7.4/lib/faraday/middleware_registry.rb:57:in `lookup_middleware': :typhoeus is not registered on Faraday::Adapter (Faraday::Error)
        from .../ruby/3.2.0/lib/ruby/gems/3.2.0/gems/faraday-2.7.4/lib/faraday/rack_builder.rb:113:in `adapter'
        from ...ruby/3.2.0/lib/ruby/3.2.0/forwardable.rb:240:in `adapter'
        from .../ruby/3.2.0/lib/ruby/gems/3.2.0/gems/elasticsearch-transport-7.4.0/lib/elasticsearch/transport/client.rb:139:in `block in initialize'

To Reproduce
Create a new Rails app (7.0.4-2).

I've linked to a gist with my Gemfile and the search init below: https://gist.github.com/wflanagan/fd861336fb1157e95976ecba40cac84d

Add the following to the Gemfile and then bundle

gem "typhoeus", "~> 1.4"
gem "elasticsearch"
gem "elasticsearch-model", github: "elastic/elasticsearch-rails", branch: "main"
gem "elasticsearch-rails", github: "elastic/elasticsearch-rails", branch: "main"
gem "faraday"
gem "faraday-httpclient", ">= 2.0.1"

Add an initializer for configuring Faraday and ElasticSearch client & Searchkick
app/initializers/search_init.rb

require 'faraday'
require 'faraday/httpclient'
Faraday.default_adapter = :httpclient
client = Elasticsearch::Client.new({hosts: ["http://localhost:9200"]})

Expected behavior
I expect that the Rails environment will boot. But, it does not currently boot.

Your Environment (please complete the following information):

• MacOS Intel Ventura
• Ruby 3.2.0
• main branch of Elasticsearch-ruby on Github
• 7.xx but the app never gets to connections to ES

Additional context
I did run this configuration successfully in a Rails 6.x and Ruby 3.0 context. Not sure what has changed about Rails 7 and Ruby 3.2 that causes this to occur.

[William Flanagan]

FYI, you can pass the adapter in the config to get around this issue. But, it's a problem for the default behavior of the gem.

def config
      {
        hosts: [elasticsearch_url],
        retry_on_failure: true,
        adapter: :httpclient,
        transport_options: {request: {timeout: 250}}
      }
end

then

Elasticsearch::Client.new(config)

[Fernando Briano]

Hi @wflanagan,
Thanks for reporting this. I think the issue here might be the use of the Rails integration with the latest client. Unfortunately we haven't updated the Rails integrations to work with Elasticsearch 8.x and Faraday 2. So I think there might be issues with dependencies getting mixed.

The elasticsearch gem uses elastic-transport, which is designed to "auto-detect" HTTP libraries installed in your project, in this order:
https://github.com/elastic/elastic-transport-ruby/blob/main/lib/elastic/transport/client.rb#L293-L306
However, if you're using Faraday 2, and adding the dependency faraday-httpclient, the client should default to that. The code wouldn't check for the presence of ::Typhoeus, and the error :typhoeus is not registered on Faraday::Adapter (Faraday::Error) shouldn't show up. But as I said, I think there might be some mix up in the dependencies to trigger this error. I couldn't reproduce it locally.

Anyway, in the case of more than one adapter being present in the code, the solution is to explicitly specify the adapter on initialization as you found out.