Warning: "Curlm already closed! quitting from process_data" in `curl_cffi`

[Jason Oh]

Hello,

Everything is working great for me after the update, but I'm encountering a warning related to curl_cffi in a python application that utilizes asynchronous operations to perform searches using the DuckDuckGoSearchResults class.

Environment:

• Python: 3.10.6
• curl_cffi==0.6.2
• duckduckgo-search==5.2.1
• orjson==3.10.0

Warning Message:

curl_cffi\aio.py:205: UserWarning: Curlm already closed! quitting from process_data

Warning Code Snippet:

  def _run(self, query):
      return DuckDuckGoSearchResults().run(query)

Additional Observations:

I noticed that this warning started appearing after updating duckduckgo-search from version 5.1.0 to versions 5.2~5.2.1. The class AsyncDDGS implementation had some changes during these updates. No such warning was observed while using version 5.1.0.

Thank you!

[deedy5]

How can I reproduce this problem?

[phamxtien]

I use curl_cffi==0.6.0b7 and it runs OK

[deedy5]

@desaiankitb I've made the changes, try the latest version v5.2.2

[Jason Oh]

@deedy5

I actually should have added that I was using langchain sorry to miss that. The package works fine overall, just saw some warning so wanted to flag it.

You can reproduce this issue by using things below. I actually get different warning message now using 5.2.2.

Environment

• Python Version: 3.10.6
• Operating System: Windows 11 Pro
• langchain==0.1.13
• duckduckgo-search==5.2.2

Steps to Reproduce

1. Install the langchain and duckduckgo-search packages using pip.

   pip install langchain duckduckgo-search
   

2. Run the following Python script:

   from langchain_community.tools.ddg_search.tool import DuckDuckGoSearchResults
   
   ddg_search = DuckDuckGoSearchResults()
   
   query = "how much calories does a banana have"
   results = ddg_search.run(query)
   
   print(results)

Warning Message

When running the script, the following warning is issued:

curl_cffi\aio.py:39: RuntimeWarning: 
    Proactor event loop does not implement add_reader family of methods required.
    Registering an additional selector thread for add_reader support.
    To avoid this warning use:
        asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())

[deedy5]

@desaiankitb
Thanks for the clarification.
I have committed the patch.
Try installing it from git and check it out.
pip install git+https://github.com/deedy5/duckduckgo_search.git

[Jason Oh]

The warning message is no longer appearing. Thank you!

[hb-webdev]

I am also getting this warning. I just ran pip install -U duckduckgo_search to update it. Did I need to do something else? (I know it successfully updated because it wasn't working at all before the update)

my code

            with DDGS() as ddgs:
                try:
                    results = [r for r in ddgs.text("Download ODBC Driver for SQL Server", max_results=1)]
                    print(results)
                    QDesktopServices.openUrl(QUrl(results[0]['href']))
                    print('Opened URL')
                except Exception as e:
                    print('Failed with error:', e)
                    QDesktopServices.openUrl(QUrl(default_url))