deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Image search sometimes stops working

doris7771 opened this issue · comments

Describe the bug
The program is encountering a httpx.RemoteProtocolError and disconnects from the server without sending a response. The program breaks when downloading a specific image and it stops downloading the rest. I encountered this while performing the image search via commands in colab. I used this command: !ddgs images -k "Audi_A3" -m 180 -s off -d

Debug log
Downloading [##########----------------------------------------] 37/180 20% 00:00:44
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
yield
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request
raise exc
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request
response = await connection.handle_async_request(request)
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection.py", line 96, in handle_async_request
return await self._connection.handle_async_request(request)
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 121, in handle_async_request
raise exc
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 99, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 164, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 214, in _receive_event
raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/ddgs", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1719, in invoke
rv.append(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 435, in images
download_results(keywords, data, images=True, proxy=proxy, threads=threads)
File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 164, in download_results
asyncio.run(_download_results(keywords, results, images, proxy))
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 157, in _download_results
await future
File "/usr/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one
return f.result() # May raise f.exception().
File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 105, in download_file
async with client.stream("GET", url) as resp:
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1573, in stream
response = await self.send(
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1617, in send
response = await self._send_handling_auth(
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1645, in _send_handling_auth
response = await self._send_handling_redirects(
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1682, in _send_handling_redirects
response = await self._send_single_request(request)
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1719, in _send_single_request
response = await transport.handle_async_request(request)
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 352, in handle_async_request
with map_httpcore_exceptions():
File "/usr/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

Screenshots
If applicable, add screenshots to help explain your problem.

Specify this information

  • Mac
  • google colab
  • duckduckgo_search v3.8.3
commented

update to v3.8.4
pip install -U duckduckgo_search