Rate limit exception
nuaimat opened this issue · comments
Hello
thanks for your time developing this amazing package, i appreciate your efforts.
using duckduckgo_search v5.2.2 I am facing a problem when using ddgs in python code, the exception was rate limit., so i decided to try it from command line to avoid any possible code issue (if i am calling the search api more than allowed), see the following please:
duckduckgo_search==5.2.2
(devika) (base) ➜ devika git:(main) ✗ pip freeze | grep -i lxml
lxml==5.1.1
(devika) (base) ➜ devika git:(main) ✗ ddgs text -k "how to embed youtube videos in html"
Traceback (most recent call last):
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search_async.py", line 94, in _aget_url
raise DuckDuckGoSearchException("Ratelimit")
duckduckgo_search.exceptions.DuckDuckGoSearchException: Ratelimit
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/nuaimat/code/ai/devika/.venv/bin/ddgs", line 8, in <module>
sys.exit(cli())
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/click/core.py", line 1719, in invoke
rv.append(sub_ctx.command.invoke(sub_ctx))
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/cli.py", line 146, in text
data = DDGS(proxies=proxy).text(
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py", line 40, in text
return self._run_async_in_thread(super().text(*args, **kwargs))
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py", line 36, in _run_async_in_thread
result = future.result()
File "/Users/nuaimat/miniconda3/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/Users/nuaimat/miniconda3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search_async.py", line 135, in text
results = await self._text_api(keywords, region, safesearch, timelimit, max_results)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search_async.py", line 214, in _text_api
await asyncio.gather(*tasks)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search_async.py", line 193, in _text_api_page
resp_content = await self._aget_url("GET", "https://links.duckduckgo.com/d.js", params=payload)
File "/Users/nuaimat/code/ai/devika/.venv/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search_async.py", line 98, in _aget_url
raise DuckDuckGoSearchException(f"_aget_url() {url} {type(ex).__name__}: {ex}") from ex
duckduckgo_search.exceptions.DuckDuckGoSearchException: _aget_url() https://links.duckduckgo.com/d.js DuckDuckGoSearchException: Ratelimit```
thanks, and let me know if more details are needed.
FYI , news API seems to work for me:
(devika) (base) ➜ devika git:(main) ✗ ddgs news -k "how to embed youtube videos in html" -m 1
1. ==============================================================================
date 2024-03-31T09:31:00+00:00
title Türkiye's National Library on go to digitalization, ease accessibility
body Efforts are underway to digitize Türkiye's National Library's vast collection,
totaling 20 million pages of resources, with plans to
url https://www.dailysabah.com/turkiye/education/turkiyes-national-library-on-go-to-digitalization-ease-accessibility
image https://idsb.tmgrup.com.tr/ly/uploads/images/2024/03/31/thumbs/800x531/321669.jpg
source Daily Sabah
same for videos:
ddgs videos -k "how to embed youtube videos in html" -m 1
1. ==============================================================================
content https://www.youtube.com/watch?v=ly36kn0ug4k
description In this tutorial, you'll learn how to add a YouTube video into your project.
Sponsor me on GitHub! https://github.com/sponsors/miguelznunez
Follow my blog: https://medium.com/@miguelznunez Email :
mignunez@csumb.edu Medium : https://medium.com/@miguelznunez
Codepen : https://codepen.io/miguelznunez GitHub :
https://github.com/miguelznunez
duration 3:39
embed_html <iframe width="1280" height="720"
src="https://www.youtube.com/embed/ly36kn0ug4k?autoplay=1"
frameborder="0" allowfullscreen></iframe>
embed_url https://www.youtube.com/embed/ly36kn0ug4k?autoplay=1
image_token e52606f0b4530dd8050422221dc6024b9b23403a8604819f2e9b0b536e007fd0
images {'large':
'https://tse1.mm.bing.net/th?id=OVP.-NECjnanmMxPIP-6jc0YbgHgFo&pid
=Api', 'medium': 'https://tse1.mm.bing.net/th?id=OVP.-NECjnanmMxPI
P-6jc0YbgHgFo&pid=Api', 'motion':
'https://tse4.mm.bing.net/th?id=OM.o8r1V4BT-
FYhRQ_1661925195&pid=Api', 'small': 'https://tse1.mm.bing.net/th?i
d=OVP.-NECjnanmMxPIP-6jc0YbgHgFo&pid=Api'}
provider Bing
published 2021-09-04T21:55:25.0000000
publisher YouTube
statistics {'viewCount': 107842}
title HTML & CSS - How to Embed a YouTube Video in Your Website
uploader Web Dev Tutorials
seems like the issue is happening mainly on text search
Hello.
Check duckduckgo_search version in .venv
its 5.2.2
(devika) (base) ➜ devika git:(main) ✗ pip freeze | grep -i duckduckgo_search
duckduckgo_search==5.2.2
it was a messed up venv, somehow using an old version of duckduckgo_search despite the pip freeze showing it.
please ignore this issue , thank you so much.