deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

HTTPStatusError: Client error '403 Forbidden' for url

hboen opened this issue · comments

commented

Describe the bug
When i used the python library and call ddgs.news and loop through the list I get most of the time the error. Sometime it's ok...I even added sleep(2) and still get the error

HTTPStatusError: Client error '403 Forbidden' for url 'https://duckduckgo.com/news.js?l=wt-wt&o=json&noamp=1&q=ING%20and%20%27Ralph%20Hamers%27&vqd=4-167272866629043070647631275929701091116&p=-2&df=m&s=0'

Debug log
Add logging.basicConfig(level=logging.DEBUG) to the beginning of your script and attach log to the issue.

Screenshots
The following code I use
with DDGS() as ddgs:
keywords = "ING and 'Ralph Hamers'"
ddgs_news_gen = ddgs.news(
keywords,
region="wt-wt",
safesearch="Off",
timelimit="m"
)
for r in ddgs_news_gen:
i += 1
if i > limit:
break
results.append(r)

Specify this information

  • Windows 10
  • python 10
  • duckduckgo_search version

This library is not reliable.. Most of the time it results in http 403 error...

commented

This is not a library problem, but a limitation of the news api.
Try to increase the waiting time between requests, or use a proxy.