Rate limit exception

[Nathan Vandeven]

Hello, I'm encountering a rate limit exception while executing my code. Could you please inform me if there's a specific rate limit I should adhere to? The search_query I use is 5 words in total.

Thanks a lot!

import pandas as pd
from duckduckgo_search import DDGS
from ratelimit import limits, sleep_and_retry

@sleep_and_retry
@limits(calls=10, period=1)
def scrape_top_urls_ddg(search_query, skip_domains, max_results=5):
    top_urls = []
    ddgs = DDGS()
    results = ddgs.text(keywords=search_query, max_results=max_results + len(skip_domains))
    
    for result in results:
        url = result.get('href')
        if url and not any(skip_domain in url for skip_domain in skip_domains):
            top_urls.append(url)
            if len(top_urls) == max_results:
                break

    return top_urls

[deedy5]

If you encounter a RatelimitException, add a delay between function calls (determine by experience) or use a proxy.