Can no longer scrape google

[Sevak Mardirosian]

Can anyone confirm this issue please ?

        gateway = ApiGateway("https://www.google.com",access_key_id='123', access_key_secret='12)
        gateway.start()
        session = requests.Session()
        session.mount("https://www.google.com", gateway)

        response = session.get("http://www.google.com/search?q=barry+bonds&tbm=nws&hl=en&num=10")        
        soup = BeautifulSoup(response.text, "html.parser")
        result_stats = soup.select_one("div#result-stats")
        result_stats = result_stats.get_text().strip().lower()
        print(result_stats)

basically returns None.

Did bunch of other tests, the results are the same. You think google blocked aws ips?

[George O]

Hey @sevmardi, thanks for your issue. I can see that the response.text returns the correct response with the google results. It seems like your HTML parsing is incorrect instead 😅