For my "Machine Learning Applied to Economic Policy Analysis" class during the 2nd year of my Masters in Dauphine - PSL, my classmate and I web scrapped data on covid related search words between May 2020 and May 2022 to create a link between Google search trends and the incidence rate of COVID-19 first at the department level, and further aggregated at a national level in France.
The main goal of the project was to understand the link between people's Google search queries and the possibility of them testing positive for COVID-19 with a 1-week lag. The results can be used for harnessing Big Data for health policy formulation and evaluation.
This was the first time we tried web scrapping. When I look back to this project now, it is certainly not the most sophisticated approach, but back then this was our best :)
We coded in python using Jupyter Notebooks.