cangurdev / scraper

Web scraping case study

Web Scraper

Case study

Setup

clone repository using git

git clone https://github.com/cvngur/scraper

Installing packages

pip install -r requirements.txt

Stack

• Python
• BeatifulSoup
• OpenPyXL

Challenges

• I have used BeautifulSoup and OpenPyXL in projects before, so I did not face a challenge

Learning From Project

• How to use concurrent.futures library
• Differences between ThreadPoolExecutor and ProcessPoolExecutor

Additional Questions

1. We can use multiple threads to decrease the scraping time of URLs.
2. API is a structure that provides the ability for applications to communicate with each other. You send a request to the API to getting data you do not have then API returns an answer which includes the data you need.