KShivendu / awesome-scrapers

Awesome list for working scrapers (with code) for popular websites

Awesome Scrapers

Repos

Tools

• https://github.com/alirezamika/autoscraper?tab=readme-ov-file Automatied scraper that recognizes patterns (no ML)
• https://github.com/PaulMcInnis/JobFunnel Job scraper (Haven't tested)

Social media:

• https://github.com/soimort/you-get Download media (video/audio/image) from sites likes Youtube, Twitter, Tumblr, etc
• https://github.com/twintproject/twint Scraper for Twitter (Archived)
• https://github.com/cwwmbm/linkedinscraper
• https://github.com/josephlimtech/linkedin-profile-scraper-api
• https://github.com/markowanga/stweet Twitter unofficial API scraper
• https://github.com/chris-greening/instascrape Instagram scraper
• https://github.com/omkarcloud/google-maps-scraper

Examples:

• https://github.com/scrapfly/scrapfly-scrapers

Services:

• https://serper.dev
• https://dataforseo.com/apis

Misc.

• https://github.com/philshem/gmaps_popular_times_scraper Google maps popular times scraper
• https://github.com/Bishalsarang/Leetcode-Questions-Scraper Leetcode scraper

Blogs:

• https://crawlbase.com/blog/how-to-scrape-tiktok/