pockees / awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Awesome Web Scraping

The list of tools, programming libraries and web services used for web scraping and data processing.

Feel free to give feedback or ask web scraping questions in Telegram groups: @grablab (English) and @grablab_ru (Russian).

Programming Libraries

• Python
• PHP
• Ruby
• JavaScript
• Go

Other Things

• Learning Web Scraping - list of articles and books teaching web scraping
• Console tools
• dhamaniasad / HeadlessBrowsers - a list of (almost) all headless web browsers in existence
• DNS over HTTPS providers - list of DNS over HTTPs providers

Captcha Solving Services

These two links point to same captcha service, it is just a different language versions

• https://2captcha.com - English UI language
• https://rucaptcha.com - Russian UI language

Proxy Server Marketplaces

Largest marketplaces in the world which contain offers from hundreds sellers and services:

• https://www.blackhatworld.com/forums/proxies-for-sale.112/
• https://forum.antichat.com/forums/147/

How to Contribute to this List

See Contributing guide.

Credits

The list is based initially on some data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript