crates / awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Awesome Web Scraping

The list of tools, programming libraries and web services used in web scraping.

Programming Libraries

• Python
• PHP
• Ruby
• JavaScript
• Go

Web Services

• Proxy Services
• Captcha Solving Services
• Web Scraping Services

Other Things

• Console tools

Other Lists

• HeadlessBrowsers - a list of (almost) all headless web browsers in existence
• awesome-python-dev - a list of tools for debugging, profiling and analyzing python programs.

About awesome-web-scraping

Make this list better! Your contributions are always welcome! See contributing how-to. To add new language to Programming Libraries section use new_language_template as starting point.

Feel free to share feedback in telegram chats about web scraping: @grablab (English) and @grablab_ru (Russian)

The list is based initially on some data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript