proxycrawler is, as the name suggests, a proxy crawler and validator. proxycrawler scrapes services (as of v0.2.0, it supports only 2 services) that offer free proxies, either displayed in a table so you can then go through the hassle of trying to copy each one of them and validate them, or by directly sending out requests to the API that these services use (like Geonode.com).
As of v0.2.0 proxycrawler usilizes two services:
-
- Python3.10+
- poetry
To install manually you will need poetry in order to build from source after you clone the repo:
git clone https://github.com/ramsy0dev/proxycrawler
cd proxycrawler
poetry build
pip install dist/proxycrawler-*.tar.gzTo see how to use proxycrawler and get a basic guide on how to get started. Please checkout the wiki
All contributations are acceptable.
GPL-3.0