 |
UEL and Econostat Scraping Software Collection |
On Linux:
python -m venv env
source env/bin/activate
python -m pip install -r requirements.txt
docker-compose up -d
python src/main.pyOn Windows:
python -m venv env
.\env\Scripts\activate
python -m pip install -r requirements.txt
docker-compose up -d
python src\main.py
- doc → Documentation
- src → Source Code
- src/spiders → Scripts that crawl websites with scrapy
- src/splash → Scripts that use splash to perform complex interactions with websites
Utilize Conventional Commits whenever possible.