Santapaji / map-website

A tool to recursively scrape websites.

Map Website

• Requirements
• Setup
  • Windows
  • Linux
• Running the Script

Requirements

• Dependencies (included in requirements.txt - see below)

  • bs4
  • requests
  • lxml

• Python Versions Tested

  • 3.8.2

Setup

Windows

python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt

Linux

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Running the Script

python map_website.py -d https://webscrapethissite.org -o test.txt