A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses websites for the purpose of Web indexing (web spidering).
T-Crawl is a web crawler (spider) written in Python. This is how it works:
- It downloads the HTML from a webpage.
- It parses the HTML to extract links.
- It prints the links collected.
- apt update && apt upgrade
- pkg install python
- pkg install git
- pkg install toilet
- pip install requests
- pip install bs4
- git clone https://github.com/ParzivalHack/T-Crawl
- cd T-Crawl
- chmod +x T-Crawl.py
- python T-Crawl.py
This tool is under the GPL v.3 License.
© 2022 Tommaso Bona