WebCrawl is a free and open source tool to crawl through the website and generate a huge dataset that can be used to train your ai
it's inspired by: CommonCrawl
you require:
requests
BeautifulSoup
tqdmthen you can just run the app with:
python main.py