nthpyrodev / wikipedia-scraper

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

wikipedia-scraper

The idea of this project was to scrape all the hyperlinks in each wikipedia page, and log how they interconnected. Then build an SVG showing the web. After running it for a day, I opened the CSV to check process, and my computer decided it couldn't handle it... So here it is for anyone with a better computer. The colour coding is incomplete, but otherwise it functions.

I did manage to process small quantities of data, and the results were good.

Keep in mind some of this is generated with AI, as I wanted to test some of it's capabilities. The results were initially catastrophic...

About


Languages

Language:Python 64.3%Language:HTML 35.7%