ChillarAnand / site_crawler

Simple spider to crawl a site and get all links from it!

Simple spider to crawl a site and get all links!

Crawled my blog http://www.avilpage.com & stored all links to links.txt