scripted web crawling on Linux
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
Usage: ./crawl.sh url.txt will list all links embedded in the pages at URLs in url.txt