Get a site links with bash
- Get all the links from a website
- For each link, get all links of the page
$ crawler <site>
Usage
$ crawler [options] <site>
Options
--depth, -d Depth of the links searched 5
--help, -h Prints this helps false
--list, -l Print a list formatted output false
--verbose, -v Verbose output false
Examples
$ crawler -d 5 www.github.com
Tests were made using bats framework, to run tests type:
$ bats --tap crawler.bats