askk / gau

Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.

getallurls (gau)

getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl for any given domain. Inspired by Tomnomnom's waybackurls.

Resources

• Usage
• Installation
• ohmyzsh note

Usage:

Examples:

$ printf example.com | gau
$ cat domains.txt | gau
$ gau example.com
$ gau -o example-urls.txt example.com
$ gau -b png,jpg,gif example.com

To display the help for the tool use the -h flag:

$ gau -h

Flag	Description	Example
-providers	providers to fetch urls from (by default, all are used)	gau -providers wayback,otx,commoncrawl example.com
-b	extensions to skip	gau -b jpg,png,gif example.com
-retries	amount of retries for http client	gau -retries 7 example.com
-subs	include subdomains of target domain	gau -subs example.com
-p	http proxy to use	gau -p http://localhost:8080 example.com
-random-agent	use a random user-agent	gau -random-agent example.com
-v	enable verbose mode (show errors)	gau -v
-o	filename to write results to	gau -o urls.txt example.com
-json	write output as json	gau -json example.com
-version	show gau version	gau -version

Installation:

From source:

$ GO111MODULE=on go get -u -v github.com/lc/gau

From binary:

You can download the pre-built binaries from the releases page and then move them into your $PATH.

$ tar xvf gau_1.1.0_linux_amd64.tar.gz
$ mv gau /usr/bin/gau

ohmyzsh note:

ohmyzsh's git plugin has an alias which maps gau to the git add --update command. This is problematic, causing a binary conflict between this tool "gau" and the zsh plugin alias "gau" (git add --update). There is currently a few workarounds which can be found in this Github issue.

Useful?