subsea www.google.com will check the first 50 links found on google.com. Isn't recursive yet.
subsea is a simple fast crawler that will hit every page in a domain or domains.
subsea www.google.com will check the first 50 links found on google.com. Isn't recursive yet.
subsea is a simple fast crawler that will hit every page in a domain or domains.