Ninir / centipede-crawler

Crawls all unique links.

symfony upgrade fixer • twig gettext extractor • wisdom • centipede • permissions handler • extraload • gravatar • locurro • country list • transliterator

Centipede Crawler

Crawls all unique links.

Usage

$urls = (new Centipede\Crawler('http://dev.umpirsky.com'))->crawl();

Asynchronous

(new Centipede\Crawler('http://dev.umpirsky.com'))->crawl(function ($url, GuzzleHttp\Message\FutureResponse $response) {
    printf('(%d) %s', $response->getStatusCode(), $url);
});