brutuscat / medusa-crawler

The Official Medusa Crawler gem

https://github.com/brutuscat/medusa-crawler

brutuscat/medusa-crawler Issues

Add example of crawling a private site with login form
Updated 3 years ago
Look into queue mem issue
Closed 4 years ago
skip_links_like does not match patterns in query strings
Closed 4 years ago
Implement storage module in medusa using moneta
Closed 4 years ago9
support phantomjs
Closed 4 years ago2