brutuscat / medusa

- THIS IS AN OLD FORK - Checkout Medusa Crawler gem instead "medusa-crawler"

https://github.com/brutuscat/medusa-crawler

brutuscat/medusa Issues

some URL fragments are being retained & encoded
Closed 4 years ago
Robots.txt not respected if first page is redirected
Closed 4 years ago8
Upgrade up to ruby 2.7
Closed 4 years ago1