oscardelben / rawler

Rawler is a tool that crawls the links of your website

http://oscardelben.com

oscardelben/rawler Issues

warning: URI.escape is obsolete
Updated 4 years ago
--ignore_fragments is not supported (but documented)
Updated 8 years ago5
Authorization header is sent even without --username and --password
Updated 8 years ago
URLs with percent are parsed incorrectly
Updated 8 years ago
Base tag is not taken into consideration
Updated 10 years ago
enable Travis
Closed 11 years ago1
http_proxy?
Closed 13 years ago3
net/http.rb:1266:in `addr_port': undefined method `+' for nil:NilClass (NoMethodError)
Closed 13 years ago4
"rawler http://geeks.aretotally.in/" - `initialize': the scheme http does not accept registry part
Closed 13 years ago2
Bug when page is redirected?
Closed 13 years ago5
Include page being checked in rawler's output
Closed 13 years ago3
Exception thrown when server doesn't answer
Closed 13 years ago2
Shouldn't say "ERROR -- : Invalid url" on "mailto:" links
Closed 13 years ago1
Returns 404 for links that contained chinese words
Closed 13 years ago4
rawler fails on some URL
Closed 13 years ago8
Fails on JavaScript URLs
Closed 14 years ago2
You should not have to specify http://
Closed 14 years ago3
Returns 404 for anchor links
Closed 14 years ago2
The app can not rescue Errno::ETIMEDOUT exception
Closed 14 years ago2