Linkcheck example and mojolicious.org
sonicepk opened this issue · comments
- Minion version: v8.12
- Perl version: 5.22.1
- Operating system: Ubuntu 16.04
Steps to reproduce the behavior
The Linkcheck example does not work for the default example website mojolicious.org. The reason why is that there is a link to https://shop.spreadshirt.com/kraih/ on the mojolicious website which results in an inactivity timeout.
wget -d https://shop.spreadshirt.com/kraih results in a:
---request begin---
GET /kraih HTTP/1.1
User-Agent: Wget/1.17.1 (linux-gnu)
Accept: /
Accept-Encoding: identity
Host: shop.spreadshirt.com
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
If I specify the user agent with wget, wget will follow the redirects and works.
wget -U "Mozilla/5.0" https://shop.spreadshirt.com/kraih
Expected behavior
Example should work.
While nothing to do with Minion, it did take me a while to realise that the issue was with a link on the mojolicious website rather than with Minion itself.
I added $ua->transactor->name('Mozilla/5.0') and $ua->request_timeout(5) to CheckLinks.pm but I still get an inactivity timeout when I run the linkcheck for the website mojolicious.org
Actual behavior
Request timeout at /home/ekenny/mojo/minion/examples/linkcheck/script/../lib/LinkCheck/Task/CheckLinks.pm line 33.
Agreed, that's not good. I've changed the default for now. dd1316f