If Crawler is "Stuck" allow option to ignore current URL in the list and move forward to the next URL
lucksp opened this issue · comments
If the crawler is Stuck on a URL, it would be really slick to allow moving forward on to the next URL in the list. This could be related to current #4 issue
- website URL 1
- website URL 2
- website URL 3
[ X ] website URL 4 (STUCK) - website URL 5 (next on list)
Agreed, shouldn't be too difficult. I gots dis.