Skip request error after retry

[Tails]

I have a site that times out, sometimes. I configured the @config to retry the errors, and to skip them if they fail, since I would like the spider to keep going. However, it seems the skip_request_errors option drops errors immediately. Is there a way to make retry_request_errors and skip_request_errors work together so errors are only dropped when the retries have been exhausted?

[Victor Afanasev]

@Tails

I think we can add skip_on_failure: true option for retry_request_errors:

@config = {
  retry_request_errors: [{ error: Net::ReadTimeout, skip_on_failure: true }]
}

[Tails]

That would be great! Or is this already possible?

[Victor Afanasev]

I'll add it today and release a new version with some other changes which already present in the master branch

[Victor Afanasev]

@Tails

Added this feature to the new 1.3.0 version. https://github.com/vifreefly/kimuraframework#retry_request_errors

Please close this issue if skip_on_failure works as expected.