matthewmueller / x-ray

The next web scraper. See through the <html> noise.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Set a local timeout

matthewmueller opened this issue · comments

Is your feature request related to a problem? Please describe.

Timeouts are confusing: #120

Describe the solution you'd like

Support for overriding with a local timeout:

x('https://dribbble.com', 'li.group', [{
  title: '.dribbble-img strong',
  image: '.dribbble-img [data-src]@data-src',
}])
  .paginate('.next_page@href')
  .limit(3)
  .timeout(30)
  .driver('phantomjs')
  .delay(100)
  .write('results.json')