vifreefly / kimuraframework

Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

uninitialized constant URI::HTTP

bjpcjp opened this issue · comments

FATAL -- github_spider: Spider: stopped: {:spider_name=>"github_spider", :status=>:failed, :error=>"#<NameError: uninitialized constant URI::HTTP>", :environment=>"development", :start_time=>2023-02-14 10:01:02.268793288 -0500, :stop_time=>2023-02-14 10:01:02.26942778 -0500, :running_time=>"0s", :visits=>{:requests=>0, :responses=>0}, :items=>{:sent=>0, :processed=>0}, :events=>{:requests_errors=>{}, :drop_items_errors=>{}, :custom=>{}}}
/var/lib/gems/3.0.0/gems/kimurai-1.4.0/lib/kimurai/base.rb:194:in request_to': uninitialized constant URI::HTTP (NameError) from /var/lib/gems/3.0.0/gems/kimurai-1.4.0/lib/kimurai/base.rb:128:in block in crawl!'
from /var/lib/gems/3.0.0/gems/kimurai-1.4.0/lib/kimurai/base.rb:124:in each' from /var/lib/gems/3.0.0/gems/kimurai-1.4.0/lib/kimurai/base.rb:124:in crawl!'
from github_spider.rb:40:in `

'

Your post isn't clear but I'm guessing you're seeing this when you try to run the GithubSpider example from the README?

If so, try adding require "uri" after the first require and see if that helps

# github_spider.rb
require 'kimurai'
require 'uri'

...