jaimeiniesta / metainspector

Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

https://github.com/metainspector/metainspector

jaimeiniesta/metainspector Issues

Unicode Normalization not appropriate for ASCII-8BIT (Encoding::CompatibilityError)
Updated 10 months ago4
Having trouble to create new instance of MetaInspector
Updated a year ago11
Move `faraday_http_cache` inside `faraday_options`
Updated 2 years ago1
Upgrade to Faraday 2
Closed 2 years ago1
Fix error in Ruby 3.1.1
Closed 2 years ago4
Can't update addressable beyond 2.8.1
Closed 2 years ago
images.best sometimes fails with undefined method `start_with?' for nil:NilClass (NoMethodError)
Updated 2 years ago2
MetaInspector is unable to scrape a particular blog post
Updated 3 years ago1
Metainspector stopped working for twitter.com
Closed 4 years ago4
Improve detection of non standard images on modern web pages
Updated 3 years ago3
Different results between local and server
Closed 3 years ago7
Update .to_hash to remove deprecation notice
Updated 3 years ago
Unexpected `ArgumentError: invalid byte sequence in UTF-8`
Closed 3 years ago4
MetaInspector::ParserError: The url provided contains text/html, text/html content instead of text/html content
Closed 4 years ago2
undefined method `headers' for nil:NilClass
Closed 4 years ago1
Error `getaddrinfo: Name or service not known` when scrape page with same host
Closed 4 years ago1
Absolutify base href
Updated 4 years ago4
MetaInspector hanging for url when timeout >= 20
Updated 4 years ago5
base_url returns ''
Closed 4 years ago2
Allow returning image type and file size from FastImage
Updated 4 years ago1
Use FastImage to detect images that can't be downloaded
Updated 4 years ago1
Many images on page slows down request when using page.images.with_size
Updated 4 years ago4
Cannot parse/get correct data from Techcrunch blog posts
Closed 4 years ago2
Unable to pick favicon
Closed 4 years ago5
Can't scrape any url from The Washington Post (even with demo)
Closed 5 years ago5
Unable to render images that have special characters in src
Closed 5 years ago6
Failed to fetch correct best title from URL with double quotes characters title
Closed 5 years ago1
Update Faraday in Gemspec
Closed 5 years ago7
twitter.com links not scraped
Closed 5 years ago5
faraday_middleware (0.13.1) is deprecating Faraday::Error::ClientError
Closed 5 years ago1
Metainspector timeout for some sites
Closed 5 years ago1
linkedin.com profile pages are not getting scraped.
Closed 5 years ago1
og:title tag not found, title returning nothing
Closed 6 years ago6
A url containing utf-8 characters is manipulated, which leads to a page redirect
Closed 6 years ago4
Image url encode problem
Updated 6 years ago13
NoMethodError: undefined method `+' for nil:NilClass
Closed 6 years ago5
Using of `Nokogiri::XML` instead of `Nokogiri::HTML`
Closed 6 years ago2
Add best_author method
Closed 6 years ago4
Metainspector not able to fetch basic title or meta elements.
Closed 6 years ago1
Some page headers missing
Closed 6 years ago8
Customize scrape for selected page
Closed 6 years ago2
ArgumentError: invalid byte sequence in UTF-8
Closed 6 years ago7
MetaInspector behaving differently on local when comparing to demo site
Closed 7 years ago3
data sources or external links
Closed 7 years ago1
RequestError: end of file reached
Closed 7 years ago2
Get all iframes in a page?
Closed 7 years ago2
Issue HTTP HEAD request first to check content type before downloading?
Updated 7 years ago1
Cannot parsing the site
Closed 7 years ago1
What's this doing on my logs?
Closed 7 years ago1
Using parsely-* fields within best_* methods
Updated 7 years ago1