alwat91 / web-scraper

A code example of a web scraper I wrote using Ruby and the Nokogiri gem.

web-scraper

A code example of a web scraper I wrote using Ruby and the Nokogiri gem.

parse_pages.rb

Iterates through each webpage and puts the desired data into a CSV file.

helper_methods.rb

Methods used by the parse_pages.rb file.

build_headers.rb

This script checks to make sure that all columns are accounted for by iterating through each page in the site.

download_pages.rb

This script downloads all pages from each URL in a CSV file.

find_address_2.rb

Iterates through all pages making sure none have an address line 2.