rendyramon / selenium_and_pagination

Amazon web scraper that takes a search query and saves all of the resulting data to a json file.

Home Page:https://medium.com/@brendanfrrs/scraping-amazon-results-with-selenium-and-python-547fc6be8bfa

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Amazon Web Scraper

Screenshot

The article describing the code for this repo can be found here.

Requirements:

To run this program you will need the following installed on your computer or virtual machine:

  1. selenium
  2. webdriver-manager
  3. selectorlib

You will also need the following file in the project directory:

  1. search_results_urls.txt

You will also need the following folder in the project directory:

  1. results

This folder will house all of the search resulting json files. The file names will be in the following format: query_date_results.jsonl

About

Amazon web scraper that takes a search query and saves all of the resulting data to a json file.

https://medium.com/@brendanfrrs/scraping-amazon-results-with-selenium-and-python-547fc6be8bfa

License:MIT License


Languages

Language:Python 100.0%