henryvps / python-common-crawl-amazon-example

Exploring Common-Crawl using Python and DynamoDB

python-common-crawl-amazon-example

Amazon PRODUCT FINDER using Python and Commoncrawl dataset Scans common crawl infomation for products and saves them to Amazon DynomoDB database. By David Cedar https://www.cedar.net.au/using-python-and-common-crawl-to-find-products-from-amazon-com/

Install

boto3: https://github.com/boto/boto3. Make sure to add AWS Credentials on our machine, see docs. & other using pip.

Test

Make sure code is working with local example - Run python productfinder_helper.py

Run

python main --domain amazon.com