chaodonghu / shopify-scraper

Shopify Scraper

A shopify products monitor that notifies users of products that have been added to sites. Enables the ability to receive notifications for certain keywords.

Connects directly with discord for notifications.

Users are notified of the product that has been added to the shopify site and can add to the user's cart if desired.

A sample notification on discord:

Usage

The repository has been dockerized to allow for easy development and deployment.

1. Install Docker

2. Copy the .env.example file and make a .env file to store environment variables/secrets/urls

Add your DB credentials and WEBHOOK_URLS to the .env file (See Configuration Options for more information)

3. Run the app with

docker-compose up

On successful run you should see a message in your discord: Monitor successfully started, time to get that bread fam

Configuration Options

Environment variables:

These should be defined in .env

1. MONGODB_URI which consists of DB_HOST, DB_NAME, DB_PASS, DB_PORT and DB_USER

   • URI that connects to your mongodb database by default utilize mongodb://db:27017/xxx with xxx being any database name you would like to specify (eg. mongodb://db:27017/shopify_database)

2. WEBHOOK_URLS

   • Discord webhook urls which will be converted into an array, seperate these urls by commas in your .env file eg. (`"https://discord.com/api/webhooks/1/,"https://discord.com/api/webhooks/2/")

Configuration:

These should be defined in /src/config.js

1. DISCORD_MESSAGE_SETTINGS
2. REQUEST_TIMING
3. SITES

Future Improvements

• Implement scraping on sites that do not utilize Shopify platform

Acknowledgements

• This is an extension of the Shopify Monitor created by https://github.com/Dam998/shopify-monitor/.

Donations

If this has helped you!