Satellaa / Dotscaper

A tool to scrape Yu-Gi-Oh! Official Card Game card information and prices.

Dotscaper

A tool to scrape Yu-Gi-Oh! Official Card Game card information and prices from YAML Yugi, Yugipedia, TCG Corner, Bigweb, etc. and store them in MongoDB.

Installing

First, you should create a virtual environment to avoid any conflicts with existing packages.

$ git clone https://github.com/Satellaa/Dotscaper.git
$ cd Dotscaper
$ pip install -r requirements.txt

Setup

See .env

Usage

You can run python main.py with the following arguments:

Name	Description	Condition
--init	Scrape card information and prices, then store it in MongoDB.	Must be used first.
--update-card-info	Scrape card information, then store it in MongoDB.	Should only be used after using --init.
--update-card-prices	Scrape card prices, then store it in MongoDB.	Should only be used after using --init.

Contributing

Please use pre-commit.