This project is a web scraper tool designed to extract product information from the Adidas Japan website. It utilizes Selenium WebDriver to automate web browsing and gather data such as product names, categories, prices, sizes, and more. The extracted data can be saved in Excel formats for further analysis and processing.
-
Scrapes product IDs from the men's category on the Adidas Japan website . For example
https://shop.adidas.jp/item/?order=11&gender=mens&limit=100&page=1
-
Supports fetching product details and meta-information
-
Saves scraped data to Excel files
-
Clone the repository:
git clone https://github.com/uzzalhcse/go-web-crawler-adidas.git
-
Install dependencies:
go mod tidy
-
Download ChromeDriver and place it in the
assets
directory.- Ensure that ChromeDriver version matches your installed Chrome browser version. You can download the ChromeDriver for your specific version from here
-
Run the main.go file:
go run main.go
-
Wait for the scraping process to complete.
-
Find the extracted data in the
dist/sheets
directories.
Special thanks to the authors of the following libraries used in this project: