hexcowboy / cryptopunks-scraper

A Python scraper that retrieves all CryptoPunk attributes form LarvaLabs website

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

CryptoPunks Scraper

This project will scrape the LarvaLabs website to retrieve all CryptoPunks species and attributes.

The completed scrape is found in punks.json.

The LarvaLabs site is protected by an IPS that will detect when too many requests are made. This means that async requests are not necessary since the IPS will just block your IP after about 70 simultaneous requests. Instead, this project just scrapes one punk at a time and can take over an hour to complete.

Requirements

  • Python 3.7+

Running

Start a Python virtual environment or use these commands:

python3 -m venv venv
source venv/bin/activate

Install the project dependencies

pip install -r requirements.txt

Run the script

python scrape.py

About

A Python scraper that retrieves all CryptoPunk attributes form LarvaLabs website


Languages

Language:Python 100.0%