Rough draft of a script for crawling the fritter network.
npx fritter-crawler
fritters & not-fritters) in the working directory and begin writing files there.
Writes all profile.json files to fritters/${key}.json.
If no profile.json is found in a dat link, writes the 404 error to not-fritters/${key}.json.
The goal is to eventually analyze and visualize the fritter network using the dataset produced by this crawler.
- retries
- better logging
- make starting key and concurrency modifiable via CLI args
- add option to pipe to STDOUT
ISC