Open Source the data scraper
Lucretiel opened this issue · comments
Unless I'm mistaken, there doesn't seem to be any code here that actually performs the data scraping and creates the commits; the existing code is only the site generator based on the scraped data. Would it be possible to open source the scraper itself?
Everything is open sourced. The functionality you're looking for is implemented using a shell script executed as a recurring workflow by GitHub Actions. Look at these files.
In particular: the original JSON dataset is grabbed from the NYT source using curl and parsed with jq before being written to results.json.