Internet archive downloader
A set of Python3 scripts and utility functions for querying and downloading data from the internet archive .
Installation instructions
Navigate to and follow the installation instructions for the internet archives python library: https://archive.org/services/docs/api/internetarchive/installation.html
Usage
Querying the internet archive
Use search.py to query the internet archive to see the total number of results found for specified search parameters:
python3 search.py --collection=metropolitanmuseumofart-gallery --subject=etching
You can specify individual years with the --year flag or a range of dates with the --year_range flag, note the date range must be in the format YYYY-YYYY:
python3 search.py --collection=metropolitanmuseumofart-gallery --subject=etching --year_range=1800-1899
Use the flag --verbose to list all of the individual search results.
Downloading from the internet archive
Once you have found a set of search parameters that give you a a set of results, you can use the downloader.py script to download all the results for a given search.
You will likely want to specify a specific format or file type type for downloading. There are three ways of doing this.
Specifying a preset data type with the flag --data_type:
python3 downloader.py --collection=metropolitanmuseumofart-gallery --subject=etching --data_type=image
Specify a specific format as listed by the internet archive database:
python3 downloader.py --collection=metropolitanmuseumofart-gallery --subject=etching --specific_format=JPEG
Use a glob pattern to search for a specific file extension:
python3 downloader.py --collection=metropolitanmuseumofart-gallery --subject=etching --glob_pattern=*.jpg