This is a companion repo to the npm-by-numbers repo hosted here:
http://github.com/iros/npm-by-numbers and hosted live here: http://npmbynumbers.bocoup.com.
This repository assumes that you have a local couchdb instance of skimdb, npm's metadata repository about its packages (everything but the binaries.)
You can set it up via the instructions here which will get you a vagrant instance up in no time: https://github.com/iros/local-npmjs-skimdb
Aside from creating a clone of npm's skimdb, this repo also contains scripts to generate
data files that can be used to analyze the data. They are all in the scripts directory.
All data goes into the data directory.
All scripts live in the scripts directory. All scripts write to data/aggregate.
get_all_docs.js- creates a file for each package underdata/packages. It is a prerequisite forget_all_packages.sh.get_all_packages.sh- createsdata/all_packages_names.jsonwhich is just an array of all available packages. It is a prerequisite for all other scripts.
build_metadata.js- Builds a general metadata file for all packages calleddata/packages_meta.csvbuild_dependencies.js- Builds two dependency tree files, one for regular dependencies, and one for dev dependencies.build_keyword_graph.js- Builds a keyword source & target pairs for all keyword co-occurance + weight (count)build_versions.js- Builds a list of all versions per package & associated timestamps.