101arrowz / fireroad-warehouse

fireroad-warehouse

A work-in-progress FireRoad-compatible API using MIT's Data Warehouse, intended to replace at least part of FireRoad's scraper when finished.

Obtaining Data Warehouse access, and setup

In order to use this, you will need credentials to access MIT's Data Warehouse. Here are steps for setting that up (on Linux x86-64; details may vary on other platforms):

1. Request access to subject enrollment data using this form; specifically, access to the tables CIS_COURSE_CATALOG, SUBJECT_OFFERED, and CIS_HASS_ATTRIBUTE is needed.
2. Once access has been granted, set a password for your Data Warehouse account.
3. Download both the "Basic" and "SQL*Plus" packages (as zip files) for Oracle Instant Client from Oracle's website.
   • As of the time of writing, MIT's Data Warehouse uses Oracle Database 10.2; consult the python-oracledb documentation for information about the versions of Instant Client that work. Typically, an Instant Client a few versions later than what is officially supported should work fine.
4. Extract both zip files; they should both create files in a new directory called instantclient_21_9.
5. Download the MIT Oracle configuration files from the MIT knowledge base entry. Extract the zip file, move ldap.ora and sqlnet.ora to instantclient_21_9/network/admin/, and add a crypto seed to sqlnet.ora as instructed in the knowledge base article.
6. Now, if $ORACLE_HOME is set to the full path of your instantclient_21_9 directory and $user and $pass are set to your Data Warehouse credentials, you should be able to run SQL*Plus with one of the following command lines (the second gives you Readline-style editing, including keeping a history file under /tmp/):

   $ LD_LIBRARY_PATH="$ORACLE_HOME" "$ORACLE_HOME/sqlplus" "$user/$pass@warehouse"
   $ LD_LIBRARY_PATH="$ORACLE_HOME" rlwrap -H /tmp/sqlplus_hist "$ORACLE_HOME/sqlplus" "$user/$pass@warehouse"

7. To be able to run the script, create a file .env at the root of this repository with the following contents:

   ORACLE_HOME="/path/to/instantclient_21_9"
   USERNAME=yourusernamehere
   PASSWORD=yourpasswordhere

Usage

$ npm install
$ npm start 6.1220

This is still a prototype script for now.