Predicting hospital admission at emergency department triage using machine learning

Woo Suk Hong, Adrian Haimovich, R. Andrew Taylor

We provide the de-identified dataset and R scripts for the paper "Predicting hospital admission at emergency department triage using machine learning". All processing scripts in the Scripts/R/ subdirectory take as input .csv files extracted from the enterprise data warehouse using SQL queries. The analysis scripts in the main directory take as input the de-identified dataset provided in this repository. The working directory should be set to the main directory with the analysis scripts. All research using this dataset should cite the original paper. "Hong WS, Haimovich AD, Taylor RA (2018) Predicting hospital admission at emergency department triage using machine learning. PLoS ONE 13(7): e0201016." (https://doi.org/10.1371/journal.pone.0201016)

Dataset

Results/5v_cleandf.RData: A 560,486 x 972 R dataframe that contains the de-identified dataset used for the analysis portion of the study. Those wishing to explore the data outside R should export the dataframe into a csv file (~ 1.7 GB) using the write_csv function in the readr package.

Pipeline and Analysis Scripts

master.R: the main processing pipeline that combines data from multiple csv files into one R dataframe, where each row is a patient visit. The pipeline calls on the scripts 1-15 in the /Scripts/R folder. While the raw data files cannot be uploaded due to protected health information, we encourage those interested to take a look into the processing scripts.
build_boost.R: trains and ouputs the average AUC on the training and validation sets for an XGBoost model, given a set of hyperparameters. Outputs the design matrix used by all subsequent scripts.
build_final.R: trains an XGBoost model using the optimized set of hyperparameters on all samples excluding the test set then outputs a test AUC with 95% CIs
build_keras.R: trains and ouputs the average accuracy on the training and validation sets for a deep neural network (DNN) model, given the network architecture and other hyperparameters
build_final_keras.R: trains a DNN model using the optimized set of hyperparameters on all samples excluding the test set then outputs a test AUC with 95% CIs
build_final_lr.R: trains a LR model with keras using a network with no hidden layer
build_importance_ci.R: trains the final XGBoost model 100 times to get the average information gain for each variable
build_boost_topvars.R: calls on /Scripts/R/24_usetopvars.R to train an XGBoost model only using ESI level, outpatient medication counts, demographics, and hospital usage statistics. Outputs the design matrix used by all subsequent "topvars" scripts.
build_final_topvars.R: trains an XGBoost model (only using the top variables) using the optimized set of hyperparameters on all samples excluding the test set then outputs a test AUC with 95% CIs
build_boost_onlytriage.R, build_final_onlytriage.R, build_boost_onlyhx.R, build_final_onlyhx.R, build_keras_onlytriage.R, build_keras_final_onlytriage.R, build_keras_onlyhx.R, build_final_keras_onlyhx.R, build_final_lr_onlytriage.R, build_final_lr_onlyhx.R: repeats the training process for "Only Triage" variables and "Only History" variables. The XGBoost training scripts should be run first to output the design matrix used for the DNN and LR training scripts.
youdens.R: uses Youden's Index to calculate the sensitivity/specificity/PPV/NPV of each model.

Files in /Scripts/R

All processing descriptions apply to each patient visit (i.e. by row) unless specified otherwise.

1_cleanbase_cleanesi_cleandemo.R: creates the initial dataframe of visit ID, patient ID, ESI level, demographic information
2_cleantime.R: extracts the arrival time (month, day, hour, 4-hr bin)
3_dispo_timeseries.R: extracts the disposition of the previous ED visit if this is not the patient's first visit within the past year
4_pmhclean_all.R: takes the list of ICD-9 codes on the patient's EHR at time of visit and recodes it using AHRQ CCS first-level categories such that each column is a CCS category taking a value of 1 or 0 depending on presence or absence of disease
6_cleanlabs.R: cleans the lab csv file to create a dataframe that contains numeric labs ordered during a visit and another dataframe containing urinalysis and culture labs.
7_countpreviousvisits.R: creates a list of vectors, where each vector contains the IDs of previous visits made by the same patient within one year of visit
7b_countadmissions.R: using the list of vectors of previous visits, counts the number of previous visits and the number of previous admissions within one year of visit
8_processhistoricallabs.R: using the list of vectors of previous visits and the dataframe of numeric labs, extracts the last, min, max, median values of numeric labs ordered within one year of visit
10_processhistoricallabs_categorical.R: using the list of vectors of previous visits and the dataframe of categorical labs, extracts the last value, number of tests, and number of positives for urinalysis and culture labs ordered within one year of visit
9_edvitalsclean.R: cleans all sets of ED vitals for all visits, replacing outliers with missing values
9b_gettriagevitals.R: using the dataframe of cleaned vitals, extracts vitals taken during triage, outputting their mean if more than one set was taken
9c_processhistoricalvitals.R: using the list of vectors of previous visits and the dataframe of vitals, extracts the last, min, max, median values of ED vitals ordered within one year of visit
11_cleanimgekg.R: recategorizes imaging and EKG orders into 9 columns
12_countprevimg.R: using the list of vectors of previous visits and the dataframe of imaging/EKG orders, counts the number orders per each modality made within one year of visit
13_cleanmeds.R: counts the number of outpatient medications in 48 therapeutic categories
14_pshclean.R: counts the number of procedures/surgeries listed in EHR at time of visit
15_chiefcomplaint.R: adds chief complaint, taking the top 200 most frequently occuring values and binning all others into one

The following scripts process the merged dataframe created by master.R into a numeric matrix for input to xgboost and keras

20_cleanmerged.R: cleans the variables in the merged dataframe by reassigning levels for categorical variables with high number of levels and replacing missing values with 0s for count and binary variables such as PMH
23_useonlytriage.R: contains two functions useonlytriage and useonlyhistory that filter variables belong to each dataset type
24_usetopvars.R: only keeps high information-gain variables identified by the full XGBoost model, namely, ESI, demographics, healthcare usuage statistics, and outpatient medications
20b_splitdataindex.R: returns a list of splits, with a held out test set of 56,000, a validation set of 56,000 and a training set with all remaining samples. The test set is held constant while the validation and training sets are split multiple times for 5-fold cross-validation
20c_splitdataindex_testplateau.R: returns a list of 6 splits with increasing proportions of the training set (1%, 10%, 30%, 50%, 80%, 100%). The held out test set remains the same for all splits.
21_makematrix.R: converts the cleaned dataframe into a numeric matrix and a response vector
25_fitboost.R: worker function for build_boost.R
26_splitandimpute.R: centers, scales, and imputes the training, validation and test sets for build_keras.R and build_final_keras.R using the median values from the training set

yaleemmlc / admissionprediction