Ensemble Network Including Transformer Models for NLP Patient Text and ED Visit Prediction
WINTER 2023
This project was conducted as Research with Stanford Medical Center and used as an end-of-term project for CS 224N: Natural Language Processing. Code is not included due to privacy and HIPAA compliance for patient data.
Stanford Medicine’s goal is to reduce the number of preventable visits to the emergency department (ED) due to many negative effects of ED congestion. This project’s goal is to leverage robust NLP BERT models predict the probability that a patient will be admitted to the ED in the next year given their past medical messages to and from their care providers in the last two years. Our experiments yielded F1 scores of up to 0.954 and suggest that patient portal messages can play a vital role in predicting patient healthcare outcomes. With these novel approaches, we hope to contribute to the growing body of research using patient portal messages in NLP and show a definitive use case in ED visit prediction.
Topics covered and tools used:
- BERT and ROBERTA
- Fine-Tuning
- Deep Semantic Embeddings
- CNNs
- Max/Min/Avg Pooling
- Ensemble Networks
- Natural Lanaguage Processing
- Medical Text
- Prediction
- Error Analysis