Welcome to the Third Annual ADLM Data Analytics Competition!
This year's competition is cohosted by the ADLM Data Analytics Steering Committee, the ADLM Health Equity and Access Division, the ADLM Informatics Division, and Washington University Section of Pathology Informatics. A representative of the winning team will be invited to speak at the 2024 ADLM Annual Scientific Meeting with travel support and free meeting registration included.
This year's competition aims to develop novel tools for monitoring fairness in laboratory testing. Competitors will be provided with a real de-identified clinical dataset documenting peripartum urine drug screening in the labor and delivery department of a single institution. The challenge is to create a dashboard that analyzes and visualizes fairness using these data. Solutions will be judged based upon functionality and adherence to best practices (see 'scoring' section). Extra points will be awarded to teams that go beyond the dataset provided by engaging local stakeholders and applying their tool to analyze fairness in their own laboratory practice.
Laboratory testing carries benefits and unintended harms that are distributed unequally among patients. Patients belonging to vulnerable subgroups often bear a disproportionate burden of harms while enjoying fewer benefits. Inequities in Laboratory Medicine may be underappreciated due to limited real time monitoring in practice.
Peripartum Urine Drug Screening (UDS) is an example of laboratory testing with benefits and unintended harms.
A recent retrospective study performed at Washington University School of Medicine identified history of isolated marijuana use (IMU) as the most frequently cited indication for peripartum UDS1. They demonstrated that 99% of the positive UDS ordered for this indication were positive for cannabis only. The vast majority (89.8%) of mothers with a positive UDS ordered for IMU were reported to CPS. Of those mothers reported to CPS, 20.9% were White and 79.1% were Black despite only about 50% of mothers identifying as Black and positivity for a nonprescribed substance other than marijuana being 3 times more likely among White mothers.
To try to address these inequities in UDS harms, a quality improvement initiative was developed. This initiative included revising the labor and delivery policies to prohibit ordering UDS for an indication of IMU. To promote accountability to these policy changes, an electronic question was added to the peripartum UDS order requiring the clinician select an appropriate testing indication.
Your task is to create a dashboard that can evaluate the impact of this quality improvement initiative by analyzing and visualizing the fairness of peripartum UDS testing over time.
Competitors will be evaluated based on the functionality of their tool, their adherence to best practices for application development, and their ability to deploy their tool to address fairness in their local laboratory practice. Scoring will be performed by expert reviewers using a standardized scoring rubric and blinded to the competitor's identity.
There are many ways to conceptualize and quantify fairness2. Points will be awarded for each concept that is correctly implemented. Additional functionality points will be awarded for quality of visualization and user experience.
- Implementation of fairness concepts (20 points)
-
Demographic parity (4 points)
Equalized odds (4 points)
Predictive parity (4 points)
Equal outcomes (4 points)
Other (4 points) - Quality of visualizations (10 points)
-
Impactful visuals (2 points)
Clean and appealing visualizations (2 points)
Axes and plots labeled (2 points)
Appropriate use of colors (2 points)
Appropriate sized fonts (2 points)
- Statistics
-
Confidence intervals provided for point estimates (2 points)
Formal hypothesis testing (3 points) - User experience (10 points)
-
Intuitive navigation (5 points)
Interactivity (5 points)
Competitors are highly encouraged to identify local stakeholders that would utilize a fairness monitor in practice. Points will be awarded for running the analytical tool on a local dataset. Please provide a screenshot of your local analysis (without PHI) and a short letter from the end user documenting their engagement. Please summarize key findings from local activation.
- Local data analysis (10 points)
- Local testing data analyzed for fairness (10 points)
- End-user engagement (5 points)
- End-user engages with fairness dashboard running on local data (5 points)
- Local insights (5 points)
- Describe insight provided by local analysis in 500 words or less (5 points)
Best practices in software development help to optimize quality, maintainability, and reusability. Note that source code and commit histories are required to be eligible for points in this section.
- Readability (4 points)
-
Code is clearly and cleanly commented (1 point)
Code is simple and not bloated (1 point)
Code utilizes a clear and consistent naming convention (1 point)
Code is organized into hierarchy of modular functional units (1 point) - Reusability (2 points)
-
Code utilizes functionalized or object oriented programming (1 point)
Code can be configured to new dataset without modification of source code (1 point)
- Version control (2 points)
-
Version control system used to track development (1 point)
Commits are modular, logical, and appropriately scoped (1 point) - Documentation and deployment (4 points)
-
Usage notes provided (1 point)
Dependencies are defined (1 point)
Virtualized or containerized environment used (1 point)
Tool accessible through web hosting (1 point)
The tools developed in this competition could be used more broadly to help laboratories to identify unfairness in laboratory testing and monitor quality improvement projects aimed to promote equity in laboratory practice.
January 15th, 2024 | Competition Begins |
---|---|
May 15th, 2024 | Competition Ends |
June 15th, 2024 | Annoucement of Winning Team |
July 29th, 2024 (Anticipated) | Presentation from Winning Team |
- Navigate to https://github.com
- Click 'Sign up'
- Follow the prompts to create your personal account
- Navigate to https://github.com/WUSM-LGM-Informatics-Section/adlm-fairlabs-competition
- Click 'Fork'
- Select 'Create a new fork'
- Set your GitHub account as the owner (defaut)
- Click 'Create fork'
-
Clone the forked repository
git clone https://github.com/myGitHubUsername/adlm-fairlabs-competition.git
Note: Replace myGitHubUsername with your GitHub handle - Make a folder in the cloned repository with your team name
- Build your solution within your team folder
-
Navigate to https://github.com/myGitHubUsername/adlm-fairlabs-competition
Note: Replace myGitHubUsername with your GitHub handle - Click 'Contribute'
-
Select 'Open pull request'
- Click 'Create pull request' Note: we will review your pull request to ensure that it contain everything needed to score your submission
- Rubin A, Zhong L, Nacke L, Woolfolk C, Raghuraman N, Carter E, Kelly J. Urine Drug Screening for Isolated Marijuana Use in Labor and Delivery Units. Obstet Gynecol. 2022 Oct 1;140(4):607-609. doi: 10.1097/AOG.0000000000004930. Epub 2022 Sep 9. PMID: 36083598.
- Azimi V, Zaydman MA. Optimizing Equity: Working towards Fair Machine Learning Algorithms in Laboratory Medicine. J Appl Lab Med. 2023 Jan 4;8(1):113-128. doi: 10.1093/jalm/jfac085. PMID: 36610413.
Important: You must submit your solution by May 15th, 2024 to be eligible to win the competition
If you are unfamiliar with GitHub, need help getting starting, or have other questions, please email mboyle@myadlm.org for assistance.