Stroke misdiagnosis, associated with poor outcomes, is estimated to occur in 9% of all stroke patients. We hypothesized that machine learning (ML) could assist in the diagnosis of ischemic stroke in emergency departments (ED). The study was conducted and reported according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline. We used structured and unstructured electronic health records (EHR) of 56,452 patient encounters from 13 hospitals in Pennsylvania, from September 2003 to December 2020. Machine learning pipelines, including natural language processing (NLP), were created using pre-event clinical data and provider notes in the EDs. We performed prospective validation, using data from pre- and post-COVID periods, and completed a case study on a small cohort of previously misdiagnosed stroke patients. This study shows how available clinical information from EHR can be used to reduce stroke misdiagnosis in real time.