Predicting acute intracranial vascular events in stroke alerts using large-language models and structured data

Predicting acute intracranial vascular events in stroke alerts using large-language models and structured data. Download PDF Abstract Acute stroke alerts are frequently activated for non-cerebrovascular conditions, creating false positives that strain resources and increase diagnostic uncertainty. We developed machine learning models integrating large-language models, structured electronic health record data, and clinical time-series features to predict acute intracranial vascular event (AIVE) at stroke alert activation.

Using retrospective stroke alerts from the Mount Sinai Health System (2011–2021), we extracted demographics, comorbidities, medications, vital signs, laboratory results, and clinical notes documented before alert activation. AIVE was defined using radiographic gold standards as a composite of acute cerebrovascular and selected acute intracranial diagnoses. Notes were modeled using Clinical-Longformer, MxBAI embeddings with a feed-forward neural network, and BioWordVec embeddings with logistic regression; structured-data models used complementary hyperparameter-tuned workflows.

Model outputs were combined in a late-fusion stacked ensemble and evaluated on internal and within-system held-out hospital test sets. Among 16,512 stroke alerts, 9,013 (55.6%) were due to AIVE. The multimodal model achieved an AUROC of 0.73 (95% CI, 0.71–0.75), positive predictive value of 0.68, sensitivity of 0.76, F1-score of 0.72, and specificity of 0.56 on internal testing, with similar held-out hospital performance.

Medical Disclaimer: This article is for informational purposes only and does not constitute medical advice. Always consult with qualified healthcare professionals for medical decisions and treatment options.