Context-aware AI assistance reduces diagnostic error in chest X-ray interpretation

Context-aware AI assistance reduces diagnostic error in chest X-ray interpretation. Download PDF Abstract Static AI assistance assumes uniform benefit, but evidence from AI-supported radiology indicates benefit varies by context. Using data from the public Collab-CXR release (104,800 pathology-level observations, or 20,960 reads, from 326 radiologists across 324 cases), we reframed AI display as a human-centered assistance problem, learning and evaluating offline a context-aware assistance policy.

The confirmatory average treatment effect on absolute diagnostic error approached null (-0.0014; SE 0.0015; p = 0.329), indicating that average effects obscure heterogeneity. In held-out evaluation, the adaptive policy achieved MAE 0.1034, versus 0.1060 for always showing AI (reduction 0.0026, 2.48%; cluster-bootstrap p = 0.0067) and 0.1088 for never showing AI (reduction 0.0054). The policy was translated into a three-variable decision tree (algorithm prediction, case difficulty, case-level AI accuracy) achieving 93.97% test concordance and a comparable held-out MAE of 0.1033.

Simulation-based sensitivity analysis showed the policy contrast remained positive under modest hidden confounding but grew sensitive at higher levels. We conclude that static assistance is suboptimal in radiology and prospective testing of adaptive, trust-aware AI support is merited. For radiologists, administrators, and AI developers, uniform AI deployment is not the best strategy; selective, context-aware use may offer modest benefit, pending prospective confirmation.

Medical Disclaimer: This article is for informational purposes only and does not constitute medical advice. Always consult with qualified healthcare professionals for medical decisions and treatment options.