A multi task deep learning framework for automated health condition prediction from retinal fundus images using hybrid attention and explainable AI

A multi task deep learning framework for automated health condition prediction from retinal fundus images using hybrid attention and explainable AI. Download PDF Abstract Although retinal fundus imaging is a non-invasive tool to assess systemic and fundus pathology, current deep learning algorithms are mostly single-task, lack clinical interpretability and are unable to handle acquisition variation between different domains. These constraints limit their clinical utility: when a patient comes to the clinic for fundus screening, they could also be diabetic, have glaucoma, AMD, and increased cardiovascular risk, with each disease having retinal vascular biomarkers that require a patient to be tested for in the clinic, which must be managed within a framework of common, interpretable, and generalizable models rather than a pipeline of independent models.

To solve these problems, we suggest a multi-task deep learning network that simultaneously classifies DR severity (five-class), glaucoma likelihood (binary), AMD stage (four-class) and cardiovascular risk (binary) from a single retinal fundus image using a single forward pass. It adopts an EfficientNet-B4 convolutional backbone pretrained on the ImageNet dataset, a six-layer multi-head self-attention Transformer encoder (d_model = 512, h = 8), a novel dual-branch spatial-channel attention module that builds on, but significantly expands, the CBAM module, and four task-specific output heads that are optimized using the homoscedastic uncertainty-weighted multi-task loss following the formulation of Kendall et al. Per-task anatomical explanations are produced by back propagating the CNN backbone via Grad-CAM in the anatomical space.

The proposed model was evaluated on EyePACS, APTOS 2019, Messidor-2 and RIM-ONE v3 and achieved an accuracy of 94.7%, AUC = 0.978, sensitivity = 93.2% and specificity = 95.8% while being statistically better than all single task models (DeLong test, p < 0.01, Bonferroni corrected). Under the reported experimental setting, the zero-shot cross-dataset evaluation achieves AUC ≥ 0.951, which is promising for its performance across datasets. Grad-CAM saliency maps are anatomically similar to established clinical grading criteria for each disease.

Medical Disclaimer: This article is for informational purposes only and does not constitute medical advice. Always consult with qualified healthcare professionals for medical decisions and treatment options.