An Efficient and Interpretable Foundation Model for Retinal Image Analysis in Disease Diagnosis
Dai, W.; Chen, Z. J.; Yao, Y.; Chen, Y.; Fang, J.; Bai, Q.; Xu, C.; Wu, H.; Ding, H.; Yang, H.; Zhuo, R.; Zhang, R.; Yuan, J.; Ye, C.; Wang, H.; Xu, L.; Yang, Y.; Yu, X.; Hospedales, T.; Qu, J.; Su, J.
Show abstract
Artificial intelligence (AI) foundation models for colour fundus photography (CFP) have been extensively studied and demonstrated great potential for advancing ocular and systemic health screening. However, their high computational demands and limited clinical interpretability constrain real-world clinical application. These models rely on self-supervised learning with massive unlabeled datasets to address the scarcity of high-quality annotations, but often generate irrelevant features and fail to improve interpretability due to the absence of medical knowledge integration. Thus, we propose HRVRL, a lightweight, knowledge-prompt foundation model that leverages a novel hierarchical representation learning framework based on retinal biological features. Over 150,000 instances were generated for pretraining through multi-level image augmentation of 267 vascular-labeled images. A progressive learning strategy enables HRVRL to capture retinal-specific features from coarse to fine scales. HRVRL demonstrates remarkable resource efficiency, requiring only 0.04 GB of memory, processing 24 images per second, and completing pretraining within one day using a single GPU. It outperforms existing foundation models in 20 of 24 downstream tasks related to ocular and systemic disease diagnosis and severity grading. HRVRL also offers high clinical interpretability, with quantitative assessments showing strong concordance between model predictions and clinical criteria and outperforming in all 10 tasks. In diabetic retinopathy (DR) analysis, HRVRL achieves superior diagnostic lesion recognition (median accuracy of 0.710 versus 0.1-0.235 for existing models; P < 0.001) and significant improvements in type-specific lesion detection under a zero-shot setting (18-fold for hemorrhages, 4-fold for microaneurysms, hard exudates, and soft exudates; P < 0.001). We demonstrate that HRVRL provides clinically interpretable predictions with transparent decision-making processes for individual cases. In conclusion, HRVRL achieves unprecedented resource efficiency and enhanced clinical interpretability, enabling practical deployment in resource-limited settings to improve ocular and systemic disease diagnosis.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Understanding the robustness of vision-language models to medical image artefacts 97%
- Cross-Platform Omics Prediction procedure enables precision medicine in patients with stage-III melanoma 92%
- Federated Target Trial Emulation using Distributed Observational Data for Treatment Effect Estimation 92%
Similar papers in this journal
- Quantification of Fundus Autofluorescence Features in a Molecularly Characterized Cohort of More Than 3500 Inherited Retinal Disease Patients from the United Kingdom 93%
- Artificial intelligence to facilitate clinical trial recruitment in age-related macular degeneration 92%
- Comparative analysis of single-cell and single-nucleus RNA-sequencing in a rabbit model of retinal detachment-related proliferative vitreoretinopathy. 90%
Similar papers in this journal
- STAIG: Spatial Transcriptomics Analysis via Image-Aided Graph Contrastive Learning for Domain Exploration and Alignment-Free Integration 94%
- Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes 94%
- PHARAOH: A collaborative crowdsourcing platform for PHenotyping And Regional Analysis Of Histology 93%
Similar papers in this journal
- Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology 94%
- Encrypted federated learning for secure decentralized collaboration in cancer image analysis 92%
- Spatial Transcriptomics Expression Prediction from Histopathology Based on Cross-Modal Mask Reconstruction and Contrastive Learning 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.