Vision-language framework for multi-sequence brain magnetic resonance imaging
Lteif, D.; Jia, S.; Bit, S.; Kaliaev, A.; Mian, A. Z.; Small, J. E.; Mangaleswaran, B.; Plummer, B. A.; Bargal, S. A.; Au, R.; Kolachalama, V. B.
Show abstract
Structural magnetic resonance imaging (MRI) is a cornerstone for diagnosing neurological disorders, yet automated interpretation of multi-sequence brain MRI remains limited by challenges in cross sequence reasoning and protocol variability. Here we present ReMIND, a vision-language modeling framework tailored for comprehensive multi-sequence and multi volumetric brain MRI analysis. Trained on over 73,000 deidentified patient visits encompassing more than 850,000 MRI sequences paired with radiology reports from diverse clinical and research cohorts, ReMIND combined large scale instruction tuning on more than one million clinically grounded question answer (QA) pairs with targeted supervised fine-tuning for radiology report generation. At inference, ReMIND employed modality aware reranking and correction, a report level decoding strategy that suppressed unsupported modality claims while preserving linguistic fluency and clinical coherence. Cross-cohort generalization was maintained on independent external datasets from different institutions. These findings represent an advance toward consistent and equitable brain MRI interpretation, meriting prospective evaluation to support diagnosis and management of neurological conditions.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Understanding the robustness of vision-language models to medical image artefacts 94%
- A Clinical Neuroimaging Platform for Rapid, Automated Lesion Detection and Personalized Post-Stroke Outcome Prediction 93%
- Biologically-informed deep neural networks provide quantitative assessment of intratumoral heterogeneity in post-treatment glioblastoma 92%
Similar papers in this journal
- Anatomy-guided, modality-agnostic segmentation of neuroimaging abnormalities 94%
- WMH-DualTasker: A weakly-supervised deep learning model for automated white matter hyperintensities segmentation and visual rating prediction 94%
- Longitudinally stable, brain-based predictive models mediate the relationships between childhood cognition and socio-demographic, psychological and genetic factors 93%
Similar papers in this journal
- Annotation-free multi-organ anomaly detection in abdominal CT using free-text radiology reports: A multi-center retrospective study 93%
- Weakly-Supervised Tumor Purity Prediction FromFrozen H&E Stained Slides 90%
- Integrative deep learning analysis improves colon adenocarcinoma patient stratification at risk for mortality 90%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.