OphthUS-GPT: Multimodal AI for Automated Reporting in Ophthalmic B-Scan Ultrasound

IMPORTANCEThe rapid advancement of AI in ophthalmology is transforming diagnostics, especially in resource-limited settings. The shortage of ophthalmologists and lack of standardized reporting creates an urgent need for AI systems capable of automated reporting and interactive decision support. OBJECTIVETo develop OphthUS-GPT, a multimodal AI system integrating BLIP and DeepSeek models for automated report generation and clinical decision support from ophthalmic B-scan ultrasound images. DESIGN, SETTING, AND PARTICIPANTSThis retrospective study at the Affiliated Eye Hospital of Jiangxi Medical College collected B-scan ultrasound reports between 2017-2024, including 54,696 images and 9,392 reports from 31,943 patients (mean age 49.14{+/-}0.124 years, 50.15% male). MAIN OUTCOMES AND MEASURESEvaluation included two components: diagnostic report generation and question-answering system assessment. Report generation was evaluated using text metrics (ROUGE-L, CIDEr), disease classification metrics (accuracy, sensitivity, specificity, precision, F1 score), and ophthalmologist ratings for accuracy and completeness. The question-answering system was assessed by ophthalmologists rating answers on accuracy, completeness, potential harm, and satisfaction. RESULTSOphthUS-GPT achieved ROUGE-L and CIDEr scores of 0.6131 and 0.9818 in report generation. For common conditions, accuracy exceeded 90% with precision >70%. Expert assessment showed >90% of reports scored [≥] 3/5 for correctness and 96% for completeness. The DeepSeek-R1-Distill-Llama-8B (DeepSeek) question-answering component performed comparably to GPT4o and OpenAI-o1, outperforming other models. CONCLUSIONS AND RELEVANCOphthUS-GPT demonstrated excellent performance in automatic report generation and intelligent Q&A, offering a novel solution for ophthalmic ultrasound interpretation and clinical decision support.