PanEcho: Complete AI-enabled echocardiography interpretation with multi-task deep learning

ImportanceEchocardiography is a cornerstone of cardiovascular care but relies on expert interpretation and manual reporting from a series of videos. We propose an artificial intelligence (AI) system, PanEcho, to automate echocardiogram interpretation with multi-task deep learning. ObjectiveTo develop and evaluate the accuracy of PanEcho on a comprehensive set of 39 echocardiographic labels and measurements on transthoracic echocardiography (TTE). Design, Setting, and ParticipantsThis study represents the development and retrospective, multi-site validation of an AI system. PanEcho was developed using a sample of TTE studies conducted at Yale-New Haven Health System (YNHHS) hospitals and clinics from January 2016-June 2022 during routine care. The trained model was internally validated in a temporally distinct YNHHS cohort from July-December 2022, externally validated across four diverse external cohorts, and made publicly available. Main Outcomes and MeasuresThe primary outcome was the area under the receiver operating characteristic curve (AUC) for diagnostic classification tasks and mean absolute error (MAE) for parameter estimation tasks, comparing AI predictions with the assessment of the interpreting cardiologist. ResultsThis study included 1.2 million echocardiographic videos from 32,265 TTE studies of 24,405 patients across YNHHS hospitals and clinics. PanEcho performed 18 diagnostic classification tasks with a median AUC of 0.91 (IQR: 0.88-0.93) and estimated 21 echocardiographic parameters with a median normalized MAE of 0.13 (0.10-0.18) in internal validation. For instance, the model accurately estimated left ventricular (LV) ejection fraction (MAE: 4.2% internal; 4.5% external) and detected moderate or higher LV systolic dysfunction (AUC: 0.98 internal; 0.99 external), RV systolic dysfunction (0.93 internal; 0.94 external), and severe aortic stenosis (0.98 internal; 1.00 external). PanEcho maintained excellent performance in limited imaging protocols, performing 15 diagnosis tasks with 0.91 median AUC (IQR: 0.87-0.94) in an abbreviated TTE cohort and 14 tasks with 0.85 median AUC (0.77-0.87) on real-world point-of-care ultrasound acquisitions by non-experts from YNHHS emergency departments. Conclusions and RelevanceWe report an AI system that automatically interprets echocardiograms, maintaining high accuracy across geography and time from complete and limited studies. PanEcho may be used as an adjunct reader in echocardiography labs or rapid AI-enabled screening tool in point-of-care settings. KEY POINTSO_ST_ABSQuestionC_ST_ABSCan artificial intelligence (AI) fully automate echocardiogram interpretation? FindingsWe report the development and validation of an automated AI system for echocardiogram analysis, called PanEcho, that performed 18 diagnostic classification tasks with a median area under the receiver operating characteristic curve (AUC) of 0.91 and 21 echocardiographic parameter estimation tasks with a median normalized mean absolute error (MAE) of 0.14. MeaningAn AI system can automate complete echocardiogram interpretation with high accuracy, potentially accelerating workflows and enabling rapid cardiovascular health screening in point-of-care settings with limited access to trained experts.