Multimodal artificial intelligence using entire electronic health record and complete pathogen genome data for patient outcome prediction from life-threatening infection: the SuperbugAI Platform

Artificial intelligence (AI) has the potential to transform healthcare, with advanced multimodal approaches showing great promise in leveraging diverse health-related data. Here, we applied multimodal AI to entire electronic health record (EHR) and complete pathogen genome data to predict patient outcomes from life-threatening infection. An automated, scalable pipeline was developed for EHR data preprocessing, quality control, and standardisation. A deep learning fusion model was trained to predict in-hospital mortality, need for ICU admission, prolonged length of stay and 30-day unplanned readmission. We then developed a novel genomic large language model (gLLM) architecture to incorporate bacterial genomic features into the multimodal fusion model. The cohort comprised 2,656 bloodstream infection hospitalisations involving 2,535 patients. Deep learning fusion models using entire structured and unstructured EHR data outperformed traditional APACHE II score mortality prediction (AUROC [95% confidence intervals] 0.93 [0.92-0.94] versus 0.77 [0.77-0.78]). The model also showed strong performance for predicting the need for ICU admission (AUROC 0.978 [0.966 - 0.986]), prolonged hospital length of stay (AUROC 0.803 [0.790 - 0.812]) and unplanned readmission (AUROC 0.696 [0.690 - 0.701]). As proof of principle, incorporating entire microbial genomic features from the causative pathogen further enhanced prediction and enabled identification of key bacterial virulence pathways relevant for human disease. Multimodal AI integrating harmonised EHR and genomic data can accurately identify hospitalised patients at risk of poor outcomes. These approaches are scalable to other subspecialities of medicine.