Conformal Prediction and Venn-ABERS Calibration for Reliable Machine Learning-Based Prediction of Bacterial Infection Focus
Schmidt, J. B.; Nielsen, K. L.; Strunin, D.; Kirkby, N. S.; Thomassen, J. Q.; Rasmussen, S. C.; Frikke-Schmidt, R.; Hertz, F. B.; Engsig-Karup, A. P.
Show abstract
Finding the focus of bacterial infections can be challenging, especially for hospitalised patients. Conventional microbiological diagnostic methods are either time consuming, expensive, or difficult to interpret due to contamination. The aim of this study was to apply machine learning (ML) to reliably predict the focus of bacterial infections. This study utilised a dataset including samples from 10,153 patients, collected from November 1, 2019, to June 3, 2023, at Rigshospitalet, Denmark. The dataset contains microbiological findings, biochemical data, and vital parameters. The dataset was analysed using ML. The ML-outputs were calibrated using Venn-ABERS calibration and model uncertainty was addressed using conformal risk control. The best performing model was the XGBoost model achieving a Log loss of 0.219 {+/-} 0.050 (mean {+/-} SD.) and an AUC of 0.93 {+/-} 0.051. Combining the model with methods from the conformal prediction framework achieves predictive capabilities that surpasses similar studies, while also accounting for model uncertainty by providing statistically robust uncertainty estimates via calibrated probabilistic predictions.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Mitigating Machine Learning Bias Between High Income and Low-Middle Income Countries for Enhanced Model Fairness and Generalizability 96%
- Predicting bloodstream infection outcome using machine learning 96%
- Developing Machine Learning Models for Predicting Intensive Care Unit Resource Use During the COVID-19 Pandemic 95%
Similar papers in this journal
- AI-MET: A Deep Learning-based Clinical Decision Support System for Distinguishing Multisystem Inflammatory Syndrome in Children from Endemic Typhus 94%
- Machine Learning Interpretability Methods to Characterize the Importance of Hematologic Biomarkers in Prognosticating Patients with Suspected Infection 93%
- Predicting the physiological effects of multiple drugs using electronic health record 93%
Similar papers in this journal
- Predicting the causative pathogen among children with pneumonia using a causal Bayesian network 95%
- Contrasting factors associated with COVID-19-related ICU admission and death outcomes in hospitalised patients by means of Shapley values 94%
- Cluster detection with random neighbourhood covering: application to invasive Group A Streptococcal disease 93%
Similar papers in this journal
- Convolutional neural networks quantify antibiotic resistance in Mycobacterium tuberculosis with diagnostic grade accuracy and predict treatment response 93%
- Deep representation learning for clustering longitudinal survival data from electronic health records 93%
- Convergence of resistance and evolutionary responses in Escherichia coli and Salmonella enterica co-inhabiting chicken farms in China 92%
Similar papers in this journal
- Modular Clinical Decision Support Networks (MoDN)—Updatable, Interpretable, and Portable Predictions for Evolving Clinical Environments 94%
- Informing antimicrobial stewardship with explainable AI 93%
- Community-acquired pneumonia identification from electronic health records in the absence of a gold standard: a Bayesian latent class analysis 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.