Rural Medical Centers Struggle to Produce Well-Calibrated Clinical Prediction Models: Data Augmentation Can Help
Brown, K.; Malin, B. A.; Davis, S. E.
Show abstract
Machine learning models support many clinical tasks; however, challenges arise with the transportability of these models across a network of healthcare sites. While there are guidelines for updating models to account for local context, we hypothesize that not all healthcare organizations, especially those in smaller and rural communities, have the necessary patient volumes to facilitate local fine tuning to ensure models are reliable for their populations. To investigate these challenges, we conducted an experiment using data from a real network of hospitals to predict 30-day unplanned hospital readmission and a simulation study using data from a multi-site ICU dataset to evaluate the utility of synthetic data generation (SDG) to augment local data volumes. Several factors associated with rurality were correlated with model miscalibration and rural sites failed to meet sample size requirements for local recalibration. Our results indicate that deep learning approaches to SDG produced the best local classifiers.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database 93%
- Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images 93%
- From theoretical models to practical deployment: A perspective and case study of opportunities and challenges in AI-driven healthcare research for low-income settings 92%
Similar papers in this journal
- Automated stratification of trauma injury severity across multiple body regions using multi-modal, multi-class machine learning models 94%
- A framework for making predictive models useful in practice 93%
- Learning from local to global - an efficient distributed algorithm for modeling time-to-event data 93%
Similar papers in this journal
- Emergency department admissions during COVID-19: explainable machine learning to characterise data drift and detect emergent health risks 94%
- A Comprehensive County Level Framework to Identify Factors Affecting Hospital Capacity and Predict Future Hospital Demand 94%
- Developing Machine Learning Models for Predicting Intensive Care Unit Resource Use During the COVID-19 Pandemic 94%
Similar papers in this journal
- Development and Validation of ‘Patient Optimizer’ (POP) Algorithms for Predicting Surgical Risk with Machine Learning 94%
- Implicit bias in Critical Care Data: Factors affecting sampling frequencies and missingness patterns of clinical and biological variables in ICU Patients 94%
- Addressing Label Noise for Electronic Health Records: Insights from Computer Vision for Tabular Data 93%
Similar papers in this journal
- Assessing the effects of data drift on the performance of machine learning models used in clinical sepsis prediction 94%
- Image and structured data analysis for prognostication of health outcomes in patients presenting to the Emergency Department during the COVID-19 pandemic 93%
- Synthetic Data Generation in Healthcare: A Scoping Review of reviews on domains, motivations, and future applications 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.