Develop and Validate A Fair Machine Learning Model to Indentify Patients with High Care-Continuity in Electronic Health Records Data

ObjectivesElectronic health record (EHR) data often missed care outside a given health system, resulting in data discontinuity. We aimed to: (1) quantify misclassification across levels of EHR data discontinuity and identify an optimal continuity threshold. (2) develop a machine learning (ML) model to predict EHR continuity and optimize fairness across racial and ethnic groups, and (3) externally validate the EHR continuity prediction model using an independent dataset. Materials and MethodsWe used linked OneFlorida+ EHR-Medicaid claims data for model development and REACHnet EHR-Louisiana Blue Cross Blue Shield (LABlue) claims data for external validation. A novel Harmonized Encounter Proportion Score (HEPS) was applied to quantify patient-level EHR data continuity and the impact on misclassification of 42 clinical variables. ML models were trained using routinely available demographic, clinical, and healthcare utilization features derived from structured EHR data. ResultsHigher EHR data continuity was associated with lower rates of misclassification. A HEPS threshold of approximately 30% effectively distinguished patients with sufficient data continuity. ML models demonstrated strong performance in predicting high continuity (AUROC=0.77). Fairness assessments showed bias against Hispanic group, which was substantially improved following bias mitigation procedures. Model performance remained robust and fair in the external validation. DiscussionOur study offers a practical metric for quantifying care continuity in EHR networks. The current ML model incorporating EHR-routinely collected information can accurately identify patients with high care continuity. ConclusionsWe developed a generalizable care-continuity classification tool that can be easily applied across EHR systems, strengthening the rigor of EHR-based research.