An external, contemporary evaluation of the Epic End of Life Care Index among hospitalized patients across two large health systems: A retrospective cohort study

BackgroundThe Epic End of Life Care Index (EOLCI) predicts one-year mortality and was developed to improve serious illness care. However, prior external EOLCI evaluations had limited sample sizes, populations, and equity evaluations. In preparation for a multi-system pragmatic clinical trial, we sought to evaluate the EOLCI performance and equity in the trials two participating health systems. ObjectiveEvaluate EOLCI model performance overall and across key subgroups. Design/Setting/PatientsRetrospective cohort study of patients hospitalized for [≥]36 hours in 2022 to 39 hospitals in the Trinity Health and Kaiser Permanente Southern California (KPSC) health systems. MeasurementsWe predicted one-year mortality risk stratified by health system using the EOLCI, a logistic regression model including age, sex, race/ethnicity, ethnicity, insurance, and diagnoses. We evaluated model performance using Scaled Brier Scores (SBS; range -1 to 1; composite measures of calibration and discrimination), calibration plots, and c-statistics. ResultsAmong 116,749 Trinity patients with 154,063 encounters, 12,054 (10.3%) patients died within one year. Among 94,489 KPSC patients with 133,043 encounters, 16,872 (17.9%) died within one year. The SBS was -0.007 at Trinity and 0.178 at KPSC. Calibration was poor for both. Trinitys discrimination was acceptable/good (c-statistic 0.76, 95% CI 0.76-0.77), and KPSCs was good/very good (c-statistic 0.81, 95% CI 0.81-0.81). Model performance across subgroups was similar to the overall cohort. LimitationsDeath data were collected exclusively within Trinity and KPSC, risking outcome misclassification; several subgroup evaluations were limited by small sample sizes. ConclusionsAn external evaluation of the widely available Epic EOLCI demonstrated adequate to very good discrimination, poor calibration, and equitable performance across sociodemographic characteristics and diagnoses in two of the nations largest health systems. Primary funding sourcePCORI PLACER-2022C3-30553.