Evaluation of the Impact of a 90-day Mortality Prediction Model Integrated in the EHR for Advanced Care Planning

Medical centers have begun to integrate AI models for mortality prediction in their electronic health records (EHR) systems to support advance care planning.1 There is evidence that advance care planning can reduce suffering and unwanted, costly treatment near the end of life.2 To better target patients at a risk of death with advance directive (AD) capture and goals of care (GOC) conversations at our comprehensive cancer center, we developed and integrated a 90-day mortality prediction model in the EHR. The model has been live since February, 2021. Performance in production has matched very closely pre-production pilots, achieving an 85% AUROC (AI Showcase Stage I). We implemented 4 clinical decision support (CDS) tools driven by the model to promote goal concordant care in inpatient and outpatient encounters (AI Showcase Stage II).3 In this document, we analyze the impact of one of the aforementioned CDS applications, implemented as an inter-ruptive alert for the clinical social workers (CSWs) to facilitate advance directive (AD) completion in the inpatient setting. We also present preliminary results on volumes of GOC conversations prompted by one of the model-driven alerts. Our analysis is based on 12 months of data, collected between March 1, 2021 and February 28, 2022. The challenges for our assessment included inconsistent EHR documentation practices, coexistence with (traditional) non-AI driven processes tasked for similar goals and many external factors that can affect target outcomes. We believe such challenges to be rather common across healthcare organizations integrating AI in their workflows.