Real-time Machine Learning Alerts to Prevent Escalation of Care: A Pragmatic Clinical Trial

ImportanceAutomated machine learning algorithms have been shown to outperform older methods in predicting clinical deterioration requiring escalation of care, but rigorous prospective data on their real-world efficacy are limited. ObjectiveWe hypothesized that real-time deterioration prediction alerts sent directly to front-line providers would reduce escalations. DesignSingle-center prospective pragmatic clinical trial conducted from July 2019 to March 2020. The trial was terminated early due to the COVID-19 pandemic. Patients were followed for 30 days post-discharge. SettingAcademic tertiary care medical center located in New York City. ParticipantsAll adult patients admitted to any of four medical-surgical units were included. Assignment to intervention or control arms was determined by initial unit admission. InterventionReal-time alerts stratified according to predicted likelihood of clinical deterioration sent to the nursing/primary team or directly to the rapid response team. Clinical care and interventions were at the discretion of the providers. For the control units, alerts were generated but not sent. Main OutcomesThe primary outcome was the incidence of escalation of care. Secondary outcomes included orders placed for cardiovascular support, in-hospital and 30-day mortality. Ad-hoc outcomes included time to ICU escalation and time to discharge. Results2,780 patients were enrolled, 1,506 in the intervention group and the 1,274 in the control group. Average age was 66.2 years and 1,446 (52%) of participants were female. There was no difference in escalation between the groups, relative risk(RR) 1.22(95% Confidence Interval[CI] (0.97,1.54),p=0.10). Patients in the intervention group were more likely to receive cardiovascular support orders RR 1.35(95% CI (1.10,1.66),p=0.022). Median time to escalation with alerts was 50.6 [21.6-103] versus 58.6 [25.4-115] hours (difference -5.70;95% CI (-10.00,-2.00),p<0.001). The hazard ratio for likelihood of ICU escalation within 12 hours of an alert was 3.36 (95% CI (1.38,8.21),p=0.01) and for faster hospital discharge was 1.10 (95% CI (1.01,1.19),p=0.02). Combined in-hospital and 30-day-mortality was lower in the intervention group, RR 0.72 (95% CI (0.54,0.94),p=0.01). Conclusions and RelevancePreliminary evidence suggests that real-time machine learning alerts do not reduce the incidence of escalation but are effective in reducing time to escalation, hospital length of stay and mortality. Trial RegistrationClinicalTrials.gov, NCT04026555, https://clinicaltrials.gov Key Points QuestionCan real-time machine learning generated alerts predicting clinical deterioration, sent directly to front-line providers, reduce escalations in care? FindingsIn this pragmatic clinical trial that included 2780 adults, that was terminated early due to the COVID-19 pandemic, the incidence of escalation among patients who received alerts was 11.2% versus 9.7% among patients who did not, a non-significant difference. Combined in-hospital and 30-day mortality in the alerts group was 6.9% versus 9.4% in the group with no alerts, a significant difference. MeaningPreliminary evidence suggests that real-time machine learning generated alerts do not reduce the incidence of escalation but may reduce mortality.