A Local Outpatient Practice-Level Prediction Model for Short-Term Psychiatric Emergency Presentation

Importance: Psychiatric emergency department (ED) presentations are difficult to predict using general medical risk stratification tools. Health information exchange (HIE) data may improve prediction by capturing fragmented care across settings. Objective: To develop and temporally validate a machine learning model using HIE and geospatial data to predict 30-day psychiatric ED presentation among outpatients receiving psychiatric care and to compare its performance with standard clinical risk scores. Design, Setting, and Participants: This retrospective cohort study included patients seen at Frontier Psychiatry with records in the Big Sky Care Connect statewide HIE. Structured clinical data were linked to zip code-level sociodemographic measures. The analytic unit was the patient snapshot, defined as all structured data available up to a given point. Models were evaluated in temporally separated train and test sets. Exposures: Predictors derived from HIE structured data, including prior utilization, diagnoses, medications, laboratory data, and zip code-linked geospatial deprivation and vulnerability measures. Main Outcomes and Measures: The primary outcome was psychiatric ED presentation within 30 days, identified from structured encounter-type fields and primary diagnosis codes for psychiatric or substance use disorders. Model discrimination was compared with a parsimonious clinical baseline model and LACE and Elixhauser scores. Results: In the test set, 343 of 16,469 snapshots (2.1%) were followed by a qualifying psychiatric ED presentation within 30 days, corresponding to 102 ED visits among 68 patients. The machine learning model showed discrimination in temporally held-out testing and outperformed the clinical baseline model as well as LACE and Elixhauser scores. At a prespecified decision threshold, the model reduced the number needed to evaluate from more than 40 with universal screening to 3.4 to identify 1 true-positive case, while identifying over two fifths of 30-day psychiatric ED presentations. Conclusions and Relevance: In this retrospective cohort study, a locally developed machine learning model using statewide HIE data showed improved prediction of 30-day psychiatric ED presentation compared with selected general-purpose risk scores. The results support the feasibility of HIE-enabled local psychiatric risk modeling and suggest other practices could develop similarly tailored models. Prospective studies are needed to assess clinical utility and effects on outcomes.