Development and validation of a personalised antipsychotic selection tool for first-line treatment in severe mental illness

BackgroundGuidance is lacking on choice of first-line antipsychotic for individuals with incident severe mental illness (SMI). Patients may try several before an effective, well-tolerated drug is identified, delaying symptom improvement. We aimed to develop a personalised selection tool to identify the optimum first-line antipsychotic, based on individual sociodemographic and clinical characteristics. MethodsRisk prediction development and validation study using electronic health records (EHRs) from primary care in England (Clinical Practice Research Datalink) linked to Hospital Episode Statistics, including 11,811 individuals with incident SMI prescribed aripiprazole, olanzapine, quetiapine or risperidone as first-line treatment between 2007-2016. The outcome was time to psychiatric hospitalisation or change to different antipsychotic within 3 years of commencing treatment. Prediction algorithms were developed using Cox proportional hazards models in a 70% training sample and validated in a 30% hold-out sample. This baseline model was compared with machine learning survival models of increasing complexity. Potential predictors included demographics, diagnoses, concomitant medications and laboratory findings. OutcomesAmong 8,225 individuals in the development cohort, 4,456 (54.2%) experienced the outcome. In model validations, 1,022 (53.3%) of 1,916 in the validation cohort did not receive the optimal antipsychotic identified by the personalised selection tool. The predicted 3-year outcome risk if all individuals received the medication assigned by the tool was 6.3% lower (95% CI 4.0%-8.5%) than the observed 3-year risk in the validation cohort, and 10.2% lower (95%CI 7.9%-12.5%) than if individuals were randomly assigned an antipsychotic (corresponding numbers need to treat of 16 and 10). Machine learning approaches did not meaningfully improve model performance. InterpretationA personalised tool based on EHR data could improve treatment success rates by optimising first-line antipsychotic selection. Machine learning did not outperform traditional prediction methods. Further research will assess the impact on adverse events and in other populations. FundingUK Research and Innovation grant MR/V023373/1. Research in contextO_ST_ABSEvidence before this studyC_ST_ABSWe searched PubMed for articles published from database inception to December 13, 2024, with no language restrictions. We searched titles and abstracts using the terms ((prediction) AND ((treatment response) OR (treatment rule) OR (treatment outcome)) AND ((psychosis) OR (severe mental illness) OR (schizophrenia) OR (bipolar disorder))). We identified 187 articles for full text screening. A number of studies exist on the prediction of lithium treatment response. A recent systematic review summarised the results of eight studies that used biomarkers, clinical and socio-demographic features to predict treatment response in psychosis, however these commonly compared responders with non-responders, rather than developing treatment selection rules. Two studies did generate treatment selection recommendations. One used a Super Learner in Taiwan National Health insurance data to optimise antipsychotic selection in first episode psychosis, resulting in a 7% improvement in estimated treatment success rate. The second examined antipsychotic selection, choosing between risperidone and aripiprazole, in children using Korean National Health insurance data and found a 1.2-1.5 times increase in antipsychotic continuation using their model compared to their allocated treatment. They found no improvement in performance when comparing machine learning with simple regression models. Neither model has been externally validated. We could not find any models that are in clinical use. Added value of this studyWe found that a simple treatment selection prediction model, based on data contained in the electronic health records at the point that an individual with severe mental illness is first prescribed an antipsychotic, could reduce treatment failure rates by 6-10%. In our validation cohort 75% of patients were switched to an alternative antipsychotic medication by the treatment selection tool. In line with the limited number of previous studies in this area, we did not observe meaningful improvements in predictive properties when machine learning approaches were compared with traditional models. Implications of all the available evidencePrediction models for optimising treatment selection in psychiatry are becoming increasingly possible with data from electronic health records. Improving treatment selection for people with SMI is low risk, compared to other prediction problems in psychiatry, and could improve long-term outcomes. Models still need full external validation and testing in new cohorts.