Average accuracy hides where a brain-imaging psychosis risk model fails across clinical cohorts
Vetter, C. S.; Grund, A.; Hahn, L.; Mena, S.; Fraza, C.; Fietz, J.; Diederichs, N. F.; Dwyer, D. B.; Lalousis, P.; Penzel, N.; Urquijo, M. F.; Schultze-Lutter, F.; Woods, S. W.; Cannon, T. D.; Haas, S. S.; Kambeitz-Ilankovic, L.; Kambeitz, J.; Ruhrmann, S.; Lichtenstein, T.; Chisholm, K.; Romer, G.; Antonucci, L. A.; Riecher-Roessler, A.; Pantelis, C.; Hietala, J.; Salokangas, R. K. R.; Lencer, R.; Dannlowski, U.; Korda, A.; Bertolino, A.; Borgwardt, S.; Brambilla, P.; Wood, S. J.; Meisenzahl, E.; Upthegrove, R.; Falkai, P.; Heekeren, K.; Roessler, W.; Theodoridou, A.; Rueckert, D.; Koutsouler
Show abstract
Brain-imaging models predicting psychiatric outcomes are rarely tested where they would be used. Clinical high risk for psychosis is a demanding test case because services differ in whom they recruit and how they record transition. We re-estimated a published structural magnetic resonance imaging (MRI) risk model and applied it unchanged to four independent cohorts (685 participants, 106 transitions). Discrimination was weak (area under the curve, 0.56) and near chance in the two cohorts not previously used to evaluate the model (0.53 and 0.56). Predicted risks did not transfer (calibration slope, 0.17). Exploratory performance landscapes, which map local discrimination across participant characteristics, showed that discrimination varied with negative-symptom severity, also in external cohorts, whereas apparent age- and brain-age-related variation depended on single cohorts; no global test survived multiple-testing correction. Reporting where a model works, not only its average accuracy, can reveal when one diagnostic label hides several prediction problems.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Atypical brain maturation in psychosis is associated with long-term cognitive decline and symptom progression 95%
- Latent subtypes of manic or irritable episode symptoms in two population-based cohorts 91%
- Analysis of diagnosis instability in electronic health records reveals diverse disease trajectories of severe mental illness 90%
Similar papers in this journal
- Cerebral cortical alterations in adolescent early-onset psychosis: a surface-based morphometry mega-analysis 94%
- Replication of a neuroimaging biomarker for striatal dysfunction in psychosis 93%
- Molecular and micro-architectural mapping of abnormal gray matter developmental trajectories in psychosis 92%
Similar papers in this journal
- Multivariate Resting-State Functional Connectivity Features Linked to Transdiagnostic Psychopathology in Early Psychosis 92%
- Striatal functional connectivity in psychosis relapse: A comparison between antipsychotic adherent and non-adherent patients at the time of relapse 92%
- Decoding Early Psychoses: Unraveling Stable Microstructural Features Associated with Psychopathology Across Independent Cohorts 92%
Similar papers in this journal
- Developmental Influences on Symptom Expression in Antipsychotic-Naive First-Episode Psychosis 92%
- Jumping To Conclusions, General Intelligence, And Psychosis Liability: Findings From The Multicentric EU-GEI Case-Control Study 92%
- Disentangling heterogeneity of psychosis expression in the general population: sex-specific moderation effects of environmental risk factors on symptom networks 91%
Similar papers in this journal
- Thinner cortex is associated with psychosis onset in individuals at Clinical High Risk for Developing Psychosis: An ENIGMA Working Group mega-analysis 94%
- Longitudinal Changes in Functional Connectivity in Antipsychotic-treated and Antipsychotic-naive Patients with First Episode Psychosis 93%
- Genetic association study of psychotic experiences in UK Biobank 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.