Information Leakage and Performance Overestimation in EEG-Based Schizophrenia Detection: Evidence from Literature and Empirical Analyses
Racz, F. S.; Csukly, G.
Show abstract
Detecting schizophrenia (SZ) from electroencephalography (EEG) signals using machine- and deep learning models gained traction lately due to potential utility in early disease detection and differential diagnosis. Classification performance reports in the range of 95% accuracy and above are common; however, review of state-of-the-art literature indicates that [~]65% of published works involve erroneous practices in the evaluation pipeline such as epoch-instead of subject-based data splitting, or ranking and selecting features before data partitioning. The consequent information leakage can result in an overestimation of SZ detection performance. Here we explicitly test this on three, open SZ-EEG datasets using gold standard classification approaches in leaky and leakage-free implementations. Results indicate that information leakage can inflate SZ classification accuracy by up to [~]30%. Accordingly, best practices regarding EEG-based SZ detection must be established and promoted before this technology can be further developed into a clinical decision-making tool.
Matching journals
The top 10 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Estimating person-specific neural correlates of mental rotation: A machine learning approach 93%
- Personalized models of Disorders of Consciousness revealcomplementary roles of connectivity and local parameters in diagnosis and prognosis 93%
- Automatic diagnostics of electroencephalography pathology based on multi-domain feature fusion 93%
Similar papers in this journal
- Comparison between EEG and MEG of static and dynamic resting-state networks 93%
- A Multimodal Vision Transformer for InterpretableFusion of Functional and Structural NeuroimagingData 93%
- Thalamic contributions to psychosis susceptibility: Evidence from co-activation patterns accounting for intra-seed spatial variability (μCAPs) 92%
Similar papers in this journal
- NeuroMark: a fully automated ICA method to identify effective fMRI markers of brain disorders 93%
- Quantification of Brain Functional Connectivity Deviations in Individuals: A Scoping Review of Functional MRI Studies 92%
- Stable Biomarker Identification For Predicting Schizophrenia in the Human Connectome 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.