Diagnosing Rejection Collapse via Uncertainty Decomposition

Standard uncertainty-informed rejection can unexpectedly trigger severe performance collapse, exposing localized vulnerabilities that common machine learning metrics typically do not show. We systematically diagnose this failure dynamic using Levodopa-Induced Dyskinesia prediction in Parkinson's Disease as a proof-of-concept. By training a heterogeneous ML ensemble, decomposing Aleatoric and Epistemic uncertainty and applying unsupervised subgroup discovery, we isolated the precise drivers of these atypical errors. Stratified error analysis revealed two divergent predictive regimes previously hidden by a global evaluation. While the models successfully extracted a predictive signal for one subgroup, the baseline features of a second subgroup lacked discriminative capacity, resulting in a high rate of confident misclassifications. Operating entirely below rejection thresholds, this single subgroup flatlined predictive metrics, driving the collapse of the global rejection curve. Ultimately, we demonstrate that atypical rejection failures stem from subgroup-specific data ambiguity rather than algorithmic deficiencies, making localized uncertainty-aware evaluation a critical methodological requirement prior to real-world deployment.