Predicting Subjective Cognitive Decline on Future BRFSS Survey Years: An Open Multi-Language Machine Learning Benchmark
Nguyen, T. T.; Nguyen, T. D.
Show abstract
Background and Objectives: Subjective cognitive decline (SCD), self-reported worsening confusion or memory over the past year, is a common early marker of cognitive concern with relevance for Alzheimer's disease prevention and population health. Population-based machine learning benchmarks that respect temporal drift in public health surveillance remain limited. We developed a reusable multi-language prediction and interpretability framework for SCD using Behavioral Risk Factor Surveillance System (BRFSS) Cognitive Decline data. Methods: We analyzed pooled national (n = 298,944) and New York (n = 30,366) cohorts with chronological train (2015-2019), validation (national: 2020-2022; New York: 2020-2021), and locked test (2023-2024) splits. Nested LASSO identified stable predictors. Sixteen machine learning algorithms were compared under year-grouped cross-validation with SMOTE restricted to training folds. Four end-to-end Python/R pipelines (single-model or soft-voting) used validation-only isotonic calibration and Youden thresholding. Primary reporting pipelines were prespecified before test unlock (national: R tidymodels single-model; New York: Python single-model); algorithms within each pipeline were chosen by validation ROC-AUC. Post-hoc GLMs (national unweighted; New York design-weighted) and two training-only knowledge-graph layers supported interpretability. Results: Locked-test discrimination was consistent across implementations (ROC-AUC approximately 0.76-0.77). Prespecified pipelines achieved test ROC-AUC 0.770 (95% CI 0.767-0.773) nationally (R gradient boosting) and 0.762 (95% CI 0.746-0.777) in New York (Python AdaBoost). Soft-voting pipelines performed similarly (national 0.770; New York 0.757) and were treated as sensitivity benchmarks. Predicted probabilities were reasonably calibrated (Brier 0.118 nationally; 0.112 in New York), and higher scores among SCD-positive respondents persisted across survey years. Difficulty deciding, mental health, and functional health items ranked highest across permutation importance, SHAP, and GLMs. Respondents who reported no difficulty deciding (DECIDE = 2) had substantially lower odds of SCD than those who reported difficulty (DECIDE = 1; aOR approximately 0.13; FDR < 0.05). Training-only knowledge graphs likewise placed difficulty deciding nearest to SCD in both cohorts. Conclusions: A temporally locked, multi-pipeline BRFSS benchmark yields stable future-year SCD risk ranking, usable calibrated probability scores that remain separated by SCD status across survey years, and convergent interpretability signals. The open implementation supports reproducible surveillance-oriented machine learning for cognitive health.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Interpretable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons in older adults 92%
- A Remote Digital Memory Composite to Detect Cognitive Impairment in Memory Clinic Samples in Unsupervised Settings using Mobile Devices 91%
- A scoping review of remote and unsupervised digital cognitive assessments in preclinical Alzheimer’s disease 91%
Similar papers in this journal
- Identifying incident dementia by applying machine learning to a very large administrative claims dataset 92%
- The Healthy Brain Initiative (HBI): A prospective cohort study protocol 92%
- Self-reported health behaviors and longitudinal cognitive performance: Results from the Wisconsin Registry for Alzheimer’s Prevention 92%
Similar papers in this journal
- Multimorbidity and risk of incident dementia: role of disease clusters and genetic risk for dementia in a cohort of 206,960 participants 90%
- Diagnostic Codes in AI prediction models and Label Leakage of Same-admission Clinical Outcomes 89%
- Estimating the Burden of Influenza on Daily Activity at Population Scale Using Commercial Wearable Sensors 88%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.