Validation of clinical diagnosis and machine learning classification of cognitive impairment

INTRODUCTION: Cognitive syndrome diagnosis (Normal, Mild Cognitive Impairment (MCI), Dementia) is important for summarizing disease status and predicting future progression. Machine learning approaches to classification might substitute for or complement clinical diagnosis but must be shown to have validity for these purposes. METHODS: A machine learning algorithm was trained in a previous study to reproduce clinical diagnosis of cognitive impairment [1]. We examined and compared concurrent validity (cross-sectional MRI measures of brain integrity) and predictive validity (longitudinal change in MRI measures of brain integrity and progression to a more impaired diagnosis/classification) of clinical diagnosis and algorithmic classification from the prior study. RESULTS: Clinical diagnosis and algorithmic classifications had robust associations with clinical and MRI outcomes and differences across diagnosis/classification types were minor. Algorithmically estimated probability of a Normal diagnosis had the strongest associations with cross-sectional and longitudinal MRI outcomes. Progression from Normal to MCI or Dementia was faster for Clinical diagnosis than Algorithmic classification but future rates of MRI measured brain degeneration were essentially the same in individuals with baseline clinical diagnosis and algorithmic classification of Normal cognition. DISCUSSION: Clinical diagnosis and algorithmic, machine learning based classification had robust and similar associations with independent validity criteria. Both forms of diagnosis/classification had utility for staging current brain degeneration and predicting future brain degeneration and clinical decline. This study demonstrates a validation design that can simultaneously evaluate the utility of both clinical diagnosis and algorithmic classification of cognitive impairment in a manner that improves understanding of both types of classification.