Odor identification task actually measures global cognition, but with 70% error

ObjectiveTo study the reliability and validity of an odor identification test. MethodsThe data come from an epidemiological cohort including 1146 non-Hispanic Caucasian, 86 Hispanic, and 12 other participants at the baseline visit (73.4% female). We tested the fit of each of three neurobiologically plausible models (validity) for responding on a 12-item odor identification task using confirmatory factor analyses (CFA); five model fit indices were assessed for each run. CFA testing fit over time (reliability) was planned for the measurement model that was found to fit across groups at the baseline visit. If a model was not found for the baseline visit, the test would be deemed "not invariant" over group, and not tested over time. In this case, we planned a post hoc Rasch analysis to further study test validity; and a multi-trait, multi-method analysis (MTMM) of the entire test battery to study reliability in terms of other, valid, cognitive and neuropsychological functional assessments. ResultsNearly 70% of the variability in odor identification scores is error, a result that was replicated over four independent samples at the baseline visit. A core of 30% of "signal" from the task was identified over time (via Rasch modeling) but was explained fully by global cognition (replicated over time). Conclusions"Odor identification" as a construct cannot be reliably or validly measured over time or group. Multiple hypothesis-driven methods and replications show that this test provides no information that a global cognition score does not also (more validly and reliably) provide.