Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Robust estimation of systemic human cognitive states is critical for many applications, from simply detecting inefficiencies in human task performance to adapting the behaviors of artificial agents to improve team performance in mixed-initiative human-machine teams. Here we use comprehensive analyses of a multi-modal dataset from a multi-tasking driving experiment to provide systematic evidence that percentage change in pupil size (PCPS) in human eye gaze is the most reliable biomarker for assessing three human cognitive states relevant to task performance: workload, sense of urgency, and mind wandering. Specifically, we performed comprehensive statistical tests that establish the superior performance of PCPS compared other physiological signals like electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), respiration, arterial blood pressure (ABP), and skin conductance. Based on the eye gaze data, we also characterized the relation between workload and sense of urgency, observing that consecutive occurrences of higher sense of urgency tended to increase overall workload. We also trained five state-of-the-art machine learning models on the data to determine their potential for predicting the three systemic cognitive states across subjects, showing that four of them had similar highest accuracy in cognitive state classification based on PCPS (with one, random forest, showing inferior performance). In combination with the statistical analyses, the comparative modeling results demonstrate that PCPS, in contrast to prior findings in the literature, represents the optimal method for estimating the three performance-relevant cognitive states.