Navigating the Maze: Identifying Potential Pitfalls in Attention State Classification from fMRI Brain Patterns

Multi-voxel pattern analysis (MVPA) is a powerful technique to decode brain states from functional magnetic resonance imaging (fMRI) activity patterns. In neurofeedback (NF) applications, it has been used to perform real-time classification of brain activity patterns, establishing a closed-loop system that provides immediate feedback to the participants, enabling them to learn to control a complex mental state. However, MVPA has many potential limitations when applied to fMRI datasets (especially in real-time analysis) arising from small effect sizes, small number of training samples, high dimensionality of the data and, more generally, design choices. All these factors might produce inaccurate classification results. In this work, we followed a previous NF paradigm for sustained attention training. Participants were presented with composite images superimposing faces and scenes. They were instructed to focus on one class (either face or scene) for an extended period. A logistic regression classifier was trained to determine whether participants were adequately focusing on the instructed category based on their fMRI data. We analysed the classification outputs of the no-feedback training runs using various classifier settings, including whole brain data and different masking approaches, combined with different methods for the computation of single-trial fMRI responses. Furthermore, a ventricle mask was used as a control condition for the classification task, and simulations were carried out to assess the influence of the class order on the classification performances. We found inflation of the decoding accuracy for several common design choices and confounders. In particular, motion artefacts and low frequency drifts coupled with the task timing might have artificially increased the accuracy scores. Furthermore, the simulations revealed that fixed order in the presentation of experimental conditions resulted in further inflation of the classification accuracies especially in GLM-based and average-based trial estimate methods. We discuss the drawbacks of applying MVPA using the analysed sustained attention paradigm and provide insights for future improvements.