Brain-Language Alignment During Naturalistic Reading and Its Disruption by Mind-Wandering
Sun, H.; Jangraw, D. C.
Show abstract
Encoding models offer a principled framework for linking computational representations of language to neural activity, but most electroencephalography (EEG) evidence for brain-language alignment comes from tightly controlled, word-by-word reading paradigms. Whether such alignment is detectable during naturalistic reading, and how it is affected by lapses in attention, remains unclear. We addressed these questions using ROAMM, a multimodal dataset containing simultaneous EEG and eye-tracking recordings with time-resolved mind-wandering (MW) annotations from 44 participants reading naturalistic texts. Ridge regression encoding models were trained to predict fixation-aligned EEG spectral power and fixation-related potentials (FRPs) from five word-embedding models (GloVe, word2vec, BERT, GPT-2, and Llama 3). Using permutation testing with false discovery rate correction, we found statistically reliable brain-language alignment across both feature types, with contextual embeddings outperforming static embeddings. Spectral alignment was strongest in the alpha and low-beta bands over parietal electrodes, while FRP-based alignment peaked 200-300 ms after fixation onset over central and parietal-occipital regions. Leveraging ROAMMs span-level MW annotations, we further show that brain-language alignment is systematically reduced during MW, an effect that was substantially larger for oscillatory (PSD) than for event-related (FRP) features. These findings demonstrate that modern language-model representations are reflected in EEG activity during naturalistic reading despite the modalitys inherent noise, and that fluctuations in attention constitute an underappreciated source of variability in brain-language encoding studies.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- All spectral frequencies of neural activity reveal semantic representation in the human anterior ventral temporal cortex 95%
- Tracing the neural trajectories of evidence accumulation and motor preparation processes during voluntary decisions 93%
- Representational similarity learning reveals a graded multi-dimensional semantic space in the human anterior temporal cortex 93%
Similar papers in this journal
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.