SIGNAL: Dataset for Semantic and Inferred Grammar Neurological Analysis of Language
Komissarenko, A.; Voloshina, E.; Cheveleva, A.; Semenkov, I.; Serikov, O.; Ossadtchi, A.
Show abstract
Recently, the idea of comparison of models representations and human brain signals has been a topic of several works. Consequently, several datasets with text data and EEG representations have been published. However, most of the datasets are based on normal reading task with grammatical sentences. At the same time, in the interpretability studies of LLMs, more and more attention is paid to thoroughly designed linguistic tasks based on acceptability measures. In this paper, we present SIGNAL, a dataset for Semantic and Inferred Grammar Neurological Analysis of Language. Our dataset contains a group of sentences with a combination of a fully acceptable sentence and a grammatically or/and semantically incongruent sentences. The dataset has been approved by native speakers and later used for an EEG experiment. In total, our dataset contains recordings of 21 participants, each of whom read 600 sentences. In addition, we present a pilot study where we compare EEG analysis with simple probing experiments.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
Similar papers in this journal
- Neural correlates of object-extracted relative clause processing across English and Chinese 94%
- Tracking lexical and semantic prediction error underlying the N400 using artificial neural network models of sentence processing 94%
- Assessing the sensitivity of EEG-based frequency-tagging as a metric for statistical learning 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.