GPT-2's activations predict the degree of semantic comprehension in the human brain
Caucheteux, C.; Gramfort, A.; King, J.-R.
Show abstract
Language transformers, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of deep translation, summarization and dialogue algorithms. However, whether these models encode information that relates to human comprehension remains controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but also predict the extent to which subjects understand narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear model to predict brain activity from GPT-2s activations, and correlate this mapping with subjects comprehension scores as assessed for each story. The results show that GPT-2s brain predictions significantly correlate with semantic comprehension. These effects are bilaterally distributed in the language network and peak with a correlation of R=0.50 in the angular gyrus. Overall, this study paves the way to model narrative comprehension in the brain through the lens of modern language algorithms.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
Similar papers in this journal
- The lexical categorization model: A computational model of left-ventral occipito-temporal cortex activation in visual word recognition 94%
- Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain 93%
- Task-evoked activity quenches neural correlations and variability across cortical areas 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.