Mapping concept and relational semantic representation in the brain using large language models
Bogdan, P. C.; Cabeza, R.; Davis, S. W.
Show abstract
How the brain organizes semantic information is one of the most challenging and expansive questions in cognitive neuroscience. To shed light on this issue, prior studies have attempted to decode how the brain represents concepts. We instead examined how relational information is encoded, which we pursued by submitting texts to a contemporary large language model and extracting relational embeddings from the model. Using behavioral data (N = 636), we found these embeddings capture independent information about scenes and objects, along with relational information on their semantic links. Turning to fMRI data (N = 60), we leveraged these embeddings for representational similarity analysis: The occipitotemporal cortex represents concepts in isolation, whereas the dorsolateral prefrontal cortex and basal ganglia principally encode relational information. Relational coding within prefrontal and striatal areas also tracks how participants reason about scenes and objects. Altogether, this research maps how information progresses from concept-level to integrative forms and how this translates into behavior.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Object representations in the human brain reflect the co-occurrence statistics of vision and language 97%
- Ventromedial prefrontal cortex compression during concept learning 96%
- Human brain representations of internally generated outcomes of approximate calculation revealed by ultra-high-field brain imaging 96%
Similar papers in this journal
- A brain-based universal measure of attention: predicting task-general and task-specific attention performance and their underlying neural mechanisms from task and resting state fMRI 95%
- Shared striatal activity in decisions to satisfy curiosity and hunger at the risk of electric shocks 94%
- Brain network dynamics predict moments of surprise across contexts 94%
Similar papers in this journal
- Widespread, perception-related information in the human brain scales with levels of consciousness 95%
- Behavior can be decoded across the cortex when individual differences are considered 95%
- Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks 95%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.