Illusions of Alignment Between Large Language Models and Brains Emerge From Fragile Methods and Overlooked Confounds
Hadidi, N.; Feghhi, E.; Song, B. H.; Blank, I. A.; Kao, J. C.
Show abstract
Emerging research seeks to draw neuroscientific insights from the neural predictivity of large language models (LLMs). However, as results continue to be generated at a rapid pace, there is a growing need for large-scale assessments of their robustness. Here, we analyze a wide range of models, methodological approaches, and neural datasets. We find that some methodological approaches, particularly the use of shuffled train-test splits, have led to many impactful yet unreliable findings, and that the method by which activations are extracted from LLMs can bias results to favor particular model classes. Moreover, we find that confounding variables, particularly positional signals and word rate, perform competitively with trained LLMs and fully account for the neural predictivity of untrained LLMs. In summary, our results suggest that theoretically interesting connections between LLMs and brains on three neural datasets are driven largely by fragile methodologies and overlooked confounds.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Representational similarity learning reveals a graded multi-dimensional semantic space in the human anterior temporal cortex 96%
- The Individualized Neural Tuning Model: Precise and generalizable cartography of functional architecture in individual brains 95%
- Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks 95%
Similar papers in this journal
Similar papers in this journal
- Artificial neural network language models predict human brain responses to language even after a developmentally realistic amount of training 97%
- Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network 97%
- Beyond Letters: Optimal Transport as a Model for Sub-Letter Orthographic Processing 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.