Brain alignment in deep neural networks emerges early and independently of object classification
Scholte, H. S.; Müller, N.; Smidi, J.; Groen, I. I. A.; van Gerven, M. A. J.
Show abstract
Deep convolutional neural networks are leading models of biological vision, largely because of their strong brain alignment: their features predict neural responses better than earlier models. Yet they are believed to recognize objects differently, relying on texture where humans rely on shape and failing on perturbations humans handle effortlessly. What then does alignment reflect? Tracking three architectures densely during training, we find that alignment with human fMRI, EEG, and macaque electrophysiology is already largely present at initialization, when networks classify at chance, and reaches a plateau within one to five epochs; thereafter it changes only modestly while classification accuracy continues to climb to 75%. Network lesioning shows that a kernel's contribution to alignment is essentially uncorrelated with its contribution to classification throughout training. Brain-network alignment therefore appears to reflect the structure of the visual environment both systems encode, rather than a shared solution for classification.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.