Beyond category-supervision: instance-level contrastive learning models predict human visual system responses to objects
Konkle, T.; Alvarez, G. A.
Show abstract
Anterior regions of the ventral visual stream have substantial information about object categories, prompting theories that category-level forces are critical for shaping visual representation. The strong correspondence between category-supervised deep neural networks and ventral stream representation supports this view, but does not provide a viable learning model, as these deepnets rely upon millions of labeled examples. Here we present a fully self-supervised model which instead learns to represent individual images, where views of the same image are embedded nearby in a low-dimensional feature space, distinctly from other recently encountered views. We find category information implicitly emerges in the feature space, and critically that these models achieve parity with category-supervised models in predicting the hierarchical structure of brain responses across the human ventral visual stream. These results provide computational support for learning instance-level representation as a viable goal of the ventral stream, offering an alternative to the category-based framework that has been dominant in visual cognitive neuroscience.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Increasing stimulus similarity drives nonmonotonic representational change in hippocampus 96%
- Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model 95%
- Stimulus representation in human frontal cortex supports flexible control in working memory 95%
Similar papers in this journal
- Neural mechanisms of credit assignment for inferred relationships in a structured world 95%
- Neural state space alignment for magnitude generalization in humans and recurrent networks 95%
- When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception 95%
Similar papers in this journal
- Sensory Processing and Categorization in Cortical and Deep Neural Networks 96%
- Understanding transformation tolerant visual object representations in the human brain and convolutional neural networks 96%
- Temporal evolution of Neural Codes: The Added Value of a Geometric Approach to Linear Coefficients 96%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.