Standardized human tests show that most vision models are face-blind
Dudipala, K. R.; Martinez-Addiego, F.; Rahnev, D.; Duchaine, B. C.; Ratan Murty, N. A.
Show abstract
Face recognition ability varies enormously across humans. Individuals with face-blindness (prosopagnosia) struggle to recognize even close family members while super-recognizers can identify strangers with exceptional accuracy. Where do ANNs fall within the human face recognition spectrum? Here, we administered the standardized tests used to characterize human face recognition ability to a diverse set of models, allowing us to contextualize a model's performance within the distribution of human behavior. We found that the majority 55% of models to be classified as face-blind and that even the best face-trained models do not cross the human threshold to be considered a super-recognizer. Interrogating the internal representations of these models showed that models that performed well on standardized face recognition tasks were more identity-selective and viewpoint-invariant. We also found that low-performing models did contain some identity information in independent representational subspaces. Removing viewpoint-dependent subspace improved face recognition abilities in 49 of the 53 models tested. Targeted unit ablations further identified opposing contributions, with viewpoint-dependent units disrupting identity coding and viewpoint-tolerant units supporting it. Together, our results show that most AI models are face-blind with worse face recognition ability than humans, and demonstrate how differences across AI models can be used generate testable hypotheses about the computational basis of human face recognition in humans which can then be probed in future studies.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression 93%
- Spatiotemporal structure and substates in emotional facial expressions 93%
- Orthogonal neural representations support perceptual judgements of natural stimuli 92%
Similar papers in this journal
Similar papers in this journal
- The face module emerged in a deep convolutional neural network selectively deprived of face experience 93%
- Implementation-independent representation for deep convolutional neural networks and humans in processing faces 93%
- Multidimensional face representation in deep convolutional neural network reveals the mechanism underlying AI racism 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.