Back

Directly interfacing brain and deep networks exposes non-hierarchical visual processing

Sexton, N. J.; Love, B. C.

2021-06-29 neuroscience
10.1101/2021.06.28.450213 bioRxiv
Show abstract

One reason the mammalian visual system is viewed as hierarchical, such that successive stages of processing contain ever higher-level information, is because of functional correspondences with deep convolutional neural networks (DCNNs). However, these correspondences between brain and model activity involve shared, not task-relevant, variance. We propose a stricter test of correspondence: If a DCNN layer corresponds to a brain region, then replacing model activity with brain activity should successfully drive the DCNNs object recognition decision. Using this approach on three datasets, we found all regions along the ventral visual stream best corresponded with later model layers, indicating all stages of processing contained higher-level information about object category. Time course analyses suggest long-range recurrent connections transmit object class information from late to early visual areas.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 0.4%
29.5%
Similar papers in this journal
2
Nature Communications
○ 5641 papers in training set
Top 15%
12.1%
Similar papers in this journal
3
PLOS Computational Biology
● 1863 papers in training set
Top 4%
9.2%
Similar papers in this journal
50% of probability mass above
4
Science Advances
● 1243 papers in training set
Top 2%
9.2%
Similar papers in this journal
5
Neuron
○ 337 papers in training set
Top 1%
6.4%
Similar papers in this journal
6
eLife
● 5828 papers in training set
Top 19%
6.4%
7
Nature Neuroscience
○ 252 papers in training set
Top 1%
6.4%
8
Science
● 477 papers in training set
Top 3%
3.1%
9
Nature
○ 645 papers in training set
Top 5%
3.1%
10
Current Biology
○ 665 papers in training set
Top 5%
2.3%
11
Cell Reports
○ 1498 papers in training set
Top 17%
2.0%
12
Communications Biology
○ 993 papers in training set
Top 24%
1.1%
13
Nature Human Behaviour
○ 95 papers in training set
Top 2%
1.1%
14
PLOS Biology
● 486 papers in training set
Top 9%
1.1%
15
Scientific Reports
○ 3612 papers in training set
Top 70%
0.9%
16
PNAS Nexus
◐ 159 papers in training set
Top 3%
0.9%
17
iScience
○ 1154 papers in training set
Top 39%
0.8%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.