Effective Ensemble of Deep Neural Networks Predicts Neural Responses to Naturalistic Videos
Yang, H.; Zhang, S.; Wu, Y.; Li, Y.; Gu, S.
Show abstract
This report provides a review of our submissions to the Algonauts Challenge 2021. In this challenge, neural responses in the visual cortex were recorded using functional neuroimaging when participants were watching naturalistic videos. The goal of the challenge is to develop voxel-wise encoding models which predict such neural signals based on the input videos. Here we built an ensemble of models that extract representations based on the input videos from 4 perspectives: image streams, motion, edges, and audio. We showed that adding new modules into the ensemble consistently improved our prediction performance. Our methods achieved state-of-the-art performance on both the mini track and the full track tasks.
Matching journals
The top 9 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Alignment massive of auditory individual artificial networks with fMRI brain data leads to generalizable improvements in brain encoding and downstream tasks 96%
- NeuroConText: Contrastive Learning for Neuroscience Meta-Analysis with Rich Text Representation 94%
- Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks 93%
Similar papers in this journal
Similar papers in this journal
- THINGSvision: a Python toolbox for streamlining the extraction of activations from deep neural networks 95%
- Contrastive Self-supervised Learning for Neurodegenerative Disorder Classification 93%
- Extended Invariant Information Clustering is Effective for Leave-One- Site-Out Cross-Validation in Resting State Functional Connectivity Modelling 92%
Similar papers in this journal
- PsychRNN: An Accessible and Flexible Python Package for Training Recurrent Neural Network Models on Cognitive Tasks 92%
- RealtimeDecoder: A fast software module for online clusterless decoding 92%
- SaLSa: a combinatory approach of semi-automatic labeling and long short-term memory to classify behavioral syllables 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.