Hierarchical recurrent temporal prediction as a model of the mammalian dorsal visual pathway

A major goal of neuroscience is to identify whether there are generalized principles that can explain the diverse structures and functions of the brain. The principle of temporal prediction provides one approach, arguing that the sensory brain is optimized to represent stimulus features that efficiently predict the immediate future input. Previous work has demonstrated that feedforward hierarchical temporal prediction models can capture the tuning properties of neurons along the visual pathway, and that recurrent temporal prediction models can explain local functional connectivity within primary visual cortex. However, the visual system is also characterized by extensive inter-areal feedback recurrency, which existing models lack. We aimed to better account for the dynamic features of neurons in the visual cortex by incorporating both local recurrency and inter-areal feedback connectivity into a hierarchical temporal prediction model. The resulting model captured tuning for pattern motion, surround suppression and elements of inter-areal functional connectivity in visual cortex. Moreover, compared with several alternative normative models, the hierarchical recurrent temporal prediction model provided the closest fit to these tuning properties and was best able to explain the emergence of neuronal response properties across the visual cortex. Accordingly, temporal prediction can account for information processing throughout the visual pathway. Author summaryTask-driven artificial neural networks have provided valuable insights into the computational operations performed by neurons in the brain. In particular, models trained on various normative principles have been shown to capture some of the feature selectivity of neurons in the visual system. However, these models generally ignore the substantial role of inter-areal feedback connectivity in neural function. This study presents an alternative approach to understanding the neural computations underlying the functional organization of the visual system, by providing compelling evidence that networks trained for temporal prediction can explain the emergence of higher-level response properties across the visual cortex and relating these features to both local and inter-areal feedback connectivity.