Learning using switching synaptic plasticity rules

1Hebbian-like learning has been repeatedly confirmed experimentally, yet computational models usually require non-local signals, such as backpropagating errors, to solve complex cognitive tasks. Recent cortical electron microscopy data suggests a model where synapses follow different plasticity rules depending on whether they are in a large or small state. Large synapses often include a spine apparatus, a calcium reservoir that influences synaptic dynamics and can alter rules of synaptic plasticity. Here, we test the computational outcomes of networks which compute with synapses switching their plasticity rules based on their strength. We designed a recurrent neural network (RNN) with synapses that switch between two learning rules: a Hebbian-like rule for weak synapses and a credit-assignment rule (backpropagation, BP) for strong synapses. We found that our plasticity-switching RNN (psRNN) learns cognitive tasks (e.g. working memory) in fewer trials than BP-only RNNs, despite fewer synapses using credit assignment. Three mechanisms underlie this advantage: BP samples multiple parameter configurations for better gradient estimation, Hebbian plasticity creates a dynamic task-relevant initialization, and the switching mechanism prevents Hebbian synapses from growing into unfavorable parameter regions. The interaction between rules also produces lower-rank, more feedforward recurrent structure, providing testable connectomic predictions and a framework for reconciling local learning rules observed in the brain with non-local rules used in computational models. Significance statementCredit assignment is essential for learning in biological and artificial systems. Credit assignment can be achieved via complex synaptic plasticity rules, yet biological data points to plasticity rules at most synapses being simpler. Here we ask if networks with a combination of simple and complex plasticity rules could solve biorealistic tasks. Surprisingly, replacing the computationally expensive error-based learning with simple, experience-dependent, changes in a large portion of synapses improves learning in our models. This substitution changes several computational properties of the model, offering new hypotheses for computing in the brain.