Reward-Bases: Dopaminergic Mechanisms for Adaptive Acquisition of Multiple Reward Types
Millidge, B. G.; Song, Y.; Lak, A.; Walton, M. E.; Bogacz, R.
Show abstract
Animals can adapt their preferences for different types for reward according to physiological state, such as hunger or thirst. To describe this ability, we propose a simple extension of temporal difference model that learns multiple values of each state according to different reward dimensions such as food or water. By weighting these learned values according to the current needs, behaviour may be flexibly adapted to present demands. Our model predicts that different dopamine neurons should be selective for different reward dimensions. We reanalysed data from primate dopamine neurons and observed that in addition to subjective value, dopamine neurons encode a gradient of reward dimensions; some neurons respond most to food rewards while the others respond more to fluids. Moreover, our model reproduces instant generalization to new physiological state seen in dopamine responses and in behaviour. Our results demonstrate how simple neural circuit can flexibly optimize behaviour according to animals needs.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.