Executive modulation of brain reward systems endows goals with value

People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from transient goals harnesses the same circuitry that supports learning from secondary reinforcers. Behavior and neuroimaging revealed that "one-shot" transient goals (abstract fractal images seen once) can act as a substitute for rewards during instrumental learning, and produce reliable reward-like signals in dopaminergic reward circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping these responses in reward circuits. These results suggest that learning from abstract goal outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow humans to perform reinforcement learning over flexible, arbitrarily abstract reward functions.