A role for acetylcholine in reinforcement learning and decision making under uncertainty

The neuromodulator acetylcholine has been suggested to govern learning under uncertainty. Here, we investigated the role of muscarinic receptors in reward-guided learning and decision making under different degrees of uncertainty. We administered the muscarinic M1 antagonist biperiden (4 mg) to healthy male participants (n = 43) in a within-subjects, placebo-controlled design. Participants performed two tasks that both involved choices between options characterized by two attributes, reward probability and magnitude. In the gambling task, both attributes were explicitly provided, whereas in the learning task, reward probabilities had to be inferred from past experience. In addition, uncertainty was manipulated within the learning task by inclusion of a stable phase with fixed reward contingencies, and a volatile phase with frequent contingency reversals. We show that biperiden did not affect decision making in the gambling task, where no learning was required. However, in the learning task, biperiden reduced the sensitivity to the learnt reward probabilities. Notably, this was primarily driven by choices under higher uncertainty in the volatile phase. Using reinforcement learning models, we reveal that the change in behaviour was caused by noisier estimates of probabilities resulting from maladaptively increased learning rates under biperiden. Together, these findings suggest that muscarinic acetylcholine transmission is involved in controlling learning in highly uncertain contexts, when the demand for carefully calibrated adjustments is highest.