Neuro-computational mechanisms of action-outcome learning under moral conflict

Predicting how actions result in conflicting outcomes for self and others is essential for social functioning. We tested whether Reinforcement Learning Theory captures how participants learn to choose between symbols that define a moral conflict between financial self-gain and other-pain. We tested whether choices are better explained by model-free learning (decisions based on combined historical values of past outcomes), or model-based learning (decisions based on the current value of separately expected outcomes) by including trials in which participants know that either self-gain or other-pain will not be delivered. Some participants favored options benefiting themselves, others, preventing other-pain. When removing the favored outcome, participants instantly altered their choices, suggesting model-based learning. Computational modelling confirmed choices were best described by model-based learning in which participants track expected values of self-gain and other-pain separately, with an individual valuation parameter capturing their relative weight. This valuation parameter predicted costly helping in an independent task. The expectations of self-gain and other-pain were also biased: the favoured outcome was associated with more differentiated symbol-outcome probability reports than the less favoured outcome. FMRI helped localize this bias: signals in the pain-observation network covaried with pain prediction errors without linear dependency on individual preferences, while the ventromedial prefrontal cortex contained separable signals covarying with pain prediction errors in ways that did and did not reflected individual preferences.