Recent history attracts and repels perceptual decisions depending on surprise

Past sensory experience shapes our perceptual decision-making in the now. Popular models frame perceptual decisions as either attracted towards or repelled away from recent sensory information, but it is unclear when and why these distinct effects emerge. We here ask whether effects turn from attractive to repulsive depending on the level of surprise elicited by the precision-weighted discrepancy between past and present sensory states. This model is based upon the idea that attraction is adaptive for optimizing efficiency and accuracy when discrepancies are small, because they likely reflect sensory noise rather than real change in the environment. In contrast, repulsion may reflect the upweighting of counterfactual evidence when discrepancies are large because they more likely signal the need for model updating. We test this model on a large amount of recently-collated trial-by-trial serial dependence data and consistently find support for it across the dataset, participant, and trial-by-trial level. Specifically, serial dependence effects are attractive at low discrepancies between past and current sensory states but turn repulsive when discrepancies are larger. Higher sensory precision is found to accelerate this flip by reducing the modal discrepancy threshold required to trigger repulsion effects. We discuss how these findings necessitate extending existing theories of serial dependence, and how they may resolve conflicts in the broader predictive processing, learning and perception literatures.