Cognition

Humans rely on multiple sensory modalities, such as vision, audition, and touch, to perceive materials in everyday life. Previous research shows that multisensory perception leads to facilitation, yet the mechanisms responsible for this facilitation remain poorly understood. One potential mechanism is crossmodal prediction, whereby input from one modality generates predictions about another. While substantial research on multisensory facilitation has focused on bottom-up processes, such as spatial, temporal, and semantic congruency, the role of crossmodal predictions, particularly in material perception, has received little attention. To address this gap, we conducted two experiments, a reaction time task and a material rating task, in which participants viewed computer-generated animations of familiar objects being dropped to the ground. The paradigm exploited the natural temporal structure of impact events: pre-impact visual appearance provides information about an objects material and therefore can generate expectations about the forthcoming impact sound. Critically, participants saw the event only until before the impact, after which the video was masked. Thus, vision and audition were temporally aligned but not presented concurrently, allowing us to isolate the influence of visually driven expectations on the incoming auditory information without a bottom-up conflict. In some trials, the sound matched the expected material, but in a subset, it was incongruent, violating expectations elicited by the preceding visual information. Across both experiments, participants took longer to respond on incongruent than congruent trials, suggesting increased processing demands. In the rating task, incongruent trials also shifted material judgments, such that ratings reflected a weighted combination of incoming auditory information and visually driven predictions, with large individual differences in relative cue weighting. These findings suggest that priors on material properties from one modality, specifically vision, not only establish high-level expectations within the modality about an objects future state, but also extend across modalities.

Attentional control balances proactive suppression of predictable distractors with reactive suppression of unexpected ones. Yet, how internal states such as alertness shape this balance is unclear. Using pupillometry and eye tracking across two probability-cueing experiments (conducted in 2024) with varying distractor prevalence, we distinguished tonic (baseline pupil size across blocks) from trial-level pupil size fluctuations (trial-by-trial residual variability in pre-stimulus pupil size). With moderate prevalence, suppression of frequent-region distractors developed gradually, whereas high prevalence induced near-immediate suppression. Behavioral measures (e.g., reaction times) were closely linked to tonic and trial-level pupil size fluctuations. Critically, both alertness components jointly influenced control: during early learning, heightened trial-level pupil size increased distractor capture and reduced target fixations, whereas later on, suppression shifted to a proactive mode resilient to trial-level fluctuations. Under high prevalence, this shift occurred faster. Notably, higher trial-level pupil size generally accelerated first target selection. These findings show that tonic alertness and trial-level alertness fluctuations dynamically regulate reactive and proactive control during statistical learning. Impact StatementThis study shows that people become better at ignoring predictable distractions over time, but that this improvement depends not only on what they have learned about the task environment, but also on their current level of alertness. By combining eye tracking and pupil measures, we found that temporary increases in alertness can sometimes help people orient more quickly to relevant information, yet during earlier stages of learning they can also make attention more vulnerable to distracting events. These findings suggest that successful focus in complex environments depends on a dynamic interplay between learned expectations and moment-to-moment fluctuations in mental state, with implications for understanding sustained attention in settings such as monitoring, driving, and other tasks that require people to stay engaged while resisting distraction.

Building upon foundational psychological theories of event segmentation, this study addresses the limitation of overreliance on temporal boundaries as the primary segmentation criterion. Drawing on two experiments of direct and indirect causation in Mandarin Chinese, this study demonstrates how cognitive segmentation granularity and semantic integration jointly shape syntactic encoding. Results reveal distinct event encoding patterns for direct and indirect causation: coarse-grained segmentation leads to compact syntactic structures (e.g., verb-resultatives), while fine-grained segmentation yields varied multi-clausal expressions. Chinese speakers update event models via prediction errors of intentionality and protagonists, and tend to establish event boundaries at goal-relevant action endpoints when construing causal chains. These conceptual dimensions exert a modulating influence on both event segmentation and semantic integration. We propose a triad model integrating event segmentation, semantic integration, and linguistic specificity, providing a unified framework for elucidating the mind-language interface in conceptual construction and event coding of causation.

During sideways movement of an observer, optic flow parsing - in which an objects speed in the world is extracted from all the other visual movement present in the scene, self-generated and otherwise - has been shown to be incomplete, leading to biases in speed perception, particularly when object and observer are moving in opposite directions. Here, we assess how judgements about the speed of objects moving in depth (judged relative to the world) towards or away from an observer (6 m/s) are affected by simultaneous movement of the observer either in the same or opposite direction as the object. In a virtual reality display, participants (n = 25) viewed a sphere simulated as moving in a corridor either while they were stationary or during visually simulated self-motion in the same or opposite direction as the object. They judged the spheres movement relative to the world by comparing its motion to a probe sphere that travelled laterally across the corridor in front of them. In a second experiment (n = 28) participants performed the same task but during faster self-motion (10 m/s). The second cohort also judged the direction in which the object was perceived to move during the same combinations of self and object speeds. Object speed was overestimated when the object travelled in the direction opposite to the observer compared to how objects motion was judged when the observer was stationary. However, object speed was also overestimated during self-motion in the same direction as the object where participants were also much more likely to misjudge the direction of motion of the object. Precision of judgements was lower when self-motion was simulated than it was for stationary observers. A simple arithmetic model of flow parsing fails to capture these results satisfactorily, suggesting that different mechanisms may be at play when the observer travels in the same direction as a moving object and is vulnerable to misperceiving its direction of travel.

Sentence comprehension requires the incremental construction of syntactic structure and semantic interpretation. Prior neural work (Nelson et al., 2017) identified key neural events at major phrase boundaries during sentence comprehension. To investigate a behavioral correlation of these processes, we used self-paced reading to examine the impact of syntactic phase boundaries, semantic congruence, and sentence structure on sentence processing. Participants read object-relative, subject-relative, and canonical control sentences one word at a time and a subsequent comprehension task. Reading times were analyzed relative to phrase boundaries, node-closing operations, and semantic congruence. Object-relative sentences produced the greatest processing difficulty, demonstrated by increased reading times and decreased comprehension accuracy. Reading times peaked at the phrase boundaries, indicating that processing costs are tied to constituent completion rather than individual lexical categories. Reading times also increased with the number of syntactic constituents completed at a phrase boundary. Agent-patient semantic congruence produced its largest effects in object-relative sentences, suggesting that semantic information interacts with syntactic computations when processing demands are greatest. These findings demonstrate that self-paced reading is sensitive to the incremental processing associated with syntactic constituent completion. Processing costs are tied more closely to phrase completion than to individual lexical categories, scale with the amount of syntactic structure completed at a boundary and interact with agent-patient semantic interpretation during object-relative sentence comprehension. Together, these findings support a view of sentence comprehension in which syntactic structure building and semantic interpretation proceed incrementally and interact continuously throughout online language processing.

Interception of moving objects requires the nervous system to compensate for sensory delays and uncertainty, yet how behavior is controlled remains debated. Key questions concern whether predictive processes play any role at all and, if so, whether they rely on simple motion extrapolation or incorporate internalized physical priors, such as gravity. Another open question is whether observers adopt a single control strategy or flexibly switch between predictive and reactive control - or between different predictive strategies - depending on task demands. To address these questions, we developed a virtual interception task in which participants intercepted moving targets under systematically varied conditions. We manipulated gravity (1g vs. 0g), visual availability (occluded vs. non-occluded), target velocity, and the initial spatial configuration of the ball and paddle (same vs. opposite side). Results indicate that interception is supported by predictive mechanisms across conditions. Behavioral patterns during occluded 0g trials suggest that participants extrapolate target motion using expectations consistent with gravity. Target velocity, visual occlusion, and task geometry modulated movement strategies, indicating that predictive control is flexibly adapted to task demands. These findings support the view that interception relies on predictive internal models incorporating structured physical priors while revealing flexible, context-dependent adaptations to sensory and task constraints.

Humans and other animals often act in environments that are at least partly familiar, where aspects of the spatial layout are known. Although such knowledge is known to support navigation and spatial cognition, its role in the online control of action remains unclear. We investigated whether drivers use knowledge of road layout to guide steering in high and low visibility and, if so, the form of such knowledge. In two simulated driving experiments (total N = 90), participants repeatedly drove winding roads containing segments with and without fog. Drivers who repeatedly experienced the same road exhibited more stable steering and lane positioning than drivers encountering novel roads, but only when visibility was reduced. These advantages were accompanied by superior performance on post-tests assessing knowledge of road geometry. We next examined the form of such knowledge by dissociating global knowledge of road layout from local associations between landmarks and road segments. Disrupting landmark-road segment associations produced the largest impairment in steering performance. The benefits of prior experience were largely preserved when road-segment order was scrambled but landmark associations remained intact. These findings show that spatial knowledge can support moment-to-moment steering control when visibility is reduced. Rather than relying on a globally coherent representation of the environment, drivers use local associations between landmarks and upcoming road geometry to anticipate future demands. More broadly, the results elucidate how familiarity with environmental structure contributes to the control of action when visual information is degraded, revealing a close interplay between spatial knowledge and visual control. Significance StatementPeople routinely act within surroundings they have encountered before, from commuting on the same streets to walking familiar hallways. Whether the spatial knowledge acquired from such experience actually shapes online visual control remains an open question. Using a simulated driving task, we show that familiarity with a road improves steering stability specifically when visibility is reduced, and that this benefit depends on learned associations between landmarks and upcoming road geometry rather than a global cognitive map. The results indicate that spatial knowledge plays a key role in moment-to-moment control, letting drivers anticipate road segments they cannot yet see. Unfamiliar roads and impaired spatial learning may compound the risks of poor visibility, suggesting a role for driver-assistance systems that leverage landmarks.

Current debates regarding the relative contribution of saliency versus semantics to gaze control often rely on comparing the predictive power of saliency and meaning maps. We argue that such indirect, global approaches are fundamentally limited because fixations arise from heterogeneous, local causes that are conflated in whole-scene comparisons. To substantiate this claim, we used a direct method where participants explicitly identified the reasons for fixation at specific clusters of high fixation density, distinguishing between low-level saliency and various semantic categories, as well as the most important one. The obtained judgments revealed that multiple factors contribute simultaneously to gaze control. Although their influence varied across fixation clusters, semantics generally dominated saliency. Notably, abstract semantic categories, particularly "unknown/unusual," proved important, highlighting the role of prior knowledge and novelty besides personal relevance in guiding attention. To interpret these findings in the context of existing models, we propose a framework distinguishing between processes highlighting interesting locations in the image from a sampling strategy translating this information into scanpaths. Within this framework, classic saliency and meaning maps are viewed as restricted inputs to the strategy, whereas deep learning-based models (e.g., DeepGaze IIE) are more general and may also implicitly encode aspects of the strategy itself. Consistent with this, we found that the predictive performance of DeepGaze IIE varied less significantly with the specific reasons for fixation than that of classic saliency and meaning map approaches.

Bayesian accounts of autism propose that perception is less influenced by prior expectations and more strongly driven by incoming sensory information in autistic than non-autistic individuals, with this altered balance cascading through the cognitive hierarchy to also influence higher cognitive functions. However, empirical support for these accounts remains mixed. Previous work has mostly tested these ideas in the context of objective environmental statistics, but recent work suggests that it may be subjective experience of structure, rather than structure itself, that shapes perceptual processing. Characterising these subjective experiences in autistic individuals is therefore crucial for understanding predictive processing in autism. In the present study, we thus examined subjective experience of statistical structure in autistic and non-autistic adults and tested how this experience relates to perceptual decisions. Participants were exposed to statistical regularities between action cues and visual stimuli (shapes), and we measured their speed and accuracy in reporting which shape they had seen. At the end of the study, participants were asked to estimate the probability and rate their surprise for each action-shape combination. Autistic and non-autistic participants showed similar subjective probability and surprise ratings and a comparable relationship between these ratings and perceptual decisions. Across participants, subjective ratings explained perceptual decisions better than objective structure. Together, these findings show that autistic and non-autistic adults experience statistical structure similarly, with these experiences exerting a similar influence on perceptual decisions - therefore suggesting that subjective experience plays a comparable role in predictive processing in autistic and non-autistic adults.

Children often outperform adults in probabilistic statistical learning tasks, yet the mechanisms underlying this developmental advantage remain poorly understood. Here, we used eye-tracking measures of belief updating to examine how children and adults acquire and update predictions in a probabilistic sequence-learning task. Using the standard (oculomotor) reaction time measure, children showed stronger statistical learning than adults, replicating previous behavioral findings while revealing a more detailed profile of developmental differences in statistical learning. Critically, children updated their predictions more frequently: they were less likely to repeat previous predictions and more likely to shift their expectations in response to new input. Adults, in contrast, showed greater persistence, tending to maintain prior predictions even when those predictions were inconsistent with the underlying statistical structure. Despite these pronounced differences in updating behavior, the processing and use of prediction errors were remarkably similar across age groups. These findings indicate that developmental differences in statistical learning do not primarily arise from how prediction errors are computed, but rather from how prior beliefs and incoming information are weighted during belief updating. Children's enhanced learning may therefore reflect reduced reliance on stable priors and greater sensitivity to current sensory evidence, supporting a more exploratory learning strategy. Adults, by contrast, appear to favor an exploitative strategy that stabilizes existing predictions but reduces flexibility in probabilistic environments. More broadly, the results suggest that developmental changes in statistical learning may reflect age-related differences in how readily learners revise their predictions in response to incoming evidence. By integrating sensitive oculomotor measures with analyses that probe the mechanisms underlying belief updating, the present study provides a more fine-grained account of how predictive learning changes across development and offers a framework for reconciling previously inconsistent developmental findings in statistical learning.

The ability to orient or attend to sensory events is generally greater in response to visual and auditory cues occurring together than in response to single-modality cues occurring alone. In such cases the perceptual fusion of cross-modal stimuli (multisensory integration) depends on low-level features (e.g., location, intensity) and follows well established principles. However, less is known about multisensory integration mechanisms when behavioral responses are less direct and require top-down control. Here we investigate this in human participants using an urgent multisensory choice task that effectively dissociates stimulus-driven and goal-driven contributions to performance based on their distinct temporal signatures. Task conditions varied the modality of the cues (auditory, visual, or both), their location (left or right), and the rule defining the correct choice (look toward or away from a given cue). When spatially coincident cues were associated with the same response rule ("look away"), we observed multisensory enhancement and performance remained close to a statistical expectation as the choice process unfolded. However, when spatially disparate cues were associated with different rules but the same target, one cue dominated performance and the other produced crossmodal capture, i.e., low-level competition. The results indicate that the efficacy of multisensory integration is dictated by the stimulus-and goal-driven signals produced by each cue, with all four factors rapidly interacting in accordance to the dynamics of spatial attention. Significance StatementAuditory and visual stimuli located near each other in space and time are typically bound into a single sensory percept that draws attention most effectively. However, it is unclear whether such "multisensory integration" occurs during behaviors that go beyond directly attending or orienting to cue stimuli and require top-down control. We investigated this using a novel task design with which stimulus-driven and goal-driven contributions to performance can be accurately identified. We found that multisensory enhancement depends not so much on the complexity of the requested cue-response associations, but rather on the timing and alignment of the stimulus-and goal-driven signals derived from each cue (auditory and visual) -- similar to the way that such signals dictate the allocation of spatial attention.

The ability to mentally rotate objects is a fundamental feature of human cognition, and humans can use this ability to make choices about objects based on their geometry. However, remarkably little is known about how such choices are reached, and what sort of visual information might facilitate them. We devised an experiment where participants had to mentally simulate an object's rotation to choose which of two objects was better for a subsequent task based on its shape alone. We also tracked their gaze while they made their choice, to see which visual information they were using to facilitate this mental simulation. We found that participants were consistently able to choose the most suitable object for the task, and, remarkably, the visual information they sampled was directly linked to their choices. Put simply, participants made better choices when they looked at more informative regions of the objects, and participants who sampled regions that were better for facilitating mental simulation made better choices overall. These findings reveal a direct link between fixations, simulation, and decision-making, suggesting that to perform any fine-grained mental simulation people need to direct their gaze at specific, informative points of an object to simulate its two-dimensional proximal image displacement.

Social decision-making depends on inferring others hidden preferences from observable behavior. Yet it remains unclear how humans combine choices with process cues such as response times and gaze when learning about others in real-time interaction. Here we combine a novel multi-attribute bargaining task with eye-tracking and show that multiple decision-process cues support preference inference. Across 75 buyer-seller dyads, buyers acceptance rates tracked offer utility, rejection speed reflected decision confidence, and first fixations preferentially targeted the highest-weighted attribute. Sellers adapted subsequent offers using choices, response times, and, when available, gaze cues. A hierarchical inference and choice model suggested that sellers balanced expected utility with expected information gain and updated their beliefs in a Bayesian manner. Although gaze access did not improve overall performance, it changed how sellers used attentional information. These findings shed light on how humans infer others hidden preferences from decision dynamics in real-time social interaction.

Eye contact is one of the most potent social signals, but it remains unclear how little visual input is sufficient to register direct gaze, and whether such registration requires conscious awareness. Here, we used a custom tachistoscope to present faces for only 1-5 ms, asking whether gaze direction can shape perception at the very limits of vision. Direct-gaze faces required less visual input to localise than averted-gaze faces and produced higher localisation sensitivity from 3 ms onwards. Crucially, this advantage emerged before participants could explicitly categorise gaze direction, and before perceptual awareness ratings showed metacognitive access to the information driving localisation, both of which appeared only at 4-5 ms. Information-theoretic analyses confirmed that localisation responses carried stimulus-location information before explicit reports and awareness ratings became informative. A second experiment showed that the earliest eye-contact advantage depended mainly on low spatial frequencies, indicating a role for coarse visual information. Finally, autistic traits were associated with a reduced localisation advantage for direct gaze. These findings show that eye contact can influence perceptual processing within just a few milliseconds, with less visual stimulation than is required for conscious access to gaze direction.

Empathy enables individuals to attune to others' experiences through shared affective, sensorimotor, and neural representations, but its influence on higher-level decision-making and social alignment remains unknown. We examined whether empathy promotes social conformity and whether this effect depends on physiological arousal, visual attention, and socio-emotional traits. Seventy-three participants completed an empathy-for-pain task followed by a modified random-dot task in which they interacted with previously seen empathy targets, while physiological signals and eye movements were recorded. Drift-diffusion modeling showed that feedback from empathy targets increased conformity, reflected in stronger decision bias toward targets, faster evidence accumulation, and reduced decision conservatism. Visual attention to the eyes or mouth was linked to greater conformity depending on the context. Heart rate variability was unrelated to conformity, whereas phasic skin conductance was associated with smaller bias changes and greater conservatism. Alexithymia and autistic traits further shaped the empathy-conformity relationships. These findings suggest that empathic shared representation extends beyond affective resonance to influence social alignment in decision-making.

Learning is typically understood as a process driven by prediction errors, when outcomes differ from expectations. Yet it remains unclear whether outcomes that perfectly match expectations are psychologically and computationally meaningful. Here, we tested whether zero prediction errors shape affect, belief updating, and neural feedback processing in human reinforcement learning. Participants repeatedly predicted rewards in environments varying in uncertainty, with a subset of trial outcomes manipulated to exactly match their predictions. Zero prediction errors produced the highest momentary happiness, and computational modeling showed that behavior was best explained by a model in which zero prediction errors induce a distinct latent belief state that guides subsequent updating, particularly under higher uncertainty and in individuals with greater intolerance of uncertainty. Outcome-locked EEG analyses further showed that zero prediction errors elicited distinct P3-like responses, with residual neural activity predicting attenuated updating after zero prediction errors but enhanced updating after standard prediction errors. These findings suggest that perfect predictions are not neutral, but informative events that actively shape affect, behavior, and neural feedback processing.

Language production requires the coordination of multiple cognitive processes. The ability to anticipate and override a habitual response in favor of a contextually-appropriate response are key subprocesses of cognitive control which enable speakers to communicate effectively. Word retrieval involves the co-activation of semantically related alternatives from which the speaker must select the appropriate target representation. Although cognitive control mechanisms have been proposed to contribute to resolving semantic interference during language production, the nature of these control processes remain unclear. Studies investigating the temporal dynamics of cognitive control during decision making tasks have led to a distinction between two operating processes: proactive control, initiated prior to the occurrence of conflict, and reactive control recruited after conflict is detected. We investigated the roles of proactive and reactive control in resolving interference between competing linguistic representations during word retrieval. We analyzed congruency sequence effects combined with delta-plot distributional analyses to dissociate potential adjustments in proactive versus reactive cognitive control in a picture-naming task manipulating semantic context compared to a minimally-linguistic Stroop-like paradigm. Reaction time distributional properties following semantically related trials revealed the engagement of proactive control in semantic interference resolution during word retrieval in the PWI task. In contrast, reactive inhibitory control was engaged in resolving semantic interference following low conflict trials. This distinction was not present in the minimally-linguistic task, which did not appear to engage adaptive control to the same extent. These findings demonstrate that both proactive and reactive cognitive control mechanisms contribute to language production, and are engaged dynamically, adjusting trial-by-trial to resolve semantic interference during word retrieval. In addition, our study provides important insight into the comparison of language with other cognitive domains and positions linguistic paradigms as being instrumental in the study of cognitive control dynamics.

Transitive inference (TI) is the ability to reason about transitive relationships in an ordered set of items (e.g., if A>B and B>C, then A>C). TI is widely held to depend on a linear representation of the serial (rank) order of those items. By what computational mechanism is such an ordering constructed during learning, and how is it used to make choices that obey transitivity? Here we take a minimalist approach, applying least-squares estimation (LSE) to a serial learning task commonly used to test TI in humans and animals. In this formulation, LSE computes a linear classifier that maps task conditions onto behavioral outcomes. This algorithm makes no explicit assumptions about transitivity or serial order, yet it reproduces key empirical features of TI; namely, the ability to generalize beyond the training set, and a symbolic distance effect (SDE) in performance accuracy. Applying the classifier to individual items produces an internally ordered representation of rank from which both generalization and the SDE naturally emerge. The approach also yields a decision mechanism, in the form of a differencing operation, for selecting the correct item from any pair. These findings reframe TI as a linear classification problem, challenging conventional assumptions about the cognitive mechanisms required for transitive reasoning.

Speech comprehension in noisy environments often requires cognitive effort, but listeners may disengage when comprehension becomes impossible. Eye movements have recently emerged as a promising new measure of listening effort, but it remains unclear whether eye movements are sensitive to the full effort profile across easy, difficult, and impossible speech comprehension. Across four experiments, participants listened to sentences at easy, difficult, and impossible levels of multi-talker background babble while pupil size and eye movements were recorded. Pupil size generally followed the expected inverted u-shaped effort profile: low for easy speech, maximal for difficult but still intelligible speech and lower again for impossible speech, although this pattern partly reflected sustained, condition-specific differences and not only sentence-evoked responses. Gaze dispersion - measuring the spread of eye movements - decreased with high temporal selectivity during difficult relative to easy and impossible speech, indicating reduced eye movements during active, effortful listening. However, gaze dispersion was also lower, but less temporally selective, during impossible compared to easy listening, especially in non-baseline-corrected analyses, suggesting that reduced eye movements do not index listening effort uniquely. Instead, eye movements appear to reflect both attentional engagement during difficult listening and disengagement or inward attention when meaningful listening is no longer possible. These findings indicate that pupil size and eye movements provide complementary indices of listening-related cognition, and highlight the integration of listening, cognition, and motor systems.

During active vision, the brain must coordinate where to move the eyes with predictions about upcoming sensory input. Before each saccade, perception is enhanced at the upcoming fixation location, but whether this enhancement depends on expectations about target features remains unknown. Here, participants prepared a saccade to a cued location while reporting the presence and orientation of a brief visual target that appeared either at the saccade goal or at the opposite location. Feature expectation was manipulated across blocks by varying the probability of the two target orientations. Perceptual sensitivity (d') increased when targets were presented at the saccade goal, consistent with presaccadic enhancement, and was also higher for less expected features. However, these effects were independent: feature probability did not alter the magnitude of presaccadic enhancement. Moreover, presaccadic enhancement increased near saccade onset, whereas the advantage for less expected features weakened as movement onset approached. Saccade latency revealed a contrasting pattern. Visual targets presented at the saccade goal delayed movement initiation. This delay depended on feature probability, with longer latencies for unexpected than for expected features only when saccades were directed towards the target. This location-specific effect persisted after accounting for perceptual report, and the latency cost for unexpected features was reproduced in a follow-up experiment. Together, these findings show that feature probability enhanced sensitivity to unexpected information independently of presaccadic enhancement, while selectively delaying saccade initiation towards targets with unexpected features. This dissociation suggests that feature expectation modulates perception and action through functionally distinct forms of visual processing.