Subthalamic Nucleus Optogenetic Inhibition Bidirectionally Regulates Social Motivation According to Familiarity and Social Hierarchy

Background: Social behavior is a core component of mental health, and its disruption characterizes many neuropsychiatric disorders such as autism. Social operant paradigms enable the quantification of volitional aspects of social motivation and interactions. While sex differences have been shown to influence social motivation, factors such as familiarity and social hierarchy are also likely to play a critical role that remain insufficiently explored. In addition, the subthalamic nucleus (STN), traditionally studied in motor circuits, has emerged as an important regulator of reward and motivational processes and may contribute to social behavior processes. Methods: In this study, we examined the influence of peer familiarity (cagemate vs. stranger) and social hierarchy (dominant vs. subordinate) 1) on operant volitional social interaction using a fixed ratio 1 (FR1) and 2) on social motivation using a progressive ratio (PR) schedule of reinforcement in non-isolated rats. To assess the causal contribution of the STN, we used optogenetic photo-inhibition during both tasks in male rats. Results: Male rats displayed a reduction of social interest and motivation toward familiar peers, mainly driven by the social hierarchy, while female did not. STN photoinhibition in males abolished the familiarity-driven reduction under FR1 but decreased motivation independently of familiarity or hierarchy in PR. Conclusions: These findings highlight sex, familiarity, and hierarchy as key modulators of volitional social behavior and demonstrate a direct role of the STN in regulating social motivation. Together, they provide mechanistic insights into processes that may be disrupted in neuropsychiatric disorders characterized by social dysfunction.