Individual bounded rationality destabilizes cooperative dynamics in human-AI groups

Cooperative dynamics arise from interactions among individuals, including humans and artificial agents, yet it remains unclear how individual-level cognition and motion shape collective dynamics in such human-AI groups. We combined a dynamic cooperative task, in which one attacker was controlled by a human and two by heuristic artificial agents facing a deep reinforcement learning defender, with counterfactual simulations of artificial groups. Across nine task conditions, groups including a human consistently underperformed relative to groups composed solely of artificial agents. Humans exhibited restricted movement options and probabilistic rather than deterministic action choices, which shifted coordination dynamics towards less symmetric, less organized patterns and reduced performance for both individuals and groups. Embedding these boundedly rational movement and decision parameters into artificial agents reproduced human-like behavior and performance declines. These results demonstrate how limitations of a single individual can destabilize cooperative dynamics and provide insights into the design of robust human-AI systems.