Strategic coexistence theory for evolutionary games

Evolutionary game theory and ecological coexistence theory both seek to predict the outcome of competition between biological entities, be they strategies or species, but the two fields have relied on largely separate approaches. Replicator equations provide a foundation for analyzing strategy competition, yet they do not explicitly separate the mechanisms that stabilize competition from those that equalize fitness differences between strategies. Here, we extend modern coexistence theory from community ecology to develop strategic coexistence theory (SCT), a framework for quantifying strategic niche and fitness differences between competing strategies. SCT recovers the classic classification of two-strategy games, distinguishing competitive exclusion, coexistence, and priority effects within a shared niche-fitness difference space. Applying SCT to five mechanisms for the evolution of cooperation further reveals that these mechanisms promote cooperation through distinct dynamical routes: kin selection, network reciprocity, and group selection primarily reduce fitness differences, whereas direct and indirect reciprocity destabilize competition and generate priority effects. Finally, applying SCT to microbial public-goods game shows that nonlinear microbial growth can both stabilize and equalize competition between cooperators and defectors, allowing coexistence. Together, these results show that SCT provides a complementary framework for comparing evolutionary games and teasing apart the coexistence mechanisms underlying strategy competition.