Emergent predictability in microbial ecosystems

Microbial ecosystems exhibit a surprising amount of functionally relevant diversity at all levels of taxonomic resolution, presenting a significant challenge for most modeling frameworks. A long-standing hope of theoretical ecology is that some patterns might persist despite community complexity - or perhaps even emerge because of it. A deeper understanding of such "emergent simplicity" could enable new approaches for predicting the behaviors of the complex ecosystems in nature. However, the concept remains partly intuitive with no consistent definition, and most empirical examples described so far afford limited predictive power. Here, we propose an information-theoretic framework for defining and quantifying emergent simplicity in empirical data based on the ability of coarsened descriptions to predict community-level functional properties. Applying this framework to two published datasets, we demonstrate that all five properties measured across both experiments exhibit robust evidence of what we define as "emergent predictability": surprisingly, as community richness increases, simple compositional descriptions become more predictive. We show that standard theoretical models of high-diversity ecosystems fail to recapitulate this behavior. This is in contrast to simple self-averaging, which is well-understood and generic across models. We propose that, counterintuitively, emergent predictability arises when physiological or environmental feedbacks oppose statistical self-averaging along some axes of community variation. As a result, these axes of variation become increasingly predictive of community function at high richness. We demonstrate this mechanism in a minimal model, and argue that explaining and leveraging emergent predictability will require integrating large-N theoretical models with a minimal notion of physiology, which the dominant modeling frameworks currently omit.