Known unknowns and model selection in ecological evidence synthesis

Quantitative evidence synthesis is a prominent path towards generality in ecology. Generality is typically discussed in terms of central tendencies, such as an average effect across a compilation of studies, and the role of heterogeneity for assessing generality is not as well developed. Heterogeneity examines the transferability of ecological effects across contexts, though between-study variance is typically assumed as constant (i.e., homoscedastic). Here, I use two case studies to show how relaxing the assumption of homoscedasticity and cross validation can combine to further the goals of evidence syntheses. First, I examine scale-dependent heterogeneity for a meta-analysis of plant native-exotic species richness relationships, quantifying the relationships among unexplained effect size variation, spatial grain and extent. Second, I examine relationships among patch size, study-level covariates and unexplained variation in species richness using a database of fragmentation studies. Heteroscedastic models quantify where effects can be transferred with more or less certainty, and provide new descriptions of transferability for both case studies. Cross validation can be applied to a single or multiple models, adapted to either the goal of intervention or generalization, and showed that assuming homoscedasticity can limit transferability for both case studies.