Quantifying the predictability of evolution by analysis of coalescent rate variation

We investigate the question of how to predict evolution from phylogenetic patterns, and we frame the problem in terms of predicting phylogenetic placement of new or circulating lineages. This leads us to develop a class of coalescent models that relax neutrality by allowing the rate of coalescence to vary as a continuous heritable trait. In this setting, each lineage has a relative propensity to coalesce, with coalescent odds ratios defined from the product of pairwise propensities. Estimated coalescent odds provide a statistic that captures variation in lineage growth and are informative about the strength of natural selection acting on individual lineages. A number of practical statistical methods are then developed: techniques to adjust for biased and non-uniform sampling; procedures to automatically calibrate hyperparameters governing the evolution of coalescent propensity; and, methods for clustering phylogenies into sets that delineate clades according to coalescent propensity. Simulations show sensitivity of these methods to detecting small selective effects acting on rare variants, and strong robustness to imbalanced sampling. We demonstrate these methods on a large set of Neisseria gonorrhoeae genomes, showing that lineages with high coalescent odds feature a unique antibiotic resistance pattern which presaged its subsequent expansion.