The genetic basis of adaptation through the evolution of mating systems

Although adaptation can be realized through the fixation of beneficial alleles that increase viability, many plant populations may adapt through the evolution of self-fertilization, especially when pollination becomes inefficient. However, the genetic basis of adaptation through the evolution of selfing remains unclear. Using population genetic models, I study adaptation through the fixation of alleles that increase the selfing rate (selfing modifiers) from new mutations or/and standing variation. For adaptive alleles unrelated to selfing, it is known that selfing promotes adaptation from a new mutation only when the beneficial alleles are recessive, and the probability of adaptation from standing variation is nearly independent of dominance, and always decreases with the selfing rate. In contrast, for adaptation through the evolution of selfing, when it occurs by fixation of a newly arisen mutation, a population that already has a high selfing rate may be more (less) likely to adapt than outcrossers even when the modifier is dominant (recessive) if the modifier is weakly (strongly) selected. Also, adaptation from standing variation is more likely through recessive modifier alleles, with the highest fixation probability found in partially selfing populations, but fixation is fastest when dominance is intermediate. When there are multiple modifiers, adaptation through new mutations is more likely when selfing is controlled by few large-effect rather than many slight-effect modifiers. This study suggests that to understand the genetic basis of adaptation, it is necessary to determine the ecological and genetic advantages of adaptive alleles. Significance statementThis study, by deriving the selective coefficient and effective population size, investigated the genetic basis of adaptation through fixation of modifier alleles that increase the selfing rate, which is shown to differ in several aspects from that through evolution of mating-unrelated alleles. Specifically, when adaptation is from new mutations, the dominance of a selfing modifier allele below which selfing increases the fixation probability depends on the strength of pollen limitation and pollen discounting. Adaptation from standing variation is more likely through recessive modifier alleles and in populations with an intermediate selfing rate. This work suggests it is important to have a mechanistic understanding of how adaptive alleles increase individual fitness in environment that is unfavorable to the population.