Negative selection on baboon admixture is strongest on chromosome X

The six admixing baboon species offer a natural experiment to study negative selection on admixture and potentially genetic incompatibility in species with divergence resembling that between anatomically modern and extinct human subspecies such as Neanderthals. We analyze 156 high-coverage genomes sampled from seven olive and four yellow baboon populations shaped by admixture in two different reticulation events. In Tanzania, olive and yellow baboons admix despite 1.3 million years of divergence. In Ethiopia, olive baboons have invaded and displaced an ancient Hamadryas-like population separated by 0.6 million years, thus mirroring the displacement of Neanderthals by modern humans. Analyzing local ancestry across whole genome, we find evidence of negative selection on minor parent ancestry in Tanzanian yellow and olive baboon populations reaching far behind the hybrid zone and in Ethiopian olive baboons. We find that evidence of selection against admixture is up to seven times stronger on the X chromosome. The proportion of minor parent ancestry is substantially higher on the X chromosome in Ethiopian olive and yellow baboon populations, which displaced the populations now representing their minor parent ancestry. This additional MPA is concentrated in a few genomic regions with high frequency. This indicates that this original ancestry was retained by negative selection on the invading ancestry, suggesting that these loci play a role in emerging reproductive barriers in the Papio as well as the Homo genus.