Data Representation Bias and Conditional Distribution Shift Drive Predictive Performance Disparities in Multi-Population Machine Learning

Machine learning frequently encounters challenges when applied to population-stratified datasets, where data representation bias and data distribution shifts can substantially impact model performance and generalizability across different subpopulations. These challenges are well illustrated in the context of polygenic prediction for diverse ancestry groups, though the underlying mechanisms are broadly applicable to machine learning with population-stratified data. Using synthetic genotype-phenotype datasets representing five continental populations, we evaluate three approaches for utilizing population-stratified data, mixture learning, independent learning, and transfer learning, to systematically investigate how data representation bias and distribution shifts influence polygenic across ancestry groups. Our results show that conditional distribution shifts, in combination with data representation bias, significantly influence machine learning model performance disparities across diverse populations and the effectiveness of transfer learning as a disparity mitigation strategy, while marginal distribution shifts have a limited effect. The joint effects of data representation bias and distribution shifts demonstrate distinct patterns under different multi-population machine learning approaches, providing important insights to inform the development of effective and equitable machine learning models for population-stratified data.