Structured Multimodal Deep Learning improves Genomic Prediction in Future Environments

The development of prediction models for phenotypes as functions of genetics and environmental inputs is a long-standing challenge in genetics and plant breeding. Deep neural networks form a promising approach to this task, due to their capacity to approximate nonlinear biological processes. Despite initial expectations, recent studies have found deep neural networks under-performing in comparison to linear methods, even for continent-scale trial datasets. We attribute this to several failure modes of deep learning, including greedy learning, the tendency of deep neural networks to over-emphasize a single type of input data. As a solution, we present the Structured Interaction Neural Network (SINN), which combines statistical decomposition of genetic, environmental and interaction effects with deep neural networks. SINN dissects phenotype prediction into isolated component modeling tasks, revealing poor generalization of learned representations to new environments to be the main limitation for both prediction of genotype-by-environment interactions and yield prediction overall. By balancing model complexity and regularization per component, we reach competitive performance on yield prediction in the next cycle of a maize multi-environment trial, including both new genotypes and new environmental conditions. SINN achieved a higher accuracy (0.63) than BLUP-based methods (0.43) and a neural network from previous literature (0.48), and surpassed the top-performing models in a public benchmark dataset with a lower RMSE (2.41 Mg/ha versus 2.46 Mg/ha, with mean yield of 9.51 Mg/ha) and higher genetic correlation (0.38 versus 0.36). By combining statistical genetics and modern deep learning, SINN enables accurate, modular and scalable genomic prediction in new environments.