Empirical decomposition of the explained variation in the variance components form of the mixed model

The coefficient of determination is a standard characteristic in linear models. It is widely used to assess the proportion of variation explained, to determine the goodness-of-fit and to compare models with different covariates. However, there has not been an agreement on a similar quantity for the class of linear mixed models yet. We introduce a natural extension of the well-known adjusted coefficient of determination in linear models to the variance components form of the linear mixed model. We propose a novel coefficient of determination which is dimensionless, has an intuitive and simple definition in terms of variance explained, is additive for several random effects and reduces to the adjusted coefficient of determination in the linear model. To this end, we prove a full decomposition of the sum of squares of the independent variable into the explained and residual variance. Based on the restricted maximum likelihood equations, we introduce a novel measure for the explained variation which we allocate specifically to the contribution of the fixed and the random covariates of the model. We illustrate on two genomic datasets that the novel coefficient of determination can be applied as an improved estimator of the heritability of complex traits. Exemplarily, we allocate the explained variation to the five chromosomes of the model plant Arabidopsis thaliana and determine the contribution of each chromosome as well as cross-correlations between the chromosomes to the trait heritability.