Do not directly compare cross-validation results between genomic and phenomic predictions

Crop breeders use both genotypic and phenotypic data to select the best candidates for new crop lines. Over the past two decades, genomic selection, a method to estimate a plant's potential performance using genetic information, has proven to be a powerful method for making selection decisions. Recently, breeders have begun to collect high-throughput phenotyping data as well, a technique where technology aids in collecting a large amount of data on complex plant traits, non-destructively. This method can also be used to improve selection decisions and is often thought of as more efficient and less expensive than genotype data. Both types of data require complex statistical models, and both are typically evaluated using cross-validation values to estimate the accuracy of each model. 

While breeders need both types of models to have high accuracy, researchers at the University of California, Davis identified a limitation of using cross-validation to compare the accuracy of these models. Specifically, accuracy means something completely different when describing a phenotypic prediction model vs. a genomic prediction model. When used to evaluate genomic prediction models, cross-validation estimates the accuracy of predicted breeding values, which are values that describe the genetic potential of individuals to pass down desirable traits to the next generation. In contrast, when used to evaluate phenotypic prediction models, cross-validation estimates the accuracy of predicted phenotypic values of the candidates themselves. Both predictions are useful for breeders, but at different stages of a breeding program. Therefore, comparing them is like “comparing apples to oranges” and doesn’t help evaluate whether either type of model is useful.

The researchers showed that by directly comparing estimated genotypic prediction accuracy to phenotypic prediction accuracy, previous studies have mistakenly concluded that phenotypic prediction was more useful. Instead, the researchers concluded that these technologies are complementary and should be evaluated separately.