- Open Access
Comparison of analyses of the XVth QTLMAS common dataset III: Genomic Estimations of Breeding Values
BMC Proceedings volume 6, Article number: S3 (2012)
The QTLMAS XVth dataset consisted of pedigree, marker genotypes and quantitative trait performances of animals with a sib family structure. Pedigree and genotypes concerned 3,000 progenies among those 2,000 were phenotyped. The trait was regulated by 8 QTLs which displayed additive, imprinting or epistatic effects. The 1,000 unphenotyped progenies were considered as candidates to selection and their Genomic Estimated Breeding Values (GEBV) were evaluated by participants of the XVth QTLMAS workshop. This paper aims at comparing the GEBV estimation results obtained by seven participants to the workshop.
From the known QTL genotypes of each candidate, two "true" genomic values (TV) were estimated by organizers: the genotypic value of the candidate (TGV) and the expectation of its progeny genotypic values (TBV). GEBV were computed by the participants following different statistical methods: random linear models (including BLUP and Ridge Regression), selection variable techniques (LASSO, Elastic Net) and Bayesian methods. Accuracy was evaluated by the correlation between TV (TGV or TBV) and GEBV presented by participants. Rank correlation of the best 10% of individuals and error in predictions were also evaluated. Bias was tested by regression of TV on GEBV.
Large differences between methods were found for all criteria and type of genetic values (TGV, TBV). In general, the criteria ranked consistently methods belonging to the same family.
Bayesian methods - A<B<C<Cπ - were the most efficient whatever the criteria and the True Value considered (with the notable exception of the MSEP of the TBV). The selection variable procedures (LASSO, Elastic Net and some adaptations) performed similarly, probably at a much lower computing cost. The TABLUP, which combines BayesB and GBLUP, generally did well. The simplest methods, GBLUP or Ridge Regression, and even worst, the fixed linear model, were much less efficient.
In 1990, Lande and Thompson  defined a two steps marker assisted selection procedure. Firstly, apparent effects of markers were estimated in a reference population. Secondly, during n generations, breeding values of candidates to selection were calculated from these estimated effects giving a so called Molecular Scores. These ideas, which founded the genomic selection, were more recently made operational by SNP chips which provide tens of thousands genotypes per individual. The seminal paper of Meuwissen et al.  presented a few statistical approaches of these Genomic Estimated Breeding Values (GEBV). A large literature followed, describing and comparing various methods.
These methods could be classified according to the assumption made concerning the variance of chromosome segments effects. The simplest assumption, assumed in BLUP methodology  or Ridge Regression , is that the variance of these effects is equal for all chromosome segments. However, this hypothesis is not consistent with classical genetic prior, and observations, that only a few chromosome segments contain QTL, with various extent of their effects, while most chromosome segments do not contain QTL.
Variable selection procedures were proposed to better fit this biological knowledge. In , a stepwise procedure, including a QTL detection step through single segments regression analyses, was envisaged in the least square framework. The efficiency for genomic evaluation of more advanced penalized regression approaches were evaluated, like sparse PLS , LASSO  or Elastic Net , which all allow the vast majority of loci to have null regression coefficients.
On the other hand, Bayesian methods were proposed to take into account the between chromosome segments variances heterogeneity. In BayesA , each chromosome segment is given its own variance, all segments contributing to the variability. This last hypothesis is made free with other Bayesian techniques which assume that only a fraction π of the segments carry QTL: BayesB keeps the between segments variance heterogeneity, while BayesC considers a single variance for the active segments. In BayesCπ, the proportion π is estimated from the data .
The QTLMAS XVth dataset consisted of the pedigree, marker genotypes and quantitative trait performances of animals with a sib family structure . Pedigree and genotypes concerned 3,000 progenies among those 2,000 were phenotyped. The trait was regulated by 8 QTLs which displayed additive, imprinting or epistatic effects. The 1,000 unphenotyped progenies were considered as candidates to selection. Participants of the XVth QTLMAS workshop were invited to predict GEBV of these 1,000 individuals and to send to the organizers the description of their methods and results before the meeting. This paper aims at comparing the GEBV estimations obtained by participants to the workshop. Comparing the results obtained by the different groups should provide insight into determining which method is best fitted to analyze this kind of data set.
The simulated data set was described by Elsen et al. . Briefly, the population comprised 3,000 individuals born from 20 sires and 200 dams , i.e. 10 dams per sire. Within each family, 10 progenies were assigned phenotypes and marker genotypes and 5 were assigned only marker genotypes. A total of 10,000 SNPs carried by 5 chromosomes of 1 Morgan each were simulated. Eight QTLs were simulated: one quadri-allelic additive QTL with a large effect on Chr1, two linked QTLs in phase on Chr2, two linked QTLs in repulsion on Chr3, one imprinted QTL on Chr4 and two interacting QTLs on Chr5. Random noise was added giving an heritability coefficient of 0.30. The marker density, linkage disequilibrium (LD) and minor allele frequency (MAF) were similar to real life parameters.
Computation of the true genotypic and breeding values
"True" genetic values of the candidates to selection were calculated from simulated QTL genotypes information. Two values were calculated for each candidate. Firstly, a True Genotypic Value (TGV) defined as the sum of the 5 chromosomal genotypic values corresponding to the candidate genotypes at each of these chromosomes. The TGV of candidate i, TGV i depends on its QTL genotypes and on the QTL effects (a j ) given in the Table 1 of Elsen et al. :
Secondly, the expectation of the genotypic value of candidate's progenies was calculated, according to the same principle, i.e. as a sum of chromosomal genotypic values. It depends on the QTL genotypes of the candidate, on the QTL effects and on the frequencies of QTL genotypes in the population, i.e. the QTL genotypes probabilities of the mate of the candidate. This breeding value was noted TBV i :
where the gj are the n QTL possible genotypes on the chromosome j (n=(10,9,9,4,9) for the chromosomes 1 to 5 respectively), is the probability of the genotype gj for the progeny of the candidate i given the candidate's genotype and a j (gj) is the genotypic value associated to that QTL genotype (Table 1 in ). Concerning the QTL6 on chromosome 4, which was imprinted, candidates were considered as those which give the allele expressed by their progeny.
The participants were sent the TGV and TBV only after the meeting.
Methods used by the participants
The participants estimated Genomic Estimated Breeding Values, noted GEBV in the following, and sent them, with a short description of the methods used, to organizers before the meeting. A total of 27 methods were studied by the participants (table 1). Most of them belong to the three categories which were presented in the introduction: (i) (G)BLUP methods - including Ridge regression , GBLUP describing dominance effect  and TABLUP  where the genomic matrix includes information about the SNP effect variance (here estimated using BayesB)  (ii) Selection variable procedures - LASSO and Elastic Net, including adaptative versions which aim at forcing the LASSO to be consistent, i.e. to correctly estimate the subset of zero coefficients with a probability tending to 1 [9, 13] and (iii) Bayesian approaches [12, 10] - including the BayesZ [14, 15] and a new two-steps Bayes procedure intermediate between the BayesA or B (one variance for each SNP) and the BayesC (a single variance for the active SNP), with a grouping of SNP based on their effect estimated with a GBLUP . This method will be given the "BayesS" acronym in the following. Mucha et al.  used simple linear models (fixed or random) with the idea of estimating haplotype rather than SNP effects, the haplotypes being inferred with the PHASE software .
Results (GEBV as given by the participants) were compared based on 4 criteria. For each criteria, the two True Values (TGV and TBV) were considered. Accuracy of GEBV was calculated as the Pearson's correlation between the TV and the GEBV. Ability to identify the best individuals was assessed from the Spearman's rank correlation between the TV and the GEBV in the top 10% of TV. Bias was assessed from the linear regression coefficient (named also the regression slope) of the TV on the GEBV. Finally, mean squared error of prediction was calculated on GEBV and TV centered on zero.
The Pearson correlation between GEBV and the TV were consistent within type of technique used. The range was large, from 0.49 (GEBV- TGV correlation, 0.47 for GEBV-TBV) for the Mucha et al.  fixed effect model to 0.94 (GEBV- TGV correlation, 0.89 for GEBV-TBV) for the Zeng et al.  BayesCπ. The highest correlations were obtained with the Bayesian approaches, with a very good performance of BayesCπ [10, 12] which overperformed BayesZ, a similar approach based on an alternative prior. The very limited number of QTL simulated in the dataset, a situation far from the BayesZ prior, is a possible explanation for this difference. The same argument could explain the lower performance of the BayesS , where SNP effects are assembled in groups of similar effects. The TABLUP, which mixes BayesB estimation and GBLUP, was intermediate between the "classical" and the new Bayesian approaches of Shariati et al. .
The variable selection procedures can work nearly as well as the BayesB or C, in particular the LASSO and Elastic net [9, 13]. However the adaptative Elastic Net did not give the expected improvement.
The GBLUP performances were more variable with a very low correlation given by the Mucha et al.  version based on haplotypes, and higher values for the Zeng et al.  and Ogutu et al.  proposals. Finally, the fixed effect linear model was far below all other methods.
Even if all tendencies were observed for both groups of correlations, the correlations between GEBV and the TBV were always lower than the correlations between GEBV and TGV. These last correlations were always lower than the former.
As compared to the Pearson's correlation, this criteria, which illustrates how methods can capture the best individuals, shows a similar range (0.32 to 0.72, i.e. 0.4 points of correlation between extreme situations), i.e. the fixed model and the BayesCπ. Globally the classification between groups of methods is the same: Bayes methods outperformed the selection variable approaches, the GBLUP family arrived last in the classification. The only exception was the TABLUP which was positioned between the two first groups. However, within some groups, differences were exacerbated. This was particularly true for the Bayes group, where the ranking BayesCπ > BayesB > BayesA > BayesZ > BayesS was preserved, and even more for the Ogutu et al.  selection variable, with a very low correlation observed for the adaptative Elastic Net. Notably the random model proposed by Mucha et al.  fell in the worst positions with this criterion.
Regression coefficient (or regression slope)
Unbiased estimators are supposed to have a regression coefficient of 1. Most of the regression coefficients observed were in the range 0.85-1.25. The ranking of the Bayesian techniques were consistently correct, while the coefficient were more variable for the other approaches. Three of the methods proposed by Mucha et al.  clearly gave biased estimations (the fixed and both random models).
Mean squared error of prediction (MSEP)
The results are still very consistent with the other criteria. The Bayesian techniques (excluding BayesS) and the selection variable techniques (LASSO or EN) gave the more precise estimations of the TGV. TABLUP was in the same range. The GBLUP and BayesS performed not as well and Mucha et al. haplotypes models  did very badly.
The MSEP of the TBV were quite different and were above or under the TGV MSEP depending on the method. The more precise estimation was given by the Mucha et al.  random model. LASSO, EN and GBLUP were satisfying, while the Bayesian approaches (in particular the BayesS) provided high Mean Squared Error of Prediction.
The very general tendency is a better ranking of the Bayesian methods, in the alphabetic order (A<B<C<Cπ) whatever the criteria and the True Value considered (with the notable exception of the MSEP of the TBV). The Selection variable procedures (LASSO, Elastic Net and some adaptations) performed similarly, probably at a much lower computing cost. The TABLUP, which combines BayesB and GBLUP, generally did well. The simplest methods, GBLUP or Ridge Regression, and even worst, the fixed linear model, were much less efficient. The approach followed by Mucha et al.  to incorporate haplotype information was not efficient.
These observations are consistent with the results presented in the previous analyses of QTLMAS data [19–21], even if the genetic architecture simulated was restricted to a quite limited (8) number of QTL. It may be that this oligogenic situation did not work in favor of methods probably more suited to highly polygenic cases, such as BayesS  or BayesZ .
Single Nucleotide Polymorphism
Quantitative Trait Locus
Minor Allele Frequency
Genomic Estimated Breeding Value
True Breeding Value
True Genomic Value
Least Absolute Shrinkage and Selection Operators
Mean Squared Error of Prediction
Genomic Best Linear Unbiased Prediction.
Lande R, Thompson R: Efficiency marker assisted selection in the improvement of quantitative traits. Genetics. 1990, 124: 743-756.
Meuwissen THE, Hayes BJ, Goddard ME: Prediction of Total Genetic Value using genome-wide dense marker maps. Genetics. 2001, 157: 1819-1829.
Piepho HP: Ridge regression and extensions for genome wide selection in maize. Crop Science. 2009, 49: 1165-1176. 10.2135/cropsci2008.10.0595.
Colombani C, Legarra A, Croiseau P, Guillaume F, Fritz S, Ducrocq V, Robert-Granié C: Application of PLS and Sparse PLS regression in genomic selection. Proceedings of the 9th World Congress of Genetics Applied to Livestock Production: 1-6 August 2010; Leipzig Germany. Edited by: Gesellschaft für Tierzuchtwissenschaften. 2010, 0439-
Usai MG, Goddard ME, Hayes BJ: LASSO with cross-validation for genomic selection. Genet Res. 2009, 91: 427-436. 10.1017/S0016672309990334.
Croiseau P, Colombani C, Legarra A, Guillaume F, Fritz S, Baur A, Dassonneville R, Patry C, Robert-Granié C, Ducrocq V: Improving genomic evaluation strategies in dairy cattle through SNP selection. Proceedings of the 9th World Congress of Genetics Applied to Livestock Production: 1-6 August 2010; Leipzig Germany. Edited by: Gesellschaft für Tierzuchtwissenschaften. 2010, 0360-
Habier D, Fernando RL, Kizilkaya K, Garrick DJ: Extension of The Bayesian alphabet for genomic selection. Proceedings of the 9th World Congress of Genetics Applied to Livestock Production: 1-6 August 2010; Leipzig Germany. Edited by: Gesellschaft für Tierzuchtwissenschaften. 2010, 0468-
Elsen JM, Tesseydre S, Filangi O, Le Roy P, Demeure O: XVth QTLMAS: simulated dataset. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Ogutu JO, Schulz-Streeck T, Piepho HP: Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Zeng J, Pszczola M, Wolc A, Strabel T, Fernando RL, Garrick DJ, Dekkers JCM: Genomic Breeding Value Prediction and QTL mapping of QTLMAS2011 data using Bayesian and GBLUP methods. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Zhang Z, Liu J, Ding X, Bijma P, de Koning DJ, Zhang Q: Best Linear Unbiased Prediction of Genomic Breeding Values Using a Trait-Specific Marker-Derived Relationship Matrix. PLoS ONE. 2010, 5 (9): e12648-10.1371/journal.pone.0012648.
Wang CL, Ma PP, Zhang Z, Ding XD, Liu JF, Fu WX, Weng ZQ, Zhang Q: Comparison of five methods for genomic breeding value estimation for the common dataset of the 15th QTL-MAS Workshop. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Usai MG, Carta A, Casu S: Alternative strategies for selecting subsets of predicting SNPs by LASSO-LARS procedure. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Janss LLG: Bayz manual version 2.03. 2010, Leiden, The Netherlands, Janss Biostatistics
Schurink A, Janss LLG, Heuven HCM: Bayesian Variable Selection to identify QTL affecting a simulated quantitative trait. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Shariati MM, Sorensen P, Janss L: A two step Bayesian approach for genomic prediction of breeding values. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Mucha A, Wierzbicki H: Linear models for breeding values prediction in haplotype-asssited selection - an analysis of QTLMAS Workshop 2011 Data. Proceedings of the XVth QTLMAS Workshop: 19-20 May 2011; Rennes France. Edited by: Demeure O, Elsen JM, Filangi O, Le Roy P. 2012
Stephens M, Smith N, Donnelly P: A new statistical method for haplotype reconstruction from population data. American Journal of Human Genetics. 2001, 68: 978-989. 10.1086/319501.
Pszczola M, Strabel T, Wolc A, Mucha S, Szydlowski M: Comparison of analyses of the QTLMAS XIV common dataset. I: genomic selection. BMC Proceedings. 2011, 5 (Suppl 3): S1-10.1186/1753-6561-5-S3-S1.
Bastiaansen J, Bink M, Coster A, Maliepaard C, Calus M: Comparison of analyses of the QTLMAS XIII common dataset. I: genomic selection. BMC Proceedings. 2010, 4 (Suppl 1): S1-10.1186/1753-6561-4-s1-s1.
Lund M, Sahana G, de Koning D-J, Su G, Carlborg Ö: Comparison of analyses of the QTLMAS XII common dataset. I: Genomic selection. BMC Proceedings. 2009, 3 (Suppl 1): S1-10.1186/1753-6561-3-s1-s1.
Zou H: The adaptive lasso and its oracle properties. Journal of the American Statistical Association. 2006, 101: 1418-1429. 10.1198/016214506000000735.
This article has been published as part of BMC Proceedings Volume 6 Supplement 2, 2012: Proceedings of the 15th European workshop on QTL mapping and marker assisted selection (QTL-MAS). The full contents of the supplement are available online at http://www.biomedcentral.com/bmcproc/supplements/6/S2.
The authors declare that they have no competing interests.
OD and OF collected and processed the data files. PLR analyzed the data. PLR and JME wrote the manuscript. All authors contributed to the ideas and methods, and read and approved the manuscript.
About this article
Cite this article
Le Roy, P., Filangi, O., Demeure, O. et al. Comparison of analyses of the XVth QTLMAS common dataset III: Genomic Estimations of Breeding Values. BMC Proc 6, S3 (2012). https://doi.org/10.1186/1753-6561-6-S2-S3
- Chromosome Segment
- Ridge Regression
- Random Model
- Marker Genotype
- Genomic Estimate Breeding Value