 Proceedings
 Open Access
 Published:
Regularized group regression methods for genomic prediction: Bridge, MCP, SCAD, group bridge, group lasso, sparse group lasso, group MCP and group SCAD
BMC Proceedingsvolume 8, Article number: S7 (2014)
Abstract
Background
Genomic prediction is now widely recognized as an efficient, costeffective and theoretically wellfounded method for estimating breeding values using molecular markers spread over the whole genome. The prediction problem entails estimating the effects of all genes or chromosomal segments simultaneously and aggregating them to yield the predicted total genomic breeding value. Many potential methods for genomic prediction exist but have widely different relative computational costs, complexity and ease of implementation, with significant repercussions for predictive accuracy. We empirically evaluate the predictive performance of several contending regularization methods, designed to accommodate grouping of markers, using three synthetic traits of known accuracy.
Methods
Each of the competitor methods was used to estimate predictive accuracy for each of the three quantitative traits. The traits and an associated genome comprising five chromosomes with 10000 biallelic Single Nucleotide Polymorphic (SNP)marker loci were simulated for the QTLMAS 2012 workshop. The models were trained on 3000 phenotyped and genotyped individuals and used to predict genomic breeding values for 1020 unphenotyped individuals. Accuracy was expressed as the Pearson correlation between the simulated true and the estimated breeding values.
Results
All the methods produced accurate estimates of genomic breeding values. Grouping of markers did not clearly improve accuracy contrary to expectation. Selecting the penalty parameter with replicated 10fold cross validation often gave better accuracy than using information theoretic criteria.
Conclusions
All the regularization methods considered produced satisfactory predictive accuracies for most practical purposes and thus deserve serious consideration in genomic prediction research and practice. Grouping markers did not enhance predictive accuracy for the synthetic data set considered. But other more sophisticated grouping schemes could potentially enhance accuracy. Using cross validation to select the penalty parameters for the methods often yielded more accurate estimates of predictive accuracy than using information theoretic criteria.
Background
Genomic prediction[1]is a method for predicting genomic breeding values for nonphenotyped individuals using molecular marker information covering the whole genome (e.g., Single Nucleotide Polymorphism, SNP) and observed phenotypic data from training populations. In essence, it involves a multiple regression of phenotypic observations on markers (SNP). The number of markers $\left(p\right)$ typically runs into thousands and often far exceeds the number of phenotypes $\left(n\right)$, leading to the classic $p\gg n$ problem. The enormous number of markers involved in genomic prediction makes regularization methods particularly attractive and convenient tools for addressing the twin problems of selection of important markers and multicollinearity in the high dimensional regressions. In particular, the high dimensional nature of highthroughput SNPmarker data sets has prompted increasing use of the power and versatility of regularization methods in genomic selection to simultaneously select important markers and account for multicollinearity. Regularized (penalized) regression methods commonly used in genomic prediction include ridge [2], lasso (least absolute shrinkage and selection operator) [3], elastic net [4]and bridge [5]regression and their extensions [6, 7].
These methods are not explicitly designed to exploit information on potential grouping structure among markers, such as that arising from the association of markers with particular Quantitative Trait Loci (QTL) on a chromosome or haplotype blocks, to enhance the accuracy of genomic prediction. The nearby SNP markers in such groups are linked, yielding highly correlated predictors. If such group structure is present but is ignored by using models that select individual predictors only, then such models may be inefficient or even inappropriate, leading to low accuracy of genomic prediction. Here, we explore if the accuracy of genomic prediction can be enhanced by explicitly accounting for potential grouping of SNP markers and using regularization methods with grouped penalties specifically designed to enable group selection. The predictive performances of the grouped methods are compared among the methods themselves and with those for corresponding but ungrouped variant of each method.
Methods
Linear regression model
Consider the linear regression model
where ${y}_{i}$ is the ith observation of the response variable, ${x}_{ij}$ is the ith observation on the jth covariate, ${\beta}_{j}$ are the regression coefficients and ${\u03f5}_{i}$ are i.i.d. random error terms with $var\left(\u03f5\right)=I{\sigma}_{e}^{2}$, where $\u03f5$ is the vector of n errors ${\u03f5}_{i}$ and $\phantom{\rule{0.1em}{0ex}}I$ is an ndimensional identity matrix. In what follows we assume, without loss of generality, that the response and the covariates in (1) are meancentered and standardized so that ${\beta}_{0}=0$, $\sum _{i=1}^{n}{y}_{i}=0,$$\sum _{i=1}^{n}{x}_{ij}=0$ and $\sum _{i=1}^{n}{x}_{ij}^{2}=1$[8]. In genomic prediction we are interested in estimating the $\phantom{\rule{0.1em}{0ex}}p$ regression coefficients ${\beta}_{j}$ which may be very many and many ${\beta}_{j}$ may be zero.
Regularization methods
All regularized regression methods estimate the vector of regression coefficients $\phantom{\rule{0.1em}{0ex}}\beta $ in (1) by minimizing an objective function F composed of the sum of a loss function (e.g. the squared error loss=Residual Sum of Squares (RSS)) and a penalty function:
where ${{p}_{\lambda}}_{,\gamma \left(.\right)}$ is a function of the vector of coefficients $\beta ={\left({\beta}_{1},{\beta}_{2},\dots ,{\beta}_{p}\right)}^{T}$ and the tuning(penalty) parameter $\lambda >0$ controls the tradeoff between minimizing the loss and the penalty terms. $\gamma >0$ is a shrinkage parameter that determines the order of the penalty function. Minimizing (2) yields a spectrum of solutions depending on the value of $\lambda $.
The gradient (first derivative) of a penalty function determines how it affects the solution in (2). To see this for bridge regression, consider the first derivative or rate of penalization of penalties of the form ${{p}_{\lambda}}_{,\gamma \left(\beta \right)}=\lambda {\beta}^{\gamma}$ with respect to $\phantom{\rule{0.1em}{0ex}}\beta $, where $\phantom{\rule{0.1em}{0ex}}\beta $ is a scalar. In ridge regression $\left(\gamma =2\right)$, the rate of penalization ${p}_{\lambda ,\gamma}^{\prime}\left(\beta \right)=2\lambda \beta $ increases with $\phantom{\rule{0.1em}{0ex}}\beta $, implying little or no penalization is applied when $\phantom{\rule{0.1em}{0ex}}\beta $ is near 0 but strong penalization is applied when $\phantom{\rule{0.1em}{0ex}}\beta $ is large. In lasso regression $\left(\gamma =1\right)$, the rate of penalization ${P}_{\lambda ,\gamma}^{\prime}\left(\beta \right)=\lambda $ is constant. In bridge regression $\left(e.g.,\phantom{\rule{2.77695pt}{0ex}}\gamma =1/2\right),$ the rate of penalization ${p}_{\lambda ,\gamma}^{\prime}\left(\beta \right)=\lambda /2\sqrt{\beta}$ is very high for values of $\phantom{\rule{0.1em}{0ex}}\beta $ near zero but declines rapidly as $\phantom{\rule{0.1em}{0ex}}\beta $ becomes large.
We consider the eight different regularized regression methods in turn below.
Bridge regression
Bridge regression minimizes the penalized least squares objective function [5, 9]
where p, ${\beta}_{j}$ and $\lambda >0$ and $\gamma >0$ are defined as in (2).
The optimal combination of $\lambda $ and $\gamma $ can be selected adaptively from the data by grid search using crossvalidation. The bridge estimator is the value of $\widehat{\beta}$ that minimizes (1) for any given $\gamma >0$[5, 9]. The bridge estimator can do automatic variable selection since some coefficients become exactly zero when $0<\gamma \le 1$ and $\lambda $ is sufficiently large. For $0<\gamma <1$, a finite number of covariates and under appropriate regularity conditions, the bridge estimator (i) is consistent and (ii) can distinguish between covariates whose coefficients are exactly zero and covariates with nonzero coefficients in sparse highdimensional settings [10].
[8]extended the results of [10] to infinite dimensional parameter settings (i.e. $p\to \infty $ as $n\to \infty $) and showed that the bridge estimator (iii) is selection consistent for any $\gamma >0$ and (iv) has the oracle property when $0<\gamma <1$.The oracle property means that:[11, 12]; (a) the bridge estimator correctly selects the nonzero coefficients with probability converging to 1 (i.e. with near certainty) and that (b) the bridge estimators of the nonzero coefficients are asymptotically normal with the same means and covariances that they would have if the zero coefficients were known in advance. The bridge estimator subsumes three important special cases. When (v) $\phantom{\rule{0.5em}{0ex}}\gamma =0$ the bridge estimator (2) simplifies to the ordinary least squares estimator (subset selection). (vi) When $\phantom{\rule{0.5em}{0ex}}\gamma =1$ the bridge estimator (2) reduces to the lasso estimator, which was introduced as a variable selection and shrinkage method [3].
(vii) When $\gamma =2$ the bridge estimator (3) simplifies to the ridge estimator (5) [1, 13–15]
(viii) Since some components of the bridge estimator can be exactly zero when $0<\gamma <1$ and $\lambda $ is sufficiently large, the bridge estimator can simultaneously estimate parameters and select variables in one step. (ix) The bridge estimator can adaptively select the penalty order $\left(\gamma \right)$ from the data and produce flexible solutions in a range of settings. (x) Bridge estimators have demonstrated robust performance in various settings relative to other penalized regression methods, including the popularly used ridge regression, lasso and the elastic net [8]. For example, the bridge estimator correctly identifies zero coefficients with higher probability than do the lasso and elastic net estimators based on simulation results [8].
MCP
The minimax concave penalty (MCP) is defined on $\left[0,\phantom{\rule{2.77695pt}{0ex}}\infty \right)$[16] as
where $\lambda \ge 0$ and $\gamma >0$. The expression for${p}_{\lambda ,\gamma}\left(.\right)$ shows that MCP initially applies the same rate of penalization as the lasso does but continuously reduces the rate of penalization until the rate becomes 0 when $\beta >\gamma \lambda $.
The MCP [17] is motivated by and is very similar to the smoothly clipped absolute deviation (SCAD, [11]) penalty function. The gradient of the SCAD penalty is given by [11]
This gradient function corresponds to a quadratic spline function with knots at $\lambda $ and $\gamma \lambda $. The penalty functions for both MCP and SCAD are concave or nonconvex. Both MCP and SCAD aim to eliminate the unimportant predictors from the model while leaving the important predictors unpenalized. This is equivalent to fitting an unpenalized model in which the truly nonzero predictors are known beforehand (i.e. the 'oracle property'). MCP and SCAD are thus asymptotically oracleefficient [11, 17]. Accordingly, as $n\to \infty $, they select the correct regression model with probability tending to one and the nonzero coefficient estimates are asymptotically normal and have the same covariance matrix as if they were known in advance [11, 18, 19]. MCP performs well when there are many rather sparse groups of predictors, i.e., when the underlying model exhibits less grouping of predictors. MCP suffers when the nonzero coefficients are clustered into tight groups because it tends to select too few groups and makes insufficient use of the grouping information. SCAD has weaker grouping behaviour than the MCP [21]
Group bridge, group lasso, sparse group lasso and group MCP methods
All the four grouped methods select the important groups of covariates. Group bridge, sparse group lasso and group MCP perform bilevel selection because they also identify the important members of each group [20, 21]. Bilevel selection is appropriate if predictors are not distinct but have common underlying grouping structure. Bilevel selection differs from simple group selection in that in bilevel selection, variable selection is carried out at the group level and at the level of the individual covariates, resulting in the selection of important groups as well as members of those groups. But in group selection, only relevant groups are selected so that the estimated coefficients within each group will be either all zero or all nonzero.
The group bridge, sparse group lasso and group MCP penalties combine two nested penalties to enable bilevel selection.
Group bridge
The group bridge estimator is [22, 23]
where ${A}_{1},\dots ,{A}_{L}$ are subsets of the set $\left\{1,\dots ,p\right\}$ indexed by $l=1,\dots ,L$ and represent known groupings of the covariates, ${\beta}_{{A}_{l}}={\left({\beta}_{j},j\u03f5{A}_{l}\right)}^{T}$ are the regression coefficients in the lth group.$\lambda >0$ is the penalty parameter and ${c}_{l}\ge 0$ are constants that adjust the different dimensions of ${\beta}_{{A}_{l}}$ and assign different weights to the different coefficients. A simple choice of ${c}_{l}$ is ${c}_{l}\propto {{\left{A}_{l}\right}^{1}}^{\gamma}$ where $\left{A}_{l}\right$ is the cardinality of ${A}_{l}$ (the length or number of unique elements in the set ${A}_{l}$). The group bridge penalty combines two penalties, namely the bridge penalty for group selection and the lasso penalty for withingroup selection. The bridge penalty is applied on the L_{1}norms of the grouped coefficients in (8). The objective criterion (8) reduces to the standard bridge criterion (3) when $\left{A}_{l}\right=1$ and $1\le l\le L$.
Group lasso
The group lasso selects groups of variables but does not select individual variables within groups. The group lasso estimator is [22]
in which ${A}_{l}$ and ${\beta}_{{A}_{l}}$$\left(l=1,\dots ,L\right)$ are defined as in (8), ${K}_{l}$ is a positive definite matrix and ${\u2225{\beta}_{{A}_{l}}\u2225}_{{K}_{l,2}}={\left({\beta}_{{A}_{l}}^{T}{K}_{l}{\beta}_{{A}_{l}}\right)}^{\frac{1}{2}}$. [24] suggest using ${K}_{l}=\left{A}_{l}\right{I}_{l}$, where $\left{A}_{l}\right$ is the cardinality of ${A}_{l}$ and ${I}_{l}$ is the $\left{A}_{l}\right\times \left{A}_{l}\right$ identity matrix.
The reason that $gLasso$ selects groups but not individual variables is made clearer by reexpressing (9) as [22]
Then minimizing $S\left(\beta ,\omega \right)$ subject to $\omega \ge 0$ for some suitably chosen constant $\stackrel{\u0303}{\omega}\ge 0$ yields $gLasso\left(\beta \right)$ in model (10) for appropriately chosen $\nu $.
The objective criterion (10) reveals that $gLasso$ behaves very much like an "adaptively weighted ridge regression" in which (i) the sum of the squared coefficients in group l is penalized by ${\omega}_{l}$, and (ii) the sum of the ${\omega}_{l}$'s is further penalized by $\nu $. If ${{\beta}_{A}}_{l}=0$ when model (10) is minimized then group l is dropped from the model. But if ${{\beta}_{A}}_{l}\ne 0$ then all the elements of ${{\beta}_{A}}_{l}$ are nonzero and all the variables in group l are retained in the model [22].
Equivalently, the group lasso penalty can also be written as [25]
where the loss (RSS) is computed using only observations of covariates in the submatrix of the matrix of all covariates with columns corresponding to covariates in group l, ${\beta}_{l}$ is the coefficient vector of that group and ${p}_{l}$ is the cardinality or length of ${\beta}_{l}$. The $\sqrt{{p}_{l}}$ terms account for the varying group sizes and ${\u2225.\u2225}_{2}$ is the Euclidean norm (not squared).
The group lasso estimator is asymptotically consistent even when model complexity increases with increasing sample size [26]. If only one variable is contained in each group then the objective function (9) simplifies to that of the usual lasso solution. $gLasso$ penalizes the grouped coefficients much like the lasso does because it uses the same tuning parameter for all groups and hence suffers from estimation inefficiency and variable selection inconsistency. The adaptive group lasso remedies these shortcomings by applying different tuning parameters and hence different amounts of shrinkage to the grouped coefficients [27] much as the adaptive lasso does to individual covariates [18]. But the adaptive group lasso does not accomplish bilevel selection[28]. The group lasso overshrinks individual coefficients when groups are sparsely populated.
Sparse group lasso
The sparse group lasso ([25, 29, 30] also performs groupwise and withingroup variable selection. The sparse group lasso penalty blends the lasso and group lasso penalties ([25, 31]:
where $\phantom{\rule{0.1em}{0ex}}\beta $is the full parameter vector, $\alpha \u03f5\left[0,1\right]$. Setting $\alpha =0$ produces the lasso fit whereas $\alpha =1$ yields the group lasso solution.
Group MCP
The group MCP estimate minimizes [20, 21]
where $\phantom{\rule{0.1em}{0ex}}\rho $ is the MCP penalty (6), the tuning parameter of the outer penalty, b, is chosen to be ${p}_{l}a\lambda /2$ to ensure that the group level penalty attains its maximum if and only if all of its components are at their maxima, ${p}_{l}$ is the size of group l,l = 1,...,L groups and $\lambda \ge 0$.
The group MCP therefore also combines two penalties to achieve bilevel, i.e., group and within group variable selection. All the methods with grouped penalties make inflexible grouping assumptions that can undermine their performance when groups are misspecified or sparsely represented [20]. SCAD displays less grouping than group MCP and is thus expected to be less suited to grouped variable selection problems.
Data set
An outbred population was simulated for the 16^{th} QTLMAS Workshop 2012. The simulation involved generating a base population (G0) of 1020 unrelated individuals (20 males and 1000 females) with a genome comprising 5 chromosomes, each having 2000 equally distributed SNPs. Each of the subsequent four nonoverlapping generations (G1G4) consisted of 20 males and 1000 females and was generated from the previous one by randomly mating each male with 51 females. Three milk production quantitative traits all of which express only in females were simulated. The traits were correlated and generated to mimic two yields and the corresponding content. Thus, the phenotypes, given as individual yield deviations, are only for the 3,000 females from G1 to G3. Young individuals (G4: individuals 3081 to 4100) have no phenotypic records. The pedigree of 4100 individuals, including the individual identity, sire, dam, sex and generation were provided as were the SNP genotypes for the 4100 individuals and the location of SNPs on each chromosome. Two alleles were given for each SNP. The marker information was coded as 1 for alleles ${A}_{1}{A}_{1}$, 1 for ${A}_{2}{A}_{2}$ and 0 for ${A}_{1}{A}_{2}$, or ${A}_{2}{A}_{1}$ and stored in a matrix $X=\left\{{x}_{ik}\right\}$, where ${x}_{ik}$ is the marker covariate for the $\phantom{\rule{0.1em}{0ex}}i$th genotype $\left(i=1,\dots ,G\right)$ and the $\phantom{\rule{0.1em}{0ex}}k$th marker $\left(k=1,2,\dots ,p\right)$. Monomorphic markers (n = 31) were identified and deleted prior to analysis, resulting in 1000031 = 9969 markers. Here, we address only the second aim of the challenge which is to predict genomic breeding values for the 1020 unphenotyped progenies using the available genomic information.
Grouping SNP markers for the grouped methods
To enable model fitting for the grouped methods we formed groups of the markers by assigning consecutive SNP markers systematically to groups of sizes 1, 10, 20,...,100 separately for each of the five chromosomes. This often resulted in the last group having fewer SNPs than the actual prescribed group size. The total number of all groups of sizes 1, 10, 20,...,100 were 9969, 978, 490,...,100.
Model fitting and selection
All the models were fit in R. Group lasso, group bridge, group MCP, and group SCAD models were fitted by the R packages grpreg. For each model and group size combination, the optimal value of $\lambda $ was selected by computing solutions along a grid of 100 $\lambda $ values spaced evenly on the log scale following the approach of [31]. The value of $\gamma $ was fixed at its recommended default value in gpreg to reduce computing time to manageable levels. The Akaike (AIC) and Schwarz Bayesian (BIC) Information Criteria were used to select the optimal value of the penalty parameter $\phantom{\rule{0.1em}{0ex}}\lambda $ along the regularization path from the set of the 100 $\lambda $ values for each model and group size combination [20]. The models with the selected best values for $\lambda $ for each group size were used to predict genomic breeding values for the 1020 unphenotyped genotypes. Pearson correlation between the predicted and true genomic breeding values was used to assess predictive accuracy. MCP and SCAD were also fitted to the ungrouped data using the R package ncvreg and the optimal value of $\lambda $ similarly selected from 100 values using 10fold crossvalidation. The 10fold crossvalidation involved partitioning the 3000 observations into 10 equal parts and estimating the prediction error in each set by using the observations in the other 9 sets to fit each of the models and predict the tenth part. Lastly, ridge regression was fitted to the ungrouped data in the R package glmnet net using 10fold crossvalidation.
Results
The predictive accuracies attained by all the methods were mostly high. Although it improved prediction accuracy, overall grouping was not associated with a consistent increase in predictive accuracy (Tables 1 to 3). Nevertheless, the method used to select the penalty parameter $\left(\lambda \right)$ often had a discernible impact on the accuracies of the regularization methods. The group bridge, group lasso and group MCP tended to produce better prediction accuracies with tuning parameters selected by AIC than by BIC. The sparse group lasso produced somewhat more accurate estimates than all the other methods for all the three synthetic traits. The best estimates of predictive accuracy for traits 2 and 3 were often slightly higher than the corresponding estimates for trait 1 (Tables 1 to 3). Results based on an alternative grouping of markers using Kmeans clustering (K = 10, results not shown) largely reproduced those for the systematic grouping and hence are omitted for the sake of brevity.
Discussion
All the regularization methods produced consistent and relatively high estimates of predictive accuracy for all the three synthetic traits. The accuracies of all the estimates are such that each could potentially provide a firm basis for making practical selection decisions. Predictive accuracy varied with the method used to select the tuning or penalty parameter. There was some evidence that the group bridge, lasso, MCP and SCAD methods tended to produce somewhat more accurate estimates of predictive accuracy when the tuning parameter was selected by AIC than by BIC. This reinforces the suggestion of [32] that AICtype criteria are often more appropriate if a model is used for prediction whereas BICtype criteria are better suited for uncovering the true underlying model. Even so, the estimated predictive accuracy was sometimes decidedly higher when the tuning parameter was selected by 10fold cross validation than by either of the information theoretic criteria. [33] recommend running crossvalidation multiple times to obtain reliable results when small signals are expected. Accordingly, we ran the 10fold cross validation 100 times, once for each of the 100 values of the tuning parameter for the grouped bridge, lasso, MCP and SCAD methods. For the sparse group lasso we replicated the 10fold cross validation 20 times, once for each value of the tuning parameter. The observed improvement in predictive accuracy in some cases when using cross validation to select the penalty parameter is thus consistent with most of the markers having small signals.
There was no compelling evidence that grouping SNP markers consistently improved predictive accuracy for these data. This could mean either that the simulated SNP markers were not strongly correlated or that they indeed were but the simple systematic or Kmeans clustering grouping methods failed to accurately capture the underlying grouping structure. If the lack of clear improvement in performance is due to failure to accurately account for the underlying grouping structure then, assuming an accurate map information is available for each chromosome, using spatial clustering methods such as Kspatial clustering that partitions the genomic or chromosomal region into disjoint and contiguous intervals, subject to the constraint that SNPs in each group are spatially adjacent, and tagging these intervals with cluster numbers (1, 2,..., K), could potentially improve performance. If adjacent SNP markers are not independent, contrary to the assumption made by most common clustering frameworks, then spatial clustering should be more informative and more powerful than simple clustering of markers. A standard clustering procedure like Kmeans should perform poorly if markers are correlated because it ignores the genomic layout of the data and considers only the similarity of the SNP markers per loci. The grouped methods will also perform suboptimally if the underlying grouping structure is too complex to accurately capture with simple clustering algorithms, including spatial clustering of groups. Such complexity may originate, for example, from overlapping of groups caused by SNPs linked to multiple QTLs.
The grouped methods we consider are not well suited to handling overlapping groups by construction. Extensions of the grouped methods would thus be needed to efficiently accommodate complications associated with overlapping groups. Existing extensions of the grouped methods designed to solve this type of complication include the overlapping group lasso that allows overlaps between groups of covariates. Some covariates are allowed to occur in more than one group but each time a covariate occurs in one group it gets a new coefficient [34, 35]. This makes it possible to select one variable without selecting all the groups containing it. A related extension is the hierarchical (overlapped) group lasso that incorporates both main effects and interactions that obey weak or strong hierarchy (nesting) patterns [36–38]. To check if allowing for overlap among groups indeed improved predictive accuracy, we fitted the hierarchical group lasso model in the glinternet package in R and used10fold crossvalidation to select the optimal $\lambda $ value from a set of 50 values [38]. The estimated predictive accuracies of 0.759, 0.815 and 0.791 for traits 1, 2 and 3, respectively, showed that using overlapping groups did not improve accuracy relative to using non overlapping groups. Other extensions of the grouped methods applicable in slightly different settings include the group lasso for logistic regression [39], generalized linear models [40] and nonparametric models [41].
Although the performance of the different methods did not differ dramatically for these data the methods often differed with respect to their relative computational efficiencies. Other studies that have compared the performance of the group lasso with other grouped methods, for example, have also found similar results and more. In particular, [24] evaluated the performance of the group lasso relative to group Lars and group nonnegative garrote. They found that the group lasso was the slowest of the three group methods because its solution path is not piecewise linear and hence requires intensive computations in large scale problems. The group Lars had comparable performance to the group lasso but was faster because its solution path is piecewise linear [24]. The group nonnegative garrote cannot be directly applied to problems in which the total number of covariates exceeds the sample size because it depends explicitly on the full least squares estimates [24].
Conclusions
All the methods produced relatively high estimates of predictive accuracy and hence can be used in genomic prediction research and practice. Systematic grouping or conventional Kmeans clustering of markers did not lead to any noticeable improvement in predictive accuracy. The grouped methods may yield better predictions with more sophisticated clustering approaches such as Kmeans spatial clustering which therefore deserve consideration in future studies. Whenever possible, the selection of the penalty parameter for the regularization methods should be done using replicated crossvalidation to enhance accuracy of estimates. Nevertheless, selecting the penalty parameter using information theoretic criteria such as AIC and BIC may occasionally yield better estimates than crossvalidation.
Abbreviations
 SNP:

Single Nucleotide Polymorphism
 QTL:

Quantitative Trait Loci
 RSS:

Residual Sum of Squares
 LASSO:

Least Absolute Shrinkage and Selection Operator
 MCP:

The Minimax Concave Penalty
 SCAD:

Smoothly clipped absolute deviation
 AIC:

Akaike Information Criterion, BIC: Schwarz Bayesian Information Criterion.
References
 1.
Meuwissen THE, Hayes BJ, Goddard ME: Prediction of total genetic value using genomewide dense marker maps. Genetics. 2001, 157: 18191829.
 2.
Kennard RW: Ridge regression: biased estimation for nonorthogonal problems. Technometrics. 1970, 12: 5567. 10.1080/00401706.1970.10488634.
 3.
Tibshirani R: Regression shrinkage and selection via the lasso. J Roy Statist Soc Ser B. 1996, 58: 267288.
 4.
Hastie T: Regularization and variable selection via the elastic net. J Roy Statist Soc Ser B. 2005, 67: 301320. 10.1111/j.14679868.2005.00503.x.
 5.
Frank IE, Friedman JH: A statistical view of some chemometrics regression tools (with discussion). Technometrics. 1993, 35: 109148. 10.1080/00401706.1993.10485033.
 6.
Heslot N, Yang HP, Sorrells ME, Jannink JL: Genomic selection in plant breeding: a comparison of models. Crop Sci. 2012, 52: 146160. 10.2135/cropsci2011.06.0297.
 7.
Ogutu JO, SchulzStreeck T, Piepho HP: Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. BMC Proceedings. 2012, BioMed Central Ltd, 6 (Suppl 2):
 8.
Huang J, Horowitz JL, Ma S: Asymptotic properties of bridge estimators in sparse highdimensional regression models. Ann Statist. 2008, 36: 587613. 10.1214/009053607000000875.
 9.
Fu WJ: Penalized regressions: The bridge versus the lasso. J Comput Graph Statist. 1998, 7: 397416.
 10.
Knight K, Fu W: Asymptotics for Lassotype estimators. Ann Statist. 2000, 28: 3561378.
 11.
Fan J, Li R: Variable selection via nonconcave penalized likelihood and its oracle Properties. J Amer Statist Assoc. 2001, 96: 13481360. 10.1198/016214501753382273.
 12.
Fan J, Peng H: Nonconcave penalized likelihood with a diverging number of parameters. Ann Stat. 2004, 32: 928961. 10.1214/009053604000000256.
 13.
Whittaker JC, Thompson R, Denham MC: Markerassisted selection using ridge regression. Genet Res. 2000, 75: 249252. 10.1017/S0016672399004462.
 14.
Piepho HP: Ridge regression and extensions for genomewide selection in maize. Crop Sci. 2009, 49: 11651176. 10.2135/cropsci2008.10.0595.
 15.
Piepho HP, Ogutu JO, SchulzStreeck T, Estaghvirou B, Gordillo A, Technow F: Efficient computation of ridgeregression best linear unbiased prediction in genomic selection in plant breeding. Crop Sci. 2012, 52: 10931104. 10.2135/cropsci2011.11.0592.
 16.
Zhang CH: Nearly unbiased variable selection under minimax concave penalty. Ann Stat. 2010, 38: 894942. 10.1214/09AOS729.
 17.
Zhang CH: Penalized linear unbiased selection. 2007, Department of Statistics and Bioinformatics, Rutgers University, Technical Report #2007003,
 18.
Zhou H: The adaptive lasso and its oracle properties. J Amer Stat Assoc. 2006, 101: 14181429. 10.1198/016214506000000735.
 19.
Breheny P, Huang J: Penalized methods for bilevel variable selection. Stat Interface. 2009, 2: 369380. 10.4310/SII.2009.v2.n3.a10.
 20.
Breheny P, Huang J: Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Ann Appl Stat. 2011, 5: 232253. 10.1214/10AOAS388.
 21.
Huang J, Breheny P, Ma S: A selective review of group selection in highdimensional models. Statist Sci. 2012, 27: 481499. 10.1214/12STS392.
 22.
Huang J, Ma S, Xie H, Zhang CH: A group bridge approach for variable selection. Biometrika. 2009, 96: 339355. 10.1093/biomet/asp020.
 23.
Park C, Yoon YJ: Bridge regression: adaptivity and group selection. J Statist Plann Inference. 2011, 141: 35063519. 10.1016/j.jspi.2011.05.004.
 24.
Yuan M, Lin Y: Model selection and estimation in regression with grouped variables. J Roy Statist Soc Ser B. 2006, 68: 4967. 10.1111/j.14679868.2005.00532.x.
 25.
Simon N, Friedman J, Hastie T, Tibshirani R: A sparsegroup lasso. J Comput Graph Statist. 2013, 22: 231245. 10.1080/10618600.2012.681250.
 26.
Nardi Y, Rinaldo A: On the asymptotic properties of the group lasso estimator for linear models. Electron J Statist. 2008, 2: 605633. 10.1214/08EJS200.
 27.
Wang H, Leng C: A note on adaptive group lasso. Comput Statist Appl Data Anal. 2008, 52: 52775286. 10.1016/j.csda.2008.05.006.
 28.
Zhang CH, Huang J: The sparsity and bias of the lasso selection in highdimensional linear regression. Ann Stat. 2008, 36: 15671594. 10.1214/07AOS520.
 29.
Peng J, Zhu J, Bergamaschi A, Han W, Noh DY, Pollack JR, Wang P: Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer. Ann Appl Stat. 2010, 4: 5377. 10.1214/09AOAS271.
 30.
Friedman J, Hastie T, Tibshirani R: A note on the group lasso and sparse group lasso. 2010, arXiv preprint arXiv:1001.0736
 31.
Friedman J, Hastie T, Tibshirani R: Regularization paths for generalized linear models via coordinate descent. 2008, [http://wwwstat.stanford.edu/~hastie/Papers/glmnet.pdf]
 32.
Yang Y: Can the strengths of AIC and BIC be shared?. Biometrika. 2005, 92: 937950. 10.1093/biomet/92.4.937.
 33.
Martinez JG, Carroll RJ, Müller S, Sampson JN, Chartterjee N: Empirical performance of crossvalidation with oracle methods in genomic context. Amer Statist. 2011, 65: 223228. 10.1198/tas.2011.11052.
 34.
Jacob L, Obozinski G, Vert JP: Group lasso with overlap and graph lasso. Proceedings of the 26th annual international conference on machine learning. Montreal, Canada. ICML 2009, 433440. ACM, New York, NY, USA
 35.
Percival D: Theoretical properties of the overlapping groups lasso. Electron J Stat. 2011, 121.
 36.
Zhao P, Rocha G, Yu B: The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat. 2009, 37: 34683497. 10.1214/07AOS584.
 37.
Bien J, Taylor J, Tibshirani R: A lasso for hierarchical interactions. Ann Stat. 2013, 41: 11111141. 10.1214/13AOS1096. 2013
 38.
Lim M, Hastie T: Learning interactions through hierarchical grouplasso regularization. [http://arxiv.org/pdf/1308.2719v1.pdf]
 39.
Meier L, van der Geer S, Bühlmann P: The group lasso for logistic regression. J Roy Statist Soc Ser B. 2008, 70: 5371. 10.1111/j.14679868.2007.00627.x.
 40.
Roth V, Fischer B: The grouplasso for generalized linear models: uniqueness of solutions and efficient algorithms. Proceedings of the 25th annual international conference on machine learning. 2009, Helsinski, Finland. ICML, 433440.
 41.
Bach F: Consistency of the group lasso and multiple kernel learning. J Mach Learn. 2008, 9: 11791225.
Acknowledgements
We thank Dr. Torben SchulzStreeck for useful discussions that helped improve this paper.
Declarations
The German Federal Ministry of Education and Research (BMBF) funded this research and publication within the AgroClustEr "Synbreed  Synergistic plant and animal breeding" (Grant ID: 0315526).
This article has been published as part of BMC Proceedings Volume 8 Supplement 5, 2014: Proceedings of the 16th European Workshop on QTL Mapping and Marker Assisted Selection (QTLMAS). The full contents of the supplement are available online at http://www.biomedcentral.com/bmcproc/supplements/8/S5
Author information
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
JOO conceived the study, conducted the statistical analysis and drafted the
manuscript. HPP read and edited the manuscript and oversaw the project. All the authors read and approved the manuscript.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Published
DOI
Keywords
 Predictive Accuracy
 Single Nucleotide Polymorphic
 Genomic Prediction
 Group Lasso
 Genomic Breeding