Skip to main content


Volume 8 Supplement 1

Genetic Analysis Workshop 18

PREST-plus identifies pedigree errors and cryptic relatedness in the GAW18 sample using genome-wide SNP data

Article metrics

  • 1555 Accesses

  • 18 Citations


Pedigree errors and cryptic relatedness often appear in families or population samples collected for genetic studies. If not identified, these issues can lead to either increased false negatives or false positives in both linkage and association analyses. To identify pedigree errors and cryptic relatedness among individuals from the 20 San Antonio Family Studies (SAFS) families and cryptic relatedness among the 157 putatively unrelated individuals, we apply PREST-plus to the genome-wide single-nucleotide polymorphism (SNP) data and analyze estimated identity-by-descent (IBD) distributions for all pairs of genotyped individuals. Based on the given pedigrees alone, PREST-plus identifies the following putative pairs: 1091 full-sib, 162 half-sib, 360 grandparent-grandchild, 2269 avuncular, 2717 first cousin, 402 half-avuncular, 559 half-first cousin, 2 half-sib+first cousin, 957 parent-offspring and 440,546 unrelated. Using the genotype data, PREST-plus detects 7 mis-specified relative pairs, with their IBD estimates clearly deviating from the null expectations, and it identifies 4 cryptic related pairs involving 7 individuals from 6 families.


Mis-specified pedigree relationship and cryptic relatedness often occur in family and population data. The potential causes of such errors are numerous, including undocumented nonpaternity, nonmaternity, adoption, mating between relatives, sample duplication or swap. It is well known that such errors, if undetected, can affect the accuracy or power of both linkage and association studies, as well as have adverse effects on other aspects of the analyses such as population stratification [16].

Genome-wide marker data can provide accurate information on the genetic relatedness among individuals. For linkage scans, a number of statistical methods have been proposed and implemented, including RELCHECK [1], RELATIVE [7], PEDCHECK [8], SIBERROR [9], PREST [10, 11], GRR [12], and ECLIPSE [13], among others. More recently, PLINK [14] and PREST-plus [5, 6, 15] have been developed for analysing the high-throughput SNP data collected from genome-wide association studies (GWAS) or next-generation sequencing (NGS) experiments.

There are 3 main differences between PREST-plus and PLINK. First, PREST-plus uses the maximum likelihood-based IBD estimation method, which has more statistical power, whereas PLINK relies on the method-of-moments approach, which is computationally more efficient. Second, PREST-plus identifies both pedigree errors and cryptic relatedness in linkage or association studies with family data, population sample, or both, whereas PLINK is primarily suitable for detecting cryptic relatedness in GWAS with population sample. Third, PREST-plus provides a formal hypothesis testing framework, which can be useful when a potential error has been being identified, whereas PLINK provides point IBD estimation only [5, 6]. The Genetic Analysis Workshop 18 (GAW18) data, similar to many emerging large-scale GWAS and NGS studies, include multigeneration large pedigrees. The genetic relationships between individuals in such data are not limited to simple types such as parent-offspring or siblings. Consequently, we focus here on the methodology implemented in PREST-plus [5, 6, 15] and discuss PLINK [14] when relevant.


Relationship estimation and testing

The relatedness between a pair of individuals can be summarized by the IBD probability distribution, p = ( p 0 , p 1 , p 2 ) . It describes the probability of a randomly sampled marker to have 0, 1, or 2 common ancestry alleles between 2 individuals. Using the available genotype data, PREST-plus estimates the most likely IBD distribution by obtaining p ^ = ( p ^ 0 , p ^ 1 , p ^ 2 ) that maximizes the quantity

L ( p ) = m = 1 M log P ( G m ; p ) = m = 1 M log i = 0 , 1 , 2 P ( G m | D m = i ) p i ,

where G m is the genotype at marker m and D m = i denotes the number of alleles shared IBD by the pair at that marker. The maximization is efficiently achieved by an application of the expectation-maximization (EM) algorithm [10, 16, 17]. In contrast, PLINK [14] estimates the IBD distribution using the method-of-moments approach, which is less powerful than the likelihood-based method.

Estimation of IBD distribution can often provide sufficient information to identify pedigree errors and cryptic relatedness [5, 6]. However, it is useful to provide statistical evidence beyond point estimation. To this end, we apply the maximum likelihood ratio test (MLRT) to formally evaluate whether the observed genotypes G are compatible with the null putative relationship type, R 0 [10]. Briefly, M L R T = log ( L ^ ( A ) ) - log ( L ( R 0 ) ) , where L ( R 0 ) is the likelihood calculated for the hypothesized R 0 , and L ^ ( A ) )=max { L ( R 1 ) } , R 1 A, is the maximum likelihood calculated over a set of alternatives as given in Table 1. Statistical significance of MLRT is then assessed using simulation, because 2*MLRT does not follow the usual Chisq distribution [10]. Efficient implementation of the test to high-throughput genotype data requires pruning of the SNPs so that they are not in linkage disequilibrium (LD) [5, 6].

Data analyses

The GWAS data of GAW18 consist of 959 individuals genotyped at 472,049 SNPs. Among the 959 individuals, 4 are removed for low genotyping rate (plink -MIND >0.8), and 141 individuals are in the "UNREL.txt" file that contains the maximum set of putatively unrelated individuals. Among the 472,049 SNPs, ~50,000 remain after minor allele frequency (MAF) and LD pruning (plink -indep-pairwise 200 50 0.2 -maf 0.05). We then conduct 3 sets of analyses, with analyses 1 and 2 focusing on relationship estimation and analysis 3 performing hypothesis testing.

Analysis 1 detects pedigree errors and cryptic relatedness in the 955 genotyped individuals from the 20 SAFS families using PREST-plus (prest -geno datafamily.ped -map -wped -aped). It estimates the IBD distribution for any pair of individuals within a pedigree (-wped), as well as for any pair of individuals across pedigrees (-aped).

Analysis 2 detects cryptic relatedness in the 141 putatively unrelated individuals, using both PREST-plus (prest -geno dataunrel.ped -map and PLINK (plink -file plink -genome).

Analysis 3 performs formal hypothesis testing on the problematic pairs identified in analyses 1 and 2. For computational efficiency, we first randomly select ~2000 SNPs from the set of ~50,000 SNPs. We then obtain the base pair (bp) physical map of the SNPs using build 36 coordinates and their corresponding centimorgan (cM) genetic map using the Rutgers combined linkage-physical map and linear interpolation. Finally, we perform the MLRT test as implemented in PREST-plus (prest -file data2k.ped -map -pair fID1 indID1 fID2 indID2 -mlrt -c).


Analysis 1: Relationship estimation within and across the 20 SAFS families

PREST-plus identifies 455,535 pairs of genotyped individuals, with the total number of genotyped SNPs (commark) ranging from 31,120 to 49,020. Most of the 455,535 pairs have the putative relationship types considered by PREST-plus (see Table 1). (See Figure 1 of reference [5] for graphical illustrations of these relationship types.)

Table 1 IBD distribution p = ( p 0 , p 1 , p 2 ) and kinship coefficient ϕ = p 1 / 4 + p 2 / 2 for the relationship types (reltype) considered by PREST-plus

Figure 1 shows the estimated IBD distributions, p ^ 1 vs p ^ 0 , stratified by the putative relationship, R 0 , with the red cross marking the the IBD distribution expected for R 0 . We observe a substantial number of pairs with their IBD estimates deviating from the null expectations. Table 2 provides detailed information for 7 clear outliers, indicating mis-specified relationships. For example, 2 putative half-sib pairs have close to (0.25, 0.5, 0.25) of full-sib, while 1 putative avuncular pair has close to (1, 0, 0) of unrelated. Analysis 3 below is to determine the statistical significance of the apparent deviation in the IBD estimates.

Figure 1

Results of analysis 1: relationship IBD estimation within and between the 20 SAFS families using PREST-plus. The figures are stratified by the null putative relationship, R 0 , as defined by the given pedigrees. The red cross marks the expected IBD distribution for R 0 as provided in Table 1. Each black dot shows the estimated p 1 vs. p 0 based on the obseved genotype data for each of the 455,535 genotyped pairs analyzed, inlucidng 1091 full-sib, 162 half-sib, 360 grandparent-grandchild, 2269 avuncular, 2717 first-cousin, 440,546 unrelated (from both within and across families), 402 half-avuncular, 559 half-first cousin, 2 half-sib+first cousin, 957 parent-offspring, and 6470 other types of pairs. Blue circles mark the obvious outliers as detailed in Table 2.

Table 2 Relationship estimation results for clear outliers in Figure 1 identified by analysis 1.

Analysis 2: Relationship estimation among individuals in the "UNREL.txt" file

In this analysis, there are 141*140/2 = 9780 putatively unrelated pairs and the commark ranges from 32,280 to 49,010. Figure 2 displays p ^ 1 vs p ^ 0 for these pairs based on PREST-plus (left) and PLINK (right). Results clearly demonstrate the statistical efficiency of PREST-plus with overall less variation in the IBD estimates as compared to PLINK: 9338 pairs with PREST p ^ 0 0 . 9 8 vs. 6538 pairs with PLINK p ^ 0 0 . 9 8 . For the 4 potential cryptic relateded pairs, the PREST-plus IBD estimates provide a better identification of the outliers (Table 3).

Figure 2

Results of analysis 2: relationship IBD estimation among the 141 genotyped putatively unrelated individuals in the "UNREL.txt" file. The red cross marks the the IBD distribution expected for unrelated. Each black dot shows the estimated p 1 vs. p 0 based on the obseved genotype data for each of the 9870 putatively unrelated pairs analyzed by PREST-plus (left) and PLINK(right). Blue circles mark the obvious outliers as detailed in Table 3.

Table 3 Relationship estimation results for clear outliers in Figure 2 identified by analysis 2

Analysis 3: Relationship hypothesis testing of problematic pairs

As a proof of principle, we focus on the 7 clear outliers identified in analysis 1 and the 4 pairs identified in analysis 2 (Table 4). Among the putatively unrelated pairs, the possible alternative relationship types range from half-first cousin to avuncular.

Table 4 Relationship testing results for clear outliers in Figure 1 identified by analysis 1, and in Figure 2 identified by analysis 2


The GAW18 "pedigree information was [previously] verified by estimated kinship coefficients, principal component analysis (PCA), and number of mendelian errors between parent and offspring samples." However, no other details are provided and our analyses show that pedigree errors and cryptic relatedness exist in the data.

The results presented here are based on the ~50,000 GWAS SNPs that have MAF greater than 5% and pair-wise LD less than 0.2. Our experience with PREST-plus shows that there is little improvement in estimation accuracy, once more than ~50,000 SNPs were used (typically with MAF >5%). Additional analyses with denser sets of SNPs confirmed this (results not shown). However, substantially more SNPs are needed for PLINK to achieve similar estimation efficiency. Although PREST-plus is more powerful than PLINK, there is a trade-off between statistical power and computational efficiency [5, 6]. For large data sets involving analyzing millions of pairs, PLINK could be used as a screening tool for further analysis with PREST-plus.

Both PREST-plus and PLINK are sensitive to mis-specified allele frequencies, therefore sensitive to population stratification and population admixture, in contrast to some recent work [eg, [18]]. However, results from other GAW18 study groups suggest that population admixture is not a major concern here. Nevertheless, robust relationship estimation and testing methods warrant further research.

The methods considered here focus on global estimation of IBD distribution, which is powerful and efficient to distinguish distinct relationships, for example, full-sibs versus unrelated. However, such global methods are not adequate to distinguish similar relationship types, for example, second-cousin versus unrelated. To this end, the recent local estimation methods [eg, [19]] provide useful research direction.


Pedigree errors and cryptic relatedness often occur in sample despite the best practice in data collection. Genome-wide marker data, collected for linkage or association studies, can provide accurate genealogy information between individuals. Using the GWAS SNP data, PREST-plus analyses the GAW18 sample that had been previously "cleaned," and it identifies 7 clearly mis-specified relative pairs in the 20 SAFS families and 4 cryptic-related pairs in the set of putatively unrelated individuals.


  1. 1.

    Boehnke M, Cox NJ: Accurate inference of relationships in sib-pair linkage studies. Am J Hum Genet. 1997, 61: 423-429. 10.1086/514862.

  2. 2.

    Voight BF, Pritchard JK: Confounding from cryptic relatedness in case-control association studies. PLoS Genet. 2005, 1: e32-10.1371/journal.pgen.0010032.

  3. 3.

    Thornton T, McPeek MS: ROADTRIPS: case-control association testing with partially or completely unknown population and pedigree structure. Am J Hum Genet. 2010, 86: 172-184. 10.1016/j.ajhg.2010.01.001.

  4. 4.

    Price AL, Zaitlen NA, Reich D, Patterson N: New approaches to population stratification in genome-wide association studies. Nat Rev Genet. 2010, 11: 459-463.

  5. 5.

    Sun L: Detecting pedigree relationship errors. Statistical Human Genetics:Methods and Protocols. Edited by: Elston R, Satagopan J, Sun S. NewYork. 2012, Humana Press, 25-46.

  6. 6.

    Sun L, Dimitromanolakis A: Identifying cryptic relationships. Statistical Human Genetics : Methods and Protocols. Edited by: Elston R, Satagopan J, Sun S. NewYork. 2012, Humana Press, 47-58.

  7. 7.

    Goring HH, Ott J: Relationship estimation in affected sib pair analysis of late-onset diseases. Eur J Hum Genet. 1997, 5: 69-77.

  8. 8.

    O'Connell JR, Weeks DE: Pedcheck: a program for identification of genotype incompatibilities in linkage analysis. Am J Hum Genet. 1998, 63: 259-266. 10.1086/301904.

  9. 9.

    Ehm M, Wagner M: A test statistic to detect errors in sib-pair relationships. Am J Hum Genet. 1998, 62: 181-188. 10.1086/301668.

  10. 10.

    McPeek MS, Sun L: Statistical tests for detection of misspecified relationships by use of genome-screen data. Am J Hum Genet. 2000, 66: 1076-1094. 10.1086/302800.

  11. 11.

    Sun L, Wilder K, McPeek MS: Enhanced pedigree error detection. Hum Hered. 2002, 54: 99-110. 10.1159/000067666.

  12. 12.

    Abecasis GR, Cherny SS, Cookson WO, Cardon LR: Grr: graphical representation of relationship errors. Bioinformatics. 2001, 17: 742-743. 10.1093/bioinformatics/17.8.742.

  13. 13.

    Sieberts SK, Wijsman EM, Thompson EA: Relationship inference from trios of individuals in the presence of typing error. Am J Hum Genet. 2002, 70: 170-180. 10.1086/338444.

  14. 14.

    Purcell S, Neale B, Todd-Brown K, Thomas L, Ferreira MAR, Bender D, Maller J, Sklar P, de Bakker PIW, Daly MJ, et al: Plink: a tool set for whole-genome association and population-based linkage analyses. Am J Hum Genet. 2007, 81: 559-575. 10.1086/519795.

  15. 15.

    Dimitromanolakis A, Paterson AD, Sun L: Accurate IBD inference identifies cryptic relatedness in 9 HapMap populations. Abstract no 1768 presented at the Annual meeting of the American Society of Human Genetics. 2009

  16. 16.

    Thompson EA: The estimation of pairwise relationships. Ann Hum Genet. 1975, 39: 173-188. 10.1111/j.1469-1809.1975.tb00120.x.

  17. 17.

    Thompson EA: Pedigree Analysis in Human Genetics. 1986, Baltimore, MD, The Johns Hopkins University Press

  18. 18.

    Thoronto T, Tang H, Hoffmann TJ, Ochs-Balcom HM, Caan BJ, Risch N: Estimating kinship in admixed populations. Am J Hum Genet. 2012, 91: 122-138. 10.1016/j.ajhg.2012.05.024.

  19. 19.

    Browning SR, Browning BL: Identity by decent between distant relatives:detection and applications. Annu Rev Genet. 2012, 46: 617-633. 10.1146/annurev-genet-110711-155534.

Download references


Genetic Analysis Workshop is supported by NIH grant R01 GM031575. The research of LS is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canadian Institutes of Health Research (CIHR).

The GAW18 whole genome sequence data were provided by the T2D-GENES Consortium, which is supported by NIH grants U01 DK085524, U01 DK085584, U01 DK085501, U01 DK085526, and U01 DK085545. The other genetic and phenotypic data for GAW18 were provided by the San Antonio Family Heart Study and San Antonio Family Diabetes/Gallbladder Study, which are supported by NIH grants P01 HL045222, R01 DK047482, and R01 DK053889. The Genetic Analysis Workshop is supported by NIH grant R01 GM031575.

This article has been published as part of BMC Proceedings Volume 8 Supplement 1, 2014: Genetic Analysis Workshop 18. The full contents of the supplement are available online at Publication charges for this supplement were funded by the Texas Biomedical Research Institute.

Author information

Correspondence to Apostolos Dimitromanolakis.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Study design: LS. Analysis: AD and LS. Manuscript drafting: LS and AD. All authors read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark


  • Unrelated Pair
  • Null Expectation
  • Relationship Estimation
  • Formal Hypothesis Testing
  • Cryptic Relatedness