We investigated powers and type I error rates for the multilevel model and FBAT approach using 3 data sets: (a) GWA SNP data, in which most of the SNPs are relatively common (MAF >0.05); (b) sequence data, which include rare and common genetic variants; and (c) rare sequence data, which include only rare genetic variants (MAF <0.05). All the results were based on the 200 simulated replicates.
Type I error rates
All the results were based on a nominal significance level of 0.05. We did not perform multiple testing corrections owing to the small number of replicates (200). For the GWA SNP data, we employed 246 noncausal SNPs with MAF >0.05 that were not in LD (r2 <0.01) with any of the 149 causal SNPs. When using SBP as the phenotype of interest, we found that of the 246 noncausal SNPs, 76 SNPs in the multilevel model analysis, and 77 SNPs in the FBAT approach had inflated type I error rates (ie, more than 10 replicates [out of 200 replicates] with p values <0.05). The average type I error rates across all noncausal SNPs were 0.05 (SE = 0.05) for the multilevel model and 0.047 (SE = 0.035) for the FBAT approach. Therefore, the type I error rates for both approaches are comparable for the GWA SNP data (ie, common variants). For the sequence data (rare and common variants), we employed 13,440 noncausal genetic variants that were not in LD (r2 <0.02) with any of the 1457 causal variants. Using SBP as the phenotype of interest, of the 13,440 noncausal variants, we observed 4357 variants with inflated type I error rates using multilevel model analysis and 3958 variants with inflated type I error rates using the FBAT approach. The average type I error rates across all noncausal variants were 0.06 (SE = 0.074) for the multilevel model and 0.046 (SE = 0.041) for the FBAT approach. From the noncausal genetic variants in the sequence data set, we selected 218 variants with MAF <0.05 (ie, rare variants). Using SBP as the phenotype of interest, of these 218 variants, we observed 68 variants with inflated type I error rates using multilevel model analysis, with an average type I error rate of 0.058 (SE = 0.075). The proportion of variants with inflated type I error rates and the average type I error rate was very similar to that in the studies of the first 2 data sets. However, for the FBAT approach, only 2 variants had inflated type I error rates; the average type I error rate using the FBAT approach to analyze rare variants was 0.007 (SE = 0.013). Because the type I error rates of the 2 approaches for the study of rare-variants-only data are not comparable, we conducted the power calculations for both approaches by adjusting for their type I errors. Specifically, using p values from the 200 replicates for noncausal variants (null distribution of p values), we identified thresholds for the multilevel model and FBAT approach that correspond to the controlled type I error rates and then computed powers based on these thresholds. We also investigated type I error rates using DBP as the phenotype of interest and obtained similar results (data not shown).
Power comparisons
The power comparison results of the multilevel model and single-time-point FBAT approach as a function of MAFs of genetic variants are shown in Figures 1 and 2 for SBP and DBP, respectively. Both figures show the powers of 2 approaches for each of the 3 data sets with and without Bonferroni corrections. The Bonferroni-corrected significance levels differed between phenotypes and between data sets because the number of causal variants differed between those scenarios. For example, in the GWA SNP data, 105 SNPs were causal for SBP and 117 SNPs were causal for DBP; therefore, the Bonferroni-corrected significance levels were 0.05/105, or 4.8 × 10−4, for SBP (Figure 1B) and 0.05/117, or 4.3 × 10−4, for DBP (Figure 2B).
From the power results for SBP, we can observe that for most of the causal genetic variants, the multilevel model analysis had powers relatively higher than or similar to those of FBAT when using the GWA SNP data set (see Figure 1A) and the sequence data set (see Figure 1C). When using the rare-variants-only data set (see Figure 1E), both approaches had very little power for identifying almost all the causal variants (less than 20% at the 0.05 significance level). When the Bonferroni-corrected significance levels were used (see Figure 1B, D, and F), both approaches had almost no power to identify any causal variants. Moreover, the MAFs of the variants did not substantially affect the power in either approach. Meanwhile, the power results for DBP were very similar to those for SBP in all data sets (see Figure 2).
We also investigated the causal variants with power of at least 20% for both the multilevel model and FBAT approach (data not shown). Most of the causal genetic variants were removed from this set because the powers using either approach were less than 20%. However, interestingly, for the rest of the causal variants, multilevel model analysis had higher power than the FBAT approach for almost all of them.
We further investigated the relationships between power, effect size (unit = mm Hg), and MAF using the GWA SNP data set with SBP as the phenotype of interest. We found that variants with very high effect sizes could be identified with high power for both approaches; for example, rs11711953 had an effect size of −9.9107, and both approaches had powers of almost 100% to identify this variant at a 0.05 significance level; however, when the Bonferroni correction was applied, the FBAT's power decreased dramatically (85.5%), while the multilevel model's power remained at 100%. We also observed that both approaches had relatively low power to identify causal variants with high effect sizes but low MAFs and relatively high power to identify causal variants with low effect sizes but high MAFs, as has been shown in our previous study [10]. For example, for SNP rs11465293, which had an effect size of −1.1227 for SBP and a MAF of 0.0148, the powers were only 39.5% for the multilevel model and 15.5% for FBAT, whereas for variant rs1131356, which had an effect size of 1.0007 for SBP and MAF of 0.4947, the powers were 95% and 48%, respectively.