When analyzing a table of statistical results, one must first decide whether adjustment of significance levels is appropriate. If the main goal is hypothesis generation or initial screening for potential conservation problems, then it may be appropriate to use the standard comparisonwise significance level to avoid Type 2 errors (not detecting real differences or trends). If, however, the main goal is rigorous testing of a hypothesis, then an adjustment for multiple tests is needed. To control the familywise Type 1 error rate (the probability of rejecting at least one true null hypothesis), sequential modifications of the standard Bonferroni Method, such as Holm’s method, will provide more statistical power than the standard Bonferroni method. Additional power may be achieved by using procedures that control the False Discovery Rate (the expected proportion of false positives among tests found to be significant). When the Holm’s method and two different false discovery rate procedures (FDR and pFDR) were applied to the results of multiple regression analyses of the relationship between habitat variables and abundance for 25 species of forest birds in Japan, the pFDR procedures provided the greatest statistical power.
Roback, P. J. and R. A. Askins. 2005. Judicious use of multiple hypothesis tests. Conservation Biology 19:261-267.
The views expressed in this paper are solely those of the author.