![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Statistical Power Analysis
Statistical power is the probability of detecting a change given that a change has truly occurred. For a reasonable test of a hypothesis, power should be >0.8 for a test. A value of 0.9 for power translates into a 10% chance that we will miss a conclude that a change has occurred when indeed it has not. This is because the statistical power of a test is equal to 1 - Type II error.
Power analysis provides a framework to translate the variability of a
biological indicator into estimates of its precision and repeatability.
After testing to be sure that the chosen indicator is truly measuring
some meaningful condition, the next step is to figure out whether the
indicator can detect a reasonable level of change. Typically this boils
down to how variable the indicator is: high variance for repeat samples
will translate into low power to detect change. Low variance (high repeatability)
of an indicator translates in good statistical power.
Statistical power is a function of the amount of change (or effect size)
that you are trying to detect, and the level of uncertainty you are willing
to accept, the sample size, the variance of the indicator, and the statistical
model you are using for testing.
In words this equation says, "Statistical power is proportional to the effect size multiplied by the alpha-level of the test, multiplied by the square root of the sample size, and divided by the standard deviation (square root of the variance)."
Effect size - The larger the difference you are trying
to detect, the greater chance you will have of detecting it. This relationship
is summarized in the equation above such that large values for the effect
size also increase the power of the test.
Alpha (Type I error) - If you are willing to get a false
positive more often, you will be more likely to detect a change. In the
equation above, a larger alpha translates into a greater statistical power.
For power analysis, the alpha-level is often relaxed from the traditional
0.05 to 0.1.
N - A larger sample size will increase the
probability of detecting a change. Power increases with N.
Variance - The estimate of variance used in power calculations
typically refers to measurement error associated with repeat sampling.
This could be the MSE from an ANOVA or the variance calculated for replicate
samples. Statistical power and variance are inversely related,
when the variance goes up the power goes down.
Keep in mind that the results of the power analysis depend on the statistical
test being used. For example, a different number of samples will likely
be needed to detect a 10% change for a regression model than for an ANOVA
model.
Power analysis: Examples
Minimum detectable difference (MDD)
Power analysis can be used to determine the size of the minimum detectable difference between a sample in hand vs. some future sample not yet collected. The MDD gives an estimate of how different the mean values of the two samples would have to be in order to be significantly different. Note that the MDD represents the smallest difference that would be significant, any larger value would also be significantly different. This is a good approach when comparisons will be made using a t test or ANOVA.
Trend detection through time
When multiple samples will be collected through time and the anticipated pattern of response is a gradual increase (or decrease), power analysis should be applied to a regression model. For example, the indicator values could be regressed against year. For this approach, the number of years can be manipulated along with the number of sites sampled each year to determine the most efficient sampling design for monitoring.