Power of a test

For any decision rule, the significance level gives the probability of an error when H0 is true — the Type I error. It is common to describe what happens if HA is true with the probability of correctly picking HA. This is called the power of the test and is one minus the probability of a Type II error.

Decision
  accept H0     reject H0  
Truth H0 is true      Significance level =
P (Type I error)
HA (H0 is false)     P (Type II error) Power =
1 - P (Type II error)

When the alternative hypothesis includes a range of possible parameter values (e.g. µ ≠ 0), the power is not a single value but depends on the actual parameter value.

Increasing the power of a test

It is clearly desirable to use a test whose power is as close to 1.0 as possible. There are three different ways to increase the power.

Change the critical value
This cannot be done if the significance level is fixed — adjusting the critical value to increase the power also increases the probability of a Type I error.
Use a different decision rule
In this e-book, we only describe the most powerful type of decision rule to test any hypotheses, so you will not be able to increase the power by changing the decision rule.
Increase the sample size
By increasing the amount of data on which we base our decision about whether to accept or reject H0, the probabilities of making both types of error can be reduced.

When the significance level is fixed, increasing the sample size is usually the only way to improve the power.