Significance level
Performing hypothesis tests through interpretation of p-values and through decision rules with fixed significance levels are closely related.
Although some of the underlying theory depends on the type of test, the decision rule for any test can be based on its p-value. For example, for a test with significance level 5%, the decision rule is always:
Decision | |
---|---|
p-value > 0.05 | accept H0 |
p-value < 0.05 | reject H0 |
For example, to conduct a test with significance level 1%, the null hypothesis, H0, should be rejected if the p-value is less than 0.01.
If computer software provides the p-value for a hypothesis test, it is therefore easy to translate it into a decision about whether to accept (or reject) the null hypothesis at the 5% or 1% significance level.
Illustration
We again investigate decision rules for testing the hypotheses
H0 : \(\mu = 10\)
HA : \(\mu \gt 10\)
based on a sample of \(n=16\) values from a normal distribution with known standard deviation \(\sigma = 4\).
In the diagram, the decision rule is based on the p-value for the test. Use the slider to adjust the critical p-value and observe that the significance level (probability of Type I error) is always equal to the p-value used in the decision rule. Adjust the critical p-value to 0.01.
Although the probability of a Type II error on the bottom row of the above table varies depending on the type of test, the top row in the diagram is the same for all kinds of hypothesis test.