Testing whether the probabilities are different
Testing for a difference between the population probabilities of 'success' in two groups is based on the sample proportions. Standardising the sample difference provides a test statistic that can be compared to the standard normal distribution to obtain an approximate p-value for the test.
Two-tailed test
Firstly, consider the two-tailed test,
H0 : π1 = π2
HA : π1 ≠ π2
The steps involved in obtaining a p-value for this test are shown in the diagram below
The p-value is interpreted in the same way as for all previous tests. A p-value close to zero is unlikely when H0 is true, but is more likely when HA holds. Small p-values therefore provide evidence of a difference between the population probabilities.
One-tailed test
For a 1-tailed test, the alternative hypothesis is that π2 is only on one side of π1.
HA : π1 − π2 > 0 or HA : π1 − π2 < 0
The test statistic is identical to that for a 2-tailed test and the p-value is obtained in a similar way, but it is found from only a single tail of the standard normal distribution.
Alternative test statistic
Most statisticians prefer to use a different formula for the standard deviation in the evaluation of the z-value above. Since π1 and π2 are equal if H0 is true, the overall proportion of successes, p, can be used in the formula for the standard deviation of p2 - p1.
This refinement makes little difference in practice, so the examples below use the 'simpler' formula that we gave earlier.
Examples
The diagram below shows how the p-values and conclusions are obtained for a selection of 1- and 2-tailed tests.