Testing whether the probabilities are different

Testing for a difference between the population probabilities of 'success' in two groups is based on the sample proportions. Standardising the sample difference provides a test statistic that can be compared to the standard normal distribution to obtain an approximate p-value for the test.

Two-tailed test

Firstly, consider the two-tailed test,

H0 :   π1  =  π2
HA :   π1  ≠  π2

The steps involved in obtaining a p-value for this test are shown in the diagram below

The p-value is interpreted in the same way as for all previous tests. A p-value close to zero is unlikely when H0 is true, but is more likely when HA holds. Small p-values therefore provide evidence of a difference between the population probabilities.

One-tailed test

For a 1-tailed test, the alternative hypothesis is that π2 is only on one side of π1.

HA :   π1  −  π2  >  0    or    HA :   π1  −  π2  <  0

The test statistic is identical to that for a 2-tailed test and the p-value is obtained in a similar way, but it is found from only a single tail of the standard normal distribution.

Alternative test statistic

Most statisticians prefer to use a different formula for the standard deviation in the evaluation of the z-value above. Since π1 and π2 are equal if H0 is true, the overall proportion of successes, p, can be used in the formula for the standard deviation of p2 - p1.

This refinement makes little difference in practice, so the examples below use the 'simpler' formula that we gave earlier.

Examples

The diagram below shows how the p-values and conclusions are obtained for a selection of 1- and 2-tailed tests.