Testing for independence

We now formally describe a hypothesis test for whether two categorical variables are independent.

H0 :  X and Y are independent
HA :  X and Y are dependent  

We have seen that the χ2 statistic

describes whether the observed counts in a contingency table, nxy, are close to those expected for independent variables.

X and Y are independent
χ2 has (approximately) a chi-squared distribution with no unknown parameters
X and Y are associated
The pattern of observed counts, nxy, is expected to be different from that of the exy, so χ2 is expected to be larger.

P-value

In a similar way to other hypothesis tests, we evaluate a p-value — the probability of getting such an extreme χ2 when the two variables are independent (H0).

If the p-value is close to zero, we conclude that the observed table would be unlikely for independent variables, so there is evidence that the variables are associated.

p-value Interpretation
over 0.1 no evidence against the null hypothesis (independence)
between 0.05 and 0.1    very weak evidence of dependence between the row and column variables
between 0.01 and 0.05    moderately strong evidence of dependence between the row and column variables
under 0.01 strong evidence of dependence between the row and column variables

Warning about low estimated cell counts

The p-value for the test can be found because the χ2 test statistic has approximately a chi-squared distribution. This approximation is close for most data sets that are encountered, but is less so when the sample size, n, is small. The guidelines that are often given suggest that the p-value can be relied on if:

If the cell counts are small enought that these conditions do not hold, the p-value is less reliable. (But advanced statistical methods are required to do better!)

Simulation: Independent variables

The diagram below shows a random sample from a model in which the row and column variables are independent. It also illustrates how the p-value is evaluated.

Click Take sample a few times to generate other samples from the model. Observe that the p-value is usually quite large (since H0 is true), but


An 'unlucky sample' might mislead you into erroneously concluding that the variables are dependent.