Testing for independence
We now formally describe a hypothesis test for whether two categorical variables are independent.
H0 : X and Y are independent
HA : X and Y are dependent
We have seen that the χ2 statistic
describes whether the observed counts in a contingency table, nxy, are close to those expected for independent variables.
P-value
In a similar way to other hypothesis tests, we evaluate a p-value — the probability of getting such an extreme χ2 when the two variables are independent (H0).
If the p-value is close to zero, we conclude that the observed table would be unlikely for independent variables, so there is evidence that the variables are associated.
p-value | Interpretation |
---|---|
over 0.1 | no evidence against the null hypothesis (independence) |
between 0.05 and 0.1 | very weak evidence of dependence between the row and column variables |
between 0.01 and 0.05 | moderately strong evidence of dependence between the row and column variables |
under 0.01 | strong evidence of dependence between the row and column variables |
Warning about low estimated cell counts
The p-value for the test can be found because the χ2 test statistic has approximately a chi-squared distribution. This approximation is close for most data sets that are encountered, but is less so when the sample size, n, is small. The guidelines that are often given suggest that the p-value can be relied on if:
If the cell counts are small enought that these conditions do not hold, the p-value is less reliable. (But advanced statistical methods are required to do better!)
Simulation: Independent variables
The diagram below shows a random sample from a model in which the row and column variables are independent. It also illustrates how the p-value is evaluated.
Click Take sample a few times to generate other samples from the model. Observe that the p-value is usually quite large (since H0 is true), but
An 'unlucky sample' might mislead you into erroneously concluding that the variables are dependent.