A general framework

The examples in earlier pages of this section involved different types of data and different analyses. Indeed, you may find it difficult to spot their common theme!

All analyses were examples of hypothesis testing. We now describe the general framework of hypothesis testing within which all of these examples fit. This general framework is the basis for important applications in later sections of CAST.

The concepts in this page are extremely important — make sure that you understand them well before moving on.

Data, model and question

Data (and model)
Each example dealt with a data set that was assumed to arise from some random mechanism. We may be able to specify some aspects of this random mechanism (model), but it also has unknown characteristics
Null hypothesis
All models had unknown characteristics, and we want to know whether the model has particular properties — the null hypothesis.
Alternative hypothesis
If the null hypothesis is not true, we say that the alternative hypothesis holds. (You can understand most of hypothesis testing without paying much attention to the alternative hypothesis however!)

Either the null hypothesis or the alternative hypothesis must be true.

Approach

We assess whether the null hypothesis is true by asking ...

Are the data consistent with the null hypothesis?

It is extremely important that you understand that hypothesis testing addresses this question — make sure that you remember it well!!

Answering the question

Test statistic
This is some function of the data that throws light on whether the null or alternative hypothesis holds.
P-value
Testing whether the data are consistent with the null hypothesis is based on the probability of obtaining a test statistic value as 'extreme' as the one recorded if the null hypothesis holds. This is called the p-value for the test.
Interpreting the p-value
Although it may be regarded as an over-simplification, the table below can be used as a guide to interpreting p-values.
p-value Interpretation
over 0.1 no evidence that the null hypothesis does not hold
between 0.05 and 0.1 very weak evidence that the null hypothesis does not hold
between 0.01 and 0.05 moderately strong evidence that the null hypothesis does not hold
under 0.01 strong evidence that the null hypothesis does not hold

Use the pop-up menu below to check how the earlier examples in this section fit into the hypothesis testing framework.

Soccer league in one season

Data (and model)
Some random mechanism underlies the actual results in the matches during a season. The probabilities of winning may vary from team to team and there may be a home-team advantage, so there are a lot of unknowns about this model! Our data are a single set of results — the league table at the end of the season.
Null hypothesis
The null hypothesis is that all teams are equally matched — i.e. that they all have the same probability of winning each match.
Alternative hypothesis
The alternative hypothesis is that all teams do not have the same probabilities of winning.
Test statistic
The standard deviation of final points is used. It will be low if the teams have the same abilities (null hypothesis) and higher otherwise (alternative hypothesis).
P-value
We simulated the soccer league, assuming that all teams had the same probability of winning. The p-value was the probability of getting a standard deviation of final points as high as 19.3 (the actual data).
Interpreting the p-value
The p-value was 0.000 (or close). Since there is virtually no chance of getting a standard deviation of points as high as that in the actual league from equally matched teams, we conclude that the teams are not equally matched — the null hypothesis is false.