Estimation asks which values for an unknown parameter are consistent with data that have been collected. Hypothesis testing usually asks whether the data are consistent with some parameter values.
Are some teams in a soccer league better than others, or is the end-of-season league table consistent with random results from equally-matched teams? A simulation can help to answer this question.
This page presents a simulation to test whether the number of observations observed in one category is consistent with a specified probability.
This page uses a simulation to test whether a sample is consistent with a particular population mean.
A method is presented to assess whether the population means in two groups are the same.
The correlation between teams' final points in two successive seasons will be relatively high if some teams are better than others. Randomising points in the second season gives an indication of whether the actual correlation is unusually high.
All hypothesis tests involve a null hypothesis, a summary statistic for testing, an alternative hypothesis and a p-value.
When the data are a random sample, the null and alternative hypotheses are usually expressed in terms of population parameters. For categorical samples, the hypotheses refer to the probabilities of the categories.
The p-value for testing a proportion is the probability of getting such an 'extreme' number of successes when the null hypothesis holds. It can be found from the binomial distribution.
An example shows how to use a binomial distribution to find the p-value for a test.
If values in both tails of the binomial distribution support the alternative hypothesis, the tail probability must be doubled to give the p-value for the test.
When the sample size is large, a normal approximation to the binomial distribution can be used to find the p-value for a hypothesis test.
The statistical distance between an estimate and hypothesised parameter value is the difference divided by the standard error of the estimate. It should have approximately a standard normal distribution if the null hypothesis is true.
The p-value for testing a proportion can be evaluated as a tail area of a standard normal distribution corresponding to values more 'extreme' than the statistical distance between p and π.
For numerical populations, the null and alternative hypotheses usually specify values for the population mean. Tests are based on the sample mean.
If the population standard deviation is known, the distribution of the sample mean can be found when the null hypothesis is true. The p-value for the test is a tail area of this distribution.
The p-value is most easily found from the 'statistical distance' between the sample and hypothesised means. The p-value can be found as a tail area of the standard normal distribution and is approximately correct even when the population distribution is not normal.
When the population standard deviation is unknown, a different test statistic must be used. It has a standard distribution known as a t distribution.
The t distribution is used to obtain a p-value for tests about a population mean when the standard deviation is unknown.
In many applications, a different action is taken if the null hypothesis is 'accepted' or 'rejected'. Two different types of error are possible from such a decision — accepting the null hypothesis when it is false, or rejecting it when it is true.
The decision rule is based on a sample statistic. For tests about some parameters, the probabilities of Type I and Type II errors can be calculated.
Many tests are conducted with a pre-specified probability of a Type I error — the significance level. The null hypothesis is rejected if the p-value for the test is lower than the significance level.
The power of a hypothesis test is one minus the probability of a Type II error. At a fixed significance level, increasing the sample size improves the power of the test.
Hypothesis tests ask whether the sample data are consistent with a statement about the parameters called the null hypothesis.
A p-value is a numerical measure of whether the sample data are consistent with the null hypothesis.
When the null hypothesis is true, any p-value between 0 and 1 is equally likely. When the alternative hypothesis is true, p-values near 0 are more likely. A p-value near 0 therefore gives evidence that the alternative hypothesis holds.
A p-value is the probability of getting such 'extreme' data when the null hypothesis holds.
P-values have the same properties and interpretation for all tests. A test for whether a population has a normal distribution is used as an example.