Revisiting p-values

A hypothesis test is based on two competing hypotheses about the value of a parameter, \(\theta\). The null hypothesis is the simpler one, restricting \(\theta\) to a single value, whereas the alternative hypothesis allows for a range of values. Initially we consider a 1-tailed alternative,

The hypothesis test is based on a test statistic,

\[ Q \;\;=\;\; g(X_1, X_2, \dots, X_n \mid \theta_0) \]

whose distribution is fully known (without any unknown parameters) when H0 is true (i.e. when \(\theta_0\) is the true parameter value). The p-value for the test is the probability of a test statistic value as extreme as that observed in the data. A p-value close to zero throw doubt on the null hypothesis.

Fixed significance level

An alternative way to perform the test involves a rule that results in a decision about which of the two hypotheses holds. Any such data-based rule can lead us to the wrong decision, so we must take into account the probability of this.

Definition

The significance level is the probability of wrongly concluding that H0 does not hold when it actually does.

For example, it might be acceptable to have a 5% chance of concluding that \(\theta \gt \theta_0\) when \(\theta\) is really equal to \(\theta_0\). This means a significance level of \(\alpha = 0.05\) for the test.

To attain this significance level, we should reject H0 when the test statistic, \(Q\), falls in the "rejection region" below.

Two-tailed tests

For a two-tailed test, values of the test statistic in both tails of its distribution usually provide evidence that H0 does not hold, so the rejection region should correspond to area \(\diagfrac {\alpha} 2\) in each tail to attain a significance level of \(\alpha\).

Saturated fat content of cooking oil

Cooking oil made from soybeans has little cholesterol and has been claimed to have only 15% saturated fat. A clinician believes that the saturated fat content is greater than 15% and randomly samples 13 bottles of soybean cooking oil for testing.

Percentage saturated fat in soybean cooking oil
15.2
12.4
15.4
13.5
15.9
17.1
16.9
14.3
19.1
18.2
15.5
16.3
20.0

Assuming that the data are a random sample from a normal distribution, the clinician wants to test the following hypotheses.

H0 :   \(\mu = 15%\)
HA :   \(\mu \gt 15%\)

What is his conclusion from testing these hypotheses with a significance level of \(\alpha = 5%\)?

(Solved in full version)