Strength of evidence against H0

We have explained how p-values describe the strength of evidence against the null hypothesis.

Saturated fat content of cooking oil

It has been claimed that the saturated fat content of soybean cooking oil is no more than 15%. A clinician believes that the saturated fat content is greater than 15% and randomly samples 13 bottles of soybean cooking oil for testing.

Percentage saturated fat in soybean cooking oil
15.2
12.4
15.4
13.5
15.9
17.1
16.9
14.3
19.1
18.2
15.5
16.3
20.0

The clinician is interested in the following hypotheses.

H0 :   μ = 15%
HA :   μ > 15%

The p-value of 0.04 means that there is moderately strong evidence against H0 — i.e. moderately strong evidence that the mean saturated fat content is greater than 15%.

Decisions from tests

We now take a different (but related) approach to hypothesis testing.

Many hypothesis tests are followed by some action that depends on whether we conclude from the test results that H0 or HA is true. This decision depends on the data.

Decision    Action
accept H0    some action (often the status quo)   
reject H0    a different action (often a change to a process)   

However the decision that is made could be wrong. There are two ways in which an error might be made — wrongly rejecting H0 when it is true (called a Type I error), and wrongly accepting H0 when it is false (called a Type II error). These are represented by the red cells in the table below:

Decision
  accept H0     reject H0  
True state of nature H0 is true    correct Type I error
HA (H0 is false)     Type II error correct

A good decision rule about whether to accept or reject H0 (and perform the corresponding action) will have small probabilities for both kinds of error.

Saturated fat content of cooking oil

The clinician who tested the saturated fat content of soybean cooking oil was interested in the hypotheses.

H0 :   μ = 15%
HA :   μ > 15%

If H0 is rejected, the clinician intends to report the high saturated fat content to the media. The two possible errors that could be made are described below.

Decision
  accept H0  
(do nothing)
  reject H0  
(contact media)
Truth H0: µ is really 15% (or less)    correct wrongly accuses manufacturers
HA: µ is really over 15%     fails to detect high saturated fat correct

Ideally the decision should be made in a way that keeps both probabilities low.