Testing for equal treatment means

In many data sets, there is no baseline treatment against which we might want to compare the other treatments. It is then unwieldy to analyse the data with confidence intervals for differences between all pairs of treatments.

Analysis should start with a single hypothesis test for whether all treatment means are equal.

Independent samples
If there is an independent sample from each treatment, we can perform a combined test for differences between the treatment means using a multi-group Analysis of Variance (anova) table, as described earlier in this chapter.
Linked samples
For data like the blocked and repeated measures examples on the previous page, this anova table should not be used. We will explain the details of a new test based on a modified anova table in the remainder of this section, but only mention at this stage that such a test exists and provides a p-value for testing whether the treatment means are equal.

In the following example, the p-value is shown and interpreted.

Grazing cattle in Uganda

As part of study on the habits of Zebu cattle in Uganda, observers recorded the time that each animal was grazing. In order to compare how accurately the observers made these measurements, an experiment was conducted in which five observers simultaneously watched the same group of 10 cattle for 88 minutes in one afternoon. Each person reported the number of minutes that each animal was observed grazing.

In this data set, we are interested in comparing the observers, but there is no 'baseline' observer.

Incorrect analysis based on independent samples
The anova table initially shows a test for whether the mean number of minutes grazing. This would have been an appropriate analysis if each observer had observed a different group of 10 cattle. The incorrect analysis would have concluded that there is no evidence of systematic differences between the observers.
Correct analysis
The same cattle were counted by all observers, so a different test should have been used. Select Correct analysis taking account of blocks to show the corresponding p-value for a correct analysis of the data. (The details of the test will be described later.) There is actually strong evidence of differences between the observers.