Testing for equal treatment means
In many data sets, there is no baseline treatment against which we might
want to compare the other treatments. It is then unwieldy to analyse the data
with confidence intervals for differences between all pairs of treatments.
Analysis should start with a single hypothesis test for whether all treatment
means are equal.
- Independent samples
- If there is an independent sample from each treatment, we can perform
a combined test for differences between the treatment means using a multi-group
Analysis of Variance (anova) table, as described earlier in this chapter.
- Linked samples
- For data like the blocked and repeated measures examples on the previous
page, this anova table should not be used. We will explain the details of
a new test based on a modified anova table in the remainder of this section,
but only mention at this stage that such a test exists and provides a p-value
for testing whether the treatment means are equal.
In the following example, the p-value is shown and interpreted.
Grazing cattle in Uganda
As part of study on the habits of Zebu cattle in Uganda, observers recorded
the time that each animal was grazing. In order to compare how accurately
the observers made these measurements, an experiment was conducted in which
five observers simultaneously watched the same group of 10 cattle for 88
minutes in one afternoon. Each person reported the number of minutes that
each animal was observed grazing.
In this data set, we are interested in comparing the observers, but there
is no 'baseline' observer.
- Incorrect analysis based on independent samples
- The anova table initially shows a test for whether the mean number of
minutes grazing. This would have been an appropriate analysis if each
observer had observed a different group of 10 cattle.
The incorrect analysis would have concluded
that there is no evidence of systematic differences between the observers.
- Correct analysis
- The same cattle were counted by all observers, so a different test should
have been used. Select Correct analysis taking account of blocks
to show the corresponding p-value for a correct analysis of the data.
(The details of the test will be described later.) There
is actually strong evidence of differences between the observers.