Unavoidable variation in experimental units

Identical experimental units result in the most accurate estimates of the effect of a factor. In practice however, we usually have little choice and the available experimental units are often very variable.

Sometimes we have no prior understanding of the variability of the experimental units. For example, we may know that the seeds in a batch of maize seeds are variable but cannot tell anything about the differences from looking at them. In this case, a completely randomised experiment is the best possible design.

Matched pairs

However if we have some knowledge about differences between the experimental units before the start of the experiment, we can use this to design the experiment better and therefore get a more accurate estimate of the effect of the experimental factor.

For example, consider an experiment involving a single factor with two levels. If the experimental units can be grouped into pairs that are similar, a better experimental design randomly allocates the two factor levels to the two experimental units in each each pair. This is called a matched pairs design.

As before, half of the experimental units get each factor level and the estimate of the effect is the difference between the mean response at the two factor levels. This estimate is however more accurate than the corresponding estimate from a completely randomised experiment.

Feed supplement and beef from cows

The diagram below simulates experiments using the same herd of calves that were used in the previous page. We now assume that the initial variability in calf weights is unavoidable.

Initially click Accumulate then click Conduct experiment several times to see the variability in the estimate of the feed supplement's effect in a completely randomised experiment. Observe that the estimate is very inaccurate.

Now select Paired by calf weight from the pop-up menu. In this experimental design, the calves are grouped into matched pairs with similar weights before the experiment is started, illustrated by the vertical bands on the scatterplot. In each of these matched pairs of calves, exactly one is randomly chosen to get the feed supplement.

Repeat this experiment several times and observe that estimate of the effect of the feed supplement is much more accurate than with the earlier completely randomised experimental design.

Matching of experimental units

In the above example, the experimental units were grouped into pairs using a numerical measurement — ages of animals or the previous year's yield from fruit trees could be used in a similar way. However pairing is often done in a less formal way — it is acceptable to construct the matched pairs of experimental units by any subjective or objective method.