Causes of variation

In a completely randomised experiment, there are two potential reasons why the response in an experiment varies between experimental units.

Treatments
The experiment is usually conducted specifically to determine how the treatments affect the response.
Random variation
This is a catch-all that refers to all variation in the response that cannot be explained in terms of the treatment. It can involve measurement errors and differences between the experimental units.

In a completely randomised experiment, all variation is caused by the treatments or is considered as random variation.

Distinguishing the treatment effect and random variation

Experiments are conducted to determine the effect of different treatments. It is therefore essential that we can distinguish the treatment effects from random variation.

There must be enough data to estimate random variation separately from variation caused by the treatments.

The easiest way to do this is with repeat measurements for each treatment — replication. Differences between these replicates are not due to treatment effects so they contain information about the amount of random variation.

Using knowledge about the amount of random variation in the experiment, we can better assess whether or not observed differences between the treatments are more than can be attributed to chance.

Crop experiment in a field

Researchers want to discover whether two varieties of wheat (A and B) have the same yield. One field is available for the experiment and it is know from previous experiments that the fertility and drainage of the soil is uniform over the whole field. The initial design for the experiment is shown below — the varieties were randomly allocated to either the left or right side of the field.

Since there was no replication, we cannot conclude that the observed difference in yields was due to the effects of the treatments.


Replication would mean growing each variety in two or more different areas. The simplest modification to the above experiment involves growing the wheat in the same way, but recording yields from smaller areas of the field.

Since we can now assess the random plot-to-plot variation, we can also assess whether the difference in yields can be attributed to the varieties of wheat.

Randomisation of replicates

In the above example, we assumed that the fertility of the soil was uniform over the whole field. In practice, this assumption can rarely be made. A natural fertility gradient across the field (left to right) would confound the variety used with fertility, making it impossible to tell whether higher yields for variety B would be caused by the variety or better soil in the plots on the right.

The risk of confounding variety and fertility makes the above experimental design bad.

Allocation of varieties to plots should be randomised to avoid the risk of fertility gradients biasing the results.


Good experimental design

The diagram below illustrates an experiment with random allocation of treatments to the 12 plots.