Random unbalance in allocation of treatments

Randomisation tries to ensure that the different treatments are on average applied to similar experimental units. However by chance there are usually some differences.

Because of randomisation, these differences are likely to be small, but they will still result in over- or under-estimation of the effect of the factor. The example below is more extreme than would be likely occur in a completely randomised experiment but it does illustrate how inbalance can affect results.

Effect of catalyst on reaction

Addition of a catalyst to a chemical reaction increases its yield by 2.1. The yield also increases as the reaction temperature increases, but the effect of the catalyst is the same whatever the temperature.

We shall pretent that the effect of temperature was unknown and observe how its variability affects our estimate of the effect of the catalyst.

Consider a completely randomised experiment involving 20 runs of the experiment that is conducted to estimate the effect of the catalyst. Temperature is not controlled but varies randomly between the runs of the experiment. Randomisation means that there is equal chance of the catalyst being used at high and low temperatures, but by chance, the runs of the experiment with the catalyst may be either hotter or colder than those without.

Initially the diagram shows the effect of temperature and the catalyst on yield. Since we are not using the covariate in the analysis, the estimated effect is simply the difference between the mean yields of the runs with and without the catalyst. Select Artificial data from the pop-up menu to see possible results. Use the slider to see the effect of 'unbalance' in the allocation of treatments.

  • If the catalyst gets used more at low temperatures than at high temperatures, its effect is underestimated.
  • If it is used more at high temperatures, its effect is overestimated.

Variation in the experimental units (caused by variation in the temperatures) can therefore have a considerable effect on the accuracy of the estimated effect of the catalyst if it is not recorded and used in the model.

Experiments in agriculture and biology usually have much more random variation. Click the checkbox Extra variation to add more unexplained random variation to the data and repeat. Observe that the same effect persists.

Completely randomised experiment

In a completely randomised experiment, the most extreme forms of unbalance are unlikely. Select Randomised experiment from the pop-up menu then click Repeat experiment a few times to observe the variation in the estimated effect of the catalyst.

Although the experimental units getting the catalyst are usually reasonably balanced between high and low temperature, even smaller degrees of unbalance add to the variability of the estimates.