Random unbalance in allocation of treatments

Randomisation tries to ensure that the different treatments are on average applied to similar experimental units. However by chance there are usually some differences.

Because of randomisation, these differences are likely to be small, but they will still result in over- or under-estimation of the effect of the factor. The example below is more extreme than would be likely occur in a completely randomised experiment but it does illustrate how inbalance can affect results.

Difference between two barley varieties

The yield for barley variety V2 is 2.1 higher than that for variety V2. The yield also increases with the moisture content of the soil but V2 always has yield 2.1 higher than V1, whatever the moisture content.

We shall pretent that the effect of soil moisture was unknown and observe how its variability affects our estimate of the difference between the two barley varieties.

Consider a completely randomised experiment involving 20 runs of the experiment that is conducted to estimate the effect of the varieties. Soil moisture is not controlled but varies randomly between experimental plots. Randomisation means that there is equal chance of the varieties being used at high and low moisture levels, but by chance, the plots getting variety V1 may be either hotter or colder than those getting V2.

Initially the diagram shows the effect of moisture and variety on yield. Since we are not using the covariate in the analysis, the estimated variety effect is simply the difference between the mean yields of the plots getting V1 and V2. Select Artificial data from the pop-up menu to see possible results. Use the slider to see the effect of 'unbalance' in the allocation of treatments.

  • If V2 gets used more in low moisture plots than in high moisture plots, its improvement over V1 is underestimated.
  • If it is used more in high moisture plots, its improvement over V1 is overestimated.

Variation in the experimental units (caused by variation in soil moisture) can therefore have a considerable effect on the accuracy of the estimated difference between the varieties if it is not recorded and used in the model.

Experiments in agriculture and biology usually have much more random variation. Click the checkbox Extra variation to add more unexplained random variation to the data and repeat. Observe that the same effect persists.

Completely randomised experiment

In a completely randomised experiment, the most extreme forms of unbalance are unlikely. Select Randomised experiment from the pop-up menu then click Repeat experiment a few times to observe the variation in the estimated difference between the varieties.

Although the experimental units getting V2 are usually reasonably balanced between high and low soil moistures, even smaller degrees of unbalance add to the variability of the estimates.