Avoiding lurking variables

The varying characteristics of the experimental units can only be lurking variables if they are associated with the allocation of treatments to the experimental units. To avoid this, ...

An important goal of experimental design is to minimise association between allocation of the treatments and characteristics of the experimental units.

The method depends on whether these varying characteristics of the experimental units are understood and measured before the experiment is conducted.

Randomisation

When the differing characteristics of the experimental units are unmeasured, the best way to avoid association between them and the treatments is to randomly allocate treatments to the experimental units. This is called randomisation of the treatments and the experimental design is called a completely randomised design.

Randomisation does not guarantee that there will be no association between the treatments and characteristics of the experimental units — by chance, there may be some association. However...

Randomisation means that is unlikely that such lurking variables will affect the conclusions.

There is no better way to allocate treatments if the varying characteristics of the experimental units are unmeasured before the experiment is conducted.

Effect of a packer on speed of checkout operators

In the experiment earlier in this section, the packer was provided for the 8 checkout operators on the Saturday morning shift. This resulted in younger and less experienced packers getting packers, so the effect of the packer was over-estimated.

Randomising the allocation of packers to 8 of the 16 operators reduces the chance of any association between the treatment and their age or experience.

Click Allocate treatments to randomly pick checkout operators to get packers then click Run experiment to find their speeds and estimate the effect of the packer.

Repeat a few times and observe that the effect of the packer is usually estimated to be between 0.07 and 0.13. The results are therefore consistent with the true effect that was built into the simulation, 0.1.

With randomisation, there is no tendency to over- or under-estimate the effect of the packer.


Mechanics of randomisation

There are several different ways to randomise allocation of treatments to the experimental units. These all use random numbers either from printed tables or, preferably, generated by a computer. The simplest method is to use a spreadsheet such as Microsoft Excel.

If there are n available experimental units,

This creates a random permutation of the numbers 1 to n. If there are two treatments, allocate the first to the experimental units in the top half of the list.

The table below illustrates this method.

The first column shows twelve random numbers, each between 0 and 1, and the list of unit numbers. Click Sort random numbers to sort the rows of the table into ascending order of the random numbers. This randomly permutes the unit numbers.

Other ways to randomise treatments (optional)

Although the above method of randomly allocating treatments to the experimental units using a spreadsheet is recommended, two alternative methods are now described. The first illustrates the randomisation better, whereas the second provides an interesting application of probability.

Randomisation of training course for 48 sales staff

The diagram below shows 48 staff, numbered 0 to 47. They will be split into three groups of 16 that will be given different courses to improve sales performance (courses A, B and C).

Click Random index to select a random number between 0 and 49. If either 48 or 49 are chosen, click the button again since these numbers do not correspond to any of the 48 staff. (We could have allowed the first digit to be 0-9 instead of 0-4, but over half of the resulting numbers would have been rejected for being 48 or higher. By restricting the first digit in the diagram, the simulation runs faster.)

The selected staff member is allocated to course A. Repeatedly click Random index to allocate more staff to course A. Once 16 people have been allocated to the group getting course A, staff are randomly allocated to course B. When 16 are in the group getting course B, the remainder get course C.

This method can be rather slow since many random indices are rejected towards the end of the method because treatments have already been allocated to these experimental units.

The final randomisation method works through all experimental units in order, picking a random treatment for each unit in turn. The only complication is that the probabilities used to generate the treatments must be adjusted after each unit gets a treatment. For example, if the first unit gets treatment A, then the probability of the second unit getting treatment A must be reduced a little. (If the same probabilities were used for each successive unit, then too many treatment A's might be allocated.)

Randomisation of training course to 30 sales staff

The diagram below illustrates this randomisation method for allocating equal numbers of a group of 30 sales staff to 3 training courses. The first person has equal probabilities for all three courses.

Click Generate Next to select a random value between 0 and 1, and hence a random treatment (i.e. course) for the first staff member. (Generation of a random category in this way was described earlier.) If the person gets treatment B, there will only be 10 A's, 9 B's and 10 C's left to allocate for the second person, so the probability of the second staff member getting treatment B should be reduced to 9/29. Click Update Probs to adjust the probabilities, then click Generate Next again to allocate a treatment to the second person.

Repeatedly click Update Probs and Generate Next until all individuals have been allocated to a treatment group.