Alternative analysis of variance tables
If an experiment has equal replicates of all treatments or if the two factors are orthogonal in other ways (as described on the previous page), the sums of squares explained by the two factors are the same whichever order the factors are added to the model.
Experiments are usually designed in this way, but poor design or missing values can result in data that are not orthogonal. If the factors are not orthogonal, there are two different analysis of variance tables corresponding to the two orders of adding the factors.
Example
The diagram below initially shows results from an experiment with two factors. Eight replicates have been used for the two treatments with X at level X1 and four replicates for the two treatments with X at level X2.
Since the two factors are initially orthogonal, their explained sums of squares do not depend on the the order of adding them to the model. Drag the red arrows to the left of the sums of squares table to change the order of adding the factors and observe that the sum of squares explained by X is the same whether it is added first to the model or after Z has already been added. (The sum of squares for Z similarly does not depend on the order of adding the terms.)
Now drag the slider to the left to reduce the number of replicates for X = X2 and Z = Z2. (These might be missing values in practice.) The resulting data are not orthogonal, so the sums of squares differ depending on the order of adding the two factors to the model. Drag to reorder the two rows of the table and observe that the sum of squares for X depends on whether the term is added first or second.
The bars to the right of the table are proportional to the sums of squares. The red section is the difference between the sum of squares when the term is added first or second.
Note that the difference between the alternative sums of squares (the length of the red section) depends on how far the design is from orthogonality. If there is only a single missing value, there is vary little difference. Some software will estimate the missing value so that a balanced anova table can be presented. The resulting sums of squares are intermediate between those of the two orders of adding the terms to the model.
Yield of chemical process
The diagram below shows results from an experiment in which two amounts of catalyst and two concentrations of the main ingredient are used. This experiment is badly unbalanced with only two replicates when both factors are at their high level and when both factors are at their low level, but seven replicates for the other two treatments.
The two red arrows to the left of the anova table can again be dragged to change the order of adding the two factors. There are major differences between the p-values with the two orderings.
We would therefore conclude that Catalyst affects the yield of the chemical process — it is important whether or not Concentration is used in the model. However there is only very weak evidence of the additional importance of Concentration.