Performs pairwise multiple comparison tests for means from an unbalanced analysis of variance, performed previously by AUNBALANCED
(D.M. Smith).
Options
PRINT = string tokens 
Controls printed output (comparisons , critical , description , lines , letters , plot , mplot , pplot) ; default lett 

METHOD = string token 
Test to be performed (flsd , bonferroni , sidak ); default flsd 
FACTORIAL = scalar 
Limit on the number of factors in each term; default 3 
COMBINATIONS = string token 
Factor combinations for which to form predicted means (present , estimable ); default esti 
ADJUSTMENT = string token 
Type of adjustment to be made when predicting means (marginal , equal , observed ); default marg 
WEIGHTS = table 
Weights classified by some or all of the factors in the model 
DIRECTION = string token 
How to sort means (ascending , descending ); default asce 
PROBABILITY = scalar 
The required significance level; default 0.05 
STUDENTIZE = string token 
Whether to use the alternative LSD test where the Studentized Range statistic is used instead of Student’s t (yes , no ); default no 
SAVE = identifier 
Save structure to provide the table of means; default uses the save structure from the most recent AUNBALANCED analysis 
Parameters
TERMS = formula 
Treatment terms whose means are to be compared 

MEANS = pointer or variate 
Saves the (sorted) means 
DIFFERENCES = pointer or symmetric matrix 
Saves differences between the (sorted) means 
LABELS = pointer or text 
Saves labels for the (sorted) means 
LETTERS = pointer or text 
Saves letters indicating groups of means that do not differ significantly 
SIGNIFICANCE = pointer or symmetric matrix 
Indicators to show significant comparisons between (sorted) means 
CIWIDTH = pointer or symmetric matrix 
Saves the width of the confidence interval for the absolute differences between the (sorted) means 
Description
AUMCOMPARISON
can be used following an analysis by AUNBALANCED
to perform all pairwise multiple comparison tests on tables of predicted means. The methodology implemented in the procedure closely follows that described in Chapter 5 of Hsu (1996).
The TERMS
parameter specifies a model formula to define the treatment terms whose means are to be compared. The means are usually taken from the most recent analysis performed by AUNBALANCED
, but you can set the SAVE
option to a save structure from another AUNBALANCED
if you want to examine means from an earlier analysis. The FACTORIAL
option sets a limit on the number of factors in each term (default 3).
The predicted means are formed using the AUPREDICT
procedure. The COMBINATIONS
, ADJUSTMENT
and WEIGHTS
options control how this is done; see AUPREDICT
for more details.
Printed output is controlled by the PRINT
option, with settings:
comparisons 
prints the differences between the pair of means, upper and lower confidence limits for the differences, tstatistics and an indication of whether or not they are significant; 

critical 
gives critical values for the tstatistic for situations where these do not vary amongst the comparisons (i.e. for the Scheffe, Bonferroni and Sidak methods, as well as the Fisher LSD methods provided all the comparisons have the same mumber of residual degrees of freedom); 
description 
provides a description including information such as the experimentwise and compartmentwise error rates; 
lines 
gives the means, with lines joining those that do not differ significantly; 
letters 
gives the means, with identical letters (a, b etc.) alongside those that do not differ significantly; 
mplot 
does a meanmean scatter plot (synonym plot ); 
pplot 
displays the probabilities in a shade plot.

By default, PRINT=letters
.
The means are usually sorted into ascending order, but you can set option DIRECTION=descending
for descending order, or DIRECTION=*
to leave them in their original order. Note, though, that the lines joining means with nonsignificant differences may then be broken.
In most unbalanced anova’s the standard errors for the differences between the means will be unequal, and the memberships of the groups defined by the lines or letters may then be inconsistent. Suppose, for example, you have ordered means A, B and C. If the s.e.d. for A vs. C is large compared to those for A vs. B and B vs C, you might find that there is no significant difference between A and C, but there are significant differences between A and B, and between B and C. So treatments A and B and treatments B and C would be in different groups. However, treatments A and C (which are further apart) would be in the same group. This contradicts the idea behind multiple comparisons, where you expect that if two means are in the same group, than any mean between them should be in that group too. If AUMCOMPARISON
finds inconsistencies like this, it gives a diagnostic and suppresses the printing of lines and letters (but not the other types of output).
The meanmean scatter plot allows you to assess the confidence region for the difference between each pair of means visually. It has grid lines from both the x and yaxis at the position of each mean, and a diagonal line at 45 degrees marking y=x. The confidence interval for each pair of means is plotted as a line at an angle of 45 degrees and centred on the intersection above the line y=x of the grid lines for the two means (so the y grid line is for the larger of the two means, and the x grid line is for the smaller mean). The difference between the means is significant if their confidence line does not intersect the line y=x. For more details, see Hsu (1996) pages 151153.
The shade plot displays the probabilities in a symmetric matrix. The colour of each cell represents the probability for the difference between the means for the treatments in the corresponding row and column.
The type of test to be performed is specified by the METHOD
option, with settings FLSD
(Fisher’s Unprotected Least Significant Difference), Bonferroni
and Sidak
. The PROBABILITY
option allows the experimentwise significance level for the intervals from the Bonferroni and Sidak tests to be changed from the default 0.05 (e.g. to 0.01). For the
Fisher’s test, it changes the pairwise significance level. The STUDENTIZE
option can specify that the Fisher’s protected or unprotected LSD tests should use the Studentized Range statistic rather than Student’s t (for further information see Hsu 1996, page 139).
The MEANS
parameter can save the means, sorted according to the DIRECTION
option and omitting any that were nonestimable. If the TERMS
parameter specifies a single term, MEANS
should be set to a variate. If TERMS
specifies several terms, you must supply a pointer which will then be set up to contain as many variates as there are terms. Similarly the LABELS
parameter can save labels to identify the means, in either a text (for a single term) or in a pointer of texts (for several). Likewise the LETTERS
parameter can save texts with the letters identifying means that do not differ significantly, and the SIGNIFICANCE
parameter can save symmetric matrices containing ones or zeros according to whether the various comparisons were significant or nonsignificant. The DIFFERENCES
parameter can save symmetric matrices containing the differences between the (sorted) means, and the CIWIDTH
parameter can save symmetric matrices containing the widths of the confidence intervals for the differences.
Options: PRINT
, METHOD
, FACTORIAL
, COMBINATIONS
, ADJUSTMENT
, WEIGHTS
, DIRECTION
, PROBABILITY
, STUDENTIZE
, SAVE
.
Parameter: TERMS
, MEANS
, DIFFERENCES
, LABELS
, LETTERS
, SIGNIFICANCE
, CIWIDTH
.
Method
The methodology implemented is based on that described in Hsu (1996).
Reference
Hsu, J.C. (1996). Multiple Comparisons Theory and Methods. Chapman & Hall, London.
See also
Procedures: AUNBALANCED
, AUDISPLAY
, AUGRAPH
, AUPREDICT
, AUKEEP
, AMCOMPARISON
, AMDUNNETT
, MCOMPARISON
, VMCOMPARISON
.
Commands for: Analysis of variance.
Example
CAPTION 'AUMCOMPARISON example',\ !t('Experiment on foster feeding of rats from Scheffe (1959)',\ 'The Analysis of Variance; also see McConway, Jones & Taylor (1999)',\ 'Statistical Modelling using GENSTAT, Example 7.6.');\ STYLE=meta,plain FACTOR [NVALUES=61; LABELS=!t('A','B','I','J')] litter READ litter; FREPRESENTATION=labels A A A A A A A A A A A A A A A A A B B B B B B B B B B B B B B B I I I I I I I I I I I I I I J J J J J J J J J J J J J J J : FACTOR [NVALUES=61; LABELS=!t('A','B','I','J')] mother READ mother; FREPRESENTATION=labels A A A A A B B B I I I I J J J J J A A A A B B B B B I I I I J J A A A B B B I I I I I J J J A A A A B B B I I I J J J J J : VARIATE [NVALUES=61] littwt READ littwt 61.5 68.2 64 65 59.7 55 42 60.2 52.5 61.8 49.5 52.7 42 54 61 48.2 39.6 60.3 51.7 49.3 48 50.8 64.7 61.7 64 62 56.5 59 47.2 53 51.3 40.5 37 36.3 68 56.3 69.8 67 39.7 46 61.3 55.3 55.7 50 43.8 54.5 59 57.4 54 47 59.5 52.8 56 45.2 57 61.4 44.8 51.5 53 42 54 : TREATMENTSTRUCTURE litter * mother AUNBALANCED [PRINT=aovtable,means; FPROBABILITY=yes] littwt AUMCOMPARISON mother