1. Home
  2. AMCOMPARISON procedure

AMCOMPARISON procedure

Performs pairwise multiple comparison tests for ANOVA means (D.M. Smith).

Options

PRINT = string tokens Controls printed output (comparisons, critical, description, lines, letters, plot, mplot, pplot); default lett
METHOD = string token Test to be performed (tukey, snk, regwmr, duncan, scheffe, fplsd, fulsd, bonferroni, sidak); default fpls
FACTORIAL = scalar Limit on the number of factors in each term; default 3
DIRECTION = string token How to sort means (ascending, descending); default asce
PROBABILITY = scalar The required significance level; default 0.05
STUDENTIZE = string token Whether to use the alternative LSD test where the Studentized Range statistic is used instead of Student’s t (yes, no); default no
SAVE = ANOVA save structure Save structure to provide the tables of means and associated information; default uses the save structure from the most recent ANOVA

Parameters

TERMS = formula Treatment terms whose means are to be compared
MEANS = pointer or variate Saves the (sorted) means
DIFFERENCES = pointer or symmetric matrix Saves differences between the (sorted) means
LABELS = pointer or text Saves labels for the (sorted) means
LETTERS = pointer or text Saves letters indicating groups of means that do not differ significantly
SIGNIFICANCE = pointer or symmetric matrix Indicators to show significant comparisons between (sorted) means
CIWIDTH = pointer or symmetric matrix Saves the width of the confidence interval for the absolute differences between the (sorted) means

Description

AMCOMPARISON performs a range of all pairwise multiple comparison tests (see Hsu 1996 and Bechhofer, Santner & Goldsman 1995). The methodology implemented in the procedure closely follows that described in Chapter 5 of Hsu (1996).

The TERMS parameter specifies a model formula to define the treatment terms whose means are to be compared. The means (and the necessary associated information) are usually taken from the most recent analysis of variance (performed by ANOVA), but you can set the SAVE option to a save structure from another ANOVA if you want to examine means from an earlier analysis. As in ANOVA, the FACTORIAL option sets a limit on the number of factors in each term (default 3).

Printed output is controlled by the PRINT option, with settings:

    comparisons prints the differences between the pair of means, upper and lower confidence limits for the differences, t-statistics and an indication of whether or not they are significant;
    critical gives critical values for the t-statistic for situations where these do not vary amongst the comparisons (i.e. for the Scheffe, Bonferroni and Sidak methods, as well as the Fisher LSD methods provided all the comparisons have the same number of residual degrees of freedom);
    description provides a description including information such as the experiment-wise and compartment-wise error rates;
    lines gives the means, with lines joining those that do not differ significantly;
    letters gives the means, with identical letters (a, b etc.) alongside those that do not differ significantly;
    mplot does a mean-mean scatter plot (synonym plot);
    pplot displays the probabilities in a shade plot.

By default, PRINT=letters.

The means are usually sorted into ascending order, but you can set option DIRECTION=descending for descending order, or DIRECTION=* to leave them in their original order. Note, though, that the lines joining means with non-significant differences may then be broken.

If the standard errors for the differences between the means are unequal (as will happen, for example, if the means have unequal replication), the memberships of the groups defined by the lines or letters may be inconsistent. Suppose, for example, you have ordered means A, B and C. If the s.e.d. for A vs. C is large compared to those for A vs. B and B vs C, you might find that there is no significant difference between A and C, but there are significant differences between A and B, and between B and C. So treatments A and B and treatments B and C would be in different groups. However, treatments A and C (which are further apart) would be in the same group. This contradicts the idea behind multiple comparisons, where you expect that if two means are in the same group, than any mean between them should be in that group too. If AMCOMPARISON finds inconsistencies like this, it gives a diagnostic and suppresses the printing of lines and letters (but not the other types of output).

The mean-mean scatter plot allows you to assess the confidence region for the difference between each pair of means visually. It has grid lines from both the x- and y-axis at the position of each mean, and a diagonal line at 45 degrees marking y=x. The confidence interval for each pair of means is plotted as a line at an angle of -45 degrees and centred on the intersection above the line y=x of the grid lines for the two means (so the y grid line is for the larger of the two means, and the x grid line is for the smaller mean). The difference between the means is significant if their confidence line does not intersect the line y=x. For more details, see Hsu (1996) pages 151-153.

The shade plot displays the probabilities in a symmetric matrix. The colour of each cell represents the probability for the difference between the means for the treatments in the corresponding row and column.

The type of test to be performed is specified by the METHOD option, with settings Tukey, SNK (Student-Newman-Keuls), REGWMR (Ryan/Einot-Gabriel/Welsch multiple range test), Duncan, Scheffe, FPLSD (Fisher’s Protected Least Significant Difference), FULSD (Fisher’s Unprotected Least Significant Difference), Bonferroni and Sidak. The PROBABILITY option allows the pair-wise significance level for the intervals from the Fisher tests to be changed from the default 0.05 (e.g. to 0.01). For the other tests, it changes the experiment-wise significance level. The STUDENTIZE option can specify that the Fisher’s protected or unprotected LSD tests should use the Studentized Range statistic rather than Student’s t (for further information see Hsu 1996, page 139).

The MEANS parameter can save the means, sorted according to the DIRECTION option and omitting any that were non-estimable. If the TERMS parameter specifies a single term, MEANS should be set to a variate. If TERMS specifies several terms, you must supply a pointer which will then be set up to contain as many variates as there are terms. Similarly the LABELS parameter can save labels to identify the means, in either a text (for a single term) or in a pointer of texts (for several). Likewise the LETTERS parameter can save texts with the letters identifying means that do not differ significantly, and the SIGNIFICANCE parameter can save symmetric matrices containing ones or zeros according to whether the various comparisons were significant or non-significant. The DIFFERENCES parameter can save symmetric matrices containing the differences between the (sorted) means, and the CIWIDTH parameter can save symmetric matrices containing the widths of the confidence intervals for the differences.

Options: PRINT, METHOD, FACTORIAL, DIRECTION, PROBABILITY, STUDENTIZE, SAVE.

Parameter: TERMS, MEANS, DIFFERENCES, LABELS, LETTERS, SIGNIFICANCE, CIWIDTH.

Method

The methodology implemented is based on that described and reviewed in Hsu (1996), and Bechhofer, Santner & Goldsman (1995). For specific details of the tests these books should be referred to.

References

Bechhofer, R.E., Santner, T.J. & Goldsman, D.M. (1995). Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons. Wiley, New York.

Hsu, J.C. (1996). Multiple Comparisons Theory and Methods. Chapman & Hall, London.

See also

Directive: ANOVA.

Procedures: AUMCOMPARISON, AMDUNNETT, CONFIDENCE, MCOMPARISON, STEEL, VMCOMPARISON.

Commands for: Analysis of variance.

Example

CAPTION        'AMCOMPARISON example',\
               !t('Experiment to study 5 diets for rats, using',\
               'a randomized-block design where the blocks are',\
               '8 litters each of 5 rats (John & Quenouille, 1977,',\
               'Experiments Design and Analysis, page 32)');\
               STYLE=meta,plain
FACTOR         [NVALUES=40; LEVELS=8] Litter
&              [LEVELS=5] Rat
&              [LABELS=!t(A,B,C,D,E)] Diet
GENERATE       Litter,Rat
READ           Diet,Gain; FREPRESENTATION=labels
E 76.0  C 70.7  D 68.3  A 57.0  B 64.8
A 55.0  D 67.1  B 66.6  C 59.4  E 74.5
C 64.5  A 62.1  D 69.1  E 76.5  B 69.5
D 72.7  B 61.1  A 74.5  C 74.0  E 86.6
A 86.7  E 94.7  B 91.8  D 90.6  C 78.5
B 51.8  C 55.8  E 43.2  A 42.0  D 44.3
D 53.8  A 71.9  C 63.0  B 69.2  E 61.1
E 54.4  D 40.9  B 48.6  C 48.1  A 51.5 :
BLOCKSTRUCTURE Litter/Rat
TREATMENTS     Diet
ANOVA          [FPROBABILITY=yes] Gain
AMCOMPARISON   Diet
CAPTION 'The unprotected test ignores the fact that Diet is not significant.'
AMCOMPARISON   [PRINT=letters,plot; METHOD=FULSD] Diet
Updated on September 13, 2019

Was this article helpful?