1. Home
  2. APERMTEST procedure

APERMTEST procedure

Does random permutation tests for analysis-of-variance tables (R.W. Payne).

Options

PRINT = string tokens Controls printed output (aovtable, critical); default aovt
PLOT = string What to plot (histogram); default *
NTIMES = scalar Number of permutations to make; default 999
EXCLUDE = factors Factors in the block model of the design whose levels are not to be randomized
SEED = scalar Seed for the random number generator used to make the permutations; default 0 continues from the previous generation or (if none) initializes the seed automatically
AOVTABLE = pointer Saves the aov-table, with permutation probabilities
CRITICAL = pointer Saves the aov-table, with critical values
SAVE = ANOVA save structure Save structure from the analysis of variance; default uses the save structure from the most recent ANOVA

No parameters

Description

Random permutation tests provide an alternative to using the F probabilities printed for variance ratios in an analysis-of-variance table in situations where the assumptions of the analysis are not satisfied. These assumptions can be assessed by studying the residual plots produced by APLOT. In particular, the use of the F distribution to calculate the probabilities is based on the assumption that the residuals from each stratum have Normal distributions with equal variances, and so the histogram of residuals produced by APLOT should look reasonably close to the Normal, bell-shaped curve. Experience shows the analysis is robust to small departures from Normality. APERMTEST can be useful if the histogram looks very non-Normal (and you are unable to redefine the analysis as a generalized linear model; see FIT).

The simplest form of use is simply to specify the command

APERMTEST

straight after the ANOVA. APERMTEST recovers the necessary information about the analysis automatically, and performs 999 random permutations (made using a default seed). The probability for each variance ratio is then determined from its distribution over the randomly permuted datasets.

The NTIMES option of APERMTEST lets you request another number of permutations, and the SEED option lets you specify another seed. APERMTEST checks whether NTIMES is greater than the number of possible permutations available for the data set. If so, APERMTEST does an exact test instead, which uses each possible permutation once.

The information about the analysis is obtained from the save structure of the most recent ANOVA (which is stored automatically within Genstat). You can save the information from any analysis of variance explicitly using the SAVE parameter of ANOVA. You can then perform permutation tests for that analysis by using the save structure as the setting of the SAVE option of APERMTEST. The EXCLUDE option allows you to restrict the randomization so that one or more of the factors in the block model is not randomized. The most common instance where this is required is when one of the treatment factors involves time-order, which cannot be randomized.

Output is controlled by the PRINT option, with settings:

    aovtable for an analysis-of-variance table with the usual F probabilities replaced by those from the permutation test; and
    critical for a table giving critical values for each variance ratio.

These can be saved using the AOVTABLE and CRITICAL parameters.

You can set PLOT=histogram to plot histograms showing the variance ratios obtained for each treatment term in the original analysis and the analyses of the permuted data sets.

Options: PRINT, PLOT, NTIMES, EXCLUDE, SEED, AOVTABLE, CRITICAL, SAVE.

Parameters: none.

Method

If there is no blocking and the treatment combinations have more than one replicate, APERMTEST uses SETALLOCATIONS to determine the number of unique permutations so that it can see whether an exact test is possible. If so, the permutations are formed using SETALLOCATIONS too. Otherwise the number of possible permutations is calculated using the FACTORIAL function, and the permutations for the exact test are formed using the PERMUTE procedure. For a permutation test, RANDOMIZE is used to perform the permutations, taking account of the block structure of the design. The AOVTABLE option of AKEEP is used to save the variance ratios, and the QUANTILES function is used to calculate the critical values.

Action with RESTRICT

APERMTEST takes account of any restrictions on the y-variate in the analysis of variance (i.e. the variate specified as the Y parameter in the earlier ANOVA command).

See also

Directive: ANOVA.

Procedures: CHIPERMTEST, RPERMTEST.

Commands for: Analysis of variance.

Example

CAPTION    'APERMTEST examples',\
           !t('1) Data from Snedecor and Cochran, Statistical Methods',\
           '(7th edition, 1980), page 216 and also see page 252.');\ 
           STYLE=meta,plain
FACTOR     [LEVELS=4; VALUES=(1...4)6] Fat
VARIATE    [VALUES=64,78,75,55, 72,91,93,66, 68,97,78,49,\ 
                   77,82,71,64, 56,85,63,70, 95,77,76,68] Absorbed
TREATMENTS Fat
ANOVA      [PRINT=aov; FPROBABILITY=yes] Absorbed
APERMTEST  [SEED=452546]
CAPTION    !t('2) Split plot design, see the Guide to Genstat,',\
           'Part 2, Section 4.2.1.')
FACTOR     [NVALUES=72; LEVELS=6] Blocks
&          [LEVELS=3] Wplots
&          [LEVELS=4] Subplots
GENERATE   Blocks,Wplots,Subplots
FACTOR     [LABELS=!T('0 cwt','0.2 cwt','0.4 cwt','0.6 cwt')] Nitrogen
&          [LABELS=!T(Victory,'Golden rain',Marvellous)]      Variety
VARIATE    Yield; DECIMALS=2; EXTRA=' of oats in cwt. per acre'
READ       [SERIAL=yes] Nitrogen,Variety,Yield
 4 3 2 1 1 2 4 3 1 2 3 4 3 1 2 4 4 1 2 3 2 1 3 4
 2 3 4 1 4 2 3 1 1 4 2 3 3 4 1 2 1 3 4 2 2 3 4 1
 4 1 3 2 3 4 1 2 3 4 2 1 3 1 4 2 4 3 1 2 1 2 3 4 :
 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2
 2 2 2 2 3 3 3 3 1 1 1 1 3 3 3 3 2 2 2 2 1 1 1 1
 2 2 2 2 1 1 1 1 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 :
156 118 140 105 111 130 174 157 117 114 161 141
104  70  89 117 122  74  89  81 103  64 132 133
108 126 149  70 144 124 121  96  61 100  91  97
109  99  63  70  80  94 126  82  90 100 116  62
 96  60  89 102 112  86  68  64 132 124 129  89
118  53 113  74 104  86  89  82  97  99 119 121 :
CALCULATE  Yield=(Yield*80)/(112*4)
VARIATE    [VALUES=0,0.2,0.4,0.6] Nitlev
TREATMENTS POL(Nitrogen;2;Nitlev) * Variety
BLOCKS     Blocks/Wplots/Subplots
ANOVA      [PRINT=aov] Yield
APERMTEST  [PRINT=aovtable,critical]
Updated on March 8, 2019

Was this article helpful?