1. Home
2. APERMTEST procedure

APERMTEST procedure

Does random permutation tests for analysis-of-variance tables (R.W. Payne).

Options

`PRINT` = string tokens Controls printed output (`aovtable`, `critical`); default `aovt` What to plot (`histogram`); default `*` Number of permutations to make; default 999 Factors in the block model of the design whose levels are not to be randomized Seed for the random number generator used to make the permutations; default 0 continues from the previous generation or (if none) initializes the seed automatically Saves the aov-table, with permutation probabilities Saves the aov-table, with critical values Save structure from the analysis of variance; default uses the save structure from the most recent `ANOVA`

Description

Random permutation tests provide an alternative to using the F probabilities printed for variance ratios in an analysis-of-variance table in situations where the assumptions of the analysis are not satisfied. These assumptions can be assessed by studying the residual plots produced by `APLOT`. In particular, the use of the F distribution to calculate the probabilities is based on the assumption that the residuals from each stratum have Normal distributions with equal variances, and so the histogram of residuals produced by `APLOT` should look reasonably close to the Normal, bell-shaped curve. Experience shows the analysis is robust to small departures from Normality. `APERMTEST` can be useful if the histogram looks very non-Normal (and you are unable to redefine the analysis as a generalized linear model; see `FIT`).

The simplest form of use is simply to specify the command

`APERMTEST`

straight after the `ANOVA`. `APERMTEST` recovers the necessary information about the analysis automatically, and performs 999 random permutations (made using a default seed). The probability for each variance ratio is then determined from its distribution over the randomly permuted datasets.

The `NTIMES` option of `APERMTEST` allows you to request another number of permutations, and the `SEED` option allows you to specify another seed. `APERMTEST` checks whether `NTIMES` is greater than the number of possible permutations available for the data set. If so, `APERMTEST` does an exact test instead, which uses each possible permutation once.

The information about the analysis is obtained from the save structure of the most recent `ANOVA` (which is stored automatically within Genstat). You can save the information from any analysis of variance explicitly using the `SAVE` parameter of `ANOVA`. You can then perform permutation tests for that analysis by using the save structure as the setting of the `SAVE` option of `APERMTEST`. The `EXCLUDE` option allows you to restrict the randomization so that one or more of the factors in the block model is not randomized. The most common instance where this is required is when one of the treatment factors involves time-order, which cannot be randomized.

Output is controlled by the `PRINT` option, with settings:

    `aovtable` for an analysis-of-variance table with the usual F probabilities replaced by those from the permutation test; and for a table giving critical values for each variance ratio.

These can be saved using the `AOVTABLE` and `CRITICAL` parameters.

You can set `PLOT=histogram` to plot histograms showing the variance ratios obtained for each treatment term in the original analysis and the analyses of the permuted data sets.

Options: `PRINT`, `PLOT`, `NTIMES`, `EXCLUDE`, `SEED`, `AOVTABLE`, `CRITICAL`, `SAVE`.

Parameters: none.

Method

If there is no blocking and the treatment combinations have more than one replicate, `APERMTEST` uses `SETALLOCATIONS` to determine the number of unique permutations so that it can see whether an exact test is possible. If so, the permutations are formed using `SETALLOCATIONS` too. Otherwise the number of possible permutations is calculated using the `FACTORIAL` function, and the permutations for the exact test are formed using the `PERMUTE` procedure. For a permutation test, `RANDOMIZE` is used to perform the permutations, taking account of the block structure of the design. The `AOVTABLE` option of `AKEEP` is used to save the variance ratios, and the `QUANTILES` function is used to calculate the critical values.

Action with `RESTRICT`

`APERMTEST` takes account of any restrictions on the y-variate in the analysis of variance (i.e. the variate specified as the `Y` parameter in the earlier `ANOVA` command).

See also

Directive: `ANOVA`.

Procedures: `CHIPERMTEST`, `RPERMTEST`.

Commands for: Analysis of variance.

Example

```CAPTION    'APERMTEST examples',\
!t('1) Data from Snedecor and Cochran, Statistical Methods',\
'(7th edition, 1980), page 216 and also see page 252.');\
STYLE=meta,plain
FACTOR     [LEVELS=4; VALUES=(1...4)6] Fat
VARIATE    [VALUES=64,78,75,55, 72,91,93,66, 68,97,78,49,\
77,82,71,64, 56,85,63,70, 95,77,76,68] Absorbed
TREATMENTS Fat
ANOVA      [PRINT=aov; FPROBABILITY=yes] Absorbed
APERMTEST  [SEED=452546]
CAPTION    !t('2) Split plot design, see the Guide to Genstat,',\
'Part 2, Section 4.2.1.')
FACTOR     [NVALUES=72; LEVELS=6] Blocks
&          [LEVELS=3] Wplots
&          [LEVELS=4] Subplots
GENERATE   Blocks,Wplots,Subplots
FACTOR     [LABELS=!T('0 cwt','0.2 cwt','0.4 cwt','0.6 cwt')] Nitrogen
&          [LABELS=!T(Victory,'Golden rain',Marvellous)]      Variety
VARIATE    Yield; DECIMALS=2; EXTRA=' of oats in cwt. per acre'
READ       [SERIAL=yes] Nitrogen,Variety,Yield
4 3 2 1 1 2 4 3 1 2 3 4 3 1 2 4 4 1 2 3 2 1 3 4
2 3 4 1 4 2 3 1 1 4 2 3 3 4 1 2 1 3 4 2 2 3 4 1
4 1 3 2 3 4 1 2 3 4 2 1 3 1 4 2 4 3 1 2 1 2 3 4 :
3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2
2 2 2 2 3 3 3 3 1 1 1 1 3 3 3 3 2 2 2 2 1 1 1 1
2 2 2 2 1 1 1 1 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 :
156 118 140 105 111 130 174 157 117 114 161 141
104  70  89 117 122  74  89  81 103  64 132 133
108 126 149  70 144 124 121  96  61 100  91  97
109  99  63  70  80  94 126  82  90 100 116  62
96  60  89 102 112  86  68  64 132 124 129  89
118  53 113  74 104  86  89  82  97  99 119 121 :
CALCULATE  Yield=(Yield*80)/(112*4)
VARIATE    [VALUES=0,0.2,0.4,0.6] Nitlev
TREATMENTS POL(Nitrogen;2;Nitlev) * Variety
BLOCKS     Blocks/Wplots/Subplots
ANOVA      [PRINT=aov] Yield
APERMTEST  [PRINT=aovtable,critical]
```
Updated on June 20, 2019