Does random permutation tests for analysis-of-variance tables (R.W. Payne).

### Options

`PRINT` = string tokens |
Controls printed output (`aovtable` , `critical` ); default `aovt` |
---|---|

`PLOT` = string |
What to plot (`histogram` ); default `*` |

`NTIMES` = scalar |
Number of permutations to make; default 999 |

`EXCLUDE` = factors |
Factors in the block model of the design whose levels are not to be randomized |

`SEED` = scalar |
Seed for the random number generator used to make the permutations; default 0 continues from the previous generation or (if none) initializes the seed automatically |

`AOVTABLE` = pointer |
Saves the aov-table, with permutation probabilities |

`CRITICAL` = pointer |
Saves the aov-table, with critical values |

`SAVE` = ANOVA save structure |
Save structure from the analysis of variance; default uses the save structure from the most recent `ANOVA` |

### No parameters

### Description

Random permutation tests provide an alternative to using the F probabilities printed for variance ratios in an analysis-of-variance table in situations where the assumptions of the analysis are not satisfied. These assumptions can be assessed by studying the residual plots produced by `APLOT`

. In particular, the use of the F distribution to calculate the probabilities is based on the assumption that the residuals from each stratum have Normal distributions with equal variances, and so the histogram of residuals produced by `APLOT`

should look reasonably close to the Normal, bell-shaped curve. Experience shows the analysis is robust to small departures from Normality. `APERMTEST`

can be useful if the histogram looks very non-Normal (and you are unable to redefine the analysis as a generalized linear model; see `FIT`

).

The simplest form of use is simply to specify the command

`APERMTEST`

straight after the `ANOVA`

. `APERMTEST`

recovers the necessary information about the analysis automatically, and performs 999 random permutations (made using a default seed). The probability for each variance ratio is then determined from its distribution over the randomly permuted datasets.

The `NTIMES`

option of `APERMTEST`

lets you request another number of permutations, and the `SEED`

option lets you specify another seed. `APERMTEST`

checks whether `NTIMES`

is greater than the number of possible permutations available for the data set. If so, `APERMTEST`

does an exact test instead, which uses each possible permutation once.

The information about the analysis is obtained from the save structure of the most recent `ANOVA`

(which is stored automatically within Genstat). You can save the information from any analysis of variance explicitly using the `SAVE`

parameter of `ANOVA`

. You can then perform permutation tests for that analysis by using the save structure as the setting of the `SAVE`

option of `APERMTEST`

. The `EXCLUDE`

option allows you to restrict the randomization so that one or more of the factors in the block model is not randomized. The most common instance where this is required is when one of the treatment factors involves time-order, which cannot be randomized.

Output is controlled by the `PRINT`

option, with settings:

`aovtable` |
for an analysis-of-variance table with the usual F probabilities replaced by those from the permutation test; and |
---|---|

`critical` |
for a table giving critical values for each variance ratio. |

These can be saved using the `AOVTABLE`

and `CRITICAL`

parameters.

You can set `PLOT=histogram`

to plot histograms showing the variance ratios obtained for each treatment term in the original analysis and the analyses of the permuted data sets.

Options: `PRINT`

, `PLOT`

, `NTIMES`

, `EXCLUDE`

, `SEED`

, `AOVTABLE`

, `CRITICAL`

, `SAVE`

.

Parameters: none.

### Method

If there is no blocking and the treatment combinations have more than one replicate, `APERMTEST`

uses `SETALLOCATIONS`

to determine the number of unique permutations so that it can see whether an exact test is possible. If so, the permutations are formed using `SETALLOCATIONS`

too. Otherwise the number of possible permutations is calculated using the `FACTORIAL`

function, and the permutations for the exact test are formed using the `PERMUTE`

procedure. For a permutation test, `RANDOMIZE`

is used to perform the permutations, taking account of the block structure of the design. The `AOVTABLE`

option of `AKEEP`

is used to save the variance ratios, and the `QUANTILES`

function is used to calculate the critical values.

### Action with `RESTRICT`

`APERMTEST`

takes account of any restrictions on the y-variate in the analysis of variance (i.e. the variate specified as the `Y`

parameter in the earlier `ANOVA`

command).

### See also

Directive: `ANOVA`

.

Procedures: `CHIPERMTEST`

, `RPERMTEST`

.

Commands for: Analysis of variance.

### Example

CAPTION 'APERMTEST examples',\ !t('1) Data from Snedecor and Cochran, Statistical Methods',\ '(7th edition, 1980), page 216 and also see page 252.');\ STYLE=meta,plain FACTOR [LEVELS=4; VALUES=(1...4)6] Fat VARIATE [VALUES=64,78,75,55, 72,91,93,66, 68,97,78,49,\ 77,82,71,64, 56,85,63,70, 95,77,76,68] Absorbed TREATMENTS Fat ANOVA [PRINT=aov; FPROBABILITY=yes] Absorbed APERMTEST [SEED=452546] CAPTION !t('2) Split plot design, see the Guide to Genstat,',\ 'Part 2, Section 4.2.1.') FACTOR [NVALUES=72; LEVELS=6] Blocks & [LEVELS=3] Wplots & [LEVELS=4] Subplots GENERATE Blocks,Wplots,Subplots FACTOR [LABELS=!T('0 cwt','0.2 cwt','0.4 cwt','0.6 cwt')] Nitrogen & [LABELS=!T(Victory,'Golden rain',Marvellous)] Variety VARIATE Yield; DECIMALS=2; EXTRA=' of oats in cwt. per acre' READ [SERIAL=yes] Nitrogen,Variety,Yield 4 3 2 1 1 2 4 3 1 2 3 4 3 1 2 4 4 1 2 3 2 1 3 4 2 3 4 1 4 2 3 1 1 4 2 3 3 4 1 2 1 3 4 2 2 3 4 1 4 1 3 2 3 4 1 2 3 4 2 1 3 1 4 2 4 3 1 2 1 2 3 4 : 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 1 1 1 1 3 3 3 3 2 2 2 2 1 1 1 1 2 2 2 2 1 1 1 1 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 : 156 118 140 105 111 130 174 157 117 114 161 141 104 70 89 117 122 74 89 81 103 64 132 133 108 126 149 70 144 124 121 96 61 100 91 97 109 99 63 70 80 94 126 82 90 100 116 62 96 60 89 102 112 86 68 64 132 124 129 89 118 53 113 74 104 86 89 82 97 99 119 121 : CALCULATE Yield=(Yield*80)/(112*4) VARIATE [VALUES=0,0.2,0.4,0.6] Nitlev TREATMENTS POL(Nitrogen;2;Nitlev) * Variety BLOCKS Blocks/Wplots/Subplots ANOVA [PRINT=aov] Yield APERMTEST [PRINT=aovtable,critical]