Does random permutation tests for the fixed effects in a `REML`

analysis (R.W. Payne).

### Options

`PRINT` = string tokens |
Controls printed output (`prwald` , `criticalwald` , `ownstatistics` , `monitoring` ); default `prwa` , `crit` |

`NTIMES` = scalar |
Number of permutation samples to make; default 99 |

`NRETRIES` = scalar |
Maximum number of extra samples to take when some `REML` analyses fail to converge; default `NTIMES` |

`BLOCKSTRUCTURE` = formula |
Model formula defining any blocking to consider during the randomization; default |

`EXCLUDE` = factors |
Factors in the block formula whose levels are not to be randomized |

`SEED` = scalar |
Seed for random number generation; default 0 continues an existing sequence or, if none, selects a seed automatically |

`WMETHOD` = string token |
Controls which Wald statistics are used (`add` , `drop` ); default `add` |

`OWNMETHOD` = string token |
Type of test required for own statistics (`twosided` , `greaterthan` , `lessthan` ); default `twos` |

`CIPROBABILITY` = scalar |
Probability level for the confidence interval for own statistics; default 0.95 |

### Parameters

`SAVE` = `REML` save structures |
Specifies the (`REML` ) save structure of the original analysis; default `*` uses the `SAVE` structure from the most recent `REML` analysis |

`WALD` = pointers |
Wald statistics saved in a pointer with a variate for each term |

`PRWALD` = pointers |
Critical values for Wald statistics saved in a pointer with a scalar for each term |

`CRITICALWALD` = pointers |
Saves a pointer with variates for the 5%, 1% and 0.1% significance levels containing the corresponding critical values for the fixed terms, obtained from the quantiles of the Wald statistics from the permuted data sets |

`NNOTCONVERGED` = scalars |
Saves the number of permutations whose `REML` analysis failed to converge |

`OWNDATA` = pointers |
Data required to calculate own statistics |

`OWNOBSERVEDVALUES` = variates |
Saves observed values of the own statistics |

`OWNPROBABILITIES` = variates |
Saves probabilities for the own statistics |

`OWNESTIMATES` = variates |
Saves estimates for the own statistics |

`OWNSES` = variates |
Saves standard errors for the own statistics |

`OWNLOWERCIS` = variates |
Saves lower values of the confidence intervals for the own statistics |

`OWNUPPERCIS` = variates |
Saves upper values of the confidence intervals for the own statistics |

`OWNSTATISTICS` = pointers |
Saves the own statistics obtained from the permutation samples, in a pointer with a variate for each statistic |

### Description

`VPERMTEST`

performs a random permutation test for fixed effects in a `REML`

analysis. The `SAVE`

parameter can supply the save structure from the original analysis; if this is not set, the tests are done for the most recent `REML`

analysis.

The test probabilities are calculated by taking the proportion of Wald statistics in the permutation samples that are larger than the observed Wald statistic of each fixed term. (As a result these should not suffer from the bias that is found in the probabilities for the Wald statistics themselves, which tend to be too low.) The `WMETHOD`

option controls whether the Wald statistics are obtained from the table where terms are added sequentially (the default), or from the table where suitable terms are dropped from the full fixed model. Note that, if you use the table where terms are dropped, the only terms that can be tested are those that are not marginal to any other term in the fixed model: for example, the main effect `A`

cannot be tested if the model contains an interaction, such as `A.B`

.

The `NTIMES`

option defines how many random permutations to perform; by default there are 99 (as well as the “null” permutation where the data keep their original order). The `NRETRIES`

option specifies the maximum number of extra samples to take when some `REML`

analyses fail to converge; the default is to use the same number as specified by `NTIMES`

. The `SEED`

option allows you to specify the seed to use for the random-number generator that is used to construct the permutation samples. The default, `SEED=0`

, continues the sequence of random numbers from a previous generation or, if this is the first use of the generator in this run of Genstat, it initializes the seed automatically. If `NTIMES`

exceed the maximum possible number of permutations for the data, an “exact” test is performed in which every permutation is used once. This is feasible only for small datasets. There are *n*! (n factorial) permutations of *n* units: 3!=6, 4!=24, 5!=120, 6!=720, 7!=5040, 8!=40320, and so on. The `NNOTCONVERGED`

parameter can save the number of samples whose analyses did not converge, in a scalar.

If the data are from a designed experiment, you may need to use the `BLOCKSTRUCTURE`

option to specify a block model to define how to do the randomization. The `EXCLUDE`

option can then restrict the randomization so that one or more of the factors in the block model is not randomized. See the `RANDOMIZE`

directive for further details.

Output is controlled by the `PRINT`

option, with settings:

`prwald`

to print the probabilities calculated from the distribution of the Wald statistics;

`criticalwald`

to print a table giving estimated critical values for each of the Wald statistics, formed from the permutation samples;

`ownstatistics`

to print the estimates, standard errors and confidence intervals for the own statistics, and

`monitoring`

to monitor the progress of the test.

The Wald statistics from the permutation tests can be saved, in a pointer with a variate for each of the `FIXEDTERMS`

, using the `WALD`

parameter. The probabilities calculated from the tests can be saved, in a pointer with a scalar for each of the `FIXEDTERMS`

, using the `PRWALD`

parameter.

You can define your own statistics to be assessed by the test. They are calculated by a procedure `_VPERMownstatistics`

, which is called by `VPERMTEST`

following the `REML`

analysis of each permutation sample. Its use is shown in the `VPERMTEST`

example, which can be modified to calculate your own statistics instead. The information required by `_VPERMownstatistics`

to do the calculations is supplied, in a pointer, by the `OWNDATA`

parameter. The `OWNMETHOD`

option specifies the type of test to be made. The default, `twosided`

tests whether the statistics differ from zero. The `greaterthan`

setting tests whether they are greater than zero, and the `lessthan`

setting tests whether they are less than zero. Standard errors and confidence intervals are also calculated, The `CIPROBABILITY`

option specifies the probability for the confidence intervals (default 0.95). The `OWNOBSERVEDVALUES`

parameter can save a variate containing the values of the own statistics from the original data set. The `OWNPROBABILITIES`

can save a variate containing the probabilities from the tests. The `OWNESTIMATES`

can save a variate containing the bootstrap estimates of the statistics (calculated as the mean of the values obtained from the bootstrap samples) The `OWNSES`

can save a variate containing standard errors of bootstrap estimates. The `OWNLOWERCIS`

and `OWNUPPERCIS`

parameters can save variates containing the lower and upper values, respectively, of the confidence intervals. Finally, the `OWNSTATISTICS`

can save the values of the own statistics obtained from the bootstrap samples, in a pointer with a variate for each statistic.

The maximum number of iterations (`MAXCYCLE`

) and number of blocks of internal memory to be (`WORKSPACE`

) to be used in the `REML`

analyses can be set by a call to the `VAOPTIONS`

procedure before you use `VPERMTEST`

.

Options: `PRINT`

, `NTIMES`

, `NRETRIES`

, `BLOCKSTRUCTURE`

, `EXCLUDE`

, `SEED`

, `WMETHOD`

, `OWNMETHOD`

, `CIPROBABILITY`

.

Parameters: `SAVE`

, `WALD`

, `PRWALD`

, `CRITICALWALD`

, `NNOTCONVERGED`

, `OWNDATA`

, `OWNOBSERVEDVALUES`

, `OWNPROBABILITIES`

, `OWNESTIMATES`

, `OWNSES`

, `OWNLOWERCIS`

, `OWNUPPERCIS`

, `OWNSTATISTICS`

.

### See also

Directives: `REML`

, `VCOMPONENTS`

.

Procedures: `VAOPTIONS`

, `VBOOTSTRAP`

, `VRPERMTEST`

.

Commands for: REML analysis of linear mixed models.

### Example

CAPTION 'VPERMTEST example',!t('Split plot design, see the',\ 'Guide to Genstat, Part 2, Section 4.2.1.'); STYLE=meta,plain SPLOAD [PRINT=*] '%gendir%/data/Oats.gsh' VCOMPONENTS [FIXED=variety*nitrogen] RANDOM=blocks/wplots/subplots REML [PRINT=model,comp,Wald] yield VPERMTEST [NTIMES=99; SEED=15405] " Own statistics: test differences (in the full model) between variety Marvellous and each of the other two varieties " PROCEDURE [WORDLENGTH=long] '_VPERMownstatistics' PARAMETER NAME=\ 'DATA', "(I: pointer) information required to calculate the statistics"\ 'STATISTICS';"(O: variate) estimated statistics"\ MODE=p; TYPE='pointer','variate' " insert commands to calculate the statistics " VKEEP DATA[1]; MEANS=means VARIATE vmeans; VALUES=means VARIATE [NVALUES=DATA[2]] STATISTICS CALCULATE STATISTICS = vmeans$[DATA[3]] - vmeans$[DATA[4]] ENDPROCEDURE "_VPERMownstatistics" TEXT [VALUES='Marvellous - Victory','Marvellous - Golden rain'] Contrast POINTER [VALUES=variety,Contrast,!(3,3),!(1,2)] Owninfo VPERMTEST [PRINT=#,ownstatistics; NTIMES=99; SEED=251015] OWNDATA=Owninfo