Uses a parametric bootstrap to estimate critical values for a fixed term in a `REML`

analysis (R.W. Payne & C.J. Brien).

### Options

`PRINT` = string tokens |
Prints the critical values (`critical` , `fcritical` , `tcritical` , `wcritical` ); default `crit` , `fcri` , `tcri` , `wcri` |
---|---|

`VPRINT` = string tokens |
Controls the output from the `REML` analyses (`model` , `components` , `effects` , `means` , `stratumvariances` , `monitoring` , `vcovariance` , `deviance` , `Waldtests` , `missingvalues` , `covariancemodels` ); default `*` i.e. none |

`TERM` = formula |
Fixed term to be tested |

`UMEANS` = variate |
Specifies the expected values for the units under the null hypothesis of no effects from the `TERM` ; default is to use the constant from the `SAVE` structure |

`UVCOVARIANCE` = symmetric matrix |
Specifies the variances and covariances of the units under the null hypothesis of no effects from the `TERM` ; default is to take this from the `SAVE` structure |

`WCRITICAL` = variate |
Saves the critical values of the Wald statistic |

`FCRITICAL` = variate |
Saves the critical values of the F statistic |

`NBOOT` = scalar |
Number of bootstrap samples to take; default 99 |

`NRETRIES` = scalar |
Maximum number of extra samples to take when some `REML` analyses fail to converge; default `NBOOT` |

`SEED` = scalar |
Seed for random number generation; default 0 continues an existing sequence or, if none, selects a seed automatically |

`PROBABILITIES` = scalar or variate |
Significance levels for which critical values are required; default 0.05 |

`METHOD` = string token |
Indicates whether to use the Fisher-scoring algorithm or the AI algorithm with sparse matrix methods (`Fisher` , `AI` ); default `AI` |

`MAXCYCLE` = scalar |
Sets a limit on the number of iterations in the `REML` analyses; default 30 |

`FMETHOD` = string token |
Controls how to calculate estimated denominator degrees of freedom when these are to be saved (`automatic` , `none` , `algebraic` , `numerical` ); default `auto` |

`WMETHOD` = string token |
Controls which Wald statistics are saved (`add` , `drop` ); default `add` |

`TMETHOD` = string token |
Type of test to be made for the contrasts (`twosided` , `greaterthan` , `lessthan` , `equivalence` , `noninferiority` ); default `twos` |

`WALD` = variate |
Saves the Wald statistics from the samples |

`FSTATISTIC` = variate |
Saves the F statistics from the samples |

`NDF` = scalar |
Saves the numerator degrees of freedom for the Wald and F statistics |

`DDF` = variate |
Saves the estimated denominator degrees of freedom for the F statistics |

`NNOTCONVERGED` = scalar |
Saves the number of bootstrap samples whose `REML` analysis failed to converge |

`WORKSPACE` = scalar |
Number of blocks of internal memory to be set up for use by the `REML` algorithm |

`SAVE` = vsave |
`REML` save structure to provide the information about the analysis |

### Parameters

`XCONTRASTS` = variates or tables |
X-variate defining a contrast to be detected |
---|---|

`CONTRASTTYPE` = string tokens |
Type of contrast (`regression` , `comparison` ) default `rege` |

`ESTIMATE` = variates |
Saves the estimated values of the contrasts from the samples |

`SE` = variates |
Saves the standard errors for the estimates of the contrasts from the samples |

`CRITICAL` = variates |
Saves the critical values for the contrasts |

`TCRITICAL` = variates |
Saves the critical values for the t-statistics of the contrasts |

### Description

The conventional way to assess fixed terms in a `REML`

analysis is to use either the Wald or the F tests, in the table of tests for fixed effects that is produced by setting option `PRINT=wald`

in either `REML`

or `VDISPLAY`

. The Wald tests have the disadvantage of being biased, i.e. they tend to generate significant results too frequently. The F tests are more reliable. However, their denominator degrees of freedom need to be estimated, using the method of Kenward & Roger (1997), and this may not be feasible for some data sets. These denominator degrees of freedom can also be used in t-tests to assess contrasts amongst the effects of a term; see procedure `VTCOMPARISONS`

. However, those tests must be used with caution, as the degrees of freedom are relevant for assessing the fixed term as a whole, and may differ over the various contrasts.

`VCRITICAL`

provides an alternative method of assessment, that may be useful if the decision from the conventional tests is not clear-cut, or if contrasts are to be assessed. It uses a parametric bootstrap, in the same way as the `VBOOTSTRAP`

procedure. However, it differs from `VBOOTSTRAP`

, in that it generates critical values, rather than assessing the significance of terms in a specific data set. These critical values can be used test hypotheses with a specific data set, and the critical values for the F, Wald and t-statistics may be useful with similar data sets. The critical values for the t-statistics also allow you to determine the size of the contrast that may be detectable in these investigations.

The model to be fitted must be defined using the `VCOMPONENTS`

and `VSTRUCTURE`

directives, in the usual way. The bootstrap samples are generated from a multivariate Normal distribution with dimension equal to the number of units in the analysis. The `UMEANS`

option supplies the expected values for the distribution. This should contain the fitted values under the null model for the term being tested. The `UVCOVARIANCE`

option supplies the variances and covariances of the units. If either `UMEANS`

or `UVCOVARIANCE`

is not specified, defaults are taken from the `REML`

analysis supplied by the `SAVE`

option, or from the most recent `REML`

if `SAVE`

is not set. For `UMEANS`

the default is a variate containing the constant estimated in that analysis. For `UVCOVARIANCE`

it is the unit-by-unit variance-covariance matrix from the analysis (see the `UVCOVARIANCE`

option of `VKEEP`

). Note: you can use the `VUVCOVARIANCE`

procedure to form the variance-covariance matrix, if you know the variance components for a `REML`

model that contains no covariance models.

The `NBOOT`

option specifies the number of bootstrap samples to take (default 99). The `NRETRIES`

option specifies the maximum number of extra samples to take when some `REML`

analyses fail to converge; the default is to use the same number as specified by `NBOOT`

. The `SEED`

option supplies the seed for the random number generator used to form the samples; default 0 continues from the previous generation or (if none) initializes the seed automatically. The `NNOTCONVERGED`

option can save the number of samples whose analyses did not converge, in a scalar.

The fixed term to be assessed is specified by the `TERM`

option. If the term is a main effect (i.e. if `TERM`

contains just one factor) you can use the `XCONTRASTS`

parameter to specify variates or tables containing the coefficients defining the contrasts amongst the effects of the term. The `CONTRASTTYPE`

option indicates whether each of these is a regression contrast (as specified in analysis of variance by the `REG`

function) or a comparison (as specified by the `COMPARISON`

function).

The `TMETHOD`

option specifies the type of test that is to be used to assess the contrasts, with the following settings.

`twosided` |
assumes a two-sided test to assess whether the contrast differs from zero (default). |
---|---|

`lessthan` |
assumes a one-sided test to assess whether the contrast is less than zero. |

`greaterthan` |
assumes a one-sided test to assess whether the contrast is greater than zero. |

`noninferiority` |
assumes a test to check that the contrast is not significantly less then zero. (See Method for more details.) |

`equivalence` |
assumes a one-sided test to check that the contrast does not differ significantly from zero; see Method for more details. |

The `PROBABILITIES`

option specifies the significance levels for which you want to obtain critical values; the default is 0.05, i.e. 5%.

Printed output is controlled buy the `PRINT`

option, with the following settings.

`critical` |
prints critical values for the contrasts, |
---|---|

`fcritical` |
prints critical values for the F statistics, |

`tcritical` |
prints critical values for the t-statistics of the contrasts, |

`wcritical` |
prints critical values for the Wald statistics, |

`nnotconverged` |
prints the number of bootstrap samples whose analysis failed to converge, and |

`monitoring` |
prints monitoring information, showing the progress of the bootstrap sampling. |

By default, all the critical values printed.

The critical values for the contrasts and their t-statistics can be saved, in variates, by the `CRITICAL`

and `TCRITICAL`

parameters, respectively. The critical values for the F and Wald statistics can be saved, again in variates by the `FCRITICAL`

and `WCRITICAL`

options.

You can also save the values estimated for the various statistics, in the analyses of the bootstrap samples, in variates (with a unit for each sample). Those for the contrasts and their standard errors can be saved the `ESTIMATES`

and `SE`

parameters, respectively. The F and Wald statistics can be saved by the `FSTATISTIC`

and `WALD`

options. The degrees of freedom for the Wald statistics and numerator degrees for the F statistics can be saved, in a scalar, using the `NDF`

option. The estimated denominator degrees of freedom for the F tests can be saved, in a variate, using the `DDF`

option.

The `VPRINT`

option controls the output from the `REML`

analyses of the bootstrap samples, with the same settings as the `PRINT`

option of `REML`

. By default, nothing is printed.

The `MAXCYCLE`

option sets a limit on the number of iterations in the `REML`

analyses (default 30). The `METHOD`

option controls whether `REML`

uses the Fisher-scoring algorithm, or the AI algorithm with sparse matrix methods (the default). The `WMETHOD`

option controls whether the Wald and F statistics are obtained from the table where terms are added sequentially (the default), or from the table where suitable terms are dropped from the full fixed model. Note that, if you use the table where terms are dropped, the `TERM`

must not be not marginal to any other term in the fixed model: for example, the main effect `A`

cannot be tested if the model contains an interaction, such as `A.B`

. The `FMETHOD`

option controls how to estimate the denominator degrees of freedom for the F tests. (This is relevant if tests for fixed effects are being printed in the `REML`

analyses of the bootstrap samples, or if the `DDF`

option is set.) The `WORKSPACE`

option specifies the number of blocks of internal memory to be set up for use by the `REML`

algorithm. The default is to use the same value as in the `SAVE`

structure, if `SAVE`

has been set. Otherwise, it uses the value from the most recent `REML`

analysis, or the standard `REML`

default if there has been no analysis.

Options: `PRINT`

, `VPRINT`

, `TERM`

, `UMEANS`

, `UVCOVARIANCE`

, `WCRITICAL`

, `FCRITICAL`

, `NBOOT`

, `NRETRIES`

, `SEED`

, `PROBABILITIES`

, `METHOD`

, `MAXCYCLE`

, `FMETHOD`

, `WMETHOD`

, `TMETHOD`

, `WALD`

, `FSTATISTIC`

, `NDF`

, `DDF`

, `NNOTCONVERGED`

, `WORKSPACE`

, `SAVE`

.

Parameters: `XCONTRASTS`

, `CONTRASTTYPE`

, `ESTIMATE`

, `SE`

, `CRITICAL`

, `TCRITICAL`

.

### Method

The critical values are calculated by taking appropriate quantiles of the statistics obtained from the bootstrap samples. For the Wald and F statistics, and the “greater-than” tests of the contrasts or their t-statistics, this is the quantile for one minus the probability. For the “less-than” tests of the contrasts or their t-statistics, it is the quantile for the probability. For the two-sided tests, the quantiles are taken over the absolute values of the contrasts and their t-statistics, and are for one minus the probability.

With an equivalence test, you define a threshold *h* below which two treatments can be assumed to be equivalent. The contrast *c* would be the difference between the treatments, and the null hypothesis that the treatments are not equivalent is that either

*c* ≤ –*t*

or

*c* ≥ *t*

with the alternative hypothesis that they are equivalent, i.e.

–*t* < *c* < t

This defines an *intersection-union* test, in which each component of the null hypothesis must be rejected separately. This implies performing two one-sided t-tests (this is known as a *TOST* procedure). If the significance level for the full test is to be α, each t-test must have significance level α (see Berger & Hsu 1996). The critical values are thus given by quantiles that are taken over the absolute values of the contrasts and their t-statistics, and are for one minus twice the probability. The hypothesis that the treatments are equivalent would be rejected if the absolute value of the estimated contrast was less than the critical value.

With a non-inferiority test, you again define the threshold *t* for the effect of the new treatment to be inferior to the standard treatment, and a contrast representing the effect of the new test minus the effect of the standard treatment. The null hypothesis is

–*c* ≥ *t*

which represents a one-sided “less-than” t-test.

### Reference

Berger, M.L. & Hsu, J.C. (1996). Bioequivalence trials, intersection-union tests and equivalence confidence sets. *Statistical Science*, 11, 283-319.

### See also

Directive: `REML`

, `VCOMPONENTS`

, `VSTRUCTURE`

.

Procedure: `VBOOTSTRAP`

, `VPOWER`

, `VUVCOVARIANCE`

.

Commands for: REML analysis of linear mixed models.

### Example

CAPTION 'VCRITICAL example',!t('Split plot design, see the',\ 'Guide to Genstat, Part 2, Section 4.2.1.'); STYLE=meta,plain SPLOAD [PRINT=*] '%gendir%/data/Oats.gsh' " Fit a model with variety and nitrogen, to get the fitted values." VCOMPONENTS [FIXED=variety+nitrogen]\ RANDOM=blocks/wplots/subplots REML yield; FITTED=fit " Fit full model to get variances & covariances of the units." VCOMPONENTS [FIXED=variety*nitrogen]\ RANDOM=blocks/wplots/subplots REML [PRINT=model,comp,Wald] yield; SAVE=fullfixed VKEEP [UVCOVARIANCE=V] " Parametric bootstrap to get critical values for variety.nitrogen" VCRITICAL [PRINT=critical,fcritical,tcritical,wcritical; NBOOT=199;\ SEED=265600; UMEANS=fit; UVCOVARIANCE=V; TERM=variety]