FDRBONFERRONI procedure

Estimates false discovery rates by a Bonferroni-type procedure (A.I. Glaser).

Options

PRINT = string token	Controls printed output (pi0); default pi0
METHOD = string token	Controls the method used for calculating π0 (smoother, bootstrap); default smoo
LOGP = string token	Whether to take logs of π0 when METHOD=smoother (yes, no); default no
DF = scalar	Degrees of freedom for smoothing spline; default 3
PLOT = string token	Controls plots (phistogram, qhistogram, pi0vslambda, qvsp, tests, expfalsepositive, inference, loginference); default phis, qhis, pi0v, qvsp, test, expf, infe, logi
WINDOW = scalar	Window for the graphs; default 1
KEYWINDOW = scalar	Window for the key (zero for none); default 2

Parameters

PROBABILITIES = variates	Significance values, must lie between 0 and 1
LAMBDA = scalars or variates	Values of tuning parameter λ, equivalent to significance levels at which to test the PROBABILITIES; default !(0, 0.05....0.9)
FDR = variates	Saves the False Discovery Rates (i.e. q-values) at the sorted p-values in PROBABILITIES
FRR = variates	Saves the False Rejection Rates at the sorted p-values in PROBABILITIES
POWER = variates	Saves the power estimates as a function of the sorted p-values in PROBABILITIES
PI0 = scalars	Saves the value of π0, i.e. the maximum value of the FDR
LOWER = scalars	Lower bound of q-values to use with PLOT settings qvsp, tests and expfalsepositive; default 0
UPPER = scalar	Upper bound of q-values to to use with PLOT settings qvsp, tests and expfalsepositive; default 1, which indicates maximum q-value

Description

When testing m multiple hypotheses there are various outcomes that can occur, summarized in the table below

where R is the total number of rejected hypotheses and W = m – R. The proportion of tests that are truly null, π₀, is m₀ divided by m. The false discovery rate (FDR), also known as the q-value of a test, is a commonly used error measure in multiple-hypotheses, defined as

FDR = E(V/R | R > 0) × Pr(R > 0)

i.e. the expected proportion of false positives findings among all the rejected hypotheses multiplied by the probability of making at least one rejection; the FDR is zero when R = 0. Similarly the false rejection rate (FRR) is defined as

FDR = E(T/W | W > 0) × Pr(W > 0),

i.e. the expected proportion of false negatives findings among all the accepted hypotheses times the probability of accepting at least one test. We also define the power to be equal to E(S/m₁ | m₁ > 0) × Pr(m₁ > 0).

The p-values from the multiple hypotheses are supplied, in a variate, using the PROBABILITIES parameter. The analysis uses a Bonferroni-type multiple-testing procedure to calculate the corresponding q-values. The p-values are assumed independent, or may be weakly dependent if there are many of them. The parameter π₀ is calculated using the method of Storey (2002). This involves a tuning parameter λ, which can be set using the LAMBDA parameter; the default is a variate containing the numbers 0, 0.05, … 0.9. Λ can be thought of as the value beyond which the individual p-values are considered null. As λ gets larger the bias of π₀ gets smaller, but its variance increases. If you set LAMBDA to a scalar, π₀ is estimated by dividing the number of null tests (i.e. the number of p-values greater than λ) by the expected number of null tests m × (1 – Λ). If you set LAMBDA to a variate with several values, two methods are available, selected by the following settings of the METHOD option:

smoother	fits a smoothing spline of λ onto initial estimates of π0 calculated as for a single λ value, and takes the estimate of π0 as the value corresponding to the largest value of λ;
bootstrap	estimates π0 by bootstrap sampling from the variate of p-values.

The default is smoother, as the bootstrap method may be time-consuming when there are many p-values. The number of degrees of freedom to use in the smoothing is specified by the DF option (default 3). Also, you can set option LOGP=yes to do the smoothing on log-transformed π₀ values.

The PRINT option controls printed output, using the settings:

pi0	prints π0, the estimate of the expected proportion of null p-values corresponding to the largest q-value.

By default PRINT=pi0.

Various graphs can be selected by the following settings of the PLOT option:

phistogram	histogram of p-values;
qhistogram	histogram of q-values (i.e. the FDR values);
pi0vslambda	λ against π0 (when only one value of λ is specified the default values of LAMBDA are used);
qvsp	q-values against p-values;
tests	plot of the sorted tests against q-values;
expfalsepositive	plot of the number of expected false positives against the sorted tests;
inference	plot of the FDR, FRR and power statistics against the sorted p-values; and
loginference	plots the FDR, FRR and power statistics on log scales, against the sorted p-values restricted to p<0.5, with a background grid to enable estimates to be read for specific probability values. Due to the small numbers used in this plot the p-values and FDR, FRR & power statistics are displayed as percentages.

By default all the plots are produced. The WINDOW option specifies the window for the graphs, and the KEYWINDOW option species the window for keys.

Options: PRINT, METHOD, LOGP, ROBUST, DF, PLOT, WINDOW, KEYWINDOW.

Parameters: PROBABILITIES, LAMBDA, FDR, FRR, POWER, PI0, LOWER, UPPER.

Method

FDRBONFERRONI uses the method of Storey (2002), with the definitions of FRR and power given in Genovese & Wasserman (2002).

Action with RESTRICT

The PROBABILITIES parameter can be restricted. All output estimates will then be based only on those unrestricted units.

References

Storey, J.D. (2002). A direct approach to false discovery rates. Journal of the Royal Statistical Society Series B, 64, 479-498.

Genovese, C. & Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. Journal of the Royal Statistical Society Series B, 64, 499-518.

See also

Procedures: FDRMIXTURE, AMCOMPARISON, QTHRESHOLD.

Commands for: Microarray data.

Example

CAPTION       'FDRBONFERRONI example'; STYLE=meta
SCALAR        ntests,seed,delta,phi,sampsize; VALUE=\
              20000,1231,0.3,0.90,50
" Create ntest data sets (rows), each of size sampsize (columns)."
GRANDOM       [NVALUES=1; SEED=seed] dum
POINTER       [NVALUES=sampsize] x
CALCULATE     x[] = GRNORMAL(ntests; 0; 1)
" Add an offset, delta, to a fraction, (1-phi), of the samples.
  Derive test statistic, z, for Test Ho: mean=0 against Ha:mean0
  and significance, p."
CALCULATE     nHa = ROUND(ntests * (1-phi))
&             nHo = ntests - nHa
VARIATE       [VALUES=#nHo(0),#nHa(delta)] vdelta
CALCULATE     xbar = vmean(x) + vdelta
&             t = ABS(xbar * SQRT(sampsize))
&             p = 2 * CUNORMAL(t; 0; 1)
" Derive FDR "
FDRBONFERRONI p; FDR=qvals; LAMBDA=!(0,0.01...0.99); UPPER=0.4