Fits a nonlinear regression model or optimizes a scalar function.
Options
PRINT = string tokens |
What to print (model, deviance, summary, estimates, correlations, fittedvalues, accumulated, monitoring, grid); default mode, summ, esti or grid if NGRIDLINES is set |
|---|---|
CALCULATION = expression structures |
Calculation of fitted values or of explanatory variates involving nonlinear parameters; default * (valid only if OWN set) |
OWN = scalar |
Option setting for OWN directive if this is to be used rather than CALCULATE; default * requests CALCULATE to be used |
CONSTANT = string token |
How to treat the constant (estimate, omit); default esti |
FACTORIAL = scalar |
Limit for expansion of model terms; default as in previous TERMS statement, or 3 if no TERMS given |
POOL = string token |
Whether to pool ss in accumulated summary between all terms fitted in a linear model (yes, no); default no |
DENOMINATOR = string token |
Whether to base ratios in accumulated summary on rms from model with smallest residual ss or smallest residual ms (ss, ms); default ss |
NOMESSAGE = string tokens |
Which warning messages to suppress (dispersion, leverage, residual, aliasing, marginality, vertical, df); default * |
FPROBABILITY = string token |
Printing of probabilities for variance and deviance ratios (yes, no); default no |
SELECTION = string tokens |
Statistics to be displayed in the summary of analysis produced by PRINT=summary, seobservations is relevant only for a Normally distributed response, and %cv only for a gamma-distributed response (%variance, %ss, adjustedr2, r2, seobservations, dispersion, %cv, %meandeviance, %deviance, aic, bic, sic); default %var, seob if DIST=normal, %cv if DIST=gamma, and disp for other distributions |
NGRIDLINES = scalar |
Number of values of each parameter for a grid of function evaluations; default * |
SELINEAR = string token |
Whether to calculate s.e.s for linear parameters (yes, no); default no |
INOWN = identifiers |
Setting to be used for the IN parameter of OWN if used in place of CALCULATE; default * |
OUTOWN = identifiers |
Setting to be used for the OUT parameter of OWN if used in place of CALCULATE; default * |
Parameter
| formula | List of explanatory variates and/or one factor to be used in linear regression, within nonlinear optimization |
|---|
Description
FITNONLINEAR can fit nonlinear regression models, or optimize a general function. However, you should check first that the model is not included in the linear and generalized linear models that can be fitted by FIT or the standard curves provided by FITCURVE, for these are much more efficient. Because the methods used by FITNONLINEAR are very general, they are neither as robust nor as automatic as, for example, the method that is used for fitting linear models.
It is better also to use FIT for nonlinear models with separable linear parameters (generalized nonlinear models), because there are fewer constraints on the model specification. FIT differs from FITNONLINEAR as follows. FIT takes account of the settings of the LINK and EXPONENT options of MODEL (but FITNONLINEAR does not). Furthermore, in FIT the parameter may be set to a general model formula, including more than one factor, interactions and functions. However, FIT cannot be used for optimisation of general functions, nor for models with no linear parameters.
Nonlinear methods make use of iterative optimization algorithms, designed to search for the minimum value of a function as the parameters vary; for nonlinear regression models, the function involved is the deviance, or minus twice the log-likelihood ratio, so the algorithm searches for the maximum-likelihood solution. It is often necessary to provide the algorithm with good starting values, to set bounds on the parameter values, and sometimes even to define the initial direction of search. Optimization is easiest with few parameters, approximately quadratic functions, small correlations between parameters, and good initial parameter estimates. Where possible, you can effectively reduce the number of parameters to be optimized by separating linear and nonlinear ones: that is, you can first fit the linear parameters, and treat the resulting residual sums of squares as functions of the nonlinear parameters alone. Problems with optimization methods are most likely to arise if you neglect the parameterization of the function. You can often transform the parameters to make the function nearly quadratic; after finding a solution, you can then use the RFUNCTION directive to estimate the original parameters. Another source of difficulty is if you try to fit inappropriately many parameters. For advice on reformulating functions to speed up optimization, see Ross (1990).
Before using FITNONLINEAR you must use the MODEL directive with either the Y parameter set to response variate, or the FUNCTION option set to a scalar that is to store the value of a general function to be optimized. You must also use the RCYCLE directive to specify the nonlinear parameters. The TERMS directive can be used as in linear regression, to list the explanatory variables to be used in modelling. The model calculations themselves are provided in expression structures which are supplied by the CALCULATION option of FITNONLINEAR. You can modify the model using the ADD, DROP and SWITCH directives as usual, except that you must give a TERMS statement first. You can use the RDISPLAY and RKEEP directives to display or save the results. The RCHECK procedure can be used to assess the fit of the model, and RGRAPH can display the fit with respect to some specified variate.
Genstat fits nonlinear regression models by maximum likelihood. The likelihood is usually from a distribution in the exponential family; this is specified using the DISTRIBUTION option of the MODEL directive. The settings of the LINK and EXPONENT options of the MODEL directive are ignored, and you are not allowed to set the GROUPS option; other options and parameters are as in linear regression.
The RCYCLE directive allows you to select the algorithm (of the three available) to be used to fit the model; these work with numerical differences and so do not require you to specify derivatives. The default algorithm is a modified Gauss-Newton method. This takes advantage of the fact that the likelihood function can be expressed as a sum of squares. However, you cannot use it for minimizing a general function. The second algorithm, a modified Newton method, is requested by setting option METHOD=Newton in the RCYCLE statement. This can be used for any nonlinear model. The third algorithm is a modified Fletcher-Powell method, specified by setting METHOD=Fletcher. In fact, this is similar to the Newton method, with an occasional step in the search being determined by the Fletcher-Powell algorithm rather than by the Newton algorithm. The modification in all these methods is to use estimated numerical differences instead of evaluating derivatives. In nonlinear regression problems, particularly ones with separable linear parameters, specification of the derivatives would be very complex, and so it is much more convenient to estimate them numerically.
You must set the PARAMETER parameter of RCYCLE to the identifiers of scalars that will be used to represent the nonlinear parameters in the model calculations. There must be at least one nonlinear parameter. There is no formal upper limit on the number of nonlinear parameters, but the greater the number of parameters the longer the time required for the search and the smaller the chance of finding a satisfactory solution.
You can set the LOWER and UPPER parameters of RCYCLE to provide fixed bounds for each parameter. By default, the values +109 are used. Where possible you should always set bounds, particularly to avoid such problems as attempting to take the log of a negative number. You can incorporate more general constraints as logical functions within the calculations. For example you could compute an extra term
(Constr > 0) * K * Constr
to impose a penalty on exceeding the constraint, controlled by setting different values of K. Often, the best way to impose a constraint is to reparameterize. For example, if a parameter α must be positive, you could replace α by exp(β), and allow β to take any value.
The STEPLENGTH parameter of RCYCLE can be used to provide initial step lengths for the search. By default the step length is 0.05 times the initial value of the corresponding parameter, or precisely 1.0 if the initial value is zero. If you set a step length to zero, Genstat treats the corresponding parameter as being fixed at its initial value. This allows complex problems in many dimensions to be tackled in stages, optimizing some parameters with others fixed, and then optimizing the others in turn.
By default, the initial value of a parameter is taken to be the current value of the scalar that represents it in the calculation, or 1.0 if the value is missing. Other values can be specified using the INITIAL parameter of RCYCLE.
If you can calculate a range within which you expect a parameter to lie, you should choose a step length of about 1% of the width of the range. If the steps are too small, numerical differencing may not work; if they are too large, gradients may be unreliable and you may get premature convergence. Genstat tests convergence by the relationship of final adjustments to step lengths.
The more parameters there are to estimate, and the more scattered are the data, the more iterations are required to find the optimum. The maximum number of iterations is set to 30 by default, but you can reset this with the MAXCYCLE option of RCYCLE. However, if convergence fails with a given setting of MAXCYCLE, you should check the data and consider reparameterizing the model before you indiscriminately increase the number of iterations.
Genstat prints a warning when convergence fails. The only sections of output that are then available are the residual degrees of freedom, the residual deviance, the fitted values, and the parameter estimates (without standard errors) for the current cycle. The EXIT parameter of the RKEEP directive allows you to obtain a numerical code indicating why convergence failed.
Many of the options of FITNONLINEAR are the same as those of FIT: PRINT, CONSTANT, FACTORIAL, POOL, DENOMINATOR, NOMESSAGE, FPROBABILITY and SELECTION are all as in FIT. The grid setting of PRINT is used with the NGRIDLINES option. If you set NGRIDLINES to n, say (with n≥2), FITNONLINEAR evaluates the likelihood at a grid of values of the nonlinear parameters, and does not search for an optimum. For each parameter, the distance between the upper and lower bounds (set by the RCYCLE directive) will be divided into (n-1) equal parts, defining a rectangular grid with n gridlines in each dimension. By setting some upper and lower bounds equal, you can look at the behaviour of the function with respect to a few parameters at a time. The default setting of the PRINT option is grid in this case, and produces a display of the function values. Other settings of the PRINT option are then ignored. The calculated grid of values is available from the GRID parameter of the RKEEP directive, and can be used to produce pictures of the surface for example with the DCONTOUR or DSURFACE directives.
You must set one of the CALCULATION and OWN options to define how the nonlinear parameters are included in the model. The CALCULATION option does this by a list of one or more expressions. The expressions are evaluated in turn at every step of the estimation process, just as if they had been given in a sequence of CALCULATE statements. For example:
EXPRESSION Diffuse[1];\
VALUE=!E(Xl,Xr=NORMAL((H+1,-1*X)/SQRT(2*D*T))
& Diffuse[2]; VALUES=!E(Z=Xl+Xr-1)
FITNONLINEAR [CALCULATION=Diffuse[1,2]] Z
Here, the CALCULATION option is set to the two expressions Diffuse[1] and Diffuse[2], to define a model for one-dimensional diffusion. Alternatively, you can set the OWN option to specify that the calculation is to be done by executing your own source code, called by a version of the subroutine G5XZXO, as for the OWN directive. Generally, using OWN is likely to be worthwhile only when calculations are very extensive, or when a particular function is needed often. The setting of the OWN option will be passed to G5XZXO in the same way as the setting of the SELECT option of the OWN directive is passed to G5XZXO. The INOWN and OUTOWN options then define data structures to provide the input and store the output from FITNONLINEAR.
There are three ways of using FITNONLINEAR. The first provides the most efficient method when the model is linear in some of the parameters. However, this can be used only if the data are Normally distributed, or if they follow a Poisson distribution and the model contains only one explanatory variable and no constant term. The linear parameters are fitted by a linear regression of the response variate (specified by the parameter of the MODEL statement) on the variates listed by the parameter of FITNONLINEAR. At least one of these variates must depend on the nonlinear parameters in the model but they need not all do so. You can define how to calculate the variates from the nonlinear parameters either by the CALCULATION option or by the OWN, INOWN and OUTOWN options of FITNONLINEAR. The parameter of FITNONLINEAR may include variates that are not changed by the calculations as well as those that are. One factor may also be included so that a separate constant is fitted for each level, giving a set of parallel nonlinear regressions. You cannot include interactions between a variate and a factor, as is allowed with FIT or FITCURVE; nor can you include POL, REG, COMPARISON, SSPLINE or LOESS functions, nor interactions between variates as allowed with FIT. However, procedure FITPARALLEL allows you to assess the various ways in which nonlinear models can be non-parallel. If there is a constant in the linear regression, as specified by the CONSTANT option, the factor will be parameterized in terms of differences from the first level – as in linear regression. If you set CONSTANT=omit, the actual constants are fitted; there is no need to set option FULL of the TERMS directive which is ignored in nonlinear models. If you have specified an offset variate using the MODEL directive, its values can also be modified by the calculations, and depend on the parameters. By default, standard errors are calculated only for nonlinear parameters. To obtain standard errors for the linear parameters as well, you can set option SELINEAR=yes. Then, after the optimum has been found, Genstat increases the number of dimensions to include the linear parameters and estimates the rate of change of the likelihood in all the dimensions.
If there are no linear parameters in the model, or if the distribution is not Normal or Poisson, you should no longer use the parameter of FITNONLINEAR. Instead you should set the FITTEDVALUES parameter in the MODEL statement to the identifier of a variate that is to contain the fitted values for any set of values of the nonlinear parameters. Then define how to calculate the fitted values from the nonlinear parameters and the explanatory variates, using either the CALCULATION or the OWN option of FITNONLINEAR. The distribution can now be any of those available from the DISTRIBUTION option of the MODEL directive, taking account of the settings of the DISPERSION and WEIGHTS options of the MODEL directive. The multinomial distribution is used rather differently from the others: it is for fitting distributions. The DISTRIBUTION directive provides a wide range of standard distributions, and is more convenient and efficient than FITNONLINEAR for these; but FITNONLINEAR allows you to fit other distributions. To specify and fit your own distribution, you should supply as response variate a set of counts of observations falling into a series of groups; the fitted values should then be a set of expected counts for the groups, calculated from the distribution being considered. The resulting multinomial likelihood is the same as that of the Poisson distribution, but with the constraint Σfi=M, where M is the sum of the counts.
The third method allows you to minimize a general function. (You can still use this to fit statistical models by supplying the deviance, which is minus twice the log-likelihood ratio.) To minimize a function, you need to start with a MODEL statement that has no response variate, but where the FUNCTION option is set to a scalar. You then specify the parameters with the RCYCLE directive as before, and perform the minimization with FITNONLINEAR, supplying an expression that calculates the function from the parameters and places the result into the scalar. When you are minimizing a general function in this way, some of the output from FITNONLINEAR is different. Genstat ignores the accumulated and fittedvalues settings of the PRINT option, and the deviance and summary settings display only the minimum function value. The correlation setting displays the inverse of the estimated matrix of second derivatives of the function with respect to the parameters, scaled by the diagonal values. Similarly, in place of the standard errors usually displayed by the estimates setting, Genstat prints the square roots of the diagonal values of twice the inverse of the second-derivative matrix. These can give a useful indication of the form of the function near the minimum. If the function is a deviance you can interpret these as asymptotic standard errors and correlations (not scaled by an estimate of dispersion). For a general function, the “s.e.” can be interpreted as the approximate change in a parameter required to increase the function by 1.0 starting from the minimum. Genstat ignores the CONSTANT option of the FITNONLINEAR directive for general functions, and you must not set the parameter. Similarly, the WEIGHTS and OFFSET options of the MODEL directive are ignored, and the GROUPS option must not be set. The only parameters of the RKEEP directive that are available are ESTIMATES, SE, INVERSE, EXIT, GRADIENTS and GRID. The minimum value of the function is of course available in the scalar specified by the FUNCTION option of the MODEL directive.
Options: PRINT, CALCULATION, OWN, CONSTANT, FACTORIAL, POOL, DENOMINATOR, NOMESSAGE, FPROBABILITY, SELECTION, NGRIDLINES, SELINEAR, INOWN, OUTOWN.
Parameter: unnamed.
Action with RESTRICT
You can restrict the units that Genstat will use for fitting the model by putting a restriction on any of the vectors involved in the MODEL statement (response variate, weight variate, offset variate or variate of binomial totals), or on any explanatory variate or factor. However, you are not allowed to have different restrictions on the different vectors.
Reference
Ross, G.J.S. (1990). Nonlinear Estimation. Springer-Verlag, New York.
See also
Directives: EXPRESSION, MODEL, RDISPLAY, RKEEP, RCYCLE, RFUNCTION, FIT, FITCURVE.
Procedures: NLAR1, HGNONLINEAR, RQUADRATIC, MINIMIZE, MIN1DIMENSION, SIMPLEX.
Commands for: Regression analysis.
Example
" Example FITN-1: Fitting a standard curve using FITNONLINEAR Modelling the relationship between yield of sugar from cane and amount of nitrogen fertilizer. NOTE: this model is fitted most efficiently in Genstat by the FITCURVE directive with option CURVE=exponential. This example is to illustrate the use of FITNONLINEAR, which is really provided for fitting models that are not in FITCURVE." READ Nitrogen,Yield 0 60 0 73 0 77 0 72 50 125 50 144 50 145 50 116 100 152 100 154 100 160 100 141 150 182 150 167 150 181 150 185 200 198 200 188 200 189 200 182 : " 1) Three parameters treated as nonlinear" MODEL Yield; FITTED=Expon RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.01,*,* EXPRESSION [VALUE=(Expon=Inter+Range*Rate**Nitrogen)] Calcexp FITNONLINEAR [CALCULATION=Calcexp] " Display the fitted curve. Note that the parameter of the RGRAPH procedure must be set to the x-variate to be used in the plot after a nonlinear fit." RGRAPH [GRAPHICS=high] Nitrogen " Plot residuals against fitted values. Note that the RCHECK procedure cannot be used to check assumptions after a nonlinear fit." RKEEP RESIDUAL=r; FITTED=f DGRAPH r; f " 2) Linear parameters separated" MODEL Yield RCYCLE PARAMETER=Rate; INITIAL=0.95; UPPER=1; STEP=0.01 EXPRESSION [VALUE=(Expon=Rate**Nitrogen)] Calcexp FITNONLINEAR [PRINT=model,summary,estimates,correl,fitted;\ CALCULATION=Calcexp] Expon " Estimate s.e.s for linear parameters as well" FITNONLINEAR [PRINT=m,s,e,c; CALCULATION=Calcexp; SELINEAR=yes] Expon " 3) Explicit likelihood" MODEL [FUNCTION=Sum] RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.001,*,* EXPRESSION [VALUE=(Sum=SUM((Yield-Inter-Range*Rate**Nitrogen)**2))]\ Calcexp FITNONLINEAR [CALC=Calcexp] " To provide true standard errors, multiply SE by residual variance." RKEEP ESTIMATES=Estimates; SE=se CALC sigma2 = Sum / (NVALUES(Yield) - 3) & se = se * SQRT(sigma2) PRINT Estimates,se