Fits a nonlinear regression model or optimizes a scalar function.
|What to print (
||Calculation of fitted values or of explanatory variates involving nonlinear parameters; default
||Option setting for
||How to treat the constant (
||Limit for expansion of model terms; default as in previous
||Whether to pool ss in accumulated summary between all terms fitted in a linear model (
||Whether to base ratios in accumulated summary on rms from model with smallest residual ss or smallest residual ms (
||Which warning messages to suppress (
||Printing of probabilities for variance and deviance ratios (
||Statistics to be displayed in the summary of analysis produced by
||Number of values of each parameter for a grid of function evaluations; default
||Whether to calculate s.e.s for linear parameters (
||Setting to be used for the
||Setting to be used for the
|formula||List of explanatory variates and/or one factor to be used in linear regression, within nonlinear optimization|
FITNONLINEAR can fit nonlinear regression models, or optimize a general function. However, you should check first that the model is not included in the linear and generalized linear models that can be fitted by
FIT or the standard curves provided by
FITCURVE, for these are much more efficient. Because the methods used by
FITNONLINEAR are very general, they are neither as robust nor as automatic as, for example, the method that is used for fitting linear models.
It is better also to use
FIT for nonlinear models with separable linear parameters (generalized nonlinear models), because there are fewer constraints on the model specification.
FIT differs from
FITNONLINEAR as follows.
FIT takes account of the settings of the
EXPONENT options of
FITNONLINEAR does not). Furthermore, in
FIT the parameter may be set to a general model formula, including more than one factor, interactions and functions. However,
FIT cannot be used for optimisation of general functions, nor for models with no linear parameters.
Nonlinear methods make use of iterative optimization algorithms, designed to search for the minimum value of a function as the parameters vary; for nonlinear regression models, the function involved is the deviance, or minus twice the log-likelihood ratio, so the algorithm searches for the maximum-likelihood solution. It is often necessary to provide the algorithm with good starting values, to set bounds on the parameter values, and sometimes even to define the initial direction of search. Optimization is easiest with few parameters, approximately quadratic functions, small correlations between parameters, and good initial parameter estimates. Where possible, you can effectively reduce the number of parameters to be optimized by separating linear and nonlinear ones: that is, you can first fit the linear parameters, and treat the resulting residual sums of squares as functions of the nonlinear parameters alone. Problems with optimization methods are most likely to arise if you neglect the parameterization of the function. You can often transform the parameters to make the function nearly quadratic; after finding a solution, you can then use the
RFUNCTION directive to estimate the original parameters. Another source of difficulty is if you try to fit inappropriately many parameters. For advice on reformulating functions to speed up optimization, see Ross (1990).
FITNONLINEAR you must use the
MODEL directive with either the
Y parameter set to response variate, or the
FUNCTION option set to a scalar that is to store the value of a general function to be optimized. You must also use the
RCYCLE directive to specify the nonlinear parameters. The
TERMS directive can be used as in linear regression, to list the explanatory variables to be used in modelling. The model calculations themselves are provided in expression structures which are supplied by the
CALCULATION option of
FITNONLINEAR. You can modify the model using the
SWITCH directives as usual, except that you must give a
TERMS statement first. You can use the
RKEEP directives to display or save the results. The
RCHECK procedure can be used to assess the fit of the model, and
RGRAPH can display the fit with respect to some specified variate.
Genstat fits nonlinear regression models by maximum likelihood. The likelihood is usually from a distribution in the exponential family; this is specified using the
DISTRIBUTION option of the
MODEL directive. The settings of the
EXPONENT options of the
MODEL directive are ignored, and you are not allowed to set the
GROUPS option; other options and parameters are as in linear regression.
RCYCLE directive allows you to select the algorithm (of the three available) to be used to fit the model; these work with numerical differences and so do not require you to specify derivatives. The default algorithm is a modified Gauss-Newton method. This takes advantage of the fact that the likelihood function can be expressed as a sum of squares. However, you cannot use it for minimizing a general function. The second algorithm, a modified Newton method, is requested by setting option
METHOD=Newton in the
RCYCLE statement. This can be used for any nonlinear model. The third algorithm is a modified Fletcher-Powell method, specified by setting
METHOD=Fletcher. In fact, this is similar to the Newton method, with an occasional step in the search being determined by the Fletcher-Powell algorithm rather than by the Newton algorithm. The modification in all these methods is to use estimated numerical differences instead of evaluating derivatives. In nonlinear regression problems, particularly ones with separable linear parameters, specification of the derivatives would be very complex, and so it is much more convenient to estimate them numerically.
You must set the
PARAMETER parameter of
RCYCLE to the identifiers of scalars that will be used to represent the nonlinear parameters in the model calculations. There must be at least one nonlinear parameter. There is no formal upper limit on the number of nonlinear parameters, but the greater the number of parameters the longer the time required for the search and the smaller the chance of finding a satisfactory solution.
You can set the
UPPER parameters of
RCYCLE to provide fixed bounds for each parameter. By default, the values +109 are used. Where possible you should always set bounds, particularly to avoid such problems as attempting to take the log of a negative number. You can incorporate more general constraints as logical functions within the calculations. For example you could compute an extra term
(Constr > 0) * K * Constr
to impose a penalty on exceeding the constraint, controlled by setting different values of
K. Often, the best way to impose a constraint is to reparameterize. For example, if a parameter α must be positive, you could replace α by exp(β), and allow β to take any value.
STEPLENGTH parameter of
RCYCLE can be used to provide initial step lengths for the search. By default the step length is 0.05 times the initial value of the corresponding parameter, or precisely 1.0 if the initial value is zero. If you set a step length to zero, Genstat treats the corresponding parameter as being fixed at its initial value. This allows complex problems in many dimensions to be tackled in stages, optimizing some parameters with others fixed, and then optimizing the others in turn.
By default, the initial value of a parameter is taken to be the current value of the scalar that represents it in the calculation, or 1.0 if the value is missing. Other values can be specified using the
INITIAL parameter of
If you can calculate a range within which you expect a parameter to lie, you should choose a step length of about 1% of the width of the range. If the steps are too small, numerical differencing may not work; if they are too large, gradients may be unreliable and you may get premature convergence. Genstat tests convergence by the relationship of final adjustments to step lengths.
The more parameters there are to estimate, and the more scattered are the data, the more iterations are required to find the optimum. The maximum number of iterations is set to 30 by default, but you can reset this with the
MAXCYCLE option of
RCYCLE. However, if convergence fails with a given setting of
MAXCYCLE, you should check the data and consider reparameterizing the model before you indiscriminately increase the number of iterations.
Genstat prints a warning when convergence fails. The only sections of output that are then available are the residual degrees of freedom, the residual deviance, the fitted values, and the parameter estimates (without standard errors) for the current cycle. The
EXIT parameter of the
RKEEP directive allows you to obtain a numerical code indicating why convergence failed.
Many of the options of
FITNONLINEAR are the same as those of
SELECTION are all as in
grid setting of
NGRIDLINES option. If you set
NGRIDLINES to n, say (with n≥2),
FITNONLINEAR evaluates the likelihood at a grid of values of the nonlinear parameters, and does not search for an optimum. For each parameter, the distance between the upper and lower bounds (set by the
RCYCLE directive) will be divided into (n-1) equal parts, defining a rectangular grid with n gridlines in each dimension. By setting some upper and lower bounds equal, you can look at the behaviour of the function with respect to a few parameters at a time. The default setting of the
grid in this case, and produces a display of the function values. Other settings of the
GRID parameter of the
RKEEP directive, and can be used to produce pictures of the surface for example with the
You must set one of the
OWN options to define how the nonlinear parameters are included in the model. The
CALCULATION option does this by a list of one or more expressions. The expressions are evaluated in turn at every step of the estimation process, just as if they had been given in a sequence of
CALCULATE statements. For example:
& Diffuse; VALUES=!E(Z=Xl+Xr-1)
FITNONLINEAR [CALCULATION=Diffuse[1,2]] Z
CALCULATION option is set to the two expressions
Diffuse, to define a model for one-dimensional diffusion. Alternatively, you can set the
OWN option to specify that the calculation is to be done by executing your own source code, called by a version of the subroutine G5XZXO, as for the
OWN directive. Generally, using
OWN is likely to be worthwhile only when calculations are very extensive, or when a particular function is needed often. The setting of the
OWN option will be passed to G5XZXO in the same way as the setting of the
SELECT option of the
OWN directive is passed to G5XZXO. The
OUTOWN options then define data structures to provide the input and store the output from
There are three ways of using
FITNONLINEAR. The first provides the most efficient method when the model is linear in some of the parameters. However, this can be used only if the data are Normally distributed, or if they follow a Poisson distribution and the model contains only one explanatory variable and no constant term. The linear parameters are fitted by a linear regression of the response variate (specified by the parameter of the
MODEL statement) on the variates listed by the parameter of
FITNONLINEAR. At least one of these variates must depend on the nonlinear parameters in the model but they need not all do so. You can define how to calculate the variates from the nonlinear parameters either by the
CALCULATION option or by the
OUTOWN options of
FITNONLINEAR. The parameter of
FITNONLINEAR may include variates that are not changed by the calculations as well as those that are. One factor may also be included so that a separate constant is fitted for each level, giving a set of parallel nonlinear regressions. You cannot include interactions between a variate and a factor, as is allowed with
FITCURVE; nor can you include
LOESS functions, nor interactions between variates as allowed with
FIT. However, procedure
FITPARALLEL allows you to assess the various ways in which nonlinear models can be non-parallel. If there is a constant in the linear regression, as specified by the
CONSTANT option, the factor will be parameterized in terms of differences from the first level – as in linear regression. If you set
CONSTANT=omit, the actual constants are fitted; there is no need to set option
FULL of the
TERMS directive which is ignored in nonlinear models. If you have specified an offset variate using the
MODEL directive, its values can also be modified by the calculations, and depend on the parameters. By default, standard errors are calculated only for nonlinear parameters. To obtain standard errors for the linear parameters as well, you can set option
SELINEAR=yes. Then, after the optimum has been found, Genstat increases the number of dimensions to include the linear parameters and estimates the rate of change of the likelihood in all the dimensions.
If there are no linear parameters in the model, or if the distribution is not Normal or Poisson, you should no longer use the parameter of
FITNONLINEAR. Instead you should set the
FITTEDVALUES parameter in the
MODEL statement to the identifier of a variate that is to contain the fitted values for any set of values of the nonlinear parameters. Then define how to calculate the fitted values from the nonlinear parameters and the explanatory variates, using either the
CALCULATION or the
OWN option of
FITNONLINEAR. The distribution can now be any of those available from the
DISTRIBUTION option of the
MODEL directive, taking account of the settings of the
WEIGHTS options of the
MODEL directive. The multinomial distribution is used rather differently from the others: it is for fitting distributions. The
DISTRIBUTION directive provides a wide range of standard distributions, and is more convenient and efficient than
FITNONLINEAR for these; but
FITNONLINEAR allows you to fit other distributions. To specify and fit your own distribution, you should supply as response variate a set of counts of observations falling into a series of groups; the fitted values should then be a set of expected counts for the groups, calculated from the distribution being considered. The resulting multinomial likelihood is the same as that of the Poisson distribution, but with the constraint Σfi=M, where M is the sum of the counts.
The third method allows you to minimize a general function. (You can still use this to fit statistical models by supplying the deviance, which is minus twice the log-likelihood ratio.) To minimize a function, you need to start with a
MODEL statement that has no response variate, but where the
FUNCTION option is set to a scalar. You then specify the parameters with the
RCYCLE directive as before, and perform the minimization with
FITNONLINEAR, supplying an expression that calculates the function from the parameters and places the result into the scalar. When you are minimizing a general function in this way, some of the output from
FITNONLINEAR is different. Genstat ignores the
fittedvalues settings of the
summary settings display only the minimum function value. The
correlation setting displays the inverse of the estimated matrix of second derivatives of the function with respect to the parameters, scaled by the diagonal values. Similarly, in place of the standard errors usually displayed by the
estimates setting, Genstat prints the square roots of the diagonal values of twice the inverse of the second-derivative matrix. These can give a useful indication of the form of the function near the minimum. If the function is a deviance you can interpret these as asymptotic standard errors and correlations (not scaled by an estimate of dispersion). For a general function, the “s.e.” can be interpreted as the approximate change in a parameter required to increase the function by 1.0 starting from the minimum. Genstat ignores the
CONSTANT option of the
FITNONLINEAR directive for general functions, and you must not set the parameter. Similarly, the
OFFSET options of the
MODEL directive are ignored, and the
GROUPS option must not be set. The only parameters of the
RKEEP directive that are available are
GRID. The minimum value of the function is of course available in the scalar specified by the
FUNCTION option of the
You can restrict the units that Genstat will use for fitting the model by putting a restriction on any of the vectors involved in the
MODEL statement (response variate, weight variate, offset variate or variate of binomial totals), or on any explanatory variate or factor. However, you are not allowed to have different restrictions on the different vectors.
Ross, G.J.S. (1990). Nonlinear Estimation. Springer-Verlag, New York.
Commands for: Regression analysis.
" Example FITN-1: Fitting a standard curve using FITNONLINEAR Modelling the relationship between yield of sugar from cane and amount of nitrogen fertilizer. NOTE: this model is fitted most efficiently in Genstat by the FITCURVE directive with option CURVE=exponential. This example is to illustrate the use of FITNONLINEAR, which is really provided for fitting models that are not in FITCURVE." READ Nitrogen,Yield 0 60 0 73 0 77 0 72 50 125 50 144 50 145 50 116 100 152 100 154 100 160 100 141 150 182 150 167 150 181 150 185 200 198 200 188 200 189 200 182 : " 1) Three parameters treated as nonlinear" MODEL Yield; FITTED=Expon RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.01,*,* EXPRESSION [VALUE=(Expon=Inter+Range*Rate**Nitrogen)] Calcexp FITNONLINEAR [CALCULATION=Calcexp] " Display the fitted curve. Note that the parameter of the RGRAPH procedure must be set to the x-variate to be used in the plot after a nonlinear fit." RGRAPH [GRAPHICS=high] Nitrogen " Plot residuals against fitted values. Note that the RCHECK procedure cannot be used to check assumptions after a nonlinear fit." RKEEP RESIDUAL=r; FITTED=f DGRAPH r; f " 2) Linear parameters separated" MODEL Yield RCYCLE PARAMETER=Rate; INITIAL=0.95; UPPER=1; STEP=0.01 EXPRESSION [VALUE=(Expon=Rate**Nitrogen)] Calcexp FITNONLINEAR [PRINT=model,summary,estimates,correl,fitted;\ CALCULATION=Calcexp] Expon " Estimate s.e.s for linear parameters as well" FITNONLINEAR [PRINT=m,s,e,c; CALCULATION=Calcexp; SELINEAR=yes] Expon " 3) Explicit likelihood" MODEL [FUNCTION=Sum] RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.001,*,* EXPRESSION [VALUE=(Sum=SUM((Yield-Inter-Range*Rate**Nitrogen)**2))]\ Calcexp FITNONLINEAR [CALC=Calcexp] " To provide true standard errors, multiply SE by residual variance." RKEEP ESTIMATES=Estimates; SE=se CALC sigma2 = Sum / (NVALUES(Yield) - 3) & se = se * SQRT(sigma2) PRINT Estimates,se