Uses the Tobit method to fit a hierarchical generalized linear model with censored Poisson data (R.W. Payne).
Options
PRINT =string tokens 
Controls printed output (model , fixedestimates , randomestimates , dispersionestimates , likelihoodstatistics , deviance , waldtests , fittedvalues , monitoring , hgmonitoring , dhgmonitoring , censored ); default mode , fixe , disp , devi , like , cens 

LMETHOD =string token 
Whether to use exact likelihood or extended quasi likelihood to obtain the yvariate and weights for the dispersion model (exact , eql ); default exac 

SEMETHOD =string token 
Method to use to calculate the se’s for the dispersion estimates (approximate , profilelikelihood ); default appr 

DMETHOD =string token 
Method to use for the adjusted profile likelihood when calculating the likelihood statistics (automatic , choleski , lrv ); default auto 

EMETHOD =string token 
Extrapolation method to use (aitken , adjustedaitken ); default aitk 

MLAPLACEORDER =scalar 
Order of Laplace approximation to use in the estimation of the mean model (0 or 1); default 0  
DLAPLACEORDER =scalar 
Order of Laplace approximation to use in the estimation of the dispersion components (0, 1 or 2); default 0  
MAXCYCLE =scalar 
Maximum number of iterations of the EM algorithm; default 100  
TOLERANCE =scalar 
Convergence criterion for the EM algorithm; default 0.001  
DIRECTION =string token 
Whether the data are left or right censored (left , right ); default righ 

HGMAXCYCLE =scalars 
Maximum number of iterations of the hierarchical generalized linear model fits, and maximum number of iterations in the fitting of the mean and dispersion models; default 99,50  
HGTOLERANCE =scalar 
Criterion for convergence; default 0.0005  
ETOLERANCE =scalar 
Maximum size of ratio of the original to the new estimates allowed in Aitken extrapolation; default 7.5  
GROUPTERM =formula 
Random term to use as groups when fitting the augmented mean model; default * i.e. none 
Parameters
Y = variate 
Response variate to be analysed; must be set  
BOUND = scalar 
Censoring threshold; must be set  
INITIAL = scalaror variate 
Scalar or a variate providing starting values for the censored observations in the EM algorithm; default BOUND+1 for rightcensored data and default BOUND1 for leftcensored data 

NEWY = variate 
Saves a copy of the response variate with the censored observations replaced by their estimates  
EXIT = scalar 
Exit status (0 for success, 1 for failure in the EM algorithm, 2 for failure to fit the generalized linear mixed model)  
SAVE = pointer 
Saves details of the analysis for use in subsequent HGDISPLAY ,HGKEEP , HGPLOT or HGPREDICT statements 
Description
When an experiment generates a mixture of small and very large counts, it may be convenient to count only the observations less than a specified boundary value, and enter that value for the larger observations. The data then come from a rightcensored Poisson distribution. In the similar (but less common) leftcensored situation, the emphasis is on the larger observations. It may then not be worth recording the small observations in detail, only that they are no larger than the boundary value. Censored Poisson data can be analysed by the Tobit method (Terza 1985), which is implemented in this procedure.
In the Tobit model, the probabilities for the uncensored observations are standard Poisson probabilities. The probabilities for rightcensored observations are cumulative upper Poisson probabilities for values greater than or equal to the boundary value. Probabilities for leftcensored observations cumulative lower Poisson probabilities for values less than or equal to the boundary value. The Tobit method uses an EM (expectationmaximization) algorithm to estimate values for the censored observations. It starts with initial estimates for the censored observations, which can be specified by the INITIAL
parameter in either a variate or a scalar. For rightcensored data the default is to use the boundary value plus one. For leftcensored data the default is the boundary value minus one. In each iteration, the method uses the HGANALYSE
procedure to fit a Poissonlog hierarchical generalized linear model, saving the resulting fitted values to provide estimated means for the Poisson distributions of the censored observations. The new estimates for the censored observations are then given by the expected values for the upper parts of those Poisson distributions. The process continues either until the updates to the estimates are less than or equal to the value specified by the TOLERANCE
option (default 0.001), or until the number of iterations equals the number specified by the MAXCYCLE
option (default 100). The EXIT
parameter can be set to a scalar that will be set to zero for a successful fit, one for failure in the EM algorithm, two if the hierarchical generalized linear model has failed to fit, or a missing value for an earlier fault.
The models for the hierarchical generalized linear model must be specified beforehand by the HGFIXEDMODEL
and HGDRANDOMMODEL
procedures, as usual, except that HGFIXEDMODEL
must specify DISTRIBUTION=poisson
and LINK=logarithm.
The response variate is specified by the Y
parameter, and the NEWY
parameter can save a variate where the censored observations are replaced by their estimates. The BOUND
option specifies the boundary value for the censoring (and the value that has been entered to indicate the censored observations in the Y
variate). The DIRECTION
option specifies whether the data are left or right censored. The default is that they are right censored.
The HGMAXCYCLE
and HGTOLERANCE
options specify the maximum number of iterations and tolerance for the fit of the hierarchical generalized linear model by the HGANALYSE
procedure, and correspond to the MAXCYCLE
and TOLERANCE
options of HGANALYSE
. The options LMETHOD
, SEMETHOD
, DMETHOD
, EMETHOD
, MLAPLACEORDER
, DLAPLACEORDER
, MAXCYCLE
, TOLERANCE
, HGMAXCYCLE
, HGTOLERANCE
, ETOLERANCE
and GROUPTERM
all operate exactly like the corresponding options of HGANALYSE
. The PRINT
option is similar. However, the monitoring
setting prints monitoring information for the EM algorithm. There is a hgmonitoring
to monitor for the fit of the hierarchical generalized linear model. There is also a setting censored
to print the estimates of the censored observations.
The SAVE
parameter can save a pointer, with information about the hierarchical generalized linear model analysis, for use by procedures like HGDISPLAY
and HGKEEP
.
Options: PRINT
, LMETHOD
, SEMETHOD
, DMETHOD
, EMETHOD
, MLAPLACEORDER
, DLAPLACEORDER
, MAXCYCLE
, TOLERANCE
, DIRECTION
, HGMAXCYCLE
, HGTOLERANCE
, ETOLERANCE
, GROUPTERM
. Parameters: Y
, BOUND
, INITIAL
, NEWY
, EXIT
, SAVE
.
Method
The hierarchical generalized linear model is fitted by the HGANALYSE
procedure. The expected values for the upper parts of the Poisson distributions are calculated by the EUPOISSON
procedure, and those for the lower parts of the distributions are calculated by the ELPOISSON
procedure.
Reference
Terza, J.V. (1985). A Tobittype estimator for the censored Poisson regression model. Economics Letters, 18, 361365.
See also
Procedures: CENSOR
, ELPOISSON
, EUPOISSON
, HGANALYSE
, HGDISPLAY
, HGDRANDOMMODEL
, HGFIXEDMODEL
, HGFTEST
, HGGRAPH
, HGKEEP
, HGNONLINEAR
, HGPLOT
, HGPREDICT
, HGRANDOMMODEL
, HGRTEST
, HGSTATUS
, HGWALD
, GLTOBITPOISSON
, RTOBITPOISSON
, TOBIT
.
Commands for: Regression analysis.
Example
CAPTION 'HGTOBITPOISSON example',\ !t('Nematode data from Cochran & Cox (1957) p.46,',\ 'analysed in Section 4.3 of the Statistics Guide,',\ 'with the unfumigated plots removed to simplify the analysis.',\ 'Suppose that counting stopped at 400.',\ 'Units 1820, 24 & 27 are then censored.'); STYLE=meta,plain SPLOAD '%data%/Nematode.gsh' SUBSET [Fumigant.IN.'Fumigated'; SETLEVELS=yes]\ Blocks,Amount,Type,Count,Priorcount CALCULATE Logpriorcount = LOG(Priorcount) HGFIXEDMODEL [DISTRIBUTION=poisson; LINK=log; DISPERSION=*]\ Logpriorcount+Amount*Type HGRANDOMMODEL [DISTRIBUTION=gamma; LINK=log] Blocks HGTOBITPOISSON [PRINT=model,fixedestimates,dispersionestimates,\ likelihoodstatistics,waldtests,censored] Count; BOUND=400