Standard error from Newton Raphson algorithm
In view of the extra complication of using the Newton Raphson algorithm to obtain the maximum likelihood estimate of a parameter when an explicit formula for the maximum likelihood estimate cannot be found, it might be expected that the standard error of the estimate would also be difficult to evaluate. Fortunately the standard error arises as a by-product of the algorithm.
The asymptotic formula for the standard error is
\[ \se(\hat {\theta}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]The second derivative of the log-likelihood is used in the Newton Raphson algorithm and its value at the MLE can be found from the last iteration of the algorithm when it converges.
Standard error for log-series distribution
Consider the following data set that is assumed to arise from a log-series distribution.
3 | 5 | 1 | 4 | 8 | 10 | 2 | 1 | 1 | 2 |
1 | 8 | 1 | 6 | 13 | 1 | 6 | 2 | 1 | 3 |
1 | 1 | 1 | 2 | 1 | 6 | 1 | 1 | 1 | 1 |
The derivatives of the log-likelihood involve the values \(n = 30\) and \(\sum x = 95\). Iterations of the algorithm from an initial guess at the value of \(\theta = 0.7\) are shown below.
From most guesses at the value of the parameter \(\theta\), the algorithm converges to the maximum likelihood estimate, \(\hat{\theta} = 0.8628\). The second derivative of the log-likelihood similarly converges to \(\ell''(\hat{\theta}) = -526.28\). The approximate standard error of the estimate is therefore
\[ \se(\hat {\theta}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \;\;=\;\; \sqrt {\frac 1 {526.28}} \;\;=\;\; 0.0436 \]