Practical problems

There are two practical problems with the approximate variance formula for \(\hat {\theta} \),

\[ \Var(\hat {\theta})\; \approx \;- \dfrac 1 {n \times E\left[\frac {\large d^2\; \log\left(p(X \;|\; \theta)\right)} {\large d\;\theta^2} \right]} \]
Difficulty evaluating the expected value
For many distributions, it is impossible to find a simple formula for the expected value.
Unknown value of \(\theta\)
Even if this expected value can be found, it is usually a function of \(\theta\) and \(\theta\) is an unknown value.

Avoiding the expected value

A numerical value for the approximate variance of \(\hat {\theta}\) can be found with a further approximation,

\[ \Var(\hat {\theta})\; \approx \;- \frac 1 {\ell''(\hat {\theta})} \]

(Justified in full version)

Its square root provides us with a numerical value for the standard error of the maximum likelihood estimator,

\[ \se(\hat {\theta}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]

This formula lets us find an approximate numerical value for the standard error of almost any maximum likelihood estimator — even when based on models in which the data are not a simple random sample.