This page applies maximum likelihood to a normal distribution.
Normal distribution with known σ
Consider a random sample,
4.2 5.2 5.6 6.1 7.3 8.5
from a normal distribution with known \(\sigma\),
\[ X \;\; \sim \; \; \NormalDistn(\mu, \;\sigma = 1.3) \]Its log-likelihood is
\[ \ell(\mu) \;\;=\;\; \sum_{i=1}^n {\log(f(x_i \;|\; \mu))} \;\;=\;\; -\frac 1 {2 \times 1.3^2} \times \sum_{i=1}^n {(x_i-\mu)^2} + K \]where \(K\) is a constant that does not depend on \(\mu\). To find the maximum likelihood estimate of \(\mu\), we solve
\[ \ell'(\mu) \;\;=\;\; \frac 1 {1.3^2} \times \sum_{i=1}^n {(x_i-\mu)} \;\;=\;\; 0 \]Giving \(\displaystyle \hat{\mu} \;\;=\;\; \frac {\sum {x_i}} n \;\;=\;\; \overline{x}\).
We now illustrate the method graphically. The likelihood function, \(L(\mu)\), is the product of the normal distribution's pdf's at the data values — the product of the bar heights at the bottom of the next diagram.
The likelihood for the normal distribution with \(\mu = 4\) is low because the pdf is so small at the highest data values, f(7.3) and f(8.5). On the other hand, when \(\mu = \overline{x} = 6.15\), there are no small pdfs the likelihood function is maximised.
Standard error
We can directly find the standard error of the MLE using the properties of sample means,
\[ \se(\hat{\mu}) \;\;=\;\; \sqrt{\Var(\overline{X})} \;\;=\;\; \frac {\sigma} {\sqrt{n}} \;\;=\;\; \frac {1.3} {\sqrt n }\]Finding the standard error from the second derivative of \(\ell(\mu)\) gives
\[ \se(\hat {\mu}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\mu})}} \;\;=\;\; \sqrt {\frac {1.3^2} n}\]For this example, the asymptotic formula gives the exact standard error of the maximum likelihood estimator.
Confidence interval
For this example, the maximum likelihood estimator is the sample mean. Since sample means from normal distributions have exactly normal distributions,
\[ \overline{X} \;\; \sim \; \; \NormalDistn(\mu,\;\; \sigma_{\overline{X}} = \frac {1.3} {\sqrt n}) \]the interval estimate
\[ \overline{x} \;\; \pm \; \; 1.96 \times \frac {1.3} {\sqrt n} \]has exactly 95% confidence level. (The confidence level is only approximate for MLEs based on other distributions.)