This page illustrates maximum likelihood for parameters of continuous distributions using a random sample from a normal distribution.

Normal distribution with known σ

Our first example of maximum likelihood relates to a random sample,

4.2   5.2   5.6   6.1   7.3   8.5

from a normal distribution whose standard deviation, \(\sigma\) is known to be 1.3.

\[ X \;\; \sim \; \; \NormalDistn(\mu, \;\sigma = 1.3) \]

This normal distribution has probability density function

\[ f(x\;|\; \mu) = \frac 1 {\sqrt{2\pi} \times 1.3} e^{-\large\frac 1 {2 \times 1.3^2}(x-\mu)^2} \quad\quad \text{for } -\infty \lt x \lt +\infty \]

Its log-likelihood is

\[ \ell(\mu) \;\;=\;\; \sum_{i=1}^n {\log(f(x_i \;|\; \mu))} \;\;=\;\; -\frac 1 {2 \times 1.3^2} \times \sum_{i=1}^n {(x_i-\mu)^2} + K \]

where \(K\) is a constant that does not depend on \(\mu\). To find the maximum likelihood estimate of \(\mu\), we set the first derivative of \(\ell(\mu)\) to zero,

\[ \ell'(\mu) \;\;=\;\; \frac 1 {1.3^2} \times \sum_{i=1}^n {(x_i-\mu)} \;\;=\;\; 0 \]

The solution to this equation is

\[ \hat{\mu} \;\;=\;\; \frac {\sum {x_i}} n \;\;=\;\; \overline{x} \]

The following diagram illustrates maximum likelihood. The bottom half shows the normal pdf with a slider that can be used to adjust the value of the parameter \(\mu\). The six data values are marked by vertical red lines.

The heights of the red bars are \( f(x \;|\; \mu) \) at the data values. The logarithm of the product of these heights is the log-likelihood, \(\ell(\mu) \), and this is shown in the top half of the diagram. Drag the slider to see how the log-likelihood depends on the heights of these bars.

Click Max likelihood to set \(\mu\) to the sample mean, 6.15, and observe that the slope of the log-likelihood function (its first derivative) is zero at the maximum likelihood estimate.

Standard error

Since \(\hat{\mu}\) is the sample mean, we can use the properties of sample means to directly find the standard error of this estimator,

\[ \se(\hat{\mu}) \;\;=\;\; \sqrt{\Var(\overline{X})} \;\;=\;\; \frac {\sigma} {\sqrt{n}} \;\;=\;\; \frac {1.3} {\sqrt n }\]

The second derivative of \(\ell(\mu)\) is

\[ \ell''(\mu) \;\;=\;\; -\frac n {1.3^2}\]

giving the 'approximation'

\[ \se(\hat {\mu}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\mu})}} \;\;=\;\; \sqrt {\frac {1.3^2} n}\]

When estimating the normal distribution's mean, this "approximation" gives the exact standard error of the maximum likelihood estimator.

Confidence interval

95% confidence intervals for maximum likelihood estimators are based on the fact that they are approximately normally distributed in large samples. In this particular example, the maximum likelihood estimator is the sample mean and sample means from normal distributions have exactly normal distributions,

\[ \overline{X} \;\; \sim \; \; \NormalDistn(\mu,\;\; \sigma_{\overline{X}} = \frac {1.3} {\sqrt n}) \]

The interval estimate

\[ \overline{x} \;\; \pm \; \; 1.96 \times \frac {1.3} {\sqrt n} \]

therefore has exactly 95% confidence level in this example.