If the estimator of any parameter, \(\theta\), is approximately unbiased and normally distributed, and if we can evaluate an approximate standard error, then the interval
\[ \hat{\theta}-1.96 \times \se(\hat {\theta}) \quad \text{ to } \quad \hat{\theta}+1.96 \times \se(\hat {\theta}) \]has approximately probability 0.95 of including the true value of \(\theta\). The resulting interval is a 95% confidence interval for \(\theta\) and we have 95% confidence that it will include the actual value of \(\theta\).
This holds for random samples from both discrete and continuous distributions. In particular, it can be used for maximum likelihood estimators, due to their asymptotic properties.
Other confidence levels
Intervals with different confidence levels can be obtained by replacing "1.96" by other quantiles of the standard normal distribution. For example, a 90% confidence interval for \(\theta\) is
\[ \hat{\theta}-1.645 \times \se(\hat {\theta}) \quad \text{ to } \quad \hat{\theta}+1.645 \times \se(\hat {\theta}) \]