Strength of a relationship
The correlation coefficient is a summary value that describes an important aspect of the strength of the relationship between two variables. Its value always lies between zero and one.
Properties of correlation
For any random variables, \(X\) and \(Y\),
\[ -1 \;\le\; \Corr(X,Y) \;\le\; +1 \]The proof starts by considering a variance (which we know must be greater than zero).
\[ \begin{align} \Var\left(\frac{X}{\sigma_X} + \frac{Y}{\sigma_Y}\right) \;&=\; \Var\left(\frac{X}{\sigma_X}\right) + \Var\left(\frac{Y}{\sigma_Y}\right) + 2\Covar\left(\frac{X}{\sigma_X}, \frac{Y}{\sigma_Y}\right) \\ &=\; \frac{\sigma_X^2}{\sigma_X^2} + \frac{\sigma_Y^2}{\sigma_Y^2} + 2\frac{\Covar(X,Y)}{\sigma_X \sigma_Y} \\[0.3em] &=\; 2 + 2 \rho_{XY} \\[0.3em] &\ge 0 \end{align} \]From this, we obtain the result that \(\rho_{XY} \ge -1\)
By similarly expanding \(\displaystyle\Var\left(\frac{X}{\sigma_X} - \frac{Y}{\sigma_Y}\right)\), it can be shown that \(\rho_{XY} \le 1\).
A slight modification to the above proof shows that the correlation coefficient between two variables can only be ±1 if the variables are linearly related.
Linear relationships and correlation
\[ \left|\Corr(X,Y)\right| = 1 \quad\text{if and only if} \quad Y = a + bX \quad\text{for some constants } a \text{ and } b. \]We first show that a correlation of ±1 can only arise when the variables are linearly related. There can only be equality in the proof at the top of this page if either
\[ \Var\left(\frac{X}{\sigma_X} + \frac{Y}{\sigma_Y}\right) = 0 \spaced{or} \Var\left(\frac{X}{\sigma_X} - \frac{Y}{\sigma_Y}\right) = 0 \]These variances can only be zero if either \(\displaystyle\Var\left(\frac{X}{\sigma_X} + \frac{Y}{\sigma_Y}\right)\) or \(\displaystyle\Var\left(\frac{X}{\sigma_X} - \frac{Y}{\sigma_Y}\right)\) is constant, making \(X\) and \(Y\) linearly related.
To complete the proof, we must also show that linearly related variables always have correlation ±1.
\[ \Covar(X, a+bX) \;=\; b\Covar(X,X) \;=\; b \sigma_X^2 \]and
\[ \Var(a+bX) \;=\; b^2\sigma_X^2 \]so
\[ \Corr(X, a+bX) \;=\; \frac{\Covar(X, a+bX)}{\sqrt{\Var(X)\Var(a + bX)}} \;=\; \pm1 \]