The concept of covariance is closely related to that of the variance of a single random variable.
Definition
The covariance of two random variables, \(X\) and \(Y\), is
\[ \Covar(X,Y) \;=\; E\left[(X - \mu_X)(Y - \mu_Y)\right] \]where \(\mu_X\) and \(\mu_Y\) are the means of the two variables. The covariance is often denoted by \(\sigma_{XY}\).
The following results can be easily proved and are simply stated here.
Properties of covariance
For any random variables, \(X\) and \(Y\), and constant \(a\),
The following is often useful when finding the covariance of two variables.
Alternative formula for covariance
For any random variables, \(X\) and \(Y\),
\[ \Covar(X, Y) \;=\; E[XY] - E[X]E[Y] \](Proved in full version)
Linear transformations
The covariance between two random variables is affected in a simple way by linear transformations of the variables.
Covariance of linear transformations of X and Y
For any random variables, \(X\) and \(Y\), and constants \(a\), \(b\), \(c\) and \(d\),
\[ \Covar(a + bX, c+dY) \;=\; bd \Covar(X, Y) \](Proved in full version)
The covariance is therefore unaffected by adding constants (\(a\) and \(c\)) to the variables. Multiplying by constants (\(b\) and \(d\)) simply multiplies their covariance by these values.