covariance

The covariance of two random variables $X_1$ and $X_2$ with mean (http://planetmath.org/ExpectedValue) ${\mu }_1$ and ${\mu }_2$ respectively is defined as

$$\mathrm{cov}(X_1,X_2):=E[(X_1-{\mu }_1)(X_2-{\mu }_2)].$$(1)

The covariance of a random variable $X$ with itself is simply the variance, $E[{(X-\mu )}^2]$.

Covariance captures a measure of the correlation of two variables. Positive covariance indicates that as $X_1$ increases, so does $X_2$. Negative covariance indicates $X_1$ decreases as $X_2$ increases and vice versa. Zero covariance can indicate that $X_1$ and $X_2$ are uncorrelated.

The correlation coefficient provides a normalized view of correlation based on covariance:

$$\mathrm{corr}(X,Y):=\frac{\mathrm{cov}(X,Y)}{\sqrt{\mathrm{var}(X)\mathrm{var}(Y)}}.$$(2)

$\mathrm{corr}(X,Y)$ ranges from -1 (for negatively correlated variables) through zero (for uncorrelated variables) to +1 (for positively correlated variables).

While if $X$ and $Y$ are independent we have $\mathrm{corr}(X,Y)=0$, the latter does not imply the former.