joint normal distribution

A finite set of random variables $X_1,\mathrm{\dots },X_n$ are said to have ajoint normal distribution or multivariate normaldistribution if all real linear combinations

$${\lambda }_1{X}_1+{\lambda }_2{X}_2+\mathrm{\cdots }+{\lambda }_n{X}_n$$

are normal (http://planetmath.org/NormalRandomVariable). This implies, in particular, that the individual random variables $X_i$ are each normally distributed. However, the converse is not not true and sets of normally distributed random variables need not, in general, be jointly normal.

If $𝑿=(X_1,X_2,\mathrm{\dots },X_n)$ is joint normal, then its probability distribution is uniquely determined by the means $𝝁\in {ℝ}^n$ and the $n\times n$ positive semidefinite covariance matrix $𝚺$,

${\mu }_i=𝔼[X_i],$

${\mathrm{\Sigma }}_{ij}=\mathrm{Cov}(X_i,X_j)=𝔼[X_i{X}_j]-𝔼[X_i]𝔼[X_j].$

Then, the joint normal distribution is commonly denoted as $\mathrm{N}(𝝁,𝚺)$. Conversely, this distribution exists for any such $𝝁$ and $𝚺$.

Figure 1: Density of joint normal variables $X,Y$ with $\mathrm{Var}(X)=2$, $\mathrm{Var}(Y)=1$ and $\mathrm{Cov}(X,Y)=-1$.

The joint normal distribution has the following properties:

1. If $𝑿$ has the $\mathrm{N}(𝝁,𝚺)$ distribution for nonsigular $𝚺$ then it has the multidimensional Gaussian probability density function

   $$f_{𝑿}(𝒙)=\frac{1}{\sqrt{{(2\pi )}^{n}det\mathbf{(}𝚺)}}\exp \left(-\frac{1}{2}{(𝒙-𝝁)}^{\mathrm{T}}{𝚺}^{-1}(𝒙-𝝁)\right).$$

2. If $𝑿$ has the $\mathrm{N}(𝝁,𝚺)$ distribution and $𝝀\in {ℝ}^n$ then

   $$𝝀\cdot 𝑿={\lambda }_1{X}_1+\mathrm{\cdots }+{\lambda }_n{X}_n\sim \mathrm{N}(𝝀\cdot 𝝁,{𝝀}^{\mathrm{T}}𝚺𝝀).$$

3. Sets of linear combinations of joint normals are themselves joint normal. In particular, if $𝑿\sim \mathrm{N}(𝝁,𝚺)$ and $A$ is an $m\times n$ matrix, then $A𝑿$ has the joint normal distribution $\mathrm{N}(A𝝁,A𝚺A^{\mathrm{T}})$.

4. The characteristic function is given by

   $${\phi }_{𝑿}(𝒂)\equiv 𝔼\left[\exp (i𝒂\cdot 𝑿)\right]=\exp \left(i𝒂\cdot 𝝁-\frac{1}{2}{𝒂}^{\mathrm{T}}𝚺𝒂\right),$$

   for $𝑿\sim \mathrm{N}(𝝁,𝚺)$ and any $𝒂\in {ℂ}^n$.

5. A pair $X,Y$ of jointly normal random variables are independent if and only if they have zero covariance.

6. Let $𝑿$ be a random vector whose distribution is jointlynormal. Suppose the coordinates of $𝑿$ are partitionedinto two groups, forming random vectors ${𝑿}_{\mathrm{𝟏}}$ and${𝑿}_{\mathrm{𝟐}}$, then the conditional distribution of${𝑿}_{\mathrm{𝟏}}$ given ${𝑿}_{\mathrm{𝟐}}=𝒄$ isjointly normal.