Gauss-Markov theorem

A Gauss-Markov linear model is a linear statistical modelthat satisfies all the conditions of a general linear model exceptthe normality of the error terms. Formally, if $𝒀$ isan $m$-dimensional response variable vector, and${𝒁}_{𝒊}=z_i(𝑿)$, $i=1,\mathrm{\dots },k$ are the$m$-dimensional functions of the explanatory variable vector$𝑿$, a Gauss-Markov linear model has the form:

$$𝒀={\beta }_0{𝒁}_{\mathrm{𝟎}}+\mathrm{\cdots }+{\beta }_k{𝒁}_{𝒌}+\mathit{ϵ},$$

with$\mathit{ϵ}$ the error vector such that

1. $\mathrm{E}[\mathit{ϵ}]=\mathrm{𝟎}$, and

2. $\mathrm{Var}[\mathit{ϵ}]={\sigma }^2𝑰$.

In other words, the observed responses $Y_i$, $i=1,\mathrm{\dots },m$ are notassumed to be normally distributed, are not correlated with oneanother, and have a common variance$\mathrm{Var}[Y_i]={\sigma }^2$.

Gauss-Markov Theorem. Suppose the response variable$𝒀=(Y_1,\mathrm{\dots },Y_m)$ and the explanatory variables$𝑿$ satisfy a Gauss-Markov linear model as describedabove. Consider any linear combination of the responses

$Y={\displaystyle \sum _{i=1}^m}{c}_i{Y}_i,$(1)

where $c_i\in ℝ$. If each ${\mu }_i$ is an estimator for response $Y_i$, parameter $\theta$ of the form

$\theta ={\displaystyle \sum _{i=1}^m}{c}_i{\mu }_i,$(2)

can be used as an estimator for $Y$. Then, among all unbiased estimators for $Y$ having form (2), the ordinary least square estimator (OLS)

$\widehat{\theta }={\displaystyle \sum _{i=1}^m}{c}_i\widehat{{\mu }_i},$(3)

yields the smallest variance. In other words, the OLS estimator is the uniformly minimum variance unbiased estimator.

Remark. $\widehat{\theta }$ in equation (3) above is morepopularly known as the BLUE, or the best linear unbiased estimatorfor a linear combination of the responses in a Gauss-Markov linearmodel.