consistent estimator

Given a set of samples $X_1,\mathrm{\dots },X_n$ from a given probabilitydistribution $f$ with an unknown parameter $\theta \in \mathrm{\Theta }$, where$\mathrm{\Theta }$ is the parameter space that is a subset of ${ℝ}^m$.Let $U(=U(X_1,\mathrm{\dots },X_n))$ be an estimator of $\theta$. Allowingthe sample size $n$ to vary, we get a sequence of estimators for$\theta$:

$U_1$$=$$U(X_1),$

$\mathrm{⋮}$

$U_n$$=$$U(X_1,\mathrm{\dots },X_n),$

$\mathrm{⋮}$

We say that the sequence of estimators $\{U_n\}$consistent (or that $U$ is a consistent estimator of$\theta$), if $U_i$ converges in probability to $\theta$ forevery $\theta \in \mathrm{\Theta }$. That is, for every $\epsilon >0$,

$$\underset{n\to \mathrm{\infty }}{lim}P(|h_n-\theta |\ge \epsilon )=0$$

for all $\theta \in \mathrm{\Theta }$.

Remark. Suppose $U$ is an estimator of $\theta$ such thatthe sequence $\{U_n\}$ is consistent. If${\alpha }_n\to \alpha \in ℝ$ and ${\beta }_n\to \beta \in {ℝ}^m$ are two convergent sequences of constants with and , then the sequence $\{V_n\}$, defined by $V_n:={\alpha }_n{U}_n+{\beta }_n$, is consistent,$V$ is an estimator of $\alpha \theta +\beta$.

Proof.

First, observe that

$|V_n-(\alpha \theta +\beta )|$$=$$|{\alpha }_n{U}_n+{\beta }_n-\alpha \theta -\beta |$

$\le$$|{\alpha }_n{U}_n-\alpha \theta |+|{\beta }_n-\beta |$

$=$$|{\alpha }_n{U}_n-{\alpha }_n\theta +{\alpha }_n\theta -\alpha \theta |+|{\beta }_n-\beta |$

$\le$$|{\alpha }_n{U}_n-{\alpha }_n\theta |+|{\alpha }_n\theta -\alpha \theta |+|{\beta }_n-\beta |$

$=$$|{\alpha }_n||U_n-\theta |+|{\alpha }_n-\alpha ||\theta |+|{\beta }_n-\beta |.$

This implies

$P(|V_n-(\alpha \theta +\beta )|\ge \epsilon )$

$\le$$P(|{\alpha }_n||U_n-\theta |+|{\alpha }_n-\alpha ||\theta |+|{\beta }_n-\beta |\ge \epsilon )$

$=$$P(|U_n-\theta |\ge {\displaystyle \frac{\epsilon -|{\beta }_n-\beta |-|{\alpha }_n-\alpha ||\theta |}{|{\alpha }_n|}}).$

As $n\to \mathrm{\infty }$, $|{\beta }_n-\beta |\to 0$,$|{\alpha }_n-\alpha ||\theta |\to 0$, and $|{\alpha }_n|\to |\alpha |\ne 0$.So the last expression goes to $0$ as $n\to \mathrm{\infty }$. Therefore,

$$\underset{n\to \mathrm{\infty }}{lim}P(|V_n-(\alpha \theta +\beta )|\ge \epsilon )=0,$$

and thus $\{V_n\}$ is a consistent sequence of estimatorsof $\alpha \theta +\beta$.∎