martingale proof of Kolmogorov’s strong law for square integrable variables

We apply the martingale convergence theorem to prove the following result.

Theorem.

Let $X_{\mathrm{1}}\mathrm{,}{X}_{\mathrm{2}}\mathrm{,}\mathrm{\dots }$ be independent random variables such that . Then, setting

$$S_n=\frac{1}{n}\sum _{k=1}^n(X_k-𝔼[X_k])$$

we have $S_n\mathrm{\to }\mathrm{0}$ as $n\mathrm{\to }\mathrm{\infty }$, with probability one.

To prove this, we start by constructing a martingale,

$$M_n=\sum _{k=1}^n\frac{X_k-𝔼[X_k]}{k}.$$

If ${ℱ}_n$ is the $\sigma$-algebra (http://planetmath.org/SigmaAlgebra) generated by $X_1,\mathrm{\dots }{X}_n$ then

$$𝔼[M_{n+1}\mid {ℱ}_n]=M_n+\frac{𝔼[X_{n+1}\mid {ℱ}_n]-𝔼[X_{n+1}]}{n+1}=M_n.$$

Here, the independence of $X_{n+1}$ and ${ℱ}_n$ has been used to imply that $𝔼[X_{n+1}\mid {ℱ}_n]=𝔼[X_{n+1}]$. So, $M$ is a martingale with respect to the filtration ${({ℱ}_n)}_{n\in \mathbb{N}}$.

Also, by the independence of the $X_n$, the variance of $M_n$ is

So, the inequality $𝔼[|M_n|]\le \sqrt{𝔼[M_n^2]}=\sqrt{\mathrm{Var}[M_n]}$ shows that $M$ is an $L^1$-bounded martingale, and the martingale convergence theorem says that the limit $M_{\mathrm{\infty }}={lim}_{n\to \mathrm{\infty }}{M}_n$ exists and is finite, with probability one.

The strong law now follows from Kronecker’s lemma, which states that for sequences of real numbers $x_1,x_2,\mathrm{\dots }$ and such that $b_n$ strictly increases to infinity and ${\sum }_n{x}_n/b_n$ converges to a finite limit, then $b_n^{-1}{\sum }_{k=1}^n{x}_k$ tends to $0$ as $n\to \mathrm{\infty }$. In our case, we take $x_n=X_n-𝔼[X_n]$ and $b_n=n$ to deduce that $n^{-1}{\sum }_{k=1}^n(X_k-𝔼[X_k])$ converges to zero with probability one.