Gaussian distribution maximizes entropy for given covariance

Theorem 1

Let $f\mathrm{:}{\mathrm{R}}^n\mathrm{\to }\mathrm{R}$ be a continuous probabilitydensity function. Let $X_{\mathrm{1}}\mathrm{,}\mathrm{\dots }\mathrm{,}{X}_n$ be random variables with density $f$and with covariance matrix $\mathrm{K}$, $K_{i\mathit{}j}\mathrm{=}\mathrm{cov}\mathit{}\mathrm{(}{X}_i\mathrm{,}{X}_j\mathrm{)}$. Let $\varphi$ be the distribution of themultidimensional Gaussian (http://planetmath.org/JointNormalDistribution) with mean $\mathrm{0}$ and covariance matrix $\mathrm{K}$. Then the Gaussian distribution maximizes the differential entropy for a given covariance matrix $\mathrm{K}$. That is, $h\mathit{}\mathrm{(}\varphi \mathrm{)}\mathrm{\ge }h\mathit{}\mathrm{(}f\mathrm{)}$.