relation between positive function and its gradient when its Hessian matrix is bounded

Let $f:R^n\to R$ a positive function, twicedifferentiable everywhere. Furthermore, let ${\parallel {𝐇}_f(𝐱)\parallel }_2\le M,M>0$ $\forall 𝐱\in R^n$, where ${𝐇}_f(𝐱)$ is the Hessian matrix of $f(𝐱)$.Then, for any $𝐱\in R^n$,

$${\parallel \nabla f(𝐱)\parallel }_2\le \sqrt{2Mf(𝐱)}$$

Proof.

Let $𝐱,$ ${𝐱}_0\in R^n$ be arbitrary points. Bypositivity of $f(𝐱)$, writing Taylor expansion of $f(𝐱)$with Lagrange error formula around ${𝐱}_0$, a point $𝐜\in R^n$ exists such that:

$0$$\le$$f(𝐱)$

$=$$f({𝐱}_0)+\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)+{\displaystyle \frac{1}{2}}{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)$

$=$$\left|f({𝐱}_0)+\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)+{\displaystyle \frac{1}{2}}{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)\right|$

$\le$$f({𝐱}_0)+\left|\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)\right|+{\displaystyle \frac{1}{2}}\left|{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)\right|$

$\le$$f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+{\displaystyle \frac{1}{2}}{\parallel {𝐇}_f(𝐜)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2^2\text{ (by Cauchy-Schwartzinequality)}$

$\le$$f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+{\displaystyle \frac{1}{2}}M{\parallel 𝐱-{𝐱}_0\parallel }_2^2$

The rightest side is a second degree polynomial in variable ${\parallel 𝐱-{𝐱}_0\parallel }_2$; for it to be positive for anychoice of ${\parallel 𝐱-{𝐱}_0\parallel }_2$ (that is,for any choice of $𝐱$), the discriminant

$${\parallel \nabla f({𝐱}_0)\parallel }_2^2-4\cdot \frac{1}{2}Mf({𝐱}_0)$$

must be negative, whence the thesis.∎

Note: The condition on the boundedness of the Hessian matrix is actually needed. In fact, in the Lagrange form remainder, the constant $𝐜$ depends upon the point $𝐱$. Thus, if we couldn’t rely on the condition ${\parallel {𝐇}_f(𝐱)\parallel }_2\le M$, we could only state$f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+\frac{1}{2}{\parallel {𝐇}_f(𝐜(𝐱))\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2^2\ge 0$which, not being a second degree polynomial, wouldn’t imply any particular further condition.Moreover, in the case $n=1$, the lemma assumes the simpler form:Let $f:ℝ\to ℝ$ a positive function, twice differentiable everywhere.Furthermore, let $f^{\prime \prime }(x)\le M,M>0$ $\forall x\in ℝ$. Then,for any $x\in ℝ$,$\left|f^{\prime }(x)\right|\le \sqrt{2Mf(x)}$.