PTAH inequality

Let $\sigma =\{({\theta }_1,\mathrm{\dots },{\theta }_n)\in {ℝ}^n|{\theta }_i\ge 0,{\sum }_{i=1}^n{\theta }_i=1\}$.

Let $X$ be a measure space with measure $m$.Let $a_i:X\to ℝ$ be measurable functions such that$a_i(x)\ge 0$ a.e [m] for $i=1,\mathrm{\dots },n$.

Note: the notation “a.e. [m]” means that the condition holds almost everywhere with respect to the measure $m$.

Define $p:X\times \sigma \to ℝ$ by

$$p(x,\lambda )=\prod _{i=1}^n\lambda _i{}^{a_i(x)}$$

where $\lambda =({\lambda }_1,\mathrm{\dots },{\lambda }_n)$.

And define $P:\sigma \to ℝ$ and $Q:\sigma \times \sigma \to ℝ$by

$$P(\lambda )=\int p(x,\lambda )𝑑m(x)$$

and

$$Q(\lambda ,{\lambda }^{\prime })=\int p(x,\lambda )\log p(x,{\lambda }^{\prime })𝑑m(x).$$

Define $\overline{{\lambda }_i}$ by

$$\overline{{\lambda }_i}=\frac{{\lambda }_i\partial P/\partial {\lambda }_i}{{\sum }_j{\lambda }_j\partial P/\partial {\lambda }_j}.$$

Theorem. Let $\lambda \in \sigma$ and $\overline{\lambda }$ be definedas above. Then for every ${\lambda }^{\prime }\in \sigma$ we have

$$Q(\lambda ,{\lambda }^{\prime })\le Q(\lambda ,\overline{\lambda })$$

with strict inequality unless ${\lambda }^{\prime }=\overline{\lambda }$. Also,

$$P(\lambda )\le P(\overline{\lambda })$$

with strict inequality unless $\lambda =\overline{\lambda }$.The second inequality is known as the PTAH inequality.

The significance of the PTAH inequality is that some of theclassical inequalities are all special cases of PTAH.

Consider:

(A) The arithmetic-geometric mean inequality:

$$\prod x_i{}^{\frac{1}{n}}\le \sum \frac{x_i}{n}$$

(B) the concavity of $\log x$:

$$\sum {\theta }_i\log x_i\le \log \sum {\theta }_i{x}_i$$

(C) the Kullback-Leibler inequality:

$$\prod \theta _i{}^{r_i}\le \prod {(\frac{r_i}{\sum r_j})}^{r_i}$$

(D) the convexity of $x\log x$:

$$(\sum {\theta }_i{x}_i)\log (\sum {\theta }_i{x}_i)\le \sum {\theta }_i{x}_i\log x_i$$

(E)

$$Q(\lambda ,{\lambda }^{\prime })\le Q(\lambda ,\overline{\lambda })$$

(F)

$$Q(\lambda ,{\lambda }^{\prime })-Q(\lambda ,\lambda )\le P(\lambda )\log \frac{P({\lambda }^{\prime })}{P(\lambda )}$$

(G) the maximum-entropy inequality (in logarithmic form)

$$-\sum _{i=1}^n{p}_i\log p_i\le \log n$$

(H) Hölder’s generalized inequality (http://planetmath.org/GeneralizedHolderInequality)

$$\sum _{j=1}^n\prod _{i=1}^m{a}_{i,j}^{{\theta }_i}\le \prod _{i=1}^m{\left(\sum _{j=1}^n{a}_{i,j}\right)}^{{\theta }_i}$$

(P) The PTAH inequality:

$$P(\lambda )\le P(\overline{\lambda })$$

All the sums and products range from 1 to $n$, all the${\theta }_i,x_i,r_i$ are positive and$({\theta }_i),\lambda ,{\lambda }^{\prime }$ are in $\sigma$ and the set $X$ isdiscrete, so that

$$P(\lambda )=\sum _{x}p(x,\lambda )m(x)$$

$$Q(\lambda ,{\lambda }^{\prime })=\sum _i{r}_i(\lambda )\log \lambda _i{}^{\prime }$$

where $m(x)>0$ $p(x,\lambda )={\prod }_i\lambda _i{}^{a_i(x)}$and $a_i(x)\ge 0$,$r_i(\lambda )=\sum a_i(x)p(x,\lambda )m(x)$and

$$\overline{{\lambda }_i}=\frac{{\lambda }_i\partial P/\partial {\lambda }_i}{\sum {\lambda }_j\partial P/\partial {\lambda }_j},$$

and $\overline{\lambda }=(\overline{{\lambda }_i})$.Then it turns out that (A) to (G) are all special cases of (H), andin fact (A) to (G) are all equivalent, in the sense that given any two of them,each is a special case of the other.(H) is a special case of (P), However, it appears that none of the reverseimplications holds.According to George Soules:

”The folklore at the Institute for Defense Analyses in Princeton NJis that the first program to maximize a function P(z) by iterating the growth transformation

$$z\to \overline{z}$$

was written while the programmer was listening to the opera Aida,in which the Egyptian god of creation Ptah is mentioned, and that became the name of the program (and of the inequality). The name is in upper case because the word processor in use in the middle 1960’shad no lower case.”