Engel’s theorem

Before proceeding, it will be useful to recall the definition of anilpotent Lie algebra. Let $\\mathfrak{g}$ be a Lie algebra. The lower central series of $\\mathfrak{g}$ is defined to be the filtration of ideals

\\mathcal{D}_{0}\\mathfrak{g}\\supset\\mathcal{D}_{1}\\mathfrak{g}\\supset\\mathcal{D%}_{2}\\mathfrak{g}\\supset\\ldots,

where

\\mathcal{D}_{0}\\mathfrak{g}=\\mathfrak{g},\\qquad\\mathcal{D}_{k+1}\\mathfrak{g}=[%\\mathfrak{g},\\mathcal{D}_{k}\\mathfrak{g}],\\quad k\\in\\mathbb{N}.

To say that $\\mathfrak{g}$ is nilpotent is to say that the lowercentral series has a trivial termination, i.e. that there exists a $k$ such that

\\mathcal{D}_{k}\\mathfrak{g}=0,

or equivalently, that $k$ nested bracket operationsalways vanish.

Theorem 1 (Engel)

Let $\\mathfrak{g}\\subset\\mathop{\\mathrm{End}}\olimits V$ be a Lie algebra of endomorphisms of afinite-dimensional vector space $V$ . Suppose that all elements of $\\mathfrak{g}$ are nilpotent transformations. Then, $\\mathfrak{g}$ is a nilpotent Liealgebra.

Lemma 1

Let $X:V\\rightarrow V$ be a nilpotent endomorphism of a vectorspace $V$ . Then, the adjoint action

\\mathop{\\mathrm{ad}}\olimits(X):\\mathop{\\mathrm{End}}\olimits V\\rightarrow%\\mathop{\\mathrm{End}}\olimits V

is also a nilpotent endomorphism.

Proof.

Suppose that

X^{k}=0

for some $k\\in\\mathbb{N}$ . We will show that

\\mathop{\\mathrm{ad}}\olimits(X)^{2k-1}=0.

Note that

\\mathop{\\mathrm{ad}}\olimits(X)=l(X)-r(X),

where

l(X),r(X):\\mathop{\\mathrm{End}}\olimits V\\rightarrow\\mathop{\\mathrm{End}}%\olimits V,

are the endomorphismscorresponding, respectively, to left and right multiplication by $X$ .These two endomorphisms commute, and hence we can use the binomialformula to write

\\mathop{\\mathrm{ad}}\olimits(X)^{2k-1}=\\sum_{i=0}^{2k-1}(-1)^{i}\\,l(X)^{2k-1-%i}\\,r(X)^{i}.

Each of terms in the above sum vanishes because

l(X)^{k}=r(X)^{k}=0.

QED

Lemma 2

Let $\\mathfrak{g}$ be as in the theorem, and suppose, in addition, that $\\mathfrak{g}$ is a nilpotent Lie algebra. Then the joint kernel,

\\ker\\mathfrak{g}=\\bigcap_{a\\in\\mathfrak{g}}\\ker a,

is non-trivial.

Proof.

We proceed by induction on the dimension of $\\mathfrak{g}$ .The claim is true for dimension 1, because then $\\mathfrak{g}$ is generated bya single nilpotent transformation, and all nilpotent transformationsare singular.

Suppose then that the claim is true for all Lie algebras of dimensionless than $n=\\dim\\mathfrak{g}$ . We note that $\\mathcal{D}_{1}\\mathfrak{g}$ fits the hypothesesof the lemma, and has dimension less than $n$ , because $\\mathfrak{g}$ isnilpotent. Hence, by the induction hypothesis

V_{0}=\\ker\\mathcal{D}_{1}\\mathfrak{g}

is non-trivial. Now, if we restrict allactions to $V_{0}$ , we obtain a representation of $\\mathfrak{g}$ by abeliantransformations. This is because for all $a,b\\in\\mathfrak{g}$ and $v\\in V_{0}$ we have

abv-bav=[a,b]v=0.

Now a finite number of mutuallycommuting linear endomorphisms admits a mutual eigenspacedecomposition. In particular, if all of the commuting endomorphismsare singular, their joint kernel will be non-trivial. We applythis result to a basis of $\\mathfrak{g}/\\mathcal{D}_{1}\\mathfrak{g}$ acting on $V_{0}$ ,and the desired conclusion follows. QED

Proof of the theorem.

We proceed by induction on the dimension of $\\mathfrak{g}$ . The theorem istrue in dimension 1, because in that circumstance $\\mathcal{D}_{1}\\mathfrak{g}$ istrivial.

Next, suppose that the theorem holds for all Lie algebras of dimensionless than $n=\\dim\\mathfrak{g}$ . Let $\\mathfrak{h}\\subset\\mathfrak{g}$ be a properly containedsubalgebra of minimum codimension. We claim that there exists an $a\\in\\mathfrak{g}$ but not in $\\mathfrak{h}$ such that $[a,\\mathfrak{h}]\\subset\\mathfrak{h}$ .

By the induction hypothesis, $\\mathfrak{h}$ is nilpotent. To prove the claimconsider the isotropy representation of $\\mathfrak{h}$ on $\\mathfrak{g}/\\mathfrak{h}$ . ByLemma 1, the action of each $a\\in\\mathfrak{h}$ on $\\mathfrak{g}/\\mathfrak{h}$ is a nilpotentendomorphism. Hence, we can apply Lemma 2 to deduce that the jointkernel of all these actions is non-trivial, i.e. there exists a $a\\in\\mathfrak{g}$ but not in $\\mathfrak{h}$ such that

[b,a]\\equiv 0\\mod\\mathfrak{h},

for all $b\\in\\mathfrak{h}$ . Equivalently, $[\\mathfrak{h},a]\\subset\\mathfrak{h}$ and the claimis proved.

Evidently then, the span of $a$ and $\\mathfrak{h}$ is a subalgebra of $\\mathfrak{g}$ .Since $\\mathfrak{h}$ has minimum codimension, we infer that $\\mathfrak{h}$ and $a$ span all of $\\mathfrak{g}$ , and that

\\mathcal{D}_{1}\\mathfrak{g}\\subset\\mathfrak{h}.

(1)

Next, we claim that all the $\\mathcal{D}_{k}\\mathfrak{h}$ are ideals of $\\mathfrak{g}$ . It isenough to show that

[a,\\mathcal{D}_{k}\\mathfrak{h}]\\subset\\mathcal{D}_{k}\\mathfrak{h}.

We argue by induction on $k$ . Suppose theclaim is true for some $k$ . Let $b\\in\\mathfrak{h},c\\in\\mathcal{D}_{k}\\mathfrak{h}$ be given. Bythe Jacobi identity

[a,[b,c]]=[[a,b],c]+[b,[a,c]].

The first term on the right hand-side in $\\mathcal{D}_{k+1}\\mathfrak{h}$ because $[a,b]\\in\\mathfrak{h}$ . The second term is in $\\mathcal{D}_{k+1}\\mathfrak{h}$ by the inductionhypothesis. In this way the claim is established.

Now $a$ is nilpotent, and hence by Lemma 1,

\\mathop{\\mathrm{ad}}\olimits(a)^{n}=0

(2)

for some $n\\in\\mathbb{N}$ . We now claim that

\\mathcal{D}_{n+1}\\mathfrak{g}\\subset\\mathcal{D}_{1}\\mathfrak{h}.

By (1) it suffices to show that

[\\overbrace{\\mathfrak{g},[\\ldots[\\mathfrak{g}}^{n\\text{ times}},\\mathfrak{h}]%\\ldots]]\\subset\\mathcal{D}_{1}\\mathfrak{h}.

Putting

\\mathfrak{g}_{1}=\\mathfrak{g}/\\mathcal{D}_{1}\\mathfrak{h},\\quad\\mathfrak{h}_{1%}=\\mathfrak{h}/\\mathcal{D}_{1}\\mathfrak{h},

this is equivalent to

[\\overbrace{\\mathfrak{g}_{1},[\\ldots[\\mathfrak{g}_{1}}^{n\\text{ times}},%\\mathfrak{h}_{1}]\\ldots]]=0.

However, $\\mathfrak{h}_{1}$ is abelian, and hence, the above follows directlyfrom (2).

Adapting this argument in the obvious fashion we can show that

\\mathcal{D}_{kn+1}\\mathfrak{g}\\subset\\mathcal{D}_{k}\\mathfrak{h}.

Since $\\mathfrak{h}$ is nilpotent, $\\mathfrak{g}$ must be nilpotent as well. QED

Historical remark.

In the traditional formulation of Engel’s theorem, the hypotheses arethe same, but the conclusion is that there exists a basis $B$ of $V$ ,such that all elements of $\\mathfrak{g}$ are represented by nilpotent matricesrelative to $B$ .

Let us put this another way. The vector space of nilpotent matrices $\\mathop{\\mathrm{Nil}}\olimits$ , is a nilpotent Lie algebra, and indeed all subalgebras of $\\mathop{\\mathrm{Nil}}\olimits$ arenilpotent Lie algebras. Engel’s theorem asserts that the converseholds, i.e. if all elements of a Lie algebra $\\mathfrak{g}$ are nilpotenttransformations, then $\\mathfrak{g}$ is isomorphic to a subalgebra of $\\mathop{\\mathrm{Nil}}\olimits$ .

The classical result follows straightforwardly from our version of theTheorem and from Lemma 2. Indeed, let $V_{1}$ be the joint kernel $\\mathfrak{g}$ . We then let $U_{2}$ be the joint kernel of $\\mathfrak{g}$ acting on $V/V_{0}$ , and let $V_{2}\\subset V$ be the subspace obtained by pulling $U_{2}x$ back to $V$ . We do this a finite number of times and obtain aflag of subspaces

0=V_{0}\\subset V_{1}\\subset V_{2}\\subset\\ldots\\subset V_{n}=V,

such that

\\mathfrak{g}V_{k+1}=V_{k}

for all $k$ . The choose an adapted basisrelative to this flag, and we’re done.