Lagrange multiplier method

The Lagrange multiplier method is used when one needs to find the extreme or stationary points of a function on a set which is a subset of the domain.

Method

Suppose that $f(\\mathbf{x})$ and $g_{i}(\\mathbf{x}),i=1,...,m$ ( $\\mathbf{x}\\in\\mathbbmss{R}^{n}$ ) are differentiable functions that map $\\mathbbmss{R}^{n}\\mapsto\\mathbbmss{R}$ , and we want to solve

\\min f(\\mathbf{x}),\\max f(\\mathbf{x})\\quad\\mbox{such that}\\quad g_{i}(\\mathbf{%x})=0,\\quad i=1,\\ldots,m

By a calculus theorem, if the constaints are independent, the gradient of $f$ , $\abla f$ , must satisfy the following equation at the stationary points:

\abla f=\\sum_{i=1}^{m}\\lambda_{i}\abla g_{i}

The constraints are said to be independent iff all the gradients of each constraint are linearly independent, that is:

$\\left\\{\abla g_{1}(\\mathbf{x}),\\ldots,\abla g_{m}(\\mathbf{x})\\right\\}$ is a set of linearly independent vectors on all points where the constraints are verified.

Note that this is equivalent to finding the stationary points of:

f(\\mathbf{x})-\\sum_{i=1}^{m}\\lambda_{i}(g_{i}(\\mathbf{x}))

for $\\mathbf{x}$ in the domain and the Lagrange multipliers $\\lambda_{i}$ without restrictions.

After finding those points, one applies the $g_{i}$ constraints to get the actual stationary points subject to the constraints.