§ The geometry of Lagrange multipliers
If we want to minise a function subject to the constraints ,
one uses the method of lagrange multipliers. The idea is to consider a new
function . Now, if one has a local maxima
, then the conditions:
Equation (2) is sensible: we want our optima to satisfy the constraint that
we had originally imposed. What is Equation (1) trying to say?
Geometrically, it's asking us to keep parallel to .
Why is this a good ask?
Let us say that we are at an which is a feasible point ().
We are interested in wiggling
- : .
- : .
- is still feasible: .
- is an improvement: .
If and are parallel, then attempting to improve
by change , and thereby violate the constraint
- If we want to not change, then we need .
- If we want to be larger, we need .