Optimization with Lagrangian Multipliers

In summary, the conversation discusses finding the function f(x,y,z)=x subject to the constraint equation g(x,y,z)=6, and how it is derived. It is explained that the function is chosen to be parallel to the normal vector of the constraint surface, in order to satisfy the conditions for an extremum under the given constraint.
  • #1
Leo Liu
353
156
Homework Statement
.
Relevant Equations
##\nabla f=\lambda\nabla g##
Problem:
1615503170806.png

Solution:
1615503188617.png

My question:
My reasoning was that if x is max at the point then the gradient vector of g at the point has only x component; that is ##g_y=0,\, g_z=0##. This way I got:
$$\begin{cases}
4y^3+x+z=0\\
\\
4z^3+x+y=0\\
\\
\underbrace{x^4+y^4+z^4+xy+yz+zx=6}_\text{constraint equation}
\end{cases}$$
which produces the same solutions as the list of equations in the official answer does.

What puzzles me is why the answer defines that ##f(x,y,z)=x##. I understand the gradient vector should be parallel to the vector ##<1,0,0>##, and therefore the equation f is x. But this step is reverse engineered. Can someone please explain where ##f(x,y,z)=x## comes from?

Many thanks.
 
Physics news on Phys.org
  • #2
Leo Liu said:
Can someone please explain where ##f(x,y,z)=x## comes from?
That's just the function you're extremising, ##f(\boldsymbol{x}) = x## (i.e. the function taking a point ##\in \mathbb{R}^3## to its ##x## co-ordinate, which is what you're trying to maximise) subject to the constraint ##g(\boldsymbol{x}) = 6##. It's essentially by construction.

To give some more context, the constraint defines a 2-dimensional surface with normal vector ##\nabla g(\boldsymbol{x})##; displacements ##\mathrm{d}\boldsymbol{x}## within this surface must satisfy ##\mathrm{d}\boldsymbol{x} \cdot \nabla g (\boldsymbol{x} )= 0##, i.e. they must be parallel to the surface. Furthermore, if ##\boldsymbol{x}_0## is the point corresponding to the extremum of ##f(\boldsymbol{x})## under the given constraint, you must have ##\mathrm{d}\boldsymbol{x} \cdot \nabla f (\boldsymbol{x}_0) = 0##, since the change in the value of the function will be zero to first order under a displacement ##\mathrm{d}\boldsymbol{x}##. The only way these two things are possible is if $$\nabla f(\boldsymbol{x}_0) = \lambda \nabla g(\boldsymbol{x}_0)$$It's because if they were not parallel, you would have$$\nabla f(\boldsymbol{x}_0) = \lambda \nabla g(\boldsymbol{x}_0) + \mathbf{a}(\boldsymbol{x}_0) \implies \mathrm{d}\boldsymbol{x} \cdot \nabla f(\boldsymbol{x}_0) = \mathrm{d}\boldsymbol{x} \cdot \mathbf{a}(\boldsymbol{x}_0)$$for some ##\mathbf{a}(\boldsymbol{x}_0)## which is perpendicular to ##\nabla g(\boldsymbol{x}_0)##. But since ##\mathrm{d}\boldsymbol{x}## can be a displacement of any direction in the constraint surface, we can consider letting ##\mathrm{d}\boldsymbol{x} = \mathbf{a}(\boldsymbol{x}_0) \mathrm{d}s##, in which case ##\mathrm{d}\boldsymbol{x} \cdot \mathbf{a}(\boldsymbol{x}_0) = |\mathbf{a}(\boldsymbol{x}_0)|^2 \mathrm{d}s > 0##, which means that ##\mathrm{d}\boldsymbol{x} \cdot \nabla f(\boldsymbol{x}_0)## would no longer be zero.
 
Last edited by a moderator:
  • Informative
Likes Leo Liu

FAQ: Optimization with Lagrangian Multipliers

What is the purpose of using Lagrangian multipliers in optimization?

Lagrangian multipliers are used in optimization to find the maximum or minimum value of a function subject to a set of constraints. This method allows for the incorporation of constraints into the optimization process, which can be useful in real-world applications where there are limitations or restrictions on the variables.

How do Lagrangian multipliers work?

Lagrangian multipliers work by creating a new function, called the Lagrangian, which combines the original objective function with the constraints using a set of multipliers. This new function is then optimized with respect to both the original variables and the multipliers, resulting in a set of equations known as the Lagrange equations. These equations can be solved to find the optimal values for both the variables and the multipliers.

What are the benefits of using Lagrangian multipliers in optimization?

Using Lagrangian multipliers allows for the inclusion of constraints in the optimization process, which can lead to more realistic and practical solutions. It also simplifies the optimization process by converting a constrained problem into an unconstrained one, making it easier to solve. Additionally, Lagrangian multipliers can be applied to a wide range of optimization problems, including both single and multi-variable functions.

What are the limitations of using Lagrangian multipliers?

One limitation of using Lagrangian multipliers is that it may not always result in the global optimal solution. This is because the method relies on finding the critical points of the Lagrangian function, which may not always correspond to the global optimal solution. Additionally, the Lagrangian method can become computationally intensive for complex optimization problems, making it less practical in some cases.

Can Lagrangian multipliers be used for non-linear optimization problems?

Yes, Lagrangian multipliers can be used for both linear and non-linear optimization problems. However, the process becomes more complex for non-linear problems, as the Lagrange equations may not have a simple closed-form solution. In these cases, numerical methods or approximations may be used to find the optimal solution.

Back
Top