Lagrange multipliers understanding

In summary, Lagrange multipliers are a mathematical technique used to find the extrema of a function subject to constraints. This method involves introducing auxiliary variables (the multipliers) that transform a constrained optimization problem into an unconstrained one. By defining a new function that incorporates the original function and the constraints, the critical points can be found by setting the gradient of this new function to zero. This approach is particularly useful in economics, engineering, and physics, where optimization with constraints is often required.
  • #1
lys04
113
4
Homework Statement
constraints in Lagrange multiplier questions and the nature of the Extrema
Relevant Equations
∇f= λ∇g
Here’s my basic understanding of Lagrange multiplier problems:

A typical Lagrange multiplier problem might be to maximise f(x,y)=x^2-y^2 with the constraint that x^2+y^2=1 which is a circle of radius 1 that lie on the x-y plane. The points on the circle are the points (x,y) that satisfy the constraint equation. The problem is to find which of these (x,y) points maximises the function f.
Can view x^2+y^2=1 as the level curve when c=1 for the function g(x,y)=x^2+y^2

And this might happen when the gradient of g is parallel to the gradient of f. This is because the gradient of f, a vector quantity, indicates the direction to move the (x,y) point to get the greatest rate of change of f. If this vector is perpendicular to the surface of the level curve, that means no component of the gradient vector is in the direction of the tangent line at the point (x,y). And because the gradient is perpendicular to the level curve at any point, this means the gradient of f is parallel to the gradient of g.

Here’s where I get confused,
Take the problem to be maximise f(x,y,z)=x^2-y^2+yz with the constraint z=y^2.
I know that I can rewrite the constraint equation into z-y^2=0, which can be viewed as the level curve c=0 of the function g(x,y,z)=z-y^2 or g(y,z)=z-y^2? I get a bit confused here but I believe the first one, g(x,y,z) is correct because only then the gradient will have 3 components and can be parallel to the gradient of f which also has 3 components?
So basically here I have the level SURFACE of g(x,y,z)=z-y^2 at c=0, and finding out which of these points on the level SURFACE gives me the maximum value of f?

And secondly, I would like to clarify Lagrange multiplier problems with two constraints.
Lagrange multiplier problems with two constraints, i.e maximising f with constraints g=c_1 and h=c_2 (assume that f, g and h are all functions of three variables). Then I want to find points (x,y,z) that satisfy both constraints, and out of these points I want to find ones that maximises/minimises f.

Given that the level curves of g and h intersect, call it C, along the curve, the gradients of g and h will be perpendicular to the tangent lines.

And when f is at a maximum/minimum, there is no direction I can move so that f will continue to increase/decrease. This means that the gradient of f at the maximum/minimum must be perpendicular to the tangent of C. This means that it’s also parallel to the gradients of g and h.

Using this equation below, I get 5 equations by taking the partial derivative with respect to each variable. (First image)

Here’s what I am confused about this, from the last 3 equations, how do we know that the gradient of f is parallel to that of g and h?
And I think it’s because:
And since from before I know that the gradient of g and h are parallel, this means (second image)
So if I plug this into the equations above for the partial derivatives of x, y and z then I’ll get that the partial derivative of f with respect to x, y and z is equal to a scalar multiple times the partial derivative of h? (Not actually gonna do this, just an idea for how the partial derivative of f equally a linear combination of the partial derivatives of g and h means that they are parallel).

And I can’t use the second derivative test on the solutions I found to tell whether it’s a maximum or a minimum because these are not necessarily the critical points of f, although they are the critical points of L?
 

Attachments

  • IMG_0114.jpeg
    IMG_0114.jpeg
    35.3 KB · Views: 57
  • IMG_0115.jpeg
    IMG_0115.jpeg
    8.6 KB · Views: 50
Physics news on Phys.org
  • #2
lys04 said:
Take the problem to be maximise f(x,y,z)=x^2-y^2+yz with the constraint z=y^2.
I’m no expert but would hate you to feel ignored!

You have:
Maximise: ##f(x,y,z)=x^2-y^2+yz##
Constraint: ##z=y^2##

The constraint does not contain ##x##. That means ##x^2## (hence ##f(x,y,z)##) can be as large as you want without being limited by the constraint.

Therefore, IMO, I’d say the question is faulty. But I will be pleased to be corrected if I've misunderstood.

Also worth noting - you’d probably get more replies if you typed your equations using LaTeX. (Link to LaTex guide at bottom/left of editting window.)

Edited.
 
Last edited:
  • #3
Thanks for replying! Appreciate it lots.
Yeah I got confused to by the fact that the constraint did not contain x, but this is a problem actually from the vector calculus booklet from my uni and the answer is that the Extrema are at points (0,0,0) and (0,2/3,4/9).
Although I agree with what you said, z=y^2 looks like a bunch of parabola cross sections, so if some z and y value minimises/maximises f then I can choose any x I want to make it even bigger/smaller?
 
  • Like
Likes Steve4Physics
  • #4
lys04 said:
Thanks for replying! Appreciate it lots.
Yeah I got confused to by the fact that the constraint did not contain x, but this is a problem actually from the vector calculus booklet from my uni and the answer is that the Extrema are at points (0,0,0) and (0,2/3,4/9).

Although I agree with what you said, z=y^2 looks like a bunch of parabola cross sections, so if some z and y value minimises/maximises f then I can choose any x I want to make it even bigger/smaller?
You can make ##f## bigger with any non-zero value of ##x##. But you can't make ##f## smaller because you are adding ##x^2## which is non-negative (assuming ##x## is real).

The official answer is only true if there is an additional constraint that ##x=0##.

For example, the points ##(100, 0, 0)## and ##(100, \frac 23 , \frac 49)## also satisfy the constraint but give much bigger values of ##f## than ##(0, 0, 0)## and ##(0, \frac 23 , \frac 49)##.

So there is a mistake in the question and/or the official answer. It happens!
 
Last edited:
  • Like
Likes lys04
  • #5
Is the problem to maximise the function or to find the stationary points? As mentioned, the x^2 term means that there cannot be any local maxima.
 
  • #6
I'll attach a picture of the original problem,
1707782003618.jpeg
 
  • #7
Steve4Physics said:
You can make ##f## bigger with any non-zero value of ##x##. But you can't make ##f## smaller because you are adding ##x^2## which is non-negative (assuming ##x## is real).

The official answer is only true if there is an additional constraint that ##x=0##.

For example, the points ##(100, 0, 0)## and ##(100, \frac 23 , \frac 49)## also satisfy the constraint but give much bigger values of ##f## than ##(0, 0, 0)## and ##(0, \frac 23 , \frac 49)##.

So there is a mistake in the question and/or the official answer. It happens!
Oh yeah forgot that x is squared.
Yeah thats true, not sure whats going on with this problem.

Thanks for your help! :)
 
  • #8
Steve4Physics said:
You can make ##f## bigger with any non-zero value of ##x##. But you can't make ##f## smaller because you are adding ##x^2## which is non-negative (assuming ##x## is real).

The official answer is only true if there is an additional constraint that ##x=0##.

For example, the points ##(100, 0, 0)## and ##(100, \frac 23 , \frac 49)## also satisfy the constraint but give much bigger values of ##f## than ##(0, 0, 0)## and ##(0, \frac 23 , \frac 49)##.

So there is a mistake in the question and/or the official answer. It happens!
I just checked the answers again and (0,0,0) is a saddle point and (0,2/3,4/9) is a local minimum, so that problem wouldn’t exist right
 
  • #9
lys04 said:
I'll attach a picture of the original problem,
View attachment 340250
So indeed it asks for the critical points and not for the maximum of f.
 

FAQ: Lagrange multipliers understanding

What are Lagrange multipliers?

Lagrange multipliers are a mathematical tool used in optimization problems to find the local maxima and minima of a function subject to equality constraints. They introduce additional variables (multipliers) to transform a constrained problem into a form where traditional optimization techniques can be applied.

How do Lagrange multipliers work?

Lagrange multipliers work by converting a constrained optimization problem into an unconstrained one. This is achieved by constructing a new function, called the Lagrangian, which incorporates the original function and the constraints using the multipliers. The critical points of this Lagrangian function, found by setting its gradient to zero, correspond to the solutions of the original constrained problem.

When should I use Lagrange multipliers?

Lagrange multipliers should be used when you need to optimize a function subject to one or more equality constraints. They are particularly useful in cases where the constraints make it difficult to apply standard optimization techniques directly.

What is the Lagrangian function?

The Lagrangian function is a combination of the original objective function and the constraints. For a function \( f(x, y) \) subject to a constraint \( g(x, y) = 0 \), the Lagrangian is defined as \( \mathcal{L}(x, y, \lambda) = f(x, y) + \lambda(g(x, y)) \), where \( \lambda \) is the Lagrange multiplier. This function is then used to find the critical points that satisfy both the objective function and the constraints.

Can Lagrange multipliers be used for inequality constraints?

While traditional Lagrange multipliers are typically used for equality constraints, there is a related method called the Karush-Kuhn-Tucker (KKT) conditions that extends the concept to handle inequality constraints. The KKT conditions provide necessary conditions for a solution to be optimal when both equality and inequality constraints are present.

Back
Top