How to Determine the Minimizer for a Complex Optimization Problem?

  • Thread starter math8
  • Start date
In summary, the problem is to minimize the expression \left\| \nabla f(x) + \lambda ^T \nabla h(x) + \mu ^T \right\|^2 + \left\| \phi (\mu ) \right\|^2 with the constraints \lambda , \mu \geq 0, by finding the critical points of the function (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 or \quad x^2 \right\} , where g= \nabla f(x) + \lambda ^T \nabla h(x) and \phi _i (\mu
  • #1
math8
160
0
I am trying to find the minimizer of the function

[itex]\left\| \nabla f(x) + \lambda ^T \nabla h(x) + \mu ^T \right\|^2[/itex]
s.t. [itex]\mu _i \geq 0[/itex] , [itex]\mu _i = 0[/itex] if [itex]x_i > 0[/itex]

We use the function [itex]\phi _i (\mu ) = min \left\{ \mu _i , x_i \right\}[/itex]
we have that [itex]\mu ^T x = 0 \Leftrightarrow \phi ( \mu ) =0[/itex]

So we can actually solve the problem

Minimize [itex]\left\| \nabla f(x) + \lambda ^T \nabla h(x) + \mu ^T \right\|^2 + \left\| \phi (\mu ) \right\|^2 [/itex]
s.t. [itex]\lambda , \mu \geq 0[/itex]

Now my reasoning is, by letting [itex]g= \nabla f(x) + \lambda ^T \nabla h(x) [/itex] , the problem becomes:

Minimize [itex] (g+ \mu ) ^2 + \phi (\mu ) ^2 [/itex] i.e. Minimize [itex] (g+ \mu ) ^2 + min \left\{ \mu , x \right\}^2 [/itex]
or Minimize [itex] (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 or \quad x^2 \right\} [/itex]

Now, I think I should first find the critical points for the function [itex] (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 or \quad x^2 \right\} [/itex] . But, should I consider this function as a function of [itex] \mu [/itex] and [itex]g[/itex], or as a function of [itex]\mu [/itex]and [itex]x [/itex]?

Another way of thinking about this problem. If I use the 'Fischer-Burmeister' function which is:

I am trying to find the minimizer of the function

[itex]\left\| \nabla f(x) + \lambda ^T \nabla h(x) + \mu ^T \right\|^2[/itex]
s.t. [itex]\mu _i \geq 0[/itex] , [itex]\mu _i = 0[/itex] if [itex]x_i > 0[/itex]

We use the function [itex]\phi _i (\mu ) = min \left\{ \mu _i , x_i \right\}[/itex]
we have that [itex]\mu ^T x = 0 \Leftrightarrow \phi ( \mu ) =0[/itex]

So we can actually solve the problem

Minimize [itex]\left\| \nabla f(x) + \lambda ^T \nabla h(x) + \mu ^T \right\|^2 + [itex]\left\| \phi (\mu ) \right\|^2 [/itex]
s.t. [itex]\lambda , \mu \geq 0[/itex]

Now my reasoning is, by letting [itex]g= \nabla f(x) + \lambda ^T \nabla h(x) [/itex] , the problem becomes:

Minimize [itex] (g+ \mu ) ^2 + \phi (\mu ) ^2 [/itex] i.e. Minimize [itex] (g+ \mu ) ^2 + min \left\{ \mu , x \right\}^2 [/itex]
or Minimize [itex] (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 , x^2 \right\} [/itex]

Now, I think I should first find the critical points for the function [itex] (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 , x^2 \right\} [/itex] . But, should I consider this function as a function of [itex] \mu [/itex] and [itex]g[/itex], or as a function of [itex]\mu [/itex]and [itex]x [/itex]?

Another way of thinking about this problem. We can use the 'Fischer-Burmeister' function which is:

[itex]\Phi (\mu , x ) = \mu + x - \sqrt{\mu ^2 + x^2} [/itex] instead of the function [itex]\phi _i (\mu ) = min \left\{ \mu _i , x_i \right\}[/itex] because for the 'Fischer-Burmeister' function,
[itex]\Phi (\mu , x ) = 0 \Leftrightarrow \mu x =0 [/itex] just like for the previous function [itex]\phi _i (\mu )[/itex] .

Now, the problem would be to

Minimize [itex] (g+ \mu ) ^2 + \Phi (\mu ) ^2 [/itex] .

Again, how should I go about finding this minimum?
 
Physics news on Phys.org
  • #2
Is this a function of \mu and g, or \mu and x? The answer to your question is that the problem should be minimized as a function of \mu and g. The Fischer-Burmeister function does not change the way the problem is minimized. The problem is still to find the critical points of the function (g^2+ 2g \mu +\mu ^2 ) + \left\{ \mu ^2 , x^2 \right\}. The only difference is that now you are using the Fischer-Burmeister function instead of the min function.
 

FAQ: How to Determine the Minimizer for a Complex Optimization Problem?

What is a minimizer?

A minimizer is a mathematical concept that refers to the value or point at which a function reaches its lowest possible value. It is also known as the minimum value or minimum point.

Why is finding the minimizer important?

Finding the minimizer is important because it allows us to identify the optimal solution or the best possible outcome for a given problem. It is often used in optimization and decision-making processes to determine the most efficient or effective course of action.

How do you find the minimizer?

The process of finding the minimizer involves using mathematical techniques such as calculus and optimization algorithms. In some cases, the minimizer can also be determined analytically by taking the derivative of the function and setting it equal to 0.

Are there different types of minimizers?

Yes, there are different types of minimizers depending on the type of function and the constraints of the problem. Some common types include absolute minimizers, relative minimizers, and global minimizers.

Can the minimizer change depending on the inputs or parameters?

Yes, the minimizer can change depending on the inputs or parameters of the function. In some cases, the minimizer may remain constant, while in others, it may vary depending on the values of the inputs or parameters.

Back
Top