Finding Global Minima in Likelihood Functions

In summary: Other than that, there are many global optimizers that can be used but for a problem like this, you can also consider using Bayesian optimization methods. These methods can provide a posterior distribution instead of just a point estimate, which can be helpful in some cases. However, if the function is computationally expensive, you may want to consider other options such as MCMC sampling optimization. Keep in mind that the number of parameters may increase in the future, so it's important to choose an optimization method that can handle higher dimensions.
  • #1
tworitdash
108
26
I have a likelihood function that has one global minima, but a lot of local ones too. I attach a figure with the likelihood function in 2D (it has two parameters). I have added a 3D view and a surface view of the likelihood function. I know there are many global optimizers that can be used to obtain the location of the global minimum point in the likelihood function. However, I want to know what basic optimizer principles that I can use (that I can also derive and implement myself) for a problem like this. If you see the 3D view, you may find many local minima. I am also open to suggestions that involve Bayesian type of optimization where I will get a posterior and not just a point estimate. I am open to that as well. I have tried MCMC type sampling optimization, however, they are computationally expensive. The number of parameters may increase later.
 

Attachments

  • cKJT2.png
    cKJT2.png
    19.1 KB · Views: 108
  • 5KeQE.png
    5KeQE.png
    29.2 KB · Views: 115
Mathematics news on Phys.org
  • #2
Is it literally just this function you want to optimize?

You already did it, by drawing a graph. More formally if that's unsatisfying, for low dimensions and fast evaluation functions you can just evaluate the function at every point on a fine grid and pick the point with the best value. If you want a little extra precision you can run any optimizer from there to find the local extremum near that point.
 
  • Like
Likes WWGD and tworitdash

FAQ: Finding Global Minima in Likelihood Functions

What is a global minimum in the context of likelihood functions?

A global minimum in the context of likelihood functions refers to the point in the parameter space where the likelihood function attains its lowest value across all possible parameter values. This is significant in statistical modeling because it indicates the parameter set that best explains the observed data, often corresponding to the most probable parameter estimates under a given model.

Why is finding the global minimum important in statistical modeling?

Finding the global minimum is crucial because it ensures that the parameter estimates obtained are the most accurate and reliable. If only local minima are found, the resulting parameter estimates may not reflect the true underlying relationships in the data, leading to biased or suboptimal models that can misinform decision-making and predictions.

What methods are commonly used to find global minima in likelihood functions?

Common methods for finding global minima in likelihood functions include optimization algorithms such as grid search, simulated annealing, genetic algorithms, and Bayesian optimization. These techniques explore the parameter space more thoroughly than traditional gradient descent methods, which may be trapped in local minima.

What challenges are associated with finding global minima in likelihood functions?

Challenges include the presence of multiple local minima, high-dimensional parameter spaces, and the computational cost associated with evaluating the likelihood function over a wide range of parameters. Additionally, the likelihood surface can be complex, making it difficult to apply standard optimization techniques effectively.

How can one verify if a minimum found is a global minimum?

To verify if a minimum found is a global minimum, one can employ techniques such as comparing results from multiple optimization algorithms, using sensitivity analysis to assess the stability of the solution, or conducting a thorough exploration of the parameter space with different starting points. Additionally, one can analyze the likelihood surface visually, if feasible, to confirm the nature of the minima.

Similar threads

I Gradient descent algorithm and optimizers
Replies
5
Views
1K
Solve the Global Minima Problem in Two Variable Functions
Replies
3
Views
1K
Find global minimum of f(x1,x2, xn), with many local minima
Replies
1
Views
2K
I What Does the Likelihood Function Tell Us?
Replies
16
Views
2K
I Discussion about least squares method
Replies
2
Views
1K
A Fisher information from likelihood function for quantum circuits
Replies
0
Views
643
MHB Finding MLE for $\theta$ of Statistical Product Model
Replies
6
Views
2K
I Finding global minima of nth degree polynomials
Replies
2
Views
1K
A Correlation between parameters in a likelihood fit
Replies
3
Views
1K
Finding Absolute Minima & Maxima of a Function
Replies
8
Views
1K

Hot Threads

Recent Insights

Back
Top