Laplace approximation in Bayesian inference

In summary, the conversation discusses the use of Bayesian inference for estimating multiple parameters in a Python project. The Laplace approximation in n-dimensions is suggested for evaluating the evidence, and the formula for calculating the evidence is provided. The speaker is having trouble obtaining the maximum value of the posterior, FX0, for more than 4 parameters and asks for suggestions. A potential solution using corner.py is mentioned.
  • #1
BRN
108
10
Hello everybody,
I am working on a Python project in which I have to make Bayesian inference to estimate 4 or more parameters using MCMC.
I also need to evaluate the evidence and I thought to do so through the Laplace approximation in n-dimensions:

$$ E = P(x_0)2\pi^{n/2}|C|^{1/2} $$

Where C is the parameter's covariance matrix and ##P(x_0)## is the maximum value that assumes the posterior.
Getting the covariance matrix is not a problem, but I don't know how get FX0. If they were only 2 parameters I could use matplotlib.hist2d, but being more than 4 parameters...
How could I do?
Some idea?

Thank you!
 
Technology news on Phys.org
  • #2
BRN said:
FX0.
What is this ?
Are you looking for something like this

https://corner.readthedocs.io/en/latest/pages/quickstart.html
 

FAQ: Laplace approximation in Bayesian inference

What is Laplace approximation in Bayesian inference?

Laplace approximation is a method used in Bayesian inference to approximate a complex probability distribution with a simpler one, often a Gaussian distribution. It is based on the Taylor expansion of the logarithm of the original distribution around its mode.

When is Laplace approximation useful in Bayesian inference?

Laplace approximation is useful when the original probability distribution is difficult to work with mathematically, but its mode and curvature can be easily determined. It is also useful when the original distribution is multi-modal, as it can approximate each mode separately.

How accurate is Laplace approximation compared to other methods?

Laplace approximation is known to be a good approximation for unimodal distributions, as it captures the shape and location of the distribution well. However, it may not be as accurate for distributions with multiple modes or heavy tails.

What are the limitations of Laplace approximation?

Laplace approximation relies on the assumption that the original distribution is well-approximated by a Gaussian near its mode. Therefore, it may not be accurate for highly skewed or asymmetric distributions. Additionally, it does not provide any information about the tails of the distribution.

How is Laplace approximation implemented in practice?

In practice, Laplace approximation involves calculating the mode and curvature of the logarithm of the original distribution, then using these values to approximate the original distribution with a Gaussian. This can be done analytically for some distributions, but may require numerical optimization for others.

Similar threads

How are hyperparameters determined in Bayesian optimization?
Replies
2
Views
1K
Python Re: Python’s Sympy Module and the Cayley-Hamilton Theorem
Replies
3
Views
724
Bernoulli and Bayesian probabilities
Replies
4
Views
1K
Python Minimization likelihood function with parameters
Replies
6
Views
2K
Python Transpose of a non-square matrix (without using ndarray.transpose)
Replies
15
Views
4K
MacKay Textbook Example: Laplace Approximation
Replies
1
Views
1K
Python Solving an Integral equation with uncertainties
Replies
1
Views
941
Python Finding a local max/min of a function in Python
Replies
7
Views
4K
Python What is the Nelder-Mead algorithm and how can it be implemented in pure Python?
Replies
3
Views
4K
I Likelihood, posterior, prior interpretation and credibility/confidence
Replies
1
Views
1K

Hot Threads

Recent Insights

Back
Top