Estimating joint distributions from marginal

In summary, you are trying to approximate the joint probability distribution for two variables, A and B, using Baye's theorem. However, you need to use special knowledge to do this and the approximation is not perfect.
  • #1
exmachina
44
0
Suppose I have the marginal probability density functions of two random variables A and B, P(A), and P(B). Suppose I modeled P(A) and P(B) using a mixture model from some dataset D and obtained a closed form pdf for each.

I am interested in finding their joint density function P(A and B) and associated properties such as maximas, minimas, etc.

Ideally the joint density is expressed as a closed form 2D mixture model as well, but this is not critical.

I could do something perhaps by brute force by use of Baye's theorem:

ie. I can approximate

P(A and B) = P(A) P(B | A) = P(B) | P(A | B)

But eventually I need to extend this to higher dimensions, eg. P( A and B and C and D... etc) and this is certainly no trivial task.
 
Physics news on Phys.org
  • #2
In general, you cannot determine a joint probability distribution when given only the marginal probability distributions, so if your problem can be solved the solution depends on special circumstances or information that you haven't mentioned. To get the best advice, you should describe the situation completely.

exmachina said:
I could do something perhaps by brute force by use of Baye's theorem:

ie. I can approximate

P(A and B) = P(A) P(B | A) = P(B) | P(A | B)

That isn't a mere approximation. It is a theorem.

Are you saying that you have data that could be used estimate the conditional probability distributions?
 
  • #3
Well A and B are two variables that specify (completely) the state of the system. Suppose I've sampled a whole bunch of data points (a,b) s.t. I can generate their PDFs.

I can approximate P(B | A=a1) and P(A | B=b1) as well by taking a slice of my dataset, (eg. B= b1+-0.1) and count the occurrences of A. However, this can be bad because my entire dataset may be quite small, and using only a subset of it will result in a lot of noise and error.
 
  • #4
exmachina said:
this can be bad because my entire dataset may be quite small, and using only a subset of it will result in a lot of noise and error.

I think the only convenient cure for small amounts of data is to build-in a lot of structure to the answer - for example, you might assume the distribution you are trying to determine is from a family of distributions that are defined by only a few parameters and estimate those parameters from the data. To do this you must employ any expert knowledge that you have about the situation. For example you may know that certain families of distributions have a plasusible shape and others don't.

You haven't described the problem clearly, but from your remarks, I conjecture that you are dealing with continuous variates. Some technicalities about your terminology: The value of a continuous probability density function does not give "the probability of" particular values. (For example, think about the uniform probability density function on the interval [0, 1/2] which has constant value 2.) However, I agree that it is often helpful to think about density functions informally that way. Observed frequences of values in a sample are not probabilities (unless you are taking about randomly selecting a value from the sample itself.) So you shouldn't use the p(A|B) notation for them. Of course, observed frequencies can be used as estimators of probabilities.
 
  • #5


I understand your interest in finding the joint density function and associated properties of two random variables A and B. It is a crucial step in understanding the relationship between these variables and making predictions based on their values.

Using a mixture model to model the marginal probability density functions of A and B is a valid approach. However, as you mentioned, finding the joint density function for higher dimensions can be a challenging task. One potential solution is to use Bayes' theorem, as you suggested, to approximate the joint density. This can be a useful approach, but it may not always yield accurate results.

Another approach could be to use numerical methods, such as Monte Carlo simulations, to estimate the joint density function. This involves generating a large number of random samples from the marginal distributions and calculating the joint probability for each sample. By repeating this process multiple times, you can obtain a more accurate estimation of the joint density function.

Additionally, there are other statistical methods, such as copulas, that can be used to model joint distributions from marginal distributions. These methods take into account the dependence between variables and can provide a more accurate representation of the joint density function.

In summary, estimating joint distributions from marginal distributions can be a challenging task, especially for higher dimensions. It is important to consider various methods and approaches, such as Bayes' theorem, numerical methods, and statistical models, to obtain the most accurate estimation of the joint density function and its associated properties.
 

FAQ: Estimating joint distributions from marginal

1. What is the goal of estimating joint distributions from marginal?

The goal is to understand the relationship between two variables by examining their joint distribution, which shows how the variables are related to each other.

2. How are marginal distributions and joint distributions related?

Marginal distributions are formed by summing or integrating over the joint distribution, showing the individual distributions of each variable. The joint distribution is the product of the marginal distributions and the conditional probability of one variable given the other.

3. What methods are commonly used to estimate joint distributions from marginal?

Common methods include maximum likelihood estimation, Bayesian estimation, and non-parametric methods such as kernel density estimation.

4. What are some challenges in estimating joint distributions from marginal?

One challenge is the curse of dimensionality, where the number of variables increases the complexity and computational burden of estimating the joint distribution. Additionally, the choice of method and assumptions made can greatly impact the accuracy of the estimates.

5. How can joint distributions from marginal be used in practical applications?

Joint distributions from marginal can be used to model and predict the behavior of complex systems, such as weather patterns or financial markets. They can also be used in decision-making processes and risk analysis by providing a comprehensive understanding of the relationship between variables.

Similar threads

Replies
2
Views
1K
Replies
2
Views
1K
Replies
1
Views
1K
Replies
4
Views
3K
Replies
1
Views
2K
Replies
2
Views
1K
Back
Top