Maximize A Posteriori: Countable Hypotheses

[OhMyMarkov]

Hello everyone!

Suppose we have multiple hypothesis, $H_1, H_2,\dots ,H_N$ of equal likelihood, and we wish to choose the unobserved parameter $\theta _m$ according to the following decision rule: $m _0 = arg \max _m p(x|H_m)$.

What if there are infinitely many hypotheses? (the case is countable but infinite)

 

[CaptainBlack]

Hello everyone!

Suppose we have multiple hypothesis, $H_1, H_2,\dots ,H_N$ of equal likelihood, and we wish to choose the unobserved parameter $\theta _m$ according to the following decision rule: $m _0 = arg \max _m p(x|H_m)$.

What if there are infinitely many hypotheses? (the case is countable but infinite)

In principle there is no difference, if you want to know more you will need to be more specific.

CB

 

[OhMyMarkov]

Hello CaptainBlack,

Let's start by two hypothesis of equally likely probability ("flat normal distribution"):

$H_0: X = \theta _0 + N$
$H_1: X = \theta _1 + N$

where N is a normal random variable (lets say of variance << $\frac{a+b}{2}$)

then the solution is $\operatorname{arg\, max}_m p(x|H_m)$.

But what if there were infinitely many hypothesis, i.e. $\theta$ is a real variable. How to estimate $\theta$?

 

[CaptainBlack]

Hello CaptainBlack,

Let's start by two hypothesis of equally likely probability ("flat normal distribution"):

$H_0: X = \theta _0 + N$
$H_1: X = \theta _1 + N$

where N is a normal random variable (lets say of variance << $\frac{a+b}{2}$)

then the solution is $\operatorname{arg\, max}_m p(x|H_m)$.

But what if there were infinitely many hypothesis, i.e. $\theta$ is a real variable. How to estimate $\theta$?

I see no difference between a finite and countably infinite number of hypotheses in principle. That is other than you cannot simply pick the required hypothesis out of a list of likelihoods, that is.

But you cannot have a completely disordered collection of hypotheses there must be some logic to their order, and so there will be some logic to the order of the likelihoods and it will be that logic that will allow you to find the hypothesis with the maximum likelihood.

CB

 

[blue_raver22]

In this case, we can use the Maximize A Posteriori (MAP) principle to select the most probable hypothesis. The MAP principle takes into account both the likelihood of the data and the prior probability of each hypothesis. This allows us to incorporate our prior knowledge or beliefs about the hypotheses into the decision-making process.

If there are infinitely many countable hypotheses, we can still apply the MAP principle by assigning a prior probability to each hypothesis. This can be done by using a probability distribution function that assigns probabilities to each hypothesis. Then, we can use the MAP principle to select the hypothesis with the highest posterior probability.

However, it is important to note that the choice of prior probability distribution can greatly influence the final outcome. Therefore, it is crucial to carefully consider and justify the choice of prior distribution in order to make an informed decision.

In summary, while the MAP principle can still be applied to select the most probable hypothesis in the case of infinitely many countable hypotheses, it is important to carefully consider the prior probabilities assigned to each hypothesis in order to make a robust decision.