Radon-Nikodym Derivative and Bayes' Theorem

In summary, Bayes' theorem states that if a random variable X has a parametric family of distributions with a parameter space, and if the probability of X given its parameter is absolutely continuous with respect to some measure, and if we have a prior distribution for the parameter, then the posterior distribution of the parameter given X is also absolutely continuous with respect to the prior distribution. The Radon-Nikodym derivative of the posterior with respect to the prior is the ratio of the conditional density of X given the parameter to the integral of the conditional density over the parameter space. This holds for all values of X except for those where the integral is zero or infinite.
  • #1
Jatex
1
0
Theorem 1.31 (Bayes' theorem):
Suppose that ##X## has a parametric family ##\mathcal{P}_0## of distributions with parameter space ##\Omega##.
Suppose that ##P_\theta \ll \nu## for all ##\theta \in \Omega##, and let ##f_{X\mid\Theta}(x\mid\theta)## be the conditional density (with respect to ##\nu##) of ##X## given ##\Theta = \theta##.
Let ##\mu_\Theta## be the prior distribution of ##\Theta##.
Let ##\mu_{\Theta\mid X}(\cdot \mid x)## denote the conditional distribution of ##\Theta## given ##X = x##.
Then ##\mu_{\Theta\mid X} \ll \mu_\Theta##, a.s. with respect to the marginal of ##X##, and the Radon-Nikodym derivative is
$$
\frac{d\mu_{\Theta\mid X}}{d\mu_\Theta}(\theta \mid x)
= \frac{f_{X\mid \Theta}(x\mid \theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, d\mu_\Theta(t)}
$$
for those ##x## such that the denominator is neither ##0## nor infinite.
The prior predictive probability of the set of ##x## values such that the denominator is ##0## or infinite is ##0##, hence the posterior can be defined arbitrarily for such ##x## values.

I tried to derive the right hand side of the Radon-Nikodym derivative above but I got different result, here is my attempt:

\begin{equation} \label{eq1}
\begin{split}
\frac{\mathrm d\mu_{\Theta\mid X}}{\mathrm d\mu_\Theta}(\theta \mid x) &= f_{\Theta\mid X}(\theta\mid x) \mathrm \space \space[1]\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{f_X(x)}\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \cdot f_{\Theta}(t) \space \mathrm dt}\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \mathrm d\mu_\Theta(t)}
\end{split}
\end{equation}

but now, where does ##f_{\Theta}(\theta)## go?

for ##[1]## see slide ##10## of the following document: http://mlg.eng.cam.ac.uk/mlss09/mlss_slides/Orbanz_1.pdf

Thanks in advance.
 
Last edited by a moderator:
Physics news on Phys.org
  • #2
Jatex said:
but now, where does ##f_{\Theta}(\theta)## go?

The ##f_{\Theta}(\theta)## is present in the expression for ##f_{\Theta|X}(t|x)## according to http://web.stanford.edu/class/stats200/Lecture20.pdf eq. 20.1.

Is the error in saying ##\frac{ d \mu_{\Theta|X}}{d \mu_\Theta} (t,x)= f_{\Theta|X}(t,x) ##? What is he definition of ##\frac{ d \mu_{\Theta|X}}{d \mu_\Theta}##?

Or is there a typo in the notes you quoted?
 

FAQ: Radon-Nikodym Derivative and Bayes' Theorem

What is the Radon-Nikodym Derivative?

The Radon-Nikodym Derivative, also known as the Radon-Nikodym derivative or the Radon-Nikodym derivative, is a mathematical concept used in measure theory and probability theory. It is used to describe the relationship between two probability measures, and it allows for the comparison of the measures on a common probability space.

How is the Radon-Nikodym Derivative used in probability theory?

In probability theory, the Radon-Nikodym Derivative is used to determine the likelihood of an event occurring, given a set of observed data. It is particularly useful in Bayesian statistics, where it is used to update prior beliefs about the probability of an event occurring based on new evidence.

What is Bayes' Theorem and how is it related to the Radon-Nikodym Derivative?

Bayes' Theorem is a mathematical formula that describes the relationship between conditional probabilities. It is used to calculate the probability of an event occurring, given certain evidence. The Radon-Nikodym Derivative is used in Bayes' Theorem to update the prior probability of an event based on new evidence.

What are some practical applications of the Radon-Nikodym Derivative and Bayes' Theorem?

The Radon-Nikodym Derivative and Bayes' Theorem have many practical applications in various fields, including statistics, economics, and machine learning. They are used in risk assessment, medical diagnosis, and predictive modeling, among others.

Are there any limitations to using the Radon-Nikodym Derivative and Bayes' Theorem?

While the Radon-Nikodym Derivative and Bayes' Theorem are powerful tools in probability theory, they do have some limitations. They assume that the underlying data is independent and identically distributed, which may not always be the case in real-world scenarios. Additionally, they require prior knowledge or assumptions, which can affect the accuracy of the results.

Similar threads

Back
Top