- #1
NATURE.M
- 301
- 0
Homework Statement
We have a Markov Random Field with the log likelihood as such:
$$ l(\theta) = \sum\limits_{i=1}^L \log p(x^{(i)}|\theta) = \sum\limits_{i=1}^L \left( \sum\limits_{s \in V} \theta_{s} x_{s}^{(i)} - \log \sum\limits_{x} \exp \left\lbrace \sum\limits_{s \in V} \theta_{s} x_{s} \right\rbrace \right) $$
Note L is the number of data examples.
Also each x(i) is a vector where each component xs(i) is a binary variable taking on a value 0 or 1.
The set V denotes the set of vertices/nodes in the Markov Random Field. The nodes are the components xs of the vector x. Each node or xs has a parameter denoted θs.
And the sum over x, is the sum over every possible arrangement of the values in the vector x.
I then take the derivative with respect to θs to determine the ML estimates.
\begin{align*}
\frac{\partial l(\theta)}{\partial \theta_{s}} &= \sum\limits_{i=1}^L \left( x_{s}^{(i)} - \frac{\partial}{\partial \theta_{s}} \log \sum\limits_{x} \exp \left\lbrace \sum\limits_{s \in V} \theta_{s} x_{s} \right\rbrace \right)
\\ &= \sum\limits_{i=1}^L x_{s}^{i} - \dfrac{L \sum\limits_{x} \exp \left( \sum\limits_{s \in V} \theta_{s} x_{s} \right)x_{s}}{\sum\limits_{x} \exp \left( \sum\limits_{s \in V} \theta_{s} x_{s} \right)}
\end{align*}
At this point I'm not really sure how to approach the problem. I'm having difficulty directly obtaining a closed form for θs. It has been suggested to consider cases when xs = 0 and 1 separately. But I don't understand what that entails.