- #1
ZetaOfThree
Gold Member
- 110
- 23
Homework Statement
A 6 sided dice is loaded such that 6 occurs twice as often as 1. What is the total probability of rolling a 6 if the Shannon entropy is a maximum?
Homework Equations
Shannon Entropy:
$$S=-\sum_i p_i \ln{p_i}$$
where ##p_i## is the probability that we roll ##i##.
The Attempt at a Solution
We know that ##\sum_i p_i = 1 ## and we are given that ##p_1=p_6/2##. So $$p_2+p_3+p_4+p_5+\frac{3}{2} p_6 =1 \Rightarrow p_5 = 1-p_2-p_3-p_4-\frac{3}{2} p_6$$ There we can write the Shannon entropy as $$S=-\left( \frac{p_6}{2} \ln{\frac{p_6}{2}} + p_2 \ln{p_2}+p_3 \ln{p_3}+p_4 \ln{p_4}+(1-p_2-p_3-p_4-\frac{3}{2} p_6) \ln(1-p_2-p_3-p_4-\frac{3}{2} p_6) + p_6 \ln{p_6}\right)$$
$$=-\left( \frac{3 p_6}{2} \ln{p_6} + p_2 \ln{p_2}+p_3 \ln{p_3}+p_4 \ln{p_4}+(1-p_2-p_3-p_4-\frac{3}{2} p_6) \ln(1-p_2-p_3-p_4-\frac{3}{2} p_6) - \frac{p_6}{2} \ln{2}\right)$$
To find an extremum, we partial differentiate and set it equal to zero:
$$\frac{\partial S}{\partial p_2} = - \left(\ln{p_2} +1 - \ln(1-p_2-p_3-p_4-\frac{3}{2} p_6) -1 \right) = -\left(\ln{p_2} - \ln(1-p_2-p_3-p_4-\frac{3}{2} p_6) \right) =0$$
$$\Rightarrow \ln p_2 = \ln(1-p_2-p_3-p_4-\frac{3}{2} p_6)$$
Differentiating with respect to ##p_3## and ##p_4##, we find the same condition, so we conclude that ##p_2=p_3=p_4=p_5##. We now write the Shannon entropy as $$S=- \left(\frac{3 p_6}{2} \ln{p_6}+4p_2 \ln{p_2} - \frac{p_6}{2} \ln{2}\right)$$ So $$\frac{\partial S}{\partial p_6}= -\left(\frac{3}{2} \ln{p_6}+\frac{3}{2} - \frac{1}{2} \ln{2}\right) = 0$$
Therefore we find that ##p_6 = \frac{2^{1/3}}{e}##. But this is wrong, because if we plug in the probabilities into the Shannon entropy formula, we do not get a maximum. For example, we get a higher Shannon entropy if we plug in ##p_1=p_2=p_3=p_4=p_5=1/7## and ##p_6=2/7##. Where did I go wrong? Maybe I found a minimum or something? If so, how do I get the maximum?
Thanks for any help.