- #71
PeterDonis
Mentor
- 47,480
- 23,758
entropy1 said:P(X=1) in the binary case is the ratio of the #bits equal to 1 relative to the total #samples.
P(A=1,B=1) is the ratio of pairs of bits that are both 1 compared to the total #samples (pairs).
Ok. But "pairs of bits" here means (or should mean--if you are defining it differently, you are doing it wrong) "pairs of bits measured in the same run of the experiment". So these probabilities, to be meaningful, require a certain way of "lining up" the two bit sequences next to each other: bits 0 and 0, bits 1 and 1, bits 2 and 2, etc., of each sample. Otherwise you are making meaningless comparisons; there is no physical meaning to comparing bit 0 from one sample and bit 1 of the other, because they are from different runs of the experiment.
Similarly, if you pick only the "1" bits out of each sample and match them up with each other, you are making a meaningless comparison.
entropy1 said:I ment it as example of a non-random cause.
How you meant it doesn't change the fact that it's meaningless. See above.
entropy1 said:You can claim that the entropy of the random content decreases in fractional bits
I have said no such thing. You are confused.
What I said was that if you have a pair of bits that are correlated, the entropy of that pair of bits, as a system, will be less than the entropy of two random uncorrelated bits; and if the correlation is only partial, the entropy of the two-bit system will not be an integral number of bits. But that doesn't mean we made the correlated pair of bits out of the two random uncorrelated bits and thereby decreased their entropy.