What Is the Counterintuitive Probability in the Bag of Balls Problem?

[Thecla]

TL;DR Summary

Probability of drawing white/black balls from a bag

In Aubrey Clayton's book" Bernoulli's Fallacy" which documents the conflict between frequentists and Bayesian interpretations of probability, he describes a problem that was proposed in the 19th century that gives a counterintuitive result.
The Problem:
"Infer the state of a bag of 3 balls each either black or white given that 2 successive white balls being drawn without replacement".
My first guess is the bag had 3 white balls or 2 white and one black ball,each equally possible. He says the answer is counterintuitive. The answer is not given, but what is the answer?
Dr. Clayton compares this problem to the problem first proposed by Martin Gardner in Scientific American(1959):
Mr. Smith says :I have 2 children and at least one of them is a boy. What is the probability that the other child is a boy?

 

[Halc]

With the ball case, each ball color is independent, and thus the probability of the final ball is the same as any single ball, 50/50.
But the child problem is worded differently. There's only a third chance that the other child is a boy since only one out of three sets of two children with at least one boy are both boys.

Had it been worded equivalently, I have three children, the older two being boys. What's the chance that the youngest is a boy? That one is 50/50.

I don't see where any of these results is counterintuitive, but you get into cases like the sleeping beauty problem and the intuitions can get in the way.

 

[Dale]

I agree. The wording for the bag is not equivalent to the wording for the children.

Perhaps the Bayesian is treating the probability of some color as an unknown. I would probably treat it that way. After all, bags of marbles are not like coins. You don’t expect “fair” distributions of colors.

 

[PeroK]

TL;DR Summary: Probability of drawing white/black balls from a bag

In Aubrey Clayton's book" Bernoulli's Fallacy" which documents the conflict between frequentists and Bayesian interpretations of probability, he describes a problem that was proposed in the 19th century that gives a counterintuitive result.
The Problem:
"Infer the state of a bag of 3 balls each either black or white given that 2 successive white balls being drawn without replacement".
My first guess is the bag had 3 white balls or 2 white and one black ball,each equally possible. He says the answer is counterintuitive. The answer is not given, but what is the answer?
Dr. Clayton compares this problem to the problem first proposed by Martin Gardner in Scientific American(1959):
Mr. Smith says :I have 2 children and at least one of them is a boy. What is the probability that the other child is a boy?

Neither problem is well defined, IMO. How do you know the bag has three balls? How do you know the balls are black or white?

Why did Mr Smith say what he said? This leads to the problem of dealing with unsolicited information. The second problem needs to be worded so that Mr Smith is asked direct questions. The simplest formulation is:

Qu 1. How many children do you have? Ans: 2.
Qu 2. Are they both girls? Ans: No.

Then you can proceed.

Notice also that saying at least one child is a boy doesn't identify an individual child. It's equivalent to saying it's not two girls. So, the "other" boy is not well defined until you have identified one.

Moreover, and this is critical: if Mr Smith randomly thinks of one of his children and says it's a boy, then that biases Mr Smith towards having two boys. This makes the probability of the other child 50-50.

Even people like Aubrey Clayton fall into this trap, it seems.

 

[FactChecker]

My first guess is the bag had 3 white balls or 2 white and one black ball,each equally possible.

To say that, you are assuming that the probabilities of the unknown ball being black or white are equal. What if there were 1,000 balls and you drew 999 white balls. Wouldn't you seriously doubt the assumption of equal probabilities? To some extent, that can also change your assumption in your three ball example.
Given no information regarding the initial probabilities, I suspect that if you wanted to maximize the conditional probability, you would have to assume that all the balls were white. But I am not an expert in this subject.

 

[Dale]

So, rather than assume that the probability of each ball being black or white is 0.5, I would do a Bayesian analysis as follows. Since there are 3 balls the probability could be 0, 1/3, 2/3, or 1. Call those models 0, 1, 2, and 3 respectively. Then we can start with a uniform prior over the models (each model prior probability is 1/4). Then each time we draw a ball we update the probability of each model by Baye's theorem $$P(M_i|E)=\frac{P(E|M_i) \ P(M_i)}{\Sigma_j \ P(E|M_j) \ P(M_j)}$$ So we start with our model priors $$P(M_0)=0.250,\ P(M_1)=0.250,\ P(M_2)=0.250,\ P(M_3)=0.250$$ After observing the first white ball we update our probabilities to $$P(M_0|E_1)=0.000,\ P(M_1|E_1)=0.167,\ P(M_2|E_1)=0.333,\ P(M_3|E_1)=0.500$$ After observing the second white ball we update our probabilities to $$P(M_0|E_2)=0.000,\ P(M_1|E_2)=0.071,\ P(M_2|E_2)=0.286,\ P(M_3|E_2)=0.643$$ Finally, to get the probability of white for the last ball we can marginalize over the models to get $$P(W|E_2)=0.857$$

This is probably a little wrong because of sampling without replacement, but I don't know how to model that. It should at least be in the ballpark.

 

[martinbn]

The Problem:
"Infer the state of a bag of 3 balls each either black or white given that 2 successive white balls being drawn without replacement".
My first guess is the bag had 3 white balls or 2 white and one black ball,each equally possible. He says the answer is counterintuitive. The answer is not given, but what is the answer?

Intuitively the three white balls seems more likely. Because to get two white from three is 100%, while to pull two white from a mixed bag is less likely. So given that you've pulled two white ones it should be more likely that you had three white ones in the bag.

 

[Office_Shredder]

Intuitively the three white balls seems more likely. Because to get two white from three is 100%, while to pull two white from a mixed bag is less likely. So given that you've pulled two white ones it should be more likely that you had three white ones in the bag.

If each ball is independently 50/50 to be white or black, then the last ball is also 50/50 here. You can see this by either using the thing I just said (they're all independently 50/50, you just learned the first two random coin flip outcomes) or by doing bayes rule on the distribution of bag outcomes (2W 1B is 3 times as likely as 3W, but 1/3 as likely to result in 2W drawn immediately, so the two end up equally likely)

 

[PeroK]

This is why we really need explicit assumptions. Just saying there are three balls in the bag and each may be black or white is nor enough.

 

[Dale]

If each ball is independently 50/50 to be white or black

We are not given that.

 

[martinbn]

If each ball is independently 50/50 to be white or black, then the last ball is also 50/50 here. You can see this by either using the thing I just said (they're all independently 50/50, you just learned the first two random coin flip outcomes) or by doing bayes rule on the distribution of bag outcomes (2W 1B is 3 times as likely as 3W, but 1/3 as likely to result in 2W drawn immediately, so the two end up equally likely)

Depends how you read the question. I think that most probability questions/puzzles need to state everything explicitly.

 

[Dale]

This is probably a little wrong because of sampling without replacement, but I don't know how to model that.

My model is definitely wrong for sampling without replacement. $P(M_1|E_2)$ should be $0$ instead of $0.071$. I am not sure how to model that correctly

 

[Office_Shredder]

Depends how you read the question. I think that most probability questions/puzzles need to state everything explicitly.

Sure I could read it as the balls are black or white, which is vacuous unless there's at least one of each, so 100% the next ball is black. But that doesn't forget our knowledge of math

 

[Thecla]

Let me quote from the paragraph on page 167 of " Bernoulli's Fallacy" by Aubrey Clayton which mentions this problem:

" In a little known paper in " Transactions of the Actuarial Society of Edinburgh(1891)",though,Chrystal explained that his objections were to Bayes' theorem itself because he thought he found an example where it gave an illogical answer. The problem was to infer the state of a bag of balls, each either black or white, given two successive white balls being drawn without replacement.. Basically he got stumped by a probability brainteaser of the same caliber as the Boy or Girl paradox we saw in chapter 1[mentioned in an earlier post]. He computed the right answer using Bayes theorem,but then considered it illogical because it was counterintuitive."

 

[Agent Smith]

Using Bayes' $\text{P(3rd ball is white)} = \frac{3}{4}$

 

[Hornbein]

To say that, you are assuming that the probabilities of the unknown ball being black or white are equal. What if there were 1,000 balls and you drew 999 white balls. Wouldn't you seriously doubt the assumption of equal probabilities? To some extent, that can also change your assumption in your three ball example.
Given no information regarding the initial probabilities, I suspect that if you wanted to maximize the conditional probability, you would have to assume that all the balls were white. But I am not an expert in this subject.

The way I would do it would be to use the empirical probability so far. With the three balls I'd say the probability that the next ball is white is 2/3. I would however have little confidence in this estimate. There is too little to go on. But it is the best I can do with this skimpy sample and absence of knowledge of the situation. The frequentist approach doesn't make much sense here, since there is no indication that this is a repeatable experiment even in theory. For all I know someone would deliberately try to mess me up based on my guesses.

With the 999 balls I'd say the probability is .999 and feel much more confident, assuming the sampling is honest.

 

[Hornbein]

Using Bayes' $\text{P(3rd ball is white)} = \frac{3}{4}$

I suppose you are right. Bayes is the way to go. I have learned something here.

I never liked pulling priors out of thin air but in this case it is the best one can do.

 

[Agent Smith]

I suppose you are right. Bayes is the way to go. I have learned something here.

I never liked pulling priors out of thin air but in this case it is the best one can do.

It's non liquet, but is it true that $0.5$ is linked to "maximum entropy"?

Desmos

 

[Hornbein]

It's non liquet, but is it true that $0.5$ is linked to "maximum entropy"?

Desmos

Yes that is the case. I don't really understand it, but it is clear that 0 and 1 are minimal entropy and it increases as we move away from them. The situation is symmetric so 0.5 has got to be maximal. It is the value in which every permutation has equal chance.