Hypothesis Testing - Basic Theory Question

[quarklet]

Homework Statement

Karen asks John to guess the date of her birth, not including the year (i.e. the day and month she was born; discount leap years). If John guessed correctly on his first try, would you believe his claim that he made a lucky guess or would you be suspicious that he already knew your birth date? Explain why, relating to chance and hypothesis testing.

2. The attempt at a solution

The probability of John guessing correctly on his first try is 1 correct day/365 possible days, or 0.0027.

I chose to set the alpha level at 0.05, because none is given in the question.

I know the probability of a correct guess due to chance is less than the alpha level (.0027 < 0.05).

Here is where I get confused. Would my alternative hypothesis be that John is psychic, or a lucky guesser? i.e. What exactly am I proving by the fact that "p-value < alpha?" I feel like this should seem obvious, but I am really frustrated.

Please help guide me in the right direction.

 

[clamtrox]

A word of warning: this might just confuse you more.

So hypothesis A says that John guessed the date and hypothesis B says he knew it. Let's denote the conditional probability that John guessed the date, given that he answered it correctly by P(A|C). This is given by Bayes' theorem,

$$P(A|C) = \frac{1}{N} P(C|A) P(A)$$

where P(A|C) is the conditional probability that A is true, given that you know C (that he guessed right) and P(C|A) is the reversed conditional probability: if you know that John doesn't know the answer, what are the odds of guessing. P(A) is the prior probability you give to the hypothesis and N is a normalization constant that does not depend on the hypothesis.

Now, P(C|A) is just 1/365, and P(A) is probability you assign to the hypothesis. If they are complementary (as in this case) you'd have P(A) + P(B) = 1, etc.

Then take hypothesis B: now you have P(C|B) = 1, so P(B|C) = [1-P(A)]/N. You can compare the two cases easily

$$\frac{P(A|C)}{P(B|C)} = \frac{P(A)}{365 (1-P(A))}.$$

You see that the answer depends on whether or not you think John might have known the answer. If you think that it's completely impossible that John could have known it, you'd assign P(A) = 1 and then you'd have P(B|C) = 0 (since P(B) = 0). If you think there's a one in million chance of John being a psychic, you'd plug in P(A) = 0.999999, and so forth.

Now, in this case you obviously don't have any knowledge on P(A). In this case, the usual assumption is just that the hypotheses are equally likely, P(A) = P(B) = 1/2. In that case, you'd end up with P(A|C) = 0.0027 P(B|C), which is clearly very significant no matter what alpha you choose.

 

[Architeuthis Dux]

I would approach this situation by conducting a hypothesis test to determine the likelihood of John guessing Karen's birth date correctly on his first try. The null hypothesis would be that John's guess was purely due to chance, and the alternative hypothesis would be that John already knew Karen's birth date.

To conduct the test, I would first calculate the probability of a correct guess due to chance (0.0027) and compare it to the alpha level (0.05). If the p-value (probability value) is less than the alpha level, it would suggest that the null hypothesis is not true and that there is a low probability that John's guess was due to chance. In this case, I would be suspicious that John already knew Karen's birth date.

However, it is important to note that this does not prove that John actually knew Karen's birth date. It simply suggests that the likelihood of his guess being due to chance is low. Additional evidence would be needed to support the alternative hypothesis.

In conclusion, as a scientist, I would not automatically believe John's claim that he made a lucky guess. The results of the hypothesis test would suggest that there is a low probability of this being true, and further investigation would be needed to determine the true reason behind John's correct guess.