Statistical Hypothesis Testing

[Agent Smith]

TL;DR Summary

Chi-Square Test

$H_0$: The probability of an obese person using chopsticks = the probability of a normal-weight person using chopsticks
$H_a$: The probability of an obese person using chopsticks $\ne$ the probability of a normal-weight person using chopsticks

"Partial" Chi-Square Test: I focused only on row 1 (Chopsticks) and computed the $\chi ^2 = \frac{(7 - 16.5)^2}{16.5} + \frac{(26 - 16.5)^2}{16.5} = 10.9$

Degree of Freedom DF = Number of components - 1 = 2 - 1 = 1
The associated P-value < 0.005.
Significance level, $\alpha = 0.05$
We can reject $H_0$

1. The probability of an obese person using chopsticks $\ne$ the probability of a normal-weight person using chopsticks (from our P-value to $\alpha$ comparison)
2. The probability of an obese person using chopsticks < the probability of a normal-weight person using chopsticks. From the fact that the observed probability of an obese person using chopsticks (7/100) < the observed probability of a normal-weight person using chopsticks (26/100)

Have I got it right? Are there any errors?

 

[Dale]

Yes, this is how it would be used. However, there is a bit of a correction. In statement 1 you are accepting the alternative hypothesis. This test does not do that, it only allows you to reject the null hypothesis. So you can only technically say something convoluted like "This data does not provide evidence that the probability is the same" (rejecting the null hypothesis).

In practice, however, what you say is exactly how most scientists would report this work in most journals.

Similarly, in statement 2 you are neither discussing the null hypothesis nor the alternative hypothesis. This is a 3rd hypothesis.

Again, in practice what you say is what scientists would say also, even though it is technically not allowed by the tests.

 

[Agent Smith]

@Dale

So for the second statement:
1. I need a population proportion $p$ of those who use chopsticks
2. I then need to take an adequate sample $n$ of obese people and compute the proportion $p_h$ that use chopsticks.
3. If we have the population standard deviation $\sigma$ well and good. Otherwise we can use the sample standard deviation $\sigma_s$
4. We then compute the z score: $\frac{p_h - p}{\frac{\sigma}{\sqrt n}}$ or $\frac{p_h - p}{\frac{\sigma_s}{\sqrt n}}$
5. Our hypotheses are: $H_0: p_h = p$ and $H_a: p_h < p$
6. We use our z score to compute the P-value
7. If our P-value < $\alpha$ (the significance level), we reject $H_0$ and accept $H_a$.

Correct?

 

[Dale]

In standard statistics you only reject $H_0$, you do not then accept $H_a$. So in step 7 you stop at "we reject $H_0$".

The issue is what the p-value itself means. The p-value is calculated assuming that $H_0$ is true. More formally it is $P(data|H_0)$. It is not calculated with any information from $H_a$. So while you can make the claim that the data is unlikely under $H_0$, you cannot make the claim that the data is likely under $H_a$, that is simply not something that is represented by the p-value. Especially in this case where $H_a$ and $H_0$ are not mutually exclusive and collectively exhaustive.

However, in practice actual scientists in actual scientific papers do not stop at "we reject $H_0$" and very often add "and accept $H_a$" as you did.

 

[Agent Smith]

What's the point to $H_a$ if we don't accept it?

 

[Dale]

It is supposed to represent the hypothesis that we are actually interested in, although it rarely does and it is usually trivial. Let me explain that last statement in detail.

First, $H_0$ is a point hypothesis. In this case we are hypothesizing that these two probabilities are exactly equal. That is a very strong claim. It would actually be amazing if they were exactly equal, that neither obesity nor anything that correlates with obesity, like nationality, affects the propensity to use chopsticks. So $H_0$, like most point hypotheses, is not really believable. They are only used because they are easy to compute.

The converse is that $H_a$, that it is literally any probability other than exactly equal, is trivial. No matter how close to equal you show that they are, no matter how tight the confidence interval you make, there are still infinitely more numbers within that range that satisfy $H_a$. The p value for a “everything other than a single point” hypothesis winds up being 1, regardless of the data. Accepting such a hypothesis doesn’t actually tell you anything interesting about your data or nature. It is a type of statement that cannot be disproven, so it is trivial in that sense.

So why do we do all of that? For many decades that was all we could do, computationally. And as shaky as it is, it is better than nothing. Experience has shown lots of instances where this approach has led to major advances in our understanding. And experience has also started to show us lots of instances where this approach has tricked us or allowed unscrupulous actors to mislead. But due to the numerous successes, it has become the cornerstone of scientific testing for many topics.