Equivalence Test: Finding Appropriate Statistical Test for Different Populations

[rbeale98]

What is an appropriate statistical test for equivalence of two population means? I'd like the null hypothesis to be that the populations are different.

The problem with the t-test is that the null hypothesis says that the populations are the same. It is more appropriate in my application that I assume the populations are different unless I can proove otherwise.

Example: treatment A1 is well understood and the distribution is well known. Treatment A2
is a newer and cheaper version, and we only accept that it is as good as treatment A1 if the null hypothesis is rejected. in other words it will be assumed that A2 is not as good as A1 unless there is enough data to show otherwise. Any thoughts on how to do this?

 

[statdad]

Your wording is a little confusing: are these the hypotheses in which you're interested?

$$\begin{align*} H_0 \colon &\mu_{A1} \ge \mu_{A2}\\ H_a \colon & \mu_{A1} < \mu_{A2} \end{align*}$$

I'm assuming larger means indicate better performance: the alternative here says that A2's mean is larger than that of A1.

If these are appropriate you can apply the two-sample t-test to your data (if the data sets are reasonably symmetric and free of outliers)

 

[rbeale98]

The goal of the t-test is prove a significant difference between 2 samples. It is not appropriate to prove equivalence.

The same paradigm is used in a court of law: You are innocent until proven guilty. If there is no evidence, you are considered "not guilty." The reason they don't call you "innocent," is because the system is not designed to prove innocense, it is assumed from the beginning (the null hypothesis).

The problem with the t-test is the same: if there is a lack of evidence, the statistician will fail to prove a significant difference and arrive at the misleading conclusion that the samples are equivalent. This method is great in clinical trials where a treatment is compared to a placebo. You need to keep gathering evidence until you have enough to prove the treatment has a different effect.

But what if the goal is to prove that two treatments are equivalent? Then you cannot use "equivalence" as the null hypothesis! If you don't have enough data to prove anything, you will always arrive at the null hypothesis.

I hope this better describes my dilemma.

 

[statdad]

I understand what a test of hypotheses is for; I didn't understand your question as originally posed.
You seem to be asking for an equivalence test procedure - the process I'll outline is one we discuss in our biostat course. It uses a sequence of two uses of the independent sample t test

As a setup for my explanation, suppose we want to determine whether a new drug is as effective a currently used drug (new drug may not have side effects that are as bad as the current one, or may be cheaper to produce, so if it is equivalent that is a point in its favor). We have decided that if it can be shown that the difference in mean responses for the two drugs is smaller than 4 units, the two drugs are ``equivalent''. The generic notation is that we want to test this ``null hypothesis''

$$H_{0E} \colon \mu_1 - \mu_2 \le -4 \text{ or } \mu_1 - \mu_2 \ge 4$$

versus the ``alternative hypothesis''
$$H_{aE} \colon -4 < \mu_1 - \mu_2 < 4$$

(the E is for Equivalence)

If the null hypothesis is rejected, then by our criterion, we have shown the two drugs are equivalent in effectiveness.

How is the test actually carried out? With a PAIR of t-tests. Perform both of these hypothesis tests.

$$\begin{align*} H_{01} \colon & \mu_1 - \mu_2 = 4\\ H_{a1} \colon & \mu_1 - \mu_2 < 4 \end{align*}$$

and

$$\begin{align*} H_{02} \colon & \mu_1 - \mu_2 = -4\\ H_{a2} \colon & \mu_1 - \mu_2 > -4 \end{align*}$$

If you reject both of these null hypotheses, you will have concluded that the mean difference is > -4 and < 4, which means that it is between - 4 and 4, which, according to our criterion, mean the two drugs are equivalent. (If only one null is rejected you cannot claim the drugs are equivalent.)

Does this sound like your type of problem?

 

[rbeale98]

Thanks a lot. I think this helps and will look at it more later.