When Should Multiple Comparisons Correction Be Applied in Clinical Trials?

[straycat]

I've been thinking about when to correct for multiple comparisons, especially in the context of clinical trials.

Consider the following situations:

1. Company X-1 does a clinical trial on Compound A-1, and finds that it does in fact have clinical efficacy, with p value < 0.05.

2. Company X-1 does a clinical trial on 20 compounds, A-1 through A-20, and finds that one of them has clinical efficacy with p value < 0.05, and 19 don't (without correction for multiple comparisons).

3. Twenty companies, X-1 through X-20, independently discover and do clinical trials on 20 different compounds (X-i tests A-i, i = 1,...,20). One of the compounds turns out to demonstrate efficacy with p value < 0.05, 19 don't. (without correction for multiple comparisons).

So here's my question: in which of the above scenarios should we correct for multiple comparisons? Or asked another way: does the evidence support the use of none, some, or all of the above three drugs?

David

http://en.wikipedia.org/wiki/Multiple_comparisons

 

[Valkarie]

_problemIn all three scenarios, it is recommended to correct for multiple comparisons, as this will allow you to more accurately determine which of the drugs have clinical efficacy. This can be done by using a Bonferroni correction or a False Discovery Rate (FDR) procedure. These methods help to reduce the risk of false positives and provide a more reliable result.

 

[tacman]

It is important to consult with a healthcare professional or a statistician for specific guidance on when to correct for multiple comparisons in clinical trials. However, here are some general considerations:

1. In scenario 1, where a single compound (A-1) is being tested, it is appropriate to use a p-value of < 0.05 as an indicator of significant clinical efficacy, without correction for multiple comparisons.

2. In scenario 2, where 20 compounds are being tested, it is important to correct for multiple comparisons to avoid false positives. This can be done using methods such as Bonferroni correction or false discovery rate control.

3. In scenario 3, where multiple companies are independently testing different compounds, it is also important to correct for multiple comparisons to avoid false positives. This can be done by considering the overall evidence from all 20 compounds, rather than just looking at individual p-values.

It is also worth considering the potential consequences of false positives and false negatives in clinical trials. In some cases, it may be more important to avoid false positives (i.e. claiming a compound is effective when it is not) than false negatives (i.e. missing a potentially effective compound). This should also be taken into account when deciding whether to correct for multiple comparisons.

Ultimately, the decision to correct for multiple comparisons should be based on the specific research question and the potential consequences of false results. It is important to carefully consider these factors and consult with experts before making a decision.