Statistical Errors, Type I and Type II

[FactChecker]

Is there any difference in assuming something is or isn't safe? If you compare it to a safe population the process is the same. in one case you look for evidence that it can be a member of the safe population and there is assumed to be safe and in the other you look for evidence that it is not a member of the safe population and therefore is assumed to be unsafe.

Good question. There is a big difference because you are giving the null hypothesis every advantage. You start by picking one hypothesis as the null hypothesis, giving it all the benefit of the doubt by using its distribution and parameters and saying that you will only change that assumption if there is strong test indications (over 95%, 99%, etc.) that it might be wrong.
In the case of testing the safety and effectiveness of a drug, they should assume that it is not safe or not effective and run tests that would convince even a skeptical audience that it is safe and effective. The burden of proof must be on the drug company to prove its drug is probably (95%, 99%, etc.) safe and effective. Otherwise, many unsafe and/or ineffective drugs would pass a minimal test and be approved for public use.

 

[gleem]

We are talking about a procedure, I don't think, that is used by drug companies that the endpoint of toxicity studies is death or estimates of death but instead some noticeable change in a physiological characteristic that could be detrimental if excessive like anemias, constipation, vomiting, reduced liver or kidney function. Typically they will start at a dose believed to have no untoward effects and increase the dose until side effects occur. They decide on what they think are acceptable side effects at an effective dose. If they do a comparison between equivalent populations of those who take the drug and those who don't at a reasonable confidence level and see no difference then what? There are side effects some serious but there is a benefit from taking the drug, Sometimes the risk can result in death. The patient must make a choice under guidance from their physician.

 

[FactChecker]

Good point. The actual decision process is complicated. My point is that, in the simplest terms, the drug company has the burden of proof to convince a skeptical audience that their drug has a net benefit. They can only do that if the original assumption, the null hypotheses, is that the drug is not beneficial and then show that the data results are strong enough to convince the skeptics otherwise.

On the other hand, if they start by assuming that the drug is beneficial and then set a very high standard (95%, 99%, etc.) to statistically indicate otherwise, they will not convinced anyone.