Fundamentals of Statistics

View

2. What Is Hypothesis?

2.2. What Is Hypothesis Testing?

Statistical hypothesis testing, also called confirmatory data analysis, is often used to decide whether experimental results contain enough information to cast doubt on established facts. Whenever we want to make claims about whether one set of results are different from another set of results, we must rely on statistical hypothesis tests.

A Hypothesis Test evaluates two mutually exclusive statements, the null hypothesis and the alternative hypothesis, about a population to determine which statement is best supported by data from a sample. Hypothesis tests typically examine a random sample from the population for which statements formulated in the hypotheses should be applicable (valid). The selected samples can range in how representative of the population they are. This is why hypothesis testing on samples can never verify (or disprove) a hypothesis with certainty (i.e. probability in decimal notation = 1) and can only say that a hypothesis has a certain probability to be true or false.

Taking the example from above, H1 stating that a new treatment ‘B’ for a disease is better than the existing standard of care treatment ‘A’, one might presume that scientists would set about proving this hypothesis, but that is not the case. Instead, and somewhat confusingly, this objective is approached indirectly. Rather than trying to prove the B hypothesis, scientific method assumes that in fact A is true – that there is no difference between the standard of care and the new treatment (observed changes are due to chance). The scientists then try to disprove A. This is also known as proving the null hypothesis false. If they can do this – prove that hypothesis A is false, – it follows that B is true, and that the new treatment is better than the standard of care treatment.

An attempt to get behind the reasoning underlying this unusual approach at this point may be to quote Albert Einstein:

“No amount of experimentation can ever prove me right; a single experiment can prove me wrong.”

This seems to suggest that trying to prove the null hypothesis false or wrong is a more rigorous, and achievable, objective than trying to prove the alternative hypothesis is right. Please note that this does NOT properly explain why science adopts this approach, but perhaps it can help us to comprehend and accept a tricky concept more easily.

Statistical tests against the likelihood of chance (or luck) being at the core of observed differences are used in determining what outcomes of a study would lead to a rejection of the null hypothesis and acceptance of the alternative hypothesis. In other words, the result of a hypothesis test is to either ‘reject the null hypothesis in favour of the alternative hypothesis’ with a certain statistically significant probability. As such, the outcome proves the alternative hypothesis. If you fail to reject the null hypothesis, you must conclude that you did not find an effect or difference in your study or that there is ‘not enough evidence to reject the null hypothesis’.

Distinguishing between the null hypothesis and the alternative hypothesis is done with the help of two conceptual types of errors (type I and type II).