5. Patient involvement in different phases of HTA

Providing evidence on patient aspects: Interpretation of effect size

Note: For more details about Statistics please see lesson Statistics in the Clinical Development module.

When assessing effect, patients should pay careful attention to how effect size is presented and statistical method in general. It is important information, as robustness of the statistical outcome is directly related to the statistical methods used as well as number of patients included in the study. Below sections provide a short overview of some of the commonly used ways of presenting effect and examples to illustrate what to pay attention to for non-statisticians.

Firstly, effect size measurements depend to a large extent on the endpoints designated in a study, for instance as difference in mean blood pressure (continuous endpoint) or difference in mortality (categorical endpoint). Secondly, effect size can be expressed in many different ways, depending on the focus of a study. For instance, the same quantitative effect can be presented , as relative risk reduction (RRR), absolute risk reduction (ARR), odds ratio (OR) or number needed to treat (NNT). The example in the table below shows the same effect as an odds ratio of 0.45, a relative risk reduction of 54%, an absolute risk reduction of 1.6% and number needed to treat of 62.

Calculation of different endpoints

Data from a randomized study

Calculations based on data from above randomized study

The same quantitative effect is expressed as an odds-ratio of 0.45, a relative risk reduction of 54%, an absolute risk reduction of 1.6% and number needed to treat of 62.

It is very important for the perception of the effect size whether ARR or RRR is used. Take as an example, a study in which 1% of the patients in the placebo group and 0.6% of the patients in the intervention group died. ARR for death is 0.4% (1%-0.6%), whereas RRR for death is as much as 40% (100x(1%-0.6%)/1%). Since RRR is a larger number, RRR is naturally often used in abstracts and marketing material.

The example shows how a moderate absolute reduction resulting in a moderate death rate of 0.4% can be presented as a fairly large relative risk reduction of 40%. Both values are correct and both should be stated instead of just the one. This is because the way of presenting an effect might lead to an erroneous interpretation of a result and should be avoided.

It is important to note that just because an effect in a study is statistically significant this does not mean that it is of a clinically relevant. Consequently, it is not sufficient to state that a statistically significant effect has been found, or state the p-value. The effect size must be stated in relation to its clinical relevance. To ensure the best possible transparency the experimental and control event rates as well as ARR should be stated. The following article offers an example illustrating that the minimum clinically important difference varies in different studies depending on a number of factors: Pain relief that matters to patients: systematic review of empirical studies assessing the minimum clinically important difference in acute pain [10] (https://pubmed.ncbi.nlm.nih.gov/28215182/).