Bias in Clinical Trials

1. If the participant eligibility criteria used in a trial are so stringent that only a small percentage of the intended patient population qualifies. For example, those with better physical condition who would be expected to be able to manage toxic treatments.
   
   
2. At treatment assignment, the eligibility criteria used to recruit and enrol participants into the treatment arms may be applied differently. This may result in some characteristics being over-expressed in one arm compared to the other. For example, if participants were selected differently according to their age or health status. The treatment outcomes may be stronger in the arm where the participants are younger and in a better shape. Therefore, any difference in outcome between the two treatment arms can no longer be attributed only to the received treatment.
   
   
3. The use of random participant allocation techniques to two or more treatment arms is key to avoiding this type of bias. Randomisation aims to ensure that the treatment arms are comparable both in terms of known and unknown prognostic factors especially over a large number of participants. Well-performed randomisation will allow the researcher to consider the observed treatment effects (response rate, survival, etc.) to be related to the treatment itself and not due to other factors. The treatment allocation should be independent from researchers’ beliefs and expectations. The optimal way is by using a central randomisation algorithm to distribute characteristics across the treatment arms.

• Participants may turn out to be ineligible after randomisation.
  
  
• Treatment may have been interrupted or modified but not according to the rules specified in the protocol.
  
  
• Disease assessments may have been delayed or not performed at all.
  
  
• A participant may decide to stop taking part in the trial, etc.
  
  This can be problematic if not properly accounted for in the analysis. For example, consider the setting of a clinical trial comparing a new experimental treatment to the standard of care. In this trial some participants taking the experimental treatment are too sick to go to the next visit within the allotted time due to side effects. A possible approach would be to include only participants with complete follow-up, so to exclude the sick participants from the analysis. However, by doing so, one selects a sub-group of patients whom, by definition, will present an artificially positive picture of the treatment under trial. This is again an example of selection bias.
  
  One potential solution to this problem is a statistical concept called an intention-to-treat (ITT) analysis. ITT analysis includes every randomised patient whether they received the treatment or not. As such, ITT analyses maintain the balance of patients' baseline characteristics between the different trial arms obtained from the randomisation. ‘Protocol deviations’ such as non-compliance to the assigned treatment (schedule, dosing, etc.) are part of daily practice. Therefore, treatment-effect estimates obtained from ITT analysis are considered to be more representative of the actual benefit of a new treatment

• Bias can also happen when measuring the endpoint(s) of interest. For instance, time to disease progression or relapse can be severely affected by the hospital visit schedule. The frequency of disease assessment should be adequate and frequent enough to capture correctly the events that could not be observed by means other than by medical examination at the hospital. The disease assessment schedules should also be the same in both arms.
  

Figure 3: Example of sample distribution by arm and sex.

• Demographic characteristics (e.g. sex, age).
  
  
• Baseline characteristics (e.g. a specific genomic profile).
  
  
• Use of concomitant therapy.

• Firstly, sub-group analyses are observational (sub-groups are defined on observed participants’ characteristics) and not based on randomised comparisons. The hindsight bias, also known as the ‘I-knew-it-all-along’-bias, is the inclination to see events that have already occurred as being more predictable than they were before they took place.
  
  
• Secondly, when multiple sub-group analyses are performed, the risk of finding a false positive result (i.e. a type I error) increases with the number of sub-group comparisons. Multiplicity issues are in general related to repeated looks at the same data set but in different ways until something ‘statistically significant’ emerges. With the wealth of data sometimes obtained, all signals should be considered carefully. Researchers must be cautious about possible over-interpretation. Techniques exist to protect against multiplicity, but they mostly require stronger evidence for statistical significance to control the overall type I error of the analysis (e.g. the Holm–Bonferroni method and the Hochberg procedure).
  
  
• Thirdly, there is a tendency to conduct analyses comparing sub-groups based on information collected while on trial. A typical example is looking at the difference in survival between patients responding (yes/no) to treatment. Participants who are responding to treatment are by definition patients who are able to spend sufficient time on treatment to allow a response. Therefore, again by definition, they may simply represent a sub-group of patients of better prognosis, and may therefore bias the analysis. This is an example of what is often referred to as lead-time bias or guarantee-time bias. One way of dealing with this is using a landmark as a starting point for the time-to-event analysis, and creating the categories based on the participants’ characteristics at the time of this landmark (e.g. did a participant respond at three months, yes/no).

Publication bias