3. Clinical effectiveness and evidence generation for HTA


Clinical effectiveness and evidence generation for HTA

The Medical Device Regulation (MDR) and the In Vitro Diagnostics Medical Devices Regulation (IVDR) require manufacturers to perform clinical studies to demonstrate the clinical effectiveness for certain classes of Medical Devices and IVDs.

Generating evidence on the clinical effectiveness of Medical Devices and IVDs also goes beyond evidence methods. In this section the term 'medical technologies' is used for both Medical Devices and IVDs.

Randomised Clinical Trials (RCT) have not been conducted for most medical technologies mainly for the following reasons:

1) There is no 'steady state period' for medical technologies, because they are in constant development/iteration and new versions are implemented continuously. Asking for RCTs to create evidence on clinical effectiveness might lead to prolong the period until the technologies become available for patients.

2) The effect of human factors on the clinical effectiveness of medical technologies is difficult to evaluate before the technology has been used for a longer period. Measuring the effect before the technology is in use risks not capturing the true effect because users' different learning curves may interfere with the results.

The EMA provides no definition of 'human factors' but requires "Usability (human factor) Studies" for drug device combinations (DDCs) [1] if the user population is new to the technology or if the setting of use is new. The purpose of these studies is: "to evaluate whether the DDC can be used safely to deliver the medicinal product to the target population… (or at least, (ed.)) a formal usability study is required to demonstrate usability of the medicinal product by the intended population." By focusing specifically on safety and usability of the product in question there is a risk of overlooking several associated human factors.

HTA bodies need to balance the demands for early assessment with the availability of data on the use of new Medical Devices. This is particularly important when considering devices associated with a ‘learning curve’ effect whereby their effectiveness can only be properly evaluated once users of the technology have adjusted their practice to incorporate the new device [2].   

How users learn to use and interact with a device and how fast they get acquainted with the device is influenced by several factors e.g., prior experiences, how well the user is trained, the setting where the device is used as well as the intention of use for the individual user.

So far, measures and methods for the human factor and learning curve implications are not (yet) standard in RCTs, and therefore any result will be difficult to place in a standard hierarchy.

FDA defines human factors for Medical Devices as the 'interactions between a user and a device, the processes performed by each, and the user interface between them' [3].

The FDA highlights the below interactions for human factor studies:
To understand the user-device system, it's important to understand the ways that people:

  • Perceive information from the device.
  • Interpret the information and make decisions about what to do.
  • Manipulate the device, its components, and/or its controls (e.g., modify a setting, replace a component, or stop the device).

It's also important to understand the ways in which devices:

  • Receive input from the user.
  • Respond and provide feedback to the user about the effects of their actions.

3) Blinding Medical Devices in RCTs is almost impossible. The operator (patient, caregiver, surgeon, cardiologist, radiologist) cannot be blinded to the type of device that is tested or even implanted, and no 'placebo' exists except for devices that can be switched on and off remotely. In addition, “sham operations” are never ethical because the patient experiences the risk of the intervention but receives no benefit.


[1] EMA. Draft Guideline on the quality requirements for drug-device combinations. Section 5.4. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-guideline-quality-requirements-drug-device-combinations-first-version_en.pdf

[2] Ramsey et al, International Journal of Technology Assessment in Healthcare, 16:4, 2000.

[3] FDA. Human Factors and Medical Deviceshttps://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/human-factors-and-medical-devices