ABHI Guest Blogs with Matthew Taylor and Michelle Jenks
Clinical evidence for medical technologies: beyond randomised control trials
In order to achieve adoption of medical technologies within the NHS, clinical evidence reporting on the efficacy of innovations is essential both to assess clinical benefit of the innovation and, if demonstrated, to provide data for use within economic analyses. Economic analyses (or cost-effectiveness models) consider many factors, including the costs of introducing the new innovation, ‘downstream’ cost implications of the technology (which may be positive or negative) and a quantification of the health benefits arising from the technology’s use. These outcomes are compared with those that would have been achieved without the technology (i.e. with current standard practice). Given that the interventions funded by the NHS are highly varied and its budget fixed, spending additional money to fund innovation adoption in one disease area could mean that funding must be cut from another. The funding that will be foregone elsewhere is known as “opportunity cost”. Conducting cost-effectiveness analyses allows decision makers to come to judgements around whether the value of the new innovation outweighs the opportunity cost. It is imperative that such decisions are based on sound clinical evidence.
Randomised control trials (RCTs), in which participants and clinicians are blinded to the intervention being received, are considered the gold standard for the generation of clinical evidence. However, there are a number of difficulties in such studies for MedTech [1] [2]. First, RCTs are expensive to conduct, but MedTech typically has a short life span with imitator devices entering the market frequently. In theory, devices can be adopted rapidly with no formal requirement for clinical evidence once a product has a CE mark. As a result, the period of revenue for an innovative device can be relatively brief and with small commercial reward to compensate for research and development costs. Second, innovative MedTech is typically developed by small-medium sized enterprises (SMEs) who do not have budgets to conduct sufficiently large RCTs. This problem is exacerbated by frequent incremental updates to MedTech meaning that such trials would need to be conducted each time there are innovations to a device. Third, it is very difficult to blind patients and clinicians to the MedTech being used, and using placebo devices (for example implantable placebo devices) may be considered unethical.
In a review by the Nuffield Trust, it was noted that “there needs to be a willingness to look at different types of evidence and an understanding of how to interpret evidence that hasn't come from RCTs”. The Medical Technologies Evaluation Programme (MTEP) at the National Institute for Health and Care Excellence (NICE) is doing this. However, the programme only considers a small proportion of the innovative technologies attempting to achieve adoption within the NHS. MTEP adapted its methods such that a lower threshold of evidence for MedTech may be accepted than would be the case for pharmaceuticals. Indeed, there have been cases in which innovative MedTech has been recommended based upon comparative observational studies or single arm case series. There are some benefits to this generally lower quality evidence in that it is less protocol driven; included patients are often more generalisable to those being treated within the NHS and outcomes such as resource use are based on actual usage, rather than trial based environments.
The MTEP is set up such that MedTech with an evidence based deemed too uncertain at present is recommended for use in research, with MTEP facilitating the prioritisation of research questions and the undertaking of future research to reduce any uncertainty in the evidence as quickly as possible. When making decisions, the programme recognises the potential bias in non-RCT evidence in estimating the innovation benefits and the uncertainty that this brings. Typically, within cost-effectiveness modelling, ‘threshold analysis’ will be undertaken to determine what benefit innovative MedTech must bring in order to be deemed cost-effective, and decision makers will utilise the existing (often non-RCT) evidence base to determine whether or not this seems feasible. Decision makers are then able to weigh up the uncertainty in their decision and determine whether collecting more information would be worthwhile. Where the value of collecting further information is not warranted, MedTech may be recommended for use earlier. Such an approach may be followed by decision makers across the NHS in order to increase uptake of innovative MedTech.
Matthew Taylor is the Director of York Health Economics Consortium (YHEC) and Michelle Jenks is a Project Director at the company. YHEC's purpose is to provide consultancy and research in health economics to the NHS and the pharmaceutical and health care industries, which combines rigorous research standards with efficient project delivery.
[1] Craig et al. A Review of the Economic Tools for Assessing New Medical Devices. Applied Health Economics and Health Policy. 2015. 13 (1): 15-27
[2] Campbell et al. A New Health Technology Assessment System for Devices: The First Five Years. Int J Technol Assess Health Care. 2017. 33 (1): 19-24