Intended for healthcare professionals

Analysis

Trials are needed before new devices are used in routine practice in Europe

BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f1646 (Published 18 March 2013) Cite this as: BMJ 2013;346:f1646
  1. Philipp Storz-Pfennig, consultant,
  2. Mechtild Schmedders, consultant,
  3. Matthias Dettloff, consultant
  1. 1GKV-Spitzenverband – Medicine, Mittelstrasse 51, Berlin D-10117, Germany
  1. Correspondence to: P Storz-Pfennig Philipp.Storz{at}gkv-spitzenverband.de
  • Accepted 8 March 2013

As the EU debates new legislation to regulate medical devices, Philipp Storz-Pfennig, Mechtild Schmedders, and Matthias Dettloff provide examples from Germany to show why the current proposals do not go far enough and call for further assessment after market approval

A growing number of examples, including metal on metal hip implants and breast implants, show the harm that can result from new devices and procedures being introduced without a rigorous assessment of their safety and efficacy.1 2 3 4 5 6 7 This has led to the wide acceptance that the system for regulating medical devices, particularly in the European Union (EU) is flawed. The European parliament is currently debating proposals put forward last year by the European Commission to reform the EU legislation for medical devices and in vitro diagnostics.8 9

However, the new proposals will not change market access to new devices appreciably.10 Notified bodies will continue to grant market approval (through CE certificates), although quality assurance will be stepped up—the European Commission will have to set up a medical devices coordination group to supervise notified bodies and advise on their assessments of new high risk devices and diagnostic tests, and there will be harmonised criteria for accreditation. After a device is approved, there will also be stronger postmarketing surveillance. The proposals include the introduction of a “unique device identifier” and unheralded inspections of device manufacturers by notified bodies.

The proposals fall short of what is required to prevent high risk devices being used without reliable evidence on their safety and efficacy—as is required for new drugs.11 And manufacturers will still be free to define and amend the intended purpose of their devices without having to get approval from any authority. EU member states will therefore still have to make decisions about using and funding new technologies without sufficient evidence to steer their decisions. The German experience illustrates the problems that this situation poses, not least for patients..

Problems with Germany’s coverage system

Most European health systems, including Germany’s system, make decisions to adopt new technologies on the basis of health technology assessments.12 However, it seems to us that Germany’s system is more permissive than that in most other countries. For devices used in hospitals, reimbursement may be established in Germany without any assessment of safety and effectiveness.

The aim of German legislation is to guarantee a quick transfer of innovative technologies into hospital practice. In German hospitals, clinicians can use new devices bearing a CE mark for the indications specified unless the German Federal Joint Committee, which is responsible for assessing medical treatments, has expressly ruled out their use. Hospitals are therefore able to use new treatments before and during any assessment.

For outpatient care, novel methods are covered by the statutory health insurance system only after the committee has explicitly sanctioned their use. Stakeholders within the federal joint committee, which is made up of representatives from hospitals, office based physicians, health insurers, and patient representatives, may apply for technologies used in outpatient or hospital sector to be assessed. Health insurers apply for hospital treatments to be assessed if they have safety concerns, while representatives of the office based physicians or the health insurers usually apply for assessments of outpatient methods. The committee then starts an assessment of whether the method is effective, reasonable, and necessary, and asks for data from trials with “patient relevant” endpoints and a high level of evidence. Additionally, if such data are available, the committee factors in other evidence including “real world” observation of effectiveness, patient need, and costs.12 However, coverage decisions will be difficult if even basic safety and efficacy data are lacking, as is the case for medical devices.

Health professionals, providers, manufacturers, and the public frequently challenge decisions of the Federal Joint Committee to exclude technologies from reimbursement (for example, regarding stem cell transplantation and positron emission tomography). Because of the conflicting interests of the various stakeholders, reimbursement decisions are often made after prolonged discussions and sometimes extended political bargaining. The average time that the committee takes to decide whether to sanction or rule out the use of a new technology is around 70 months,13 much longer than in other countries (18 months in the UK, for example).

The lack of any systematic approach to assess new hospital treatments means that the use of innovative technologies is often years ahead of published results of appropriate clinical trials, as the following example of transcatheter aortic heart valve implantation (TAVI) shows.

The TAVI experience

The use of TAVI has increased remarkably over recent years in Germany, and far exceeds usage in other European countries—almost 40% of all European TAVI treatments have been done in Germany (box 1).14 There has been no assessment by the Federal Joint Committee of TAVI in Germany, although health technology assessment reports and decision guidance documents, some of which highlighted the risk of inappropriate use, have been produced by other countries.15 16 17 18 19 20

Box 1: TAVI in Germany

  • 2007: Two devices for transcatheter aortic heart valve implantation (TAVI) become available on the European market. German hospitals can negotiate extra fees for TAVI and about 290 patients are treated by the end of the year

  • 2009: The German Cardiac Society and the German Society of Thorax, Heart and Vessel Surgery publish a position paper to clarify that only inoperable or high risk patients should be treated with TAVI.21 Over 5000 patients have TAVI

  • 2010: A separate TAVI diagnosis related group is used to reimburse hospitals (set higher than in other European countries14), which leads to increased use. 5776 patients were treated in 91 German hospitals, some of which did not even have a heart surgery unit.22 23 Only 38.3% of cases were rated appropriate in a national quality assessment22

  • 2010: One year results of PARTNER trial show that a TAVI device is superior to balloon angioplasty and medical treatment in inoperable patients,24 but the validity of the trial is criticised15 25

  • 2011: German insurance funds try to establish an agreement with hospitals on quality standards for TAVI. A total of 7664 patients receive a TAVI device in 93 hospitals, of which 18 did not have a heart surgery unit.25 Appropriateness remains low at 40.1%25

  • 2011: Additional results of PARTNER trial26 show the use of the valve is not inferior to open surgery in patients with an increased risk of complications during surgery. Patients in the surgery arm have an increased risk of bleeding while those in the TAVI arm have an increased stroke risk

  • 2011: The FDA approves the TAVI device tested in PARTNER for inoperable patients with additional requirements regarding appropriateness, provider qualifications, and accrual of additional clinical trial data

  • 2012: FDA extends approval for the treatment of operable high risk patients under certain conditions, requiring a treatment decision by a heart team and the conduct of a clinical trial to assess the risk of stroke from the procedure. In Europe, five devices are on the market.

The unrestricted use of TAVI in Germany was criticised by one of the investigators of the PARTNER trial, which compared treatment using the Edwards Sapien valve with standard therapy24 and exposed uncertainties about which patients should receive TAVI.27 The criticisms support the view that the technology was, and still is being, used beyond proper indications and in treatment facilities not adequately equipped for emergencies.23 The approach of the Food and Drug Administration, which waited until the PARTNER results were available before making a decision, shows the advantages of requiring some efficacy data before approval for market. In the US, TAVI was only ever approved for a restricted group of patients.

Collecting evidence after devices are in use

Given the failure of the proposals for reform of EU legislation to address the problem of incomplete evidence, we need other ways to obtain essential data on safety and efficacy. If suitable trials are not done before market access they can be done afterwards. Instead of making decisions about use and reimbursement directly after market authorisation, authorities could require further trials. The case for such “coverage with evidence development” is compelling.

Several countries already stipulate that the use of some new devices and interventions is dependent on the collection of further evidence to evaluate safety and efficacy (box 2). This approach raises several questions: which interventions should be evaluated, what type of study is required, who should be responsible for conducting the studies and who should pay for them, and does the new evidence affect subsequent decisions?28 29 30

Another important decision is how much coverage to allow while data are being accrued. Use of the technology could be sanctioned and reimbursed either only for patients participating in research studies or for all patients while the research is going on. A final decision about adoption would be made only after the results are available. A recent analysis suggests that if the accrual of evidence is being hindered because the intervention is being used outside trials, the “only in research” option is the best one to follow.10 31 If use outside a trial will not affect the results, wider coverage may be judged acceptable. It is often assumed that a “balance” should prevail between early coverage and evidence accrual.32 33 34 However, pressure on payers could result in nominal (“alibi”) research as a pretext for coverage.35 36

Box 2: International examples of coverage with evidence development

United States (Medicare)

The US Medicare system introduced a formal programme for coverage of new technologies with evidence development in 2006 after using it for some projects going back to the 1990s.29 37 A public consultation has recently been conducted as part of an overhaul of the programme.38 Different restrictions can be applied (only available in trials or full use with additional data collection). Among the technologies introduced under coverage with evidence are autologous bone marrow transplant, carotid artery angioplasty,30 positron emission tomography, artificial hearts, and implantable cardiodefibrillators. The variation in results between technologies makes it difficult to draw conclusions.

UK (NICE): “only in research” or “with research” recommendations39

The National Institute for Health and Clinical Excellence recommended approval of 29 technologies only in research or with further research between 2000 and 2010 (about 16% of all decisions).40 The decisions were mostly made because of lack of evidence on relative effectiveness, and NICE required experimental trials in 14 cases. Some of these trials have taken place (in laparoscopic surgery, for example) and the results have been used to review guidance. The use of such recommendations seems to have declined over time.40

Canada (Ontario)

Coverage with evidence schemes have been used since 2003. The technologies subject to the scheme include positron emission tomography, endovascular repair of aortic aneurysms, drug eluting stents, deep brain stimulation, vacuum assisted wound closure system, and hyperbaric oxygen therapy for diabetic foot ulcers.30 Many of the studies undertaken have been registry data collections or (other) observational studies, although randomised controlled trials were initiated for some technologies. Most registry studies led to later decisions to fund the treatment; the results from the randomised controlled trials are not yet available in most cases.

The Wingspan stent system for intracranial stenosis provides a good example of how coverage with evidence development can refine use. In 2005 the FDA approved the device under its “humanitarian device exemption” for the treatment of patients who had not responded to existing treatments and who had an intracranial stenosis of > 50%. The exemption means that efficacy data are not needed for approval. Yet Medicare agreed to fund the device only within the context of a clinical trial. The relevant randomised trial (SAMMPRIS) was stopped early because the rate of stroke or death within 30 days was 14.7% in patients receiving a stent plus medical treatment versus 5.8% in those receiving medical treatment only.4 As a result the FDA substantially amended its market approval.41 The Wingspan stent may now be used under very limited conditions: patients with two or more strokes despite aggressive medical treatment, most recent stroke more than seven days ago, 70-99% stenosis, and good recovery from the last stroke.

By contrast, in Europe, where the definition of the “intended purpose” lies solely with the manufacturer, the use of the Wingspan stent system was changed by the manufacturer to “for the treatment of treatment refractory patients with an intracranial stenosis of > 50%”( the stance taken by the FDA before SAMMPRIS results became available). To our knowledge, so far no regulatory authority in Europe has intervened to change the indications for use. Furthermore, there are two other stent systems available on the European market for intracranial stenosis, one of which is still intended for “the treatment of arteriosclerosis of intracranial arteries.”

The above example shows why reimbursement only within research is preferable if safety and efficacy data are completely lacking. Patient recruitment for SAMMPRIS accelerated because the intervention was only reimbursed within the trial.42

Germany’s approach to lack of safety and efficacy data

After controversial decisions to exclude certain technologies—for example, several indications for positron emission tomography and stem cell transplantation in 2010-11,43 Germany has made some amendments to try to deal with cases where evidence is uncertain,44 but some important problems remain. The new law makes it harder to refuse new technologies in the absence of evidence unless the federal committee’s assessment has shown the technology is harmful or inferior to standard treatment. If the assessment concludes that there is no evidence of benefit but there is clinical “potential” for a new treatment the committee has to release a “testing directive,” defining key parameters for a clinical trial.

Although this is a useful development, a major problem is that while the trial is under way the new technology can be used freely and has to be reimbursed, not only within but also outside the clinical trials, thus implementing a “with research” model. The stipulation creates the difficulty that a trial is under way to answer clinical uncertainty about safety and effectiveness of an intervention that is already generally available.28 31 This contradiction is difficult to communicate to the health services community and in particular to patients. It may reduce participation in trials and therefore jeopardise the entire approach. When there is considerable uncertainty the best solution would be to insist on a randomised trial, which would justify coverage only for research purposes. Concerns about whether it is ethical to provide funding only for patients in clinical trials are then irrelevant.30 33 36

Conclusion

Patients in Germany would have been better served if decisions about the implementation of TAVI and intracranial stents were made after results of the relevant trials. And, they would still benefit from action reflecting their results. The use of TAVI and intracranial stenting is still legally unlimited in Germany, exposing patients to the risk of avoidable harm from inappropriate use. These two examples underline the case for tighter regulation of new technologies based on better evidence. If requirements regarding safety and efficacy data were strengthened at the time of market entry, together with a clear definition of the intended purpose that reflects these clinical data (box 3), all European healthcare systems could make their coverage decisions for high risk (class III) devices more easily and effectively. In the absence of such requirements, the use of these devices should be restricted to appropriate trials until sufficient evidence is available for general reimbursement decisions.

Box 3: What new EU device regulation should require

  • Manufacturers of high risk (class III) devices must provide evidence of safety and efficacy from prospective (and if possible randomised controlled) trials with outcomes relevant to patients before market entry

  • The intended indications for the medical device have to be approved

  • A competent authority will mandate appropriate requirements for postmarketing surveillance, including trials assessing long term safety if necessary

  • The authority will be able to limit the intended purpose for use or withdraw market approval.

Key messages

  • New medical devices are approved for EU use without evidence of safety and efficacy

  • In Germany no further assessment is required before devices can be used in hospitals

  • Examples show that the disproportionate use of new technologies may have harmed patients

  • If devices are approved without sufficient evidence of safety and efficacy their use should be restricted until evidence from further research studies has been collected.

Notes

Cite this as: BMJ 2013;346:f1646

Footnotes

  • Contributors and sources: PS-P, MS, and MD are employees of the German National Association of Statutory Health Insurance Funds. PS-P is a sociologist originally, while MS and MD have a background as biologists. They all have experience in evidence based medicine. They are representatives on several working groups of the Federal Joint Committee and participate in the preparation of the committee’s decisions about non-pharmaceutical medical methods.

  • Competing interests: We have read and understood the BMJ Group policy on declaration of interests and have no relevant interests to declare.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

References