Changing Evidence, Changing Practice

Introduction

The pursuit of new innovations and techniques to provide optimal care to patients is both welcome and vital. Without advances in practice, we would not improve quality of life and save as many lives as we do. However patient safety is paramount and that is why there are well established practices to assess the results of clinical trials that support the preparation of clinical guidelines which provide the advice on which clinicians depend to identify optimal treatment. Withdrawing support from guidelines in this fashion is unprecedented for our Association. It was a decision taken by the whole Council with considerable care. This article explains what we did and why.

The guideline in question was prepared in 2018 by representatives from both the European Society of Cardiology (ESC) and the European Association for Cardiothoracic Surgery (EACTS), who had considered a range of evidence including the reported outcomes from the EXCEL trial to develop the recommendations for the treatment of patients with left main coronary artery disease (LMCAD) and stable angina which form part of the joint 2018 ESC-EACTS Myocardial Revascularization Guidelines.

EACTS Council made three important decisions

At our Council meeting on 7 December 2019 three important decisions were taken regarding the guidelines.

1. Council withdrew its support from the current recommendations on treatment of left main disease in the 2018 joint ESC-EACTS Myocardial Revascularization Guidelines. This decision was based on a range of scientific, statistical and professional issues that had been raised.
2. Council wanted to ensure members were notified and it was agreed to publicise the decision widely.
3. Council also requested that the ESC was invited immediately to participate in a new joint taskforce to review the LMCAD recommendations in the guidelines.

All the decisions were agreed unanimously.

Prior to the Council meeting, the BBC’s Newsnight programme contacted both EACTS and the ESC with revelations about the EXCEL trial. The BBC asked Professor Nick Freemantle, from the Institute of Clinical Trials and Methodology at University College London, to examine their findings. Professor Freemantle shared his analysis of the current evidence and the new findings with the EACTS Council. To help inform their decisions, the EACTS Council was able to consider Professor Freemantle’s comprehensive statistical analysis alongside several matters of scientific and professional propriety, including those raised by the BBC, which questioned the robustness of both the content and the guideline process.

Following the Council meeting, we proactively issued a statement so that members and clinicians knew the Council had removed its support for the LMCAD recommendations in the 2018 ESC-EACTS Myocardial Revascularization Guidelines.

So what evidence underpinned the 2018 Guideline recommendations?

The recommendations in the 2018 ESC-EACTS Myocardial Revascularization Guidelines for the treatment of LMCAD were based on SYNTAX score terciles and the conclusions were:

1. There was class IA evidence to support CABG in all groups, the highest level of recommendation.
2. In patients with SYNTAX score <22, PCI was equivalent to CABG. Class IA recommendation.
3. In patients with SYNTAX 23-32, the evidence for PCI was less clear. Class IIaA recommendation.
4. In patients with SYNTAX scores >33, PCI was not recommended. Class IIIA.

These recommendations are the same as those published in 2014, which were derived from short-term outcome data and an underpowered subgroup analysis of the SYNTAX trial.

The scientific evidence underpinning the 2018 decisions was based mostly on 3 studies (see references 1-3).

1. The patient level meta-analysis of randomised trials of CABG versus PCI (3) using up to 5-year all-cause death as the primary endpoint showed that for the overall cohort PCI was associated with a significant survival disadvantage (Hazard Ratio [HR] 1.20, 95% confidence intervals [CI] 1.06-1.37; P=0.0038). A subgroup analysis for patients with LMCAD was reported to have similar outcomes for PCI and CABG. However, this conclusion is scientifically questionable given the non-significant interaction (P=0.12) for the LMCAD subgroups. Thus, the correct scientific interpretation is that the result for the overall cohort applies also for the LMCAD subgroup (4, 5). The patient level meta-analysis was evaluated by the task force in its pre-publication status and shared with the reviewers a short time before the publication of the guidelines; this questions whether sufficient time for scrutiny had been allowed.
2. The NOBLE trial (2) showed a more frequent occurrence of the primary composite endpoint (all-cause death, non-procedural myocardial infarction (MI), stroke and repeat revascularisation in the PCI than in the CABG group (HR 1.48, 95% CI 1.11-1.96; P=0.007). There was no difference in all cause mortality.
3. The EXCEL trial which studied patient with SYNTAX Score <32 (1), reported that at 3-year follow up the primary composite endpoint (all-cause death, stroke, or MI) occurred with similar frequency in the CABG and the PCI groups (HR 1.00, 95% CI 0.79-1.26; P=0.98). This was interpreted as showing that PCI was not inferior to CABG.

But the evidence available has changed

Our primary concern is patient safety and given there is new emerging evidence, there are multiple reasons for adapting to these different circumstances and reviewing the LMCAD recommendations of the guideline:

1. More complete understanding of the 3-year data from EXCEL (1) using the standard definition of a myocardial infarction (UDMI) has reversed the perceived advantage/non-inferiority of PCI compared with CABG for LMCAD disease. These data were presented by the BBC but have not been published.
2. The 5-year data from EXCEL (10) have now demonstrated a significant survival advantage for CABG over PCI for this group of patients.

And the BBC has also highlighted the following issues

The recent BBC Newsnight investigation has raised several additional concerns with regards to the EXCEL trial and the guideline process:

1. To date, the trial authors have not published the trial outcome data using the universal definition of myocardial infarction (UDMI) despite stating they would report the findings.

 EXCEL has attracted controversy since its inception (6), particularly around the definition of MI. There is a standard Universal Definition of MI (UDMI) (7), which was included in the EXCEL protocol as a prespecified secondary endpoint. This definition has the advantage of being endorsed by most cardiovascular organizations, including the ESC, ACC, AHA and regulators. The BBC reported that the EXCEL investigators adopted a new definition for this complication (MI) which would increase the apparent occurrence of peri-operative MI after CABG, leading to results appearing to favor the PCI option. This could only be acceptable if its findings proved to be consistent with those using the UDMI, hence the importance of the secondary endpoint. The protocol of the study was not amended to reflect the decision not to publish – to date, this represents a breach of the CONSORT and Good Clinical Practice Guidelines (ICH E9) (8, 9).

2. 35% increased risk of death in the PCI group

The BBC received data for the 3-year outcomes, including the hitherto unpublished secondary outcome using the UDMI, which provided a qualitatively different outcome. In an analysis shared by the BBC, when UDMI is used PCI is associated with substantially worse outcomes at 3 years with a significantly higher risk of MI than CABG (HR 1.79, 95% CI 1.25-2.57; P=0.002). When UDMI is used in the composite primary end-point with all-cause death and stroke, PCI is associated with a 40% increased risk for this (HR 1.40, 95% CI 1.09-1.81; P=0.009). This analysis is predictive of the published 5-year EXCEL results (10) which show a 35% increased risk of death in the PCI group, (Odds Ratio 1.38, 95% CI 1.03-1.85).

3. Emerging mortality data have not been shared

The BBC revealed there were emerging data available to the EXCEL data safety monitoring board (DSMB) indicating an increased mortality for the PCI group. In its news coverage, the BBC considered why these data were not made available to those on the guideline task force. Had all the information described above been available to the task force, the conversation around the choice of guideline recommendation would probably have been different. It is of note that some of the EXCEL investigators were also members of the guidelines task forces.

4. Alleged conflicts of interest

 Public concerns have arisen over commercial conflicts of interests among both the authors and some members of the guideline committees. The BBC identified that one third of the authors of the guidelines had significant relevant commercial conflicts of interests, including at least one holding a patent for drug-eluting stents. These potential conflicts were not available to the guideline task force members during the writing of the recommendations or to the reviewers (although they had been seen by the guideline chairs) and were published at the same time as the guidelines.

Restoring confidence in our recommendations

Producing clinical guidelines provides an important opportunity for scrutinizing the available clinical evidence. For this reason, the guideline process must take care to minimize bias and the potential influence of conflict of interests. The methodology for developing “Guidelines we can Trust” is well described by The Institute of Medicine (IOM)(11) which recommends steps to minimize bias by using systematic literature search around predefined clinical questions, a transparent management of COIs, the use of statistical methodologists to prepare evidence tables and guide the interpretation of data, and more recently the use of the GRADE collaboration system (12).

The 2018 joint ESC-EACTS guideline task force did not use this methodology as it was not adopted at the time.

The way forward is clear. If vested interests add complexity to the resolution, these must be swept aside by professional, scientific and analytical integrity in the interests of our patients.

Therefore, we have taken the following action to restore confidence in the guidelines.

1. We have written to the ESC inviting our colleagues to work with us jointly to consider the evidence available and develop updated recommendations as a matter of urgency.
2. In order to ensure the trustworthiness of the guidelines, we propose to adhere to the Institute of Medicine gold standard principles for developing clinical recommendations.
3. We have invited Professor Schuenemann, Director of the DeGroote Cochrane Canada Centre and Mc Master Grade Centre, to oversee the process.

The guidelines for myocardial revascularization must be reviewed urgently. By working together collaboratively and transparently, we can restore confidence in our clinical recommendations and send a strong signal to the public that patients’ interests are at the centre of all we do and say.

D. Pagano.

Secretary General of EACTS, on behalf of the EACTS Council

-ENDS-

REFERENCES

1. Stone GW, Sabik JF, Serruys PW, Simonton CA, Généreux P, Puskas J, et al. Everolimus-Eluting Stents or Bypass Surgery for Left Main Coronary Artery Disease. N Engl J Med. 2016;375(23):2223-35.
2. Makikallio T, Holm NR, Lindsay M, Spence MS, Erglis A, Menown IB, et al. Percutaneous coronary angioplasty versus coronary artery bypass grafting in treatment of unprotected left main stenosis (NOBLE): a prospective, randomised, open-label, non-inferiority trial. Lancet. 2016;388(10061):2743-52.
3. Head SJ, Milojevic M, Daemen J, Ahn JM, Boersma E, Christiansen EH, et al. Mortality after coronary artery bypass grafting versus percutaneous coronary intervention with stenting for coronary artery disease: a pooled analysis of individual patient data. Lancet. 2018;391(10124):939-48.
4. Freemantle N, Ruel M, Gaudino MFL, Pagano D. On the pooling and subgrouping of data from percutaneous coronary intervention versus coronary artery bypass grafting trials: a call to circumspection. Eur J Cardiothorac Surg. 2018;53(5):915-8.
5. Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA. 1991;266(1):93-8.
6. Ruel M, Falk V, Farkouh ME, Freemantle N, Gaudino MF, Glineur D, et al. Myocardial Revascularization Trials. Circulation. 2018;138(25):2943-51.
7. Thygesen K, Alpert JS, White HD. Universal definition of myocardial infarction. Eur Heart J. 2007;28(20):2525-38.
8. Rennie D. CONSORT revised–improving the reporting of randomized trials. JAMA. 2001;285(15):2006-7.
9. European Medicines Agency. ICH E9 statistical principles for clinical trials. https://www.ema.europa.eu/en/ich-e9-statistical-principles-clinical-trials. (15 December 2019, data last accessed).
10. Stone GW, Kappetein AP, Sabik JF, Pocock SJ, Morice M-C, Puskas J, et al. Five-Year Outcomes after PCI or CABG for Left Main Coronary Disease. N Engl J Med. 2019;381(19):1820-30.
11. Institute of Medicine Committee on Data Standards for Patient S. In: Aspden P, Corrigan JM, Wolcott J, Erickson SM, editors. Patient Safety: Achieving a New Standard for Care. Washington (DC): National Academies Press (US)

Copyright 2004 by the National Academy of Sciences. All rights reserved.; 2004.

12. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6.