An audit of mortality by using ECMO specific scores and APACHE II scoring system in patients receiving extracorporeal membrane oxygenation in a tertiary intensive care unit in Hong Kong

Introduction

The use of extracorporeal membrane oxygenation (ECMO) specific scoring systems have been shown to better predict survival compared to general risk scores used in the intensive care unit (ICU) (1-6). These may lead to better risk stratification, selection of patients, and possibly benchmarking. Yet, outcome of patients receiving ECMO therapy risk adjusted in individual units by use of these scores has rarely been reported and comparison among different cohorts has been difficult (7-12). Traditionally, standardized mortality ratio (SMR), as a performance measure, is calculated with general ICU risk scores based on expected mortality. However, use of SMR in patient populations receiving ECMO therapy is subject to limitations, in particular lead-time bias and case mix differences. The Acute Physiology and Chronic Health Evaluation (APACHE) scoring system was developed before the widespread use of ECMO and has not been validated in this group of patients (13,14). Few investigators have reported outcomes by using ECMO specific risk scores and determination of respective SMRs to benchmark performance in individual units (15,16). In addition, the precise effects and agreements between ECMO specific scoring system and general ICU risk scores on mortality prediction remain unclear.

In this study, our primary objective was to determine the SMR of our ECMO patients by using Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP), Survival After Veno-Arterial-ECMO (SAVE) and APACHE II scores. To explore the potential beneficial effect of ECMO, we also use our cohort to explore the differences in mortality prediction between ECMO specific scores (SAVE and RESP) and traditional ICU risk score (APACHE II).

Methods

This was a retrospective study, reviewing all patients who received venous-venous ECMO (VV-ECMO) or venous-arterial ECMO (VA-ECMO) in a tertiary ICU at a university teaching hospital in Hong Kong between 1st January 2009 and 1st March 2017. Our patient cohort comes from a catchment population of approximately one million and included admissions from other regional hospitals. Patients who received ECMO during cardiopulmonary resuscitation (ECPR) were excluded. The study was done in compliance with the Declaration of Helsinki. Demographic, clinical, laboratory, imaging data were obtained from case records and electronic Clinical Management System. RESP and SAVE scores were calculated based on the patients’ conditions before initiation of ECMO. APACHE II score was calculated based on the worst physiological variables within 24 hours of ICU admission, chronic health condition and reason for ICU admission.

Selection of patients

VV-ECMO

Patients over age of 12 years were considered for VV-ECMO if they had severe reversible respiratory failure refractory to optimized mechanical ventilation. General criteria included severe acute respiratory distress syndrome (ARDS) with Murray score ≥3, and PaO₂ to FiO₂ ratio (p/f ratio) of 60–80, severe hypoxaemia with p/f ratio of less than 60 secondary to a variety of causes, and/or refractory status asthmaticus. Prior to initiation of VV-ECMO, all patients with severe ARDS were paralyzed and mechanically ventilated with low tidal volume strategy (6 mL/kg predicted body weight). Prone ventilation was attempted before ECMO unless contraindicated. In general, patients were excluded if they had severe neurological insult, intracranial bleed, advanced multi-organ failure and age more than 70. Decisions to initiate ECMO were made on individual basis by treating intensive care specialists.

VA-ECMO

Patients over age of 12 years were considered for VA-ECMO if they had severe reversible cardiogenic shock despite inotropic support with or without intra-aortic balloon counterpulsation (IABP). In general, patients were excluded if they had severe neurological insult, intracranial bleed, unwitnessed pre-ECMO cardiac arrest, advanced multi-organ failure, severe aortic regurgitation or unrepaired aortic dissection. Decisions were made on individual basis by treating intensive care specialists and/or cardiothoracic surgeons.

ECMO configurations and ECMO care

VV-ECMO was usually configured with femoral drainage and jugular return. VA-ECMO was either configured peripherally using femoral-femoral sites or central configuration. A 7-French distal perfusion catheter was inserted if lower limb ischaemia was suspected in peripheral VA-ECMO. Patients who received peripheral VA-ECMO with subsequent conversion to central VA-ECMO were counted as having central VA-ECMO. IABP was usually inserted before VA-ECMO implantation. Occasionally, IABP was inserted after VA-ECMO implantation for severe myocardial stunning without aortic valve opening. A Left ventricular vent was surgically inserted if there was persistent left ventricular distension. The circuit was connected to either Rotaflow (Quadrox PLS Jostra) or Cardiohelp (HLS module advanced 7.0, Maquet, Germany).

Patients were closely monitored by trained physicians and nurse specialists. Lung protective strategy after ECMO involved the use of pressure control ventilation with pressure control of 10 cmH₂O above positive end-expiratory pressure (PEEP) of 10 cmH₂O, and respiratory rate of 10 breaths per minute. Unfractionated heparin was infused and titrated to activated clotting time (ACT) of 150–220 seconds, depending on bleeding risk. Platelet was maintained above 80×10⁹ per liter and haemoglobin above 9 grams per deciliter. Plasma free haemoglobin, D-dimers and fibrinogen were monitored daily.

Echocardiography was performed on at least alternate days to assess for recovery and weaning in patients receiving VA-ECMO. Patients were also assessed for destination therapies including ventricular assist devices (VADs) or heart transplant if they could not be weaned from VA-ECMO. Patients with VA-ECMO were weaned by stepwise reductions of ECMO blood flow until idling flow could be maintained with stable haemodynamics for 15 minutes. Alternatively, a pump controlled retrograde trial-off method was used for up to one hour in patients with predominant right ventricular failure, or those with concomitant respiratory failure (17). In patients receiving VV-ECMO, sweep gas was turned off to assess native gas exchange during the trial-off. Decannulation of all arterial cannulae mandated surgical repair of the artery.

Antibiotic prophylaxis was not used during ECMO except for surgical site prophylaxis upon ECMO cannulation and decannulation. C-reactive protein and white cell count were monitored daily. Antibiotic was used for suspected and proven infections according to culture and sensitivity results.

Outcomes

Primary outcome was the SMR, which was expressed as a ratio of observed hospital mortality to predicted hospital mortality derived from the RESP score (RESP-SMR) for VV-ECMO and SAVE score (SAVE-SMR) for VA-ECMO, respectively. Secondary outcomes included SMR with predicted mortality derived from APACHE II (APACHE-II SMR), six-month mortality post-ECMO, complications, hospital length of stay, ECMO duration and recovery or bridge to destination therapy. Agreement between RESP and APACHE II, as well as SAVE and APACHE II was tested using the Bland Altman plot. The predicted mortality for each patient from APACHE II versus RESP or SAVE score was compared.

Statistical analysis

Continuous data were expressed as median and interquartile range (IQR) or mean ± standard deviation (SD) with 95% confidence intervals (95% CIs). Categorical data were expressed as number and percentage. Student’s t-test was used for comparison of continuous variables while chi square test was used for comparison of categorical variables. The level of significance was 0.05. SMR was calculated by dividing the observed hospital mortality by predicted hospital mortality. Bland Altman plot was used to assess the agreement of mortality risks predicted by different scoring systems. Maximum allowed difference in predicted mortality to achieve clinical significance was defined priori as 10% in Bland Altman plot. Statistical analysis was carried out using Statistical Product and Service Solutions (SPSS) Windows Version 22 and MedCalc Version 18.

Ethics review

The proposal was reviewed and approved by the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee.

Results

Study population

Between 1^st January 2009 and 1^st March 2017, 20 patients received 20 VV-ECMO runs (14 from our own hospital and 6 transferred from other referring hospitals). Fourteen patients who met inclusion criteria for VV-ECMO were transferred from other referring hospitals. ECMO was not instituted in 8 of them due to either improvement (7/14) or rapid development of multiple organ failure (1/14) resulting in 6 transfers who received VV-ECMO. During the same period, there were 42 patients receiving 43 VA-ECMO runs. Among them, six patients were transferred from other referring hospitals and all received VA-ECMO. The baseline characteristics before initiation of ECMO are shown in Table 1. Half of the patients receiving VV-ECMO had either viral or bacterial pneumonia. Post-cardiotomy cardiogenic shock was the indication for VA-ECMO in 52% (22/42) of patients.

Table 1 Pre-ECMO characteristics: VV-ECMO and VA-ECMO
Full table

VV-ECMO

Primary outcomes

Hospital mortality was 50% for patients treated with VV-ECMO. Mean RESP score was 0.6 (SD 4.86) with a corresponding RESP-SMR of 1.16 (95% CI: 0.44–1.88).

Secondary outcomes

For patients on VV-ECMO, the mean APACHE II was 19 (SD ±7.8), with corresponding APACHE II-SMR 1.4 (95% CI: 0.53–2.27). Six-month mortality was 50%. The median VV-ECMO duration was 10 days (IQR, 4.8–22.1 days). The median ICU and hospital length of stay were 15.5 (IQR, 8.5–34.5) and 33.5 (IQR, 8.3–49) days, respectively. Subgroup outcomes and complications are shown in Tables 2-4.

Table 2 Diagnosis, SMR and secondary outcomes: VV-ECMO
Full table

Table 3 Diagnosis, SMR and secondary outcomes: VA-ECMO
Full table

Table 4 Secondary outcomes and complications: VV-ECMO and VA-ECMO
Full table

VA-ECMO

Primary outcomes

Hospital mortality was 50% for patients treated with VA-ECMO. The mean SAVE score was −7.8 (SD ±5.6), with SAVE-SMR of 0.69 (95% CI: 0.39–0.98).

Secondary outcomes

For patients on VA-ECMO, mean APACHE II was 27.4 (SD ±11.7), with corresponding APACHE II-SMR 0.89 (95% CI: 0.51–1.27). Six-month mortality was 50%. The median ECMO duration was 5 days (IQR, 3–12.5 days). The median ICU and hospital length of stay were 10 (IQR, 5.3–15) and 18 (IQR, 9.3–41.8) days respectively. Two patients had VAD implantation and subsequent heart transplant due to failure to wean from VA-ECMO, and both of them survived. Subgroup outcomes and complications are shown in Tables 2-4.

Comparison of prediction models by RESP, SAVE and APACHE II

The Bland Altman plots and regression analysis between the predicted mortality from RESP and SAVE against APACHE II scores are shown in Figures 1 and 2, respectively. We assigned the following variables: Mortality_RESP and Mortality_SAVE representing the predicted percentage mortality from RESP and SAVE score respectively, Mortality_APACHEII(VV) and Mortality_APACHEII(VA) representing the predicted percentage mortality from APACHE II score for VV-ECMO and VA-ECMO, respectively.

Figure 1 Comparison of predicted mortality between RESP and APACHE II score. (A,B) Bland-Altman plot of mortality predicted by RESP and APACHE II score, with y-axis being the absolute difference in percentage mortality in (A) and the difference in percentage mortality divided by the mean in (B); (C) Scatter diagram and linear regression between the mortality predicted by RESP and APACHE II score. Dotted lines: regression line with upper and lower 95% CI. Purple solid line at y axis =0: line of equality. Error bars: 95% CI of upper limit of agreement, mean and lower limit of agreement. RESP, Respiratory Extracorporeal Membrane Oxygenation Survival Prediction.

Figure 2 Comparison of predicted mortality between SAVE and APACHE II score. (A,B) Bland-Altman plot of mortality predicted by SAVE and APACHE II score, with y-axis being the absolute difference in percentage mortality in (A) and the difference in percentage mortality divided by the mean in (B); (C) Scatter diagram and linear regression between the mortality predicted by SAVE and APACHE II score. Dotted lines: regression line with upper and lower 95% CI. Purple solid line at y axis =0: line of equality. Error bars: 95% CI of upper limit of agreement, mean and lower limit of agreement. SAVE, Survival After Veno-Arterial-ECMO.

For VV-ECMO, mean Mortality_RESP was not statistically different from mean Mortality_APACHEII(VV) (P=0.137, mean difference Mortality_RESP − Mortality_APACHEII(VV) =9.5%, 95% CI: −0.5% to 19.5%). However, RESP and APACHE II did not agree well (95% limit of agreement −44% to 62.9%) by using a predetermined 10% difference in mortality.

In contrast, for VA-ECMO, the mean Mortality_SAVE was statistically different from mean Mortality_APACHEII(VA) (mean difference Mortality_SAVE − Mortality_APACHEII(VA) =17.6%, 95% CI: 7.6% to 27.6%, P<0.0001). Predicted mortality from SAVE and APACHE II did not agree within 10% (95% limit of agreement −30.2% to 65.5%). Systemic bias was present (P<0.0001) with beta-coefficient being −1.04. Mortality_APACHEII(VA) was higher than Mortality_SAVE when the averaged mortality was above 86%, and lower when the averaged mortality was below 75%. Among the fifteen patients in whom predicted mortality were the highest, twelve of them (80%) had cardiac arrest prior to VA-ECMO.

Discussion

ECMO specific scores seem to have better discrimination and calibration than APACHE II does for mortality prediction in ECMO patients (1-6). However, the underlying mechanisms and their degree of difference are not entirely clear. The implication of this on SMR is not clear either. From our data, SMR for VA-ECMO decreases from 0.89 (95% CI: 0.51–1.27) to 0.69 (95% CI: 0.39–0.98) by changing APACHE II with SAVE scores as the denominator. As shown in Figure 2C, SAVE and APACHE II are correlated across the range of predicted mortality (r=0.69, P<0.001). However, the Bland Altman analysis shows that the two predictive scores do not agree. Initially, APACHE II underestimates mortality when compared to SAVE at lower end of predicted mortality. But as predicted mortality increases, APACHE II prediction becomes progressively more severe until it crosses SAVE prediction at around 80% (Figure 2A,B).

We propose a few reasons to account for this relationship. Firstly, APACHE II score was derived and validated in patient populations not receiving ECMO. In fact, it has been shown that APACHE II was not as good as pre-operative models in predicting hospital mortality when applied to heart surgery patients (18,19). In contrast, SAVE was derived specifically from the patient cohort receiving VA-ECMO. Among the fifteen patients who had the highest predicted mortality, twelve out of them had pre-ECMO arrest. Pre-ECMO arrest, with its severe physiological derangement, in particular low Glasgow Coma Scale, is associated with a high Mortality_{APACHEII(VA).} This group of patients, who may otherwise die in the past, is more likely to survive with ECMO support, as reflected by a lower Mortality_SAVE. Secondly, lead time bias may account for the lower Mortality_APACHEII(VA) as compared with Mortality_SAVE. In contrast to the use of pre-ECMO variables, APACHE II variables are primarily based on physiological parameters within the first 24 hours of ICU admission. A significant proportion of our patients had ECMO initiated outside ICU, including those with ECMO implantation in the operating theatre following post-cardiotomy as well as those with mobile ECMO in referring regional hospitals. This subgroup of patients could have their deranged physiology mostly reversed by the time of ICU admission and thereby not being captured well in APACHE II scoring. This effect would be particular pronounced in groups with lower average mortality and readily reversible pathophysiology. Thirdly, there is a significant variation in diagnostic subgroups in our cohort as compared to SAVE derivation cohort. The main diagnoses associated with cardiogenic shock in SAVE derivation cohort were chronic heart failure of other causes (33%), acute myocardial infarction (29%), valvular heart disease (17%), refractory VT/VF (13%), post heart or lung transplantation (6%) (3). In contrast, the major diagnostic subgroup in our cohort is post-cardiotomy cardiogenic shock (52%), which consists of a high proportion of aortic surgery (30.8%), but no post heart transplantation. This difference in patient population may have an impact on the discriminative power of SAVE scores in our cohort of patients.

In our cohort of VV-ECMO patients, SMR decreases from 1.4 (95% CI: 0.53–2.27) to 1.16 (95% CI: 0.44–1.88) by changing APACHE II with RESP score as the denominator. In contrast to our result as in VA-ECMO, Mortality_RESP and Mortality_APACHEII(VV) are poorly correlated (r=0.39, P=0.091). In the Bland Altman analysis, we could not identify any systematic bias between the Mortality_RESP and Mortality_APACHEII(VV) (95% CI: of the mean difference contains the line of equality), however, the 95% limit of agreement is more than the predetermined 10%. We conclude that the two predictive scores do not agree with each other. However, our small sample size of patients receiving VV-ECMO and the presence of a few outliers limits the interpretation. We do not exclude the possibility that systematic bias may exist if our sample size of patient increases.

The major limitations in our study are the limited number of patients in a single center and retrospective design. However, we were able to demonstrate difference between SAVE and APACHE II even with this limited cohort. Further trials with larger sample size are warranted. Secondly, while SMR is widely used in ICU as performance measures, studies on ICU SMR based on hospital mortality are confounded by multiple factors such as care given before and after ICU stay and the patient’s health status when step down care from ECMO ICU discharge to another ICU occur.

Conclusions

In our cohort of patients receiving VA-ECMO in whom severity of illness was high and mortality was lower than predicted, we demonstrated that APACHE II underestimates mortality risk in lower risk patients and overestimates mortality risk in patients at high risk of death. In this regard, the use of SAVE-SMR may better identify performance in the management of patients receiving VA-ECMO. We recommend the use of SAVE-SMR for analyzing and reporting outcome in future studies.

Acknowledgements

The authors gratefully acknowledge Miss Constance Wong, Bik-Shan Tam, Ngar-Wan Chu for their generous help in collecting and organizing the data of patients.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The proposal was reviewed and approved by the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee. Consents were waived.

References

1. Schmidt M, Bailey M, Sheldrake J, et al. Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score. Am J Respir Crit Care Med 2014;189:1374-82. [Crossref] [PubMed]
2. Schmidt M, Zogheib E, Roze H, et al. The PRESERVE mortality risk score and analysis of long-term outcomes after extracorporeal membrane oxygenation for severe acute respiratory distress syndrome. Intensive Care Med 2013;39:1704-13. [Crossref] [PubMed]
3. Schmidt M, Burrell A, Roberts L, et al. Predicting survival after ECMO for refractory cardiogenic shock: the survival after veno-arterial-ECMO (SAVE)-score. Eur Heart J 2015;36:2246-56. [Crossref] [PubMed]
4. Pappalardo F, Pieri M, Greco T, et al. Italian ECMOnet: Predicting mortality risk in patients undergoing venovenous ECMO for ARDS due to influenza A (H1N1) pneumonia: the ECMOnet score. Intensive Care Med 2013;39:275-81. [Crossref] [PubMed]
5. Muller G, Flecher E, Lebreton G, et al. Combes A: The ENCOURAGE mortality risk score and analysis of long-term outcomes after VA-ECMO for acute myocardial infarction with cardiogenic shock. Intensive Care Med 2016;42:370-8. [Crossref] [PubMed]
6. Klinzing S, Wenger U, Steiger P, et al. External validation of scores proposed for estimation of survival probability of patients with severe adult respiratory distress syndrome undergoing extracorporeal membrane oxygenation therapy: a retrospective study. Critical Care 2015;19:142. [Crossref] [PubMed]
7. Chen JS, Ko WJ, Yu HY, et al. Analysis of the outcome for patients experiencing myocardial infarction and cardiopulmonary resuscitation refractory to conventional therapies necessitating extracorporeal life support rescue. Crit Care Med 2006;34:950-7. [Crossref] [PubMed]
8. Rastan AJ, Dege A, Mohr M, et al. Early and late outcomes of 517 consecutive adult patients treated with extracorporeal membrane oxygenation for refractory postcardiotomy cardiogenic shock. J Thorac Cardiovasc Surg 2010;139:302-11. [Crossref] [PubMed]
9. Bermudez CA, Rocha RV, Toyod Y, et al. Extracorporeal Membrane Oxygenation for Advanced Refractory Shock in Acute and Chronic Cardiomyopathy. Ann Thorac Surg 2011;92:2125-31. [Crossref] [PubMed]
10. Loforte A, Montalto A, Ranocchi F, et al. Peripheral Extracorporeal Membrane Oxygenation System as Salvage Treatment of Patients With Refractory Cardiogenic Shock: Preliminary Outcome Evaluation. Artif Organs 2012;36:E53-E61. [Crossref] [PubMed]
11. Aso S, Matsui H, Fushimi K, et al. In-hospital mortality and successful weaning from venoarterial extracorporeal membrane oxygenation: analysis of 5,263 patients using a national inpatient database in Japan. Crit Care 2016;20:80. [Crossref] [PubMed]
12. Dangers L, Brechot N, Schmidt M, et al. Extracorporeal Membrane Oxygenation for Acute Decompensated Heart Failure. Crit Care Med 2017;45:1359-66. [Crossref] [PubMed]
13. Knaus WA, Draper EA, Wagner DP. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. [Crossref] [PubMed]
14. Wong DT, Knaus WA. Predicting outcome in critical care: the current status of the APACHE prognostic scoring system. Can J Anaesth 1991;38:374-83. [Crossref] [PubMed]
15. Ng WT, Chan KM. The Characteristics and Outcome of Patients Receiving Veno-venous and Veno-arterial Extracorporeal Membrane Oxygenation (VV-ECMO and VA-ECMO) in a Tertiary Hospital in Hong Kong. Eur J Heart Failure 2017;19:36-49.
16. Loveridge R, Patel S, Kakar V, et al. Nowhere else to turn? building a case for equity of access to VV ECMO in the UK for complex multi-system patients. Intensive Care Medicine Experimental 2015;3:A475. [Crossref]
17. Ling L, Chan KM. Weaning adult patients with cardiogenic shock on veno-arterial extracorporeal membrane oxygenation by pump-controlled retrograde trial off. Perfusion 2018;33:339-45. [Crossref] [PubMed]
18. Martínez-Alario J, Tuesta ID, Plasencia E, et al. Mortality prediction in cardiac surgery patients: comparative performance of Parsonnet and general severity systems. Circulation 1999;99:2378-82. [Crossref] [PubMed]
19. Tamayo E, Fierro I, Bustamante-Munguira J, et al. Development of the Post Cardiac Surgery (POCAS) prognostic score. Crit Care 2013;17:R209. [Crossref] [PubMed]