Validation of EuroSCORE II to predict mortality in post-cardiac surgery patients in East Java tertiary hospital

Yan Efrata Sembiring,1 Adhitya Ginting,1 Puruhito,1 Budiono2†

Background

The European system for cardiac operative risk evolution (EuroSCORE) II is one of the established risk models used to predict mortality after cardiac surgery. However, its application as a mortality predictor for Indonesian adult cardiac surgery is still unknown. This study aimed to examine the validation of EuroSCORE II in predicting mortality following cardiac surgery in Indonesian adults.

 

Methods

This retrospective cohort study collected data from the medical records and the database of the Department of Thoracic Cardiac and Vascular Surgery at Soetomo General Hospital, Surabaya. Data on the EuroSCORE II variables were collected for patients aged >18 years who underwent coronary artery bypass, heart valve, heart tumors, aortic surgeries, or a combination of these surgeries between January 2016 and December 2018. In-hospital mortality prediction was calculated using the online calculator at www.euroscore.org. The calibration of the EuroSCORE II model was conducted using the Hosmer–Lemeshow test, and discrimination analysis was done using receiver operating characteristic (ROC) curves with area under the ROC curve (AUC) calculation.

 

Results

A total of 213 patients met the inclusion criteria and were analyzed for this study. Mortality was 8.9%. The predicted and actual mortalities were 1.74% and 8.9%, respectively. The significance (p-value) of the Hosmer–Lemeshow test was 0.55, indicating good calibration. The AUC of ROC curve was 0.85 (95% CI = 0.77–0.92, p<0.001), suggesting good discrimination.

 

Conclusions

EuroSCORE II has a good calibration and discrimination for cardiac surgery in Indonesian adults.

 

Keywords

cardiac surgery, mortality, prognosis, treatment outcome

In recent decades, the hospital mortality for coronary artery bypass graft (CABG), which is the most common surgical procedure in the world, has fallen from 2.8% to 1.6% (shrinking by about 43%) even in a population of patients who are older and more exposed to the disease. Based on the Society for Thoracic Surgery database, the mortality rate for 774,000 CABG was 2.3%, and for 109,000 isolated valve surgeries was 3.4%.1 The recommendation of the predictor and risk stratification is important in the current practice to achieve an objective prediction of mortality risk.2 The European system for cardiac operative risk evaluation (EuroSCORE) has been used since firstly introduced in 1999.3 This system has been used extensively to estimate mortality risk and as a benchmark for evaluating cardiac surgery in >1,300 official citations in the medical literature.4 Therefore, EuroSCORE II has been available since October 2011 and has been developed to overcome previous EuroSCORE.5 In China, studies have been conducted to assess the performance of EuroSCORE II. According to the study, the EuroSCORE II was good in the group of patients with middle to low risk.6 Most importantly, pre-operative risk assessment also allows doctors to provide realistic expectations to both the patients and their families concerning the potential risks and benefits of surgery. Thus, they can decide reasonable decisions related to the management of their diseases.7

Indonesian population is generally known as late presenter, mostly due to its geographical location, which is far from our center and poverty. Furthermore, patients with valvular heart disease are dominated by rheumatic causes rather than degenerative and thus usually younger and has more calcified valves. Thus far, the data for EuroSCORE used in Indonesia are still unavailable. EuroSCORE II was developed to predict risk for in-hospital mortality after cardiac surgery. As the EuroSCORE II data in Indonesia are also unavailable, the in-hospital mortality was analyzed instead. In this study, we initially observed the validation of EuroSCORE II for adult cardiac surgery including CABG, heart valve surgery, heart tumor surgery, aorta surgery, or a combination of these surgeries. This study aimed to examine the relationship between the EuroSCORE II stratification and post-cardiac surgery in Indonesian adult mortality.

 

 

This retrospective cohort study collected secondary data from the medical records of Soetomo General Hospital in Surabaya and the database of the Department of Thoracic Cardiac and Vascular Surgery, Universitas Airlangga. This research received approval from the Ethics Committee of Soetomo General Hospital (1298/KEPK/VII/2019). The sample size was determined by total population sampling.

The inclusion criteria of this study were adults who underwent a coronary artery bypass surgery, heart valve surgery, heart tumor surgery, aorta surgery, or a combination of these surgeries treated by the Department of Thoracic Cardiac and Vascular Surgery in Soetomo General Hospital, Surabaya from January 1, 2016 to December 31, 2018. Data with incomplete medical records were excluded from this study.

Age and body mass index (BMI) were presented as mean and range data. Age was taken from medical record when a patient was admitted for surgery. BMI was calculated by dividing the body weight (kg) by height (m) squared when a patient was admitted for surgery. Renal impairment was divided into three categories based on creatinine clearance calculated using Cockcroft-Gault formula for EuroSCORE II taken when a patient was admitted for surgery: on dialysis (regardless of value), moderate (50–85 ml/min), and severe (<50 ml/min). Extracardiac arteriopathy was examined in all subjects by ultrasound with history taking and characterized by one or more of the following: claudication, carotid occlusion or stenosis of >50%, amputation for arterial disease, and history of interventions in the foot or carotid abdominal aorta. Poor mobility was determined by deterioration of mobility that resulted in musculoskeletal or neurological dysfunction. Chronic lung disease was assessed during hospitalization for surgery by pulmonologist and determined according to the use of bronchodilators or steroids for the treatment of lung disease. Endocarditis was assessed using Duke Criteria and according to EuroSCORE II then was confirmed by a continuous antibiotic therapy before the surgery. Critical preoperative conditions were determined using the patient’s history of ventricular tachycardia, ventricular fibrillation, or aborted sudden death in the preoperative period. Left ventricular (LV) function was assessed based on four categories: good (left ventricular ejection fraction [LVEF] of >50%), moderate (LVEF of 31–50%), bad (LVEF of 21–30%), and very bad (LVEF of <20%). Recent myocardial infarction was determined by the presence of a myocardial infarction <90 days from the first onset and confirmed by history taking with medical record for laboratory examination. Pulmonary hypertension assessed by cardiologist using an echocardiography and determined by the pulmonary artery systolic pressure classified into one of two classes: moderate (pulmonary artery pressure [PAP] 31–55 mmHg) or severe (PAP >55 mmHg). The level of urgency of surgery was classified into four classifications: elective (routine scheduled surgery), urgent (requires surgery when the patient was treated), emergency (requires surgery 1 day before the workday since it was decided for surgery), and saves lives (requires cardiac resuscitation—lung help along with travel to the operating room or just before induction of anesthesia). Intervention of the heart was obtained by observing the ongoing surgical types, e.g., CABG surgery, heart valves replacement or repair, partial replacement of the aorta, heart defects repair, maze procedures, and cardiac tumors resection.

The required data for EuroSCORE include gender, renal impairment, extracardiac arteriopathy, poor mobility, previous cardiac surgery, chronic lung disease, endocarditis, critical preoperative, diabetes on insulin, New York Heart Association classification, Canadian Cardiovascular Society class 4 angina, ventricular function, recent myocardial infarction, pulmonary hypertension, surgical urgency, weight of the intervention, and surgery on the thoracic aorta, which were presented as sum and percentage. In-hospital mortality prediction was done using the online calculator from the site www.euroscore.org. The statistical analyses were done using the SPSS software version 23 (IBM Corp., USA).

The calibration of the EuroSCORE II model was analyzed using the Hosmer–Lemeshow test. Calibration is a statistical step that refers to the ability of a risk capital to harmonize results of estimated figures and observations across the scattered data. The discrimination was analyzed using receiver operating characteristic (ROC) curves and calculating the area under the ROC curve (AUC). Discrimination determines the ability of a capital to sort between groups of people, for example patients who live or die during hospital treatment.

 

 

This study was performed from January 2016 to December 2018. A total of 292 adults who underwent cardiac surgery including coronary artery bypass surgery (n = 218), heart valve surgery (n = 65), heart tumors surgery (n = 8), aorta surgery (n = 1), or a combination of these surgeries in the Department of Thoracic Cardiac and Vascular Surgery at Soetomo General Hospital were retrospectively reviewed. Of 292 patients, 213 patients met the inclusion criteria for this study, and 79 patients (27.0%) had incomplete medical records which met the exclusion criteria. Demographics and comorbidity are illustrated in Table 1.

 

 

In this study, the EuroSCORE II was used to estimate the mortality. The mortality rate was 19 out of 213 patients (8.9%). The observed and predicted mortality among different group of surgery is described in Table 2.

 

 

Table 3 shows the observed and predicted mortality in the quintiles of the EuroSCORE II. It can be inferred that the EuroSCORE II predicted the mortality in the risk group of 1.31–2.3%, whereas in the risk group of <1.3%, the score tended to overestimate the mortality rate.

 

 

Median (range) of EuroSCORE II was 1.03 (0.5–11.9). Furthermore, the data were classified using cut-off point (≤1.03 and >1.03). The p-value of the Hosmer–Lemeshow test was 0.55, which indicates good calibration. The AUC of the ROC (Figure 1) was 0.85 (95% confidence interval [CI] = 0.77–0.92, and p<0.001) for the total sample, suggesting a good discrimination.

 

 

 

 

Preoperative risk assessments have been widely discussed in several studies. After being introduced in the 1990s, the EuroSCORE was used to estimate mortality in patients with complex cardiac surgery. Several similar studies have been carried out to evaluate the accuracy of the EuroSCORE on a particular population. A study involving 621 patients from a cardiac surgery institution in Greece conducted EuroSCORE II of patients who underwent cardiac surgery with an observed mortality rate of 3%. The estimated mortality rate was 1.3%, and the ROC curve showed a high discrimination ability (AUC = 0.848, 95% CI = 0.75–0.94, and p<0.001).8 In India, a study involving 911 patients was conducted to assess the EuroSCORE II in relation to patients who underwent heart surgery with an observed mortality rate of 5.7%. The estimated mortality rate was 2.93%, and the ROC curve showed a high discriminatory ability (AUC = 0.76, 95% CI = 0.69–0.82).9 In other study that involved 4,034 patients from 20 cardiac surgery institutions in Spain, the EuroSCORE II estimated mortality rate of 5.7% in patients who underwent cardiac surgery with an observed mortality rate of 6.5%. The study showed a high discrimination ability (AUC = 0.97, 95% CI = 0.76–0.82), which shows the adequacy of EuroSCORE II as an assessment for the risk of cardiac surgery.10 Other study conducted in Asia, specifically China, in 2006–2011 by Zhang et al11 involved 3,479 patients undergoing heart valve surgery assessed using the EuroSCORE and EuroSCORE II. A mortality rate of 3.32% was observed, whereas the estimated mortality rate using the EuroSCORE was 3.33%. The estimated number using the EuroSCORE II was 2.52%. The discriminatory ability was found higher in EuroSCORE II (AUC = 0.792) than EuroSCORE.11

The EuroSCORE II, published in 2011 after the previous version, tends to overestimate mortality. A database was created by collecting data on 22,381 patients from 154 institutions in 43 countries. Although Zhang et al¹¹ conducted studies with Asian characteristics that were similar to Indonesian characteristics, none of the data originated from Indonesian institutions. It should be noted that demographics, patient profiles, and surgical situations and conditions are different in each country. The data from our study showed good calibration and discrimination of the EuroSCORE II which can be used as an established modality in terms of predicting the risk of in-hospital mortality after cardiac surgery. The use of this score influences the decision making of surgeons, especially in cases of high-risk cardiac surgery, to reduce the volume of high-risk surgery.12 Apart from the pros and cons, Bridgewater et al13 suggested that its use is limited to a consideration tool for surgeons and work units in the selection at the lower risk end. Decision-making must still be considered comprehensively by the heart team and the patient. Gogbashian et al¹² added that the surgeon’s professionalism and common sense in the assessment as well as other risk factors also need to be investigated further.

Several risk assessments tools, such as EuroSCORE, have been put forward in recent years as an addition to the risk assessment after heart surgery. However, the tool was formerly made based on certain databases which came from certain populations. We found that the EuroSCORE II is a good predictor of in-hospital mortality after cardiac surgery in Indonesian adults. Age is a significant predictor of mortality according to the EuroSCORE II.5 However, the average age in our study is lower than the age in the database of the EuroSCORE II.

In our center, only 11 urgent surgeries were performed with no emergency/salvage surgery due to the priority treatment of emergency patients which was measured by the hemodynamic stability at the intensive care unit. They thus underwent cardiac surgery the following day, resulting in urgent surgery.14 The limitation of this study includes the study population of only one cardiac surgery institution; therefore, this study cannot describe the entire population of patients who underwent cardiac surgery throughout Indonesia. Further research with a broader sample of patients involving more than one cardiac surgery institutions in Indonesia is suggested. In conclusion, the EuroSCORE II has a good calibration and discrimination in Indonesian adults. However, we found a significant under-prediction between predicted mortality and the actual observed mortality.

 

 

 

 

 

 

 

1. Peterson ED. Innovation and comparative-effectiveness research in cardiac surgery. N Engl J Med. 2009;361(19):1897–9.
2. Prins C, Jonker ID, Smit FE, Botes L. Cardiac surgery riskstratification models. Cardiovasc J Afr. 2012;23(3):160–4.
3. Ad N, Holmes SD, Patel J, Pritchard G, Shuman DJ, Halpin L. Comparison of EuroSCORE II, original EuroSCORE, and The Society of Thoracic Surgeons Risk Score in cardiac surgery patients. Ann Thorac Surg. 2016;102(2):573–9.
4. Siregar S, Groenwold RH, de Heer F, Bots ML, van der Graaf Y, van Herwerden LA. Performance of the original EuroSCORE. Eur J Cardiothorac Surg. 2012;41(4):746–54.
5. Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4):734–44; discussion 744–5.
6. Bai Y, Wang L, Guo Z, Chen Q, Jiang N, Dai J, et al. Performance of EuroSCORE II and SinoSCORE in Chinese patients undergoing coronary artery bypass grafting. Interact Cardiovasc Thorac Surg. 2016;23(5):733–9.
7. Lloyd-Jones D, Adams R, Carnethon M, De Simone G, Ferguson TB, Flegal K, et al. Heart disease and stroke statistics--2009 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation. 2009;119(3):e21–181.
8. Stavridis G, Panaretos D, Kadda O, Panagiotakos DB. Validation of the EuroSCORE II in a Greek cardiac surgical population: a prospective study. Open Cardiovasc Med J. 2017;11:94–101.
9. Kar P, Geeta K, Gopinath R, Durga P. Mortality prediction in Indian cardiac surgery patients: validation of European System for Cardiac Operative Risk Evaluation II. Indian J Anaesth. 2017;61(2):157–62.
10. Garcia-Valentin A, Mestres CA, Bernabeu E, Bahamonde JA, Martín I, Rueda C, et al. Validation and quality measurements for EuroSCORE and EuroSCORE II in the Spanish cardiac surgical population: a prospective, multicentre study. Eur J Cardiothorac Surg. 2016;49(2):399–405.
11. Zhang GX, Wang C, Wang L, Lu FL, Li BL, Han L, et al. Validation of EuroSCORE II in Chinese patients undergoing heart valve surgery. Heart Lung Circ. 2013;22(8):606–11.
12. Gogbashian A, Sedrakyan A, Treasure T. EuroSCORE: a systematic review of international performance. Eur J Cardiothorac Surg. 2004;25(5):695–700.
13. Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJ, et al. Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ. 2003;327(7405):13–7.
14. Axelsson TA, Mennander A, Malmberg M, Gunn J, Jeppsson A, Gudbjartsson T. Is emergency and salvage coronary artery bypass grafting justified? The Nordic Emergency/Salvage coronary artery bypass grafting study. Eur J Cardiothorac Surg. 2016;49(5):1451–6.

Copyright @ 2021 Authors. This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are properly cited. For commercial use of this work, please see our terms at http://mji.ui.ac.id/journal/index.php/mji/copyright.