APORTEI score is a new risk prediction model for patients with infective endocarditis. It has been recently validated on a Spanish multicentric national cohort of patients. The aim of the present study is to compare APORTEI performances with logistic EuroSCORE and EuroSCORE II by testing calibration and discrimination on a local sample population underwent cardiac surgery because of endocarditis.
MethodsWe tested three prediction scores on 111 patients underwent surgery from 2014 to 2020 at our Institution because of infective endocarditis. Area under the curves and Hosmer–Lemeshow test were used to analyze discrimination and calibration respectively of logistic EuroSCORE, EuroSCORE II and APORTEI score.
ResultsThe overall observed one-month mortality rate was 21.6%. The observed-to-expected ratio was 1.27 for logistic EuroSCORE, 3.27 for EuroSCORE II and 0.94 for APORTEI. The area under the curve (AUC) value of APORTEI (0.88±0.05) was significantly higher than that one of logistic EuroSCORE (AUC 0.77±0.05; p 0.0001) and of EuroSCORE II (AUC 0.74±0.05; p 0.0005). Hosmer–Lemeshow test showed better calibration performance of the APORTEI, (logistic EuroSCORE: p 0.19; EuroSCORE II: p 0.11; APORTEI: p 0.56).
ConclusionAPORTEI risk score shows significantly higher performances in term of discrimination and calibration compared with both logistic EuroSCORE and EuroSCORE II.
El APORTEI score es un nuevo sistema de predicción de riesgo para pacientes con endocarditis infecciosa. El mismo ha sido recientemente validado en una cohorte de pacientes procedentes de un estudio nacional multicéntrico español. El objetivo del presente estudio es comparar la discriminación y la calibración del APORTEI score con las del EuroSCORE logístico y del EuroSCORE II analizando una población sometida a cirugía cardiaca por endocarditis infecciosa.
MétodosAnalizamos las propiedades de 3 sistemas de predicción de riesgo sobre una población de 111 pacientes sometidos a cirugía cardiaca desde 2014 hasta 2020 en un único hospital terciario por endocarditis infecciosa. El área bajo las curvas y la prueba de Hosmer-Lemeshow se usaron para analizar la discriminación y la calibración, respectivamente, del EuroSCORE logístico, del EuroSCORE II y del APORTEI score.
ResultadosLa mortalidad global observada a un mes fue del 21,6%. La relación mortalidad observada/mortalidad esperada fue de 1,27 para el EuroSCORE logístico, 3,27 para el EuroSCORE II y 0,94 para el APORTEI score. El valor del área bajo la curva (AUC) del APORTEI score (0,88±0,05) fue significativamente mayor que los del EuroSCORE logístico (AUC 0,77±0,05; p 0,0001) y del EuroSCORE II (AUC 0,74±0,05; p 0,0005). La prueba de Hosmer-Lemeshow mostró un mejor rendimiento en cuanto a calibración del APORTEI score, (EuroSCORE logístico: p 0,19; EuroSCORE II: p 0,11; APORTEI score: p 0,56).
ConclusiónEl sistema de predicción de riesgo APORTEI score muestra un rendimiento significativamente mejor en cuanto a discriminación y calibración en comparación con el EuroSCORE logístico y con el EuroSCORE II.
Infective endocarditis (IE), especially in surgical setting, represents a complex condition associated with high early and long-term mortality and morbidity.1 In fact, surgical mortality in patients with IE ranges from 15% to 45%.2 To approach the IE complexity in terms of diagnosis and therapeutic strategies, building of multidisciplinary ad-hoc teams and use of statistical models for prognosis prediction have been extensively advocated3.
These are, in most cases, the product of sophisticated statistical models obtained from specific patients-based datasets such as logistic EuroSCORE4 and EuroSCORE II.5
Still, several concerns have been raised about the discrimination performances of risk models obtained from dataset where patients with IE are poorly represented.6,7
Recently, a new prediction model tool, APORTEI score, obtained from an extensive meta-analysis, has been reported to provide a precise and easy tool to stratify mortality risk in patients underwent surgery because of IE.8,9
The aim of the present study is to analyze discrimination and calibration proprieties of the three risk scores, logistic EuroSCORE, EuroSCORE II and APORTEI score, over a sample population undergoing cardiac surgery at our Institution because of IE.
MethodsThe sample population was constituted by 111 patients underwent consecutively cardiac surgery because of infective endocarditis from January 2014 to March 2020 at our Institution, a tertiary-level hospital.
Data were collected prospectively through our local electronic database. Logistic EuroSCORE and EuroSCORE II were prospectively calculated using the specific online calculators (http://www.euroscore.org/calcsp.html). The APORTEI score was retrospectively calculated for each patient using excel sheet published, as supplementary material, by the score developers9 (https://academic.oup.com/ejcts/article/57/4/724/5647329#supplementary-data). Continuous variables following a normal distribution were expressed as mean±standard deviation (SD). Continuous variables not normally distributed were expressed as median (25th–75th percentiles). Dichotomous categorical variables were indicated as absolute frequency (percentage). Predicted surgical mortality was obtained for each score. Spearman rank correlation analyses were carried out between each score. Discrimination was analyzed by receive operating characteristic analyses from which area under the curve (AUC)±standard error (SE) values were obtained. The method of DeLong10 was used for the calculation of the standard error (SE) of the AUC and of the difference between two AUCs. Univariable logistic regression analysis was built introducing “30-day mortality” as dependent variable and separately each of the three scores as independent variable. From this model the Hosmer–Lemeshow test was obtained. The Hosmer–Lemeshow method, as test for goodness of fit for the three logistic regression models, was used to analyze the calibration of the scores. Statistical analyses were performed using MedCalc Statistical Software version 19.0.7 (MedCalc Software bvba, Ostend, Belgium; https://www.medcalc.org; 2019).
ResultsPatients characteristics are listed in Table 1. Mean age was 58.9 years. Most frequent causative microorganism were Staphylococcus aureus (25.2%, n=28), Staphylococcus epidermidis (14.4%, n=16), and Streptococcus viridans group (11.7%, n=13). The aortic valve was the most frequent localization (43.2%, n=48), (Table 1).
Pre and intra operative patients data of the whole sample population.
Patients characteristics | All sample population (N=111) |
---|---|
Female | 30 (27.0%) |
Age | 58.9 (±13.7) |
Body mass index | 26.5 (± 4.9) |
Diabetes mellitus | 24 (21.6%) |
Chronic lung disease | 16 (14.4%) |
Arterial hypertension | 40 (36.0%) |
Preoperative inotropic treatment | 21 (18.9%) |
Pulmonary hypertension (>60mmHg) | 15 (13.5%) |
Congestive heart failure | 54 (48.6%) |
Invasive ventilation | 25 (22.5%) |
Left ventricle ejection fraction<49% | 12 (10.8%) |
Cardiogenic or septic shock | 29 (26.1%) |
Emergency surgery | 32 (28.8%) |
Prosthetics endocarditis | 30 (27.0%) |
Peri annular abscess | 27 (24.3%) |
Nosocomial endocarditis | 16 (14.4%) |
Staphylococcus Aureus | 21 (22.6%) |
Logistic EuroSCORE | 17.1 (7.8–41.7) |
EuroSCORE II | 6.8 (3.2–24.5) |
APORTEI score | 46.4 (30.6–70.0) |
APORTEI predicted mortality | 23.0 (13–39.8) |
Location of endocarditis | |
Aortic valve | 48 (43.2%) |
Mitral valve | 32 (28.8%) |
Aortic and mitral valves | 23 (20.7%) |
Mitral and tricuspid valves | 2 (1.8%) |
Pulmonary valve | 2 (1.8%) |
Interventricular septum | 1 (0.9%) |
Tricuspid valve | 1 (0.9%) |
Aortic, mitral and pulmonary valves | 1 (0.9%) |
Aortic, mitral and tricuspid valves | 1 (0.9%) |
Thirty-day mortality was 21.6% (n=24). Predicted surgical mortality by logistic EuroSCORE, EuroSCORE II and APORTEI score were 17.1%, 6.6% and 23.0%, respectively.
Therefore, the observed-to-expected ratio was 1.27 for logistic EuroSCORE, 3.27 for EuroSCORE II and 0.94 for APORTEI score.
According to Spearman's rank correlation analysis the highest coefficient of correlation was observed between logistic EuroSCORE and EuroSCORE II (rho 0.88, p<0.0001). Still, significant correlation was documented between logistic EuroSCORE and APORTEI score (rho 0.58, p<0.0001) and between EuroSCORE II and APORTEI score (rho 0.63, p<0.0001). According to the receive operating characteristic analyses, AUC value of APORTEI score was 0.88±0.05 which was significantly higher than those of logistic EuroSCORE (AUC 0.77±0.05; difference between areas 0.17; p 0.0001) and EuroSCORE II (AUC 0.74±0.05; difference between areas 0.11; p 0.0005) (Fig. 1).
Hosmer–Lemeshow test of the regression logistic analysis showed better calibration performance of the APORTEI score, (logistic EuroSCORE: chi-squared 12.30, p value: 0.19; EuroSCORE II: chi-squared 12.98, p value: 0.11; APORTEI: chi-squared 6.77; p value: 0.56).
DiscussionThe present study shows higher discrimination and calibration proprieties of APORTEI score compared with logistic EuroSCORE and EuroSCORE II. The three scores were tested on a sample population of 111 patients underwent surgery because of IE.
In the last decade, several risk scores have been built to predict patients’ outcomes after cardiac surgery.8,11 Risk models have been used also to identify specific risk factors, to allow comparison of postoperative results and to carry out risk stratification analysis.12
Patients undergoing cardiac surgery because of infective endocarditis represents a complex subgroup of patients, with higher surgical mortality than that one of the general populations undergoing cardiac surgery and specific risk factors.13–15 For this reason, several concerns have been raised about the proprieties of logistic EuroSCORE and EuroSCORE II when applied to IE populations.6,7 In fact, EuroSCORE I prediction models4 have been built from a database containing 19,030 patients underwent cardiac surgery, but, of these, only 1.1% (173) suffered from endocarditis. EuroSCORE II5 was built on a database of 22,381 patients undergoing major cardiac surgery. Of these only 2.2% (497) presented with endocarditis. In both cases prediction scores were obtained by logistic modellings.
APORTEI score building process lied on a different methodology as it was not computed from a specific database of patients. Instead, it was obtained by meta-analysis data consisted on 16 studies and 7484 patients with IE.8 Through overall pooled odds ratio (OR) calculation, 11 risk factors were identified and posteriorly tested on a national Spanish multicenter database containing 1338 patients with IE.9
Our results show clearly than logistic EuroSCORE and especially EuroSCORE II underestimates surgical mortality risk in patients with endocarditis. Also, receive operating characteristic analyses demonstrated that APORTEI score has the largest AUC value and, hence, the best performance in term of discrimination.
Even if the present study was carried out on a small sample population, its results confirm those of a previous study9 in terms of better performance of the APORTEI score compared with logistic EuroSCORE. Also, our analysis shows that the APORTEI score has higher discrimination and calibration proprieties even when compared with EuroSCORE II as prognostic and risk stratification tool for patients undergoing surgery because of IE.
FundingThis research received no funding.
Conflict of interestNone.
The present authors sincerely thank all the remaining members of the Equipo Multidisciplinar de Endocarditis (EME) of the Hospital Universitario Dr. Negrín:
Eloy Gomez, Oscar Sanz, Rafael Granados, Xerach Bosh (Unidad de Enfermedades Infecciosas), Ana Bordes, Raúl Gilarranz (Microbiología), José Luis Romero, Adela Hernández, Sílvia Hipola, José María Gómez, (Medicina Intensiva), Verónica Quevedo, Marta Blanco, Irene Menduiña (Cardiología), Luis Ríos, Raquel Bellot (Cirugía Cardíaca).