Validation of prognostic scores for clinical outcomes in cirrhotic patients with acute variceal bleeding

Abstract

Background. The Rockall, Glasgow-Blatchford, and AIMS65 are useful and validated scoring systems for predicting the outcomes of patients with nonvariceal gastrointestinal bleeding. However, there are no validated evidence for using them to predict outcomes on variceal bleeding. The aim of this study was to evaluate and compare the prognostic accuracy of different nonvariceal bleeding scores with other liver-specific scoring systems in cirrhotic patients. Material and methods. A retrospective multicenter study that included 160 cirrhotic patients with acute variceal bleeding. The AUROC’s to predict in-hospital mortality, and rebleeding, were analyzed for each scoring system.

Results. Overall in-hospital mortality occurred in 13% and in-hospital rebleeding in 12% of patients. The systems with the best AUROC value for predicting mortality were MELD (0.828; 95% CI 0.748-0.909), and AIMS65 (0.817; 95% CI 0.724-0.909). The best score systems for predicting rebleeding were Glasgow-Blatchford (0.756; 95% CI 0.640-0.827), and Rockall (0.691; 95% CI 0.580-0.802).

Conclusions. In addition to liver-specific scores, the AIMS65 score is accurate for predicting in-hospital mortality in cirrhotic patients with acute variceal bleeding. Other scoring systems might be useful for predicting significant clinical outcomes in these patients.

Introduction

Acute variceal bleeding is one of the most dreadful complications of cirrhosis and occurs in 20-50% of patients. Despite the improvements in prognosis in the past decade, the mortality of patients with acute variceal bleeding remains high (24%) in cirrhotic patients.1

The Child-Pugh score and its components, bacterial infection, and renal failure are important predictors of short-term mortality in patients with acute variceal bleeding.2 A recent report showed that a Model for End-stage Liver Disease (MELD)-based model, including only objective variables, accurately predicted mortality in cirrhotic patients with acute variceal bleeding. MELD scores of ≥ 19 predicted a mortality rate of ≥ 20%, whereas scores of < 11 predicted a mortality rate of < 5%.3

There are other prognostic scores for assessing gastrointestinal bleeding, such as the Glasgow-Blatchford score (GBS), Rockall score, and AIMS65 score. However, most studies that have validated these scoring systems have focused on patients with nonvariceal gastrointestinal bleeding.4 These scores provide information about prognosis and help the clinician in medical decision-making. The cornerstone of predictive scores for gastrointestinal bleeding is using simple clinical and analytical data accurately in the emergency room to stratify those patients with a high risk for mortality, rebleeding, and need for early interventions.

The primary aim of this study was to compare the scores for the MELD, MELD-Sodium, Child-Pugh, GBS, Rockall, and AIMS65 systems to predict in-hospital mortality in cirrhotic patients with acute variceal bleeding. The secondary aim was to compare the accuracy of these scoring systems for predicting rebleeding.

Materials and Methods

Study cohort and data collection

We retrospectively included all cirrhotic patients admitted to the emergency room with acute variceal bleeding from three tertiary hospitals in Mexico City, Mexico. The hospitals and timeframes of patient inclusion were:

• •

  Medica Sur Clinic & Foundation from January 2008 to December 2012.

• •

  Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán from January 2009 to December 2012.

• •

  Hospital Juárez de México from January 2007 to January 2008.

All patients were older than 18 years. The diagnosis of cirrhosis was based on previous clinical history, liver biopsy, and/or unequivocal clinical data and compatible findings shown by imaging techniques. Esophageal variceal bleeding was confirmed by emergency endoscopy according to the Baveno Consensus Workshop criteria.5 The exclusion criteria were the presence of other causes of gastrointestinal bleeding (e.g., peptic or esophageal ulcer, vascular lesions, gastrointestinal neoplasia), hepatocellular carcinoma, infection documented at entry or suspicion of infection during the first 24 h after hospital admission, complete portal vein thrombosis, variceal bleeding in noncirrhotic patients, and incomplete data.

In all patients, clinical and analytical data were collected. Rescue therapies for rebleeding episodes (e.g., repeat endoscopic treatment or transjugular intrahepatic portosystemic shunt [TIPS]) and transfusion requirements were recorded. Follow-up findings during the admission and until discharge or death were recorded.

Treatments

All patients included in the study received antibiotic prophylaxis for 7 days. An initial endoscopy was performed within the first 48 h after admission, and endoscopic variceal ligation (EVL) was conducted in all cases. A Sengstaken-Blakemore tube was placed as a bridge to a repeat endoscopy when indicated. Vasoactive therapy was performed with octreotide or terlipressin and was started immediately after admission to the bleeding unit. Use of vasoactive drugs was limited due to cost and lack of availability in some centers.

Secondary prophylaxis was initiated in surviving patients passing the 5-day period. Oral propranolol was started at 40 mg/day and subsequently increased until intolerance appeared or the heart rate decreased to < 55 beats per minute or by ≥ 25% of the baseline heart rate. All patients underwent EVL at 2 to 4-week intervals until variceal eradication was achieved. In addition, patients who were on pharmacological therapy before bleeding had EVL added to the prophylactic regimen.

Definitions and scoring system calculations

Acute gastrointestinal bleeding from a variceal source was considered, if the initial endoscopy showed any of the signs of variceal hemorrhage according to the Baveno Consensus Workshop criteria.5 Uncontrolled bleeding was defined as any bleeding from esophageal varices that persisted despite intensive endoscopic treatment (i.e., > 2 therapeutic endoscopy procedures), balloon tamponade when indicated, TIPS insertion, or death. Rebleeding was defined as a new hematemesis or melena resulting in any of the following: hospital admission, blood transfusion, or 3 g drop in hemoglobin as defined in the Baveno Consensus Workshop criteria.5 In-hospital rebleeding was defined as the presence of hematemesis or melena with signs of hemodynamic instability during hospitalization. In-hospital mortality was defined as death resulting from any cause during the patient’s hospitalization. The criteria for hemodynamic instability included systolic blood pressure < 100 mmHg, heart rate > 100 bpm, and/or the presence of clinical signs of peripheral hypoperfusion. The criteria for bacterial infection included: fever > 37.5 °C for > 12 h; spontaneous bacterial peritonitis; ascitic fluid polymorphonuclear count ≥ 250/mm3; positive blood culture; urinary tract infection; and/or pneumonia on chest X-ray. Other infections were diagnosed according to clinical, radiological, and bacteriologic data.

The scores for the MELD, Child-Pugh, GBS, Rockall, and AIMS65 were calculated in each patient based on their clinical and laboratory features at admission as described on table 1.

Table 1.

Scoring systems calculations.

Score	Calculation
MELD	(0.957 × ln (PCr) + ln (PB) + 1.120 × ln (INR) + 0.643) × 10 if hemodialysis in the last 2 weeks.^1
Glasgow-Blatchford	• BUN (mg/dL): 6.5-7.9 (2 points), 8.0-9.9 (3 points), 10-25 (4 points), > 25 (6 points).
	• Hb in men (mg/L): 12-12.9 (1 point), 10-11.0 (3 points), < 10 (6 points).
	• Hb in women (mg/L): 10-11.9 (1 point), < 10 (6 points).
	• SBP (mmHg): 100-109 (1 point), 90-99 (2 points), < 90 (3 points).
	• HR >100 (1 point).
	• Presentation with melena (1 point).
	• Presentation with syncope (2 points).
	• Chronic HF failure (2 points).
	• Previous HF (2 points).^2
Rockall	• Age: < 60 (1 point), 60-79 (2 points), > 80 (3 points).
	• HR: < 100 (1 point), > 100 (2 points).
	• SBP (mmHg): > 100 (1 point), < 100 (3 points).
	• Previous comorbidity: none (1 point), cardiac disease (2 points), COPD (2 points), DM (2 points),
	CRF (3 points), cancer (3 points), cirrhosis (3 points).
	• Endoscopic signs of recent bleeding: no stigmata (1 point), fresh blood in stomach (3 points).^3
AIMS65	• P Alb (mg/dL): < 3.0 (1 point).
	• INR > 1.5 s (1 point).
	• GCS: < 14 (1 point).
	• SBP: < 90 (1 point).
	• Age > 65 years (1 point).^4

PCr: plasmatic creatinine. PB: plasmatic bilirubin. BUN: blood urea nitrogen. Hb: hemoglobin. SBP: systolic blood pressure. HR: hearth rate. HF: hearth failure. COPD: chronic obstructive pulmonary disease. DM: diabetes mellitus. CRF: chronic renal failure. P Alb: plasmatic albumin. GCS: Glasgow coma score.

1

Kamath PS, et al. A model to predict survival in patients with end-stage liver disease. Hepatology 2001; 33:464-70.

2

Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet 2000; 356: 1318-21.

3

Sanders DS, Carter MJ, Goodchap RJ, Cross SS, Gleeson DC, Lobo AJ. Prospective validation of the Rockall risk scoring system for upper GI hemorrhage in subgroups of patients with varices and peptic ulcers. Am J Gastroenterol 2002; 97: 630-5.

4

Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts inhospital mortality, length of stay, and cost in acute upper GI bleeding. Gastrointestinal Endoscopy 2011; 74: 1215-24.

Statistical analysis

Continuous variables were expressed as mean ± standard deviation, and were compared using Student’s t test or the Mann-Whitney U test, as appropriate, depending on the normality of their distribution. Differences between categorical variables were assessed by Fisher’s exact test or the χ2 test with Yeat’s correction for continuity, when necessary. A p value of < 0.05 was considered significant.

The area under the receiver-operating characteristic curve (AUROC) was calculated for each scoring system for validation purposes and comparative analysis of the accuracy of each for predicting each clinical outcome. Correlations between the different AUROC values for each outcome were performed using an estimated covariance matrix for nonparametric data to calculate the variability and to compare the differences between scoring systems.6 Calibration is how well the model tracks the outcome, it was assessed using the Hosmer-Lemeshow Goodnessof-Fit test and P < 0.05 was considered as well-calibrated.

Results

Patients’ features

During the inclusion period, 230 patients were enrolled and 70 were excluded. One hundred sixty cirrhotic patients with variceal bleeding were finally included in the study cohort (Figure 1). Sixty patients were from Hospital Juárez de México, 42 patients from Medica Sur Clinic & Foundation, and 58 patients from Instituto Nacional de Ciencias Médicas y Nutrición. The baseline demographic and clinical characteristics of patients at admission are shown in table 2.

Figure 1.

Flowchart of the enrolled patients in the study.

Table 2.

Clinical and demographic characteristics for all patients and comparison between patients who survived and those who died.

	Total (n = 160)	Survived (n = 139)	Died (n = 21)	p value
Male	88 (55)	75 (53.9)	13 (61.9)	0.6390
Age (years)	54.1±12.6	54.3±12.4	53.4±14.6	0.7678
Glasgow-Blatchford	10.8±3.7	10.4±3.81	3.5±2.2	0.0005
Rockall	5.7±1.3	5.5±1.3	6.6±1.2	0.0006
AIMS65	1.5±1.1	1.3±1.0	2.7±1.0	0.0001
Child-Pugh	8.2±2.0	7.9±1.9	10.0±2.1	0.0001
MELD	15.2±6.5	14.1±5.8	22.2±6.9	0.0001
Melena	131 (81.8)	49 (35.2)	15 (71.4)	0.0032
Ascites	68 (42.5)	116 (83.4)	14 (66.6)	0.0768
Encephalopathy	66 (41.2)	23 (16.5)	17 (80.9)	0.0001
Rebleeding	19 (11.8)	6 (4.3)	13 (61.9)	0.0001
Hospital stay (days)	3.4±5.4	2.5±3.8	8.7±9.9	0.0001
Cancer history	11 (6.8)	7 (5.0)	4 (19.0)	0.0396
Systolic pressure (mm/Hg)	109±23	111±22	90±21	0.0001
Diastolic pressure (mmHg)	64±14	66±13	51±15	0.0001
Hemoglobin (g/dL)	8.9±2.7	9.0±2.7	8.7±2.1	0.6489
Creatinine (mg/dL)	1.1 ±0.5	1.0±0.4	1.76±0.6	0.0001
Albumin (g/dL)	2.8±0.6	2.8±0.6	2.6±0.6	0.0799
INR (seconds)	1.5±0.5	1.4±0.5	1.7±0.4	0.0324
Sodium (mEq/L)	136.7±5.5	137.4±4.4	131.4±8.5	0.0001
Potassium (mEq/L)	4.3±0.8	4.2±0.7	4.7±1.1	0.0119
RBCT (patients)	80 (50)	61 (43.8)	19 (90.4)	0.0001
RBCT (units)	1.4±1.9	2.5±1.4	3.6±2.9	0.0048
Vasoactive drug use	46 (28.7)	30 (21.5)	16 (76.1)	0.0001

RBCT: red blood cell transfusion. Data are expressed as mean ± SD or n (%).

The most frequent etiologies of cirrhosis were hepatitis C virus infection (n = 47) and alcoholic liver disease (n = 47), both representing 58.7% of all causes, followed by cryptogenic cirrhosis (n = 30, 18.7%). Thirty-seven patients (23%) showed renal impairment upon arrival at the hospital as shown by a serum creatinine concentration > 1.5 mg/dL. Overt hepatic encephalopathy (grade > 2) was present in 66 patients (41%). One hundred thirty-one patients (82%) presented with melena at admission, and 68 patients (42%) had clinically detected ascites. The overall mean in-hospital stay was 3.4 ± 5.4 days.

Clinical outcomes

Overall in-hospital mortality was 13%. All three centers showed comparable in-hospital mortality rates: Hospital Juárez de Mexico with 13% mortality (n = 8), Medica Sur Clinic & Foundation with 14% (n = 6), and Instituto Nacional de Ciencias Médicas y Nutrición with 12% mortality (n = 7) (p = 0.947). The presence of rebleeding during hospitalization was 12%. This rate did not differ between the participant centers: Hospital Juárez de México with 13% (n = 8); Medica Sur Clinic & Foundation with 14% (n = 6); and Instituto Nacional de Ciencias Médicas y Nutrición with 9% (n = 5) (p = 0.62).

Eighty patients (50%) were transfused. The mean number of red blood cell packages used in each transfusion was 1.4 ± 1.9. Vasoactive drugs were used in 49 patients (30%) during hospitalization: terlipressin in 27; octreotide in 10; norepinephrine in eight; and vasopressin in four. Ninety five percent of endoscopic procedures were done before 24 h (mean time of 6 h).

Comparison of scoring systems for predicting outcomes

AUROC analysis was performed to study the accuracy for prediction of in-hospital mortality for the liver-specific and gastrointestinal bleeding scoring systems (Table 3). The higher AUROC values for predicting in-hospital mortality were those for MELD (0.828; 95% CI 0.748-0.909; Hosmer-Lemeshow test P = 0.543), and AIMS65 (0.817; 95% CI 0.724-0.909; Hosmer-Lemeshow test P = 0.851). The best cutoff values for predicting in-hospital mortality were MELD 13 points (sensitivity 95.2%, specificity 53.2%), and AIMS65 ≥ 1 point (sensitivity 85.7%, specificity 57%) (Table 4).

Table 3.

AUROC analysis of the prognostic scales for all outcomes.

Outcome	Scale	AUROC	95% CI
In-hospital mortality	MELD	0.828	0.748-0.909
	Child-Pugh	0.760	0.645-0.874
	AIMS65	0.817	0.724-0.909
	Glasgow-Blatchford	0.749	0.655-0.842
	Rockall	0.721	0.616-0.827
In-hospital rebleeding	MELD	0.677	0.563-0.792
	Child-Pugh	0.581	0.451-0.711
	AIMS65	0.664	0.532-0.795
	Glasgow-Blatchford	0.756	0.640-0.872
	Rockall	0.691	0.580-0.802

Table 4.

Diagnostic values AIM65 and MELD for mortality.

Score	Cut-off	Sensitivity	Specificity	PPV	NPV	LR+	LR−
AIMS65	≥ 1	85.7%	57%	0.22	0.96	1.99	0.25
MELD	≥13	95.2%	53.2%	0.23	0.76	2.03	0.09

PPV: positive predictive value. NPV: negative predictive value. PLR: positive likelihood ratio. NLR: negative likelihood ratio.

Finally, we analyzed the accuracy of all scoring systems for predicting in-hospital rebleeding (Table 3). The GBS has higher AUROC for predicting in-hospital rebleeding (0.756; 95% CI 0.640-0.827; Hosmer-Lemeshow test P = 0.218), followed by Rockall score (0.691; 95% CI 0.580-0.802; Hosmer-Lemeshow test 0.477).

We performed a sensitivity analysis comparing the group using the vasoactive drugs in order to evaluate if the result was similar if the patients received the adequate therapy. Area under the curve was similar in all scores except for GBS which has to be carefully interpreted.

Discussion

One of the most difficult challenges for clinicians when treating patients with acute variceal bleeding is reducing the risk of mortality. Most interventions to achieve this aim were applied systematically in all patients. However, risk classifications that could help the physician to predict the risk of death a priori and to prioritize clinical care for high-risk patients are underexplored. The ability to detect high-risk patients will be useful for developing new strategies to reduce mortality and to identify those candidates suitable for salvage measures.

In this retrospective multicenter study, we confirmed the utility of MELD for predicting short-term mortality in cirrhotic patients with acute variceal bleeding.3 Rather than trying to develop a new prognostic score, we compared the current scoring systems to explore whether these validated systems, developed for nonvariceal bleeding, could help in the stratification of high-risk patients with variceal bleeding. We found no differences in the accuracy between MELD and AIMS65 for predicting in-hospital mortality, both of them with good calibration. This is an interesting point given that the latter non-liver-specific score includes both data from other liver-specific scores (e.g., serum albumin concentration and INR) and other biologically significant variables (e.g., mental status, blood pressure, and age), which are not considered in the MELD and Child-Pugh scores. AIMS65 is based on objective and well-validated variables and is easy to use in a clinical setting because specific calculations are not needed. This approach has been previously tested1 and that report found lower accuracy for the GBS and Rockall, with AUROC values of 0.527 and 0.591, respectively. These differences underline the need for a more extensive external validation of this approach before its widespread clinical use. One important difference regarding AIMS65 is the cut-off point ≥ 1 instead of> 2 reported on original studies.

Data about the accuracy of nonliver-specific scoring systems to predict in-hospital rebleeding in cirrhotic patients with acute variceal bleeding are limited.7–9 We found no differences in the accuracy for the prediction of in-hospital rebleeding between the AIMS65 and Rockall scores with other liver-specific scoring systems. Interestingly, the GBS showed better predictive accuracy than the Child-Pugh score in a side-by-side comparison of their AUROC values (Figure 2). Further studies are needed to clarify the best scoring system for this clinical outcome in cirrhotic patients with acute variceal bleeding.

Figure 2.

AUROC values. A. AUROC for mortality. B. AUROC for rebleeding. P-value < 0.05 vs. MELD-Na for mortality. ** P-Value < 0.05 vs. Child-Pugh Score for rebleeding.

Finally, in addition to the ability of liver-specific and nonliver-specific scores to predict mortality in patients with acute variceal bleeding, critical care scoring systems have showed high accuracy for predicting in-hospital mortality in this setting.10 A comparative analysis with these critical care scores was beyond the objectives of the present study. There is already a variety of scores to assess the mortality risk of cirrhotic patients with acute variceal bleeding. We propose that the choice should be a rational selection based on the accuracy, validity, and the ease of use at the bedside.

We acknowledge that our study has limitations. First, it was a retrospective design, and some data was not available or incomplete, therefore information from Hospital Juárez was included only for one year. The second limitation is the lack of use of vasoactive drugs in all included patients because of the availability in the different centers, hence, the results cannot be extrapolated to patients receiving pharmacological standard of care. Nevertheless, we performed a sensitivity analysis and found this is not an important implication. Third is the lack of full data for comparative analysis with critical care scoring systems. This last point is interesting and deserves further investigation. Lastly, the accuracy is just enough only to predict in-hospital mortality, more data is necessary for other outcomes.

Finally it is important bring the discussion about the significant outcomes in the management of variceal bleeding. We decided to include in-hospital mortality, and all scores show better performance for this outcome, even nonvariceal scores. Mortality is a complex and multifactorial outcome, but must of other outcomes are designed for clinical trials and not for clinical practice.11 We observe a lower performance of nonvariceal scores limitating their use in clinical trials. However, the recent validation of outcomes is a great advance for better trial design, and the future discussion about consensus of hard outcomes will be seen in the future.12,13

In conclusion, additionally to liver-specific scores, gastrointestinal bleeding scoring systems are useful for predicting clinical outcomes of acute variceal bleeding. The AIMS65 is particularly accurate for predicting in-hospital mortality in this setting. Further prospective and multicentric studies are necessary to confirm our data.

Abbreviations

• •

  AUROC: area under the receiver operating characteristic curve.

• •

  EVL: endoscopic variceal ligation.

• •

  GBS: Glasgow-Blatchford score.

• •

  MELD: Model for end stage liver disease.

• •

  TIPS: transjugular intrahepatic portosystemic shunt.

Grants

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Acknowledgments

The authors declare that they have no conflict of interests.

The study was performed in accordance with the declaration of Helsinki, and was approved by the ethical committee at Medica Sur Clinic and Foundation.

NCT and MMK designed the protocol, MMK, AEA performed literature research and searched the clinical data, FTA, JA, AGA and FZD collaborated to data collection, NCT, MMK and JA analyzed data, and NCT, JA, MMK, NAO and MU wrote, reviewed and edited the anuscript. NCT, MMK, AEA, FTA, JA, AGA, FZD, NAO and MU read and approve the manuscript.