Diagnostic performance of three non-invasive fibrosis scores (Hepamet, FIB-4, NAFLD fibrosis score) in NAFLD patients from a mixed Latin American population

Abstract

Introduction and aims

Several non-invasive scoring systems have been developed and validated worldwide to predict the risk of liver fibrosis in nonalcoholic fatty liver disease (NAFLD). However, information about the performance of these systems in Latin American populations is scarce. Our aim was to evaluate the performance of the Hepamet Fibrosis Score, Fibrosis-4 (FIB-4) and the NAFLD Fibrosis Score (NFS) in a mixed Latin American group of NAFLD patients.

Methods

Clinical, laboratory and liver biopsy data collected from 379 biopsy-proven NAFLD patients from Latin American tertiary health centers were reviewed. Histological fibrosis stages were classified using the Kleiner score. Accuracy was determined, and new fibrosis score thresholds were calculated to better compare the performances of non-invasive tests and to explore their usefulness in excluding fibrosis.

Results

The distribution of fibrosis stages among the sample population was as follows: F0 (45%), F1 (27%), F2 (8%), F3 (16%) and F4 (4%). Using modified thresholds, the areas under the ROC curves (AUROC) for Hepamet and FIB-4 (0.73 and 0.74, respectively) to detect significant fibrosis were higher than that of NFS (0.58). However, the AUROCs of the three scores were not significantly different in advanced fibrosis and cirrhosis. To exclude fibrosis, we calculated lower cutoffs than standard thresholds for Hepamet, FIB-4 and NFS with similar performances.

Conclusion

Thresholds of non-invasive fibrosis scores (Hepamet, FIB-4 and NFS) can be modified to maximize diagnostic accuracy in Latin American patients with NAFLD.

1

Introduction

Non-alcoholic fatty liver disease (NAFLD) is a pathological condition characterized by inflammatory fatty liver in the absence of secondary causes of steatosis such as alcohol consumption, drugs or other metabolic disorders [1,2]. Being an emerging disease with an epidemic status, NAFLD is now the main cause of chronic liver disease in Western countries and will probably overtake viral hepatitis as the world’s leading motive for liver transplants in a few decades [2].

As the liver biopsy remains the gold standard for NAFLD diagnosis, non-invasive scores are currently used to select high-risk patients for performing liver biopsy and patient monitoring [2,3]. Therefore, previous studies have focused on the performance of tests in advanced fibrosis [1,2], usually concluding that they are more effective at ruling out than confirming fibrosis [1–4]. Furthermore, the cutoff values of the different non-invasive scores are based on distinct criteria, making their results more difficult to interpret and to apply in clinical practice.

Several scores have been validated in different studies. Fibrosis-4 (FIB-4) and the NAFLD Fibrosis Score (NFS) are among the most widely validated and recommended tests given their low cost and accessibility in routine clinical practice [5–8]. These scores have been demonstrated to be most effective in identifying advanced fibrosis (F3-F4) and ruling out significant fibrosis in NAFLD patients [2,5–8]. Nevertheless, such methods are influenced by age and body mass index (BMI), and perform poorly within obese patients [9], patients under 35 or over 65 years of age [10], and patients with type 2 diabetes mellitus (T2DM) in the case of NFS [11]. In these situations, adjusted thresholds must be used with cautious interpretation of the results. To overcome these obstacles, a new non-invasive fibrosis score (i.e. Hepamet fibrosis score) was developed and recently validated with a population of 2452 NAFLD patients from diverse medical centers (Spain, Italy, France, Cuba, and China). The Hepamet fibrosis score incorporates demographic, anthropometric, and simple laboratory parameters to predict advanced fibrosis and cirrhosis. It seems to have improved accuracy and fewer undetermined results compared to FIB-4 and NFS [12], with a test performance that is not affected by age or BMI [12,13].

As Latin America populations have highly heterogeneous genetic backgrounds [14], it is interesting to study possible singularities of fibrosis tests results with our population. Our aim was to evaluate the performance of three non-invasive fibrosis scores (Hepamet, FIB-4 and NFS) among a mixed Latin American population with NAFLD, and to find an optimal cutoff to rule out fibrosis in this sample.

2

Methods

2.1

Selection of patients

We conducted an international multicenter cross-sectional retrospective study of 379 biopsy-proven NAFLD patients from three countries (Peru, Brazil and Argentina). Data was collected between 2005 and 2018.

The inclusion criteria were: NAFLD patients between 18 and 75 years old, with biopsy-proven NAFLD. The exclusion criteria were: 1) presence of other causes of chronic liver disease, such as schistosomiasis, viral hepatitis, drug-induced fatty liver, autoimmune hepatitis, Wilson’s disease, alpha-1-antitrypsin deficiency, and hemochromatosis; 2) alcohol intake > 100 g ethanol/ week; 3) Absence of clinical and/or laboratory data required to calculate the scores; 4) Liver biopsy samples measuring less than 15 mm long or having less than 10 portal tracts. The study was performed in agreement with the Declaration of Helsinki and with local and national laws.

2.2

Histologic evaluation

All selected patients had been evaluated by liver biopsy before their inclusion in the study, indicated as part of the disease management only. Liver specimens were assessed by experienced pathologists. Fibrosis parameters were scored independently of other pathological findings, using the scoring system developed by Kleiner [15]: F0 (no fibrosis), F1 (perisinusoidal or periportal fibrosis), F2 (perisinusoidal and periportal fibrosis), F3 (septal or bridging fibrosis) and F4 (cirrhosis). We considered significant fibrosis as ≥ F2, advanced fibrosis as ≥ F3 and cirrhosis as F4 for statistical purposes.

2.3

Calculation of noninvasive fibrosis scores

Clinical and laboratorial data were collected in the interval between three months before and three months after the liver biopsy. We calculated three non-invasive fibrosis scores: Hepamet, FIB-4 and NFS. NFS and FIB-4 score were calculated using the original formulas [5,7]. NFS: -1.675 + 0.037 × age (years) + 0.094 × BMI (kg/m2) + 1.13 × diabetes (yes = 1, no = 0) + 0.99 × AST/ALT ratio - 0.013 × platelet count (x109/L) - 0.66 × albumin (g/dL). FIB-4 score: age (years) × AST (U/L) / platelets (109/L) x √ALT (U/L). The Hepamet score was calculated using a free online application (https://www.hepamet-fibrosis-score.eu/) [12].

2.4

Cutoff calculation

The Youden Index was used to determine the optimal upper cutoff, and the lower cutoff was calculated to identify 90% sensitivity to have fewer false negatives.

2.5

Statistical analysis

Statistical analysis was performed using SPSS Version 25 and Graph Pad Prism software. Continuous variables were expressed as mean ± standard deviation (SD). Qualitative data were presented as numbers with percentages. Normally distributed continuous variables were compared with a paired t-test. Fisher’s exact test and the Chi-square test were used with Yates' correction test for qualitative data.

Overall diagnostic accuracy was evaluated by determining the area under the ROC curve (AUROC). The effect of the fibrosis stage distribution on test performances was explored by calculating the DANA (Difference between advanced and non‐advanced fibrosis) [3,16].

Diagnostic performances were determined by sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Two-tailed P values < 0.05 were considered statistically significant. ROC curves were compared according to Hanley and McNeil’s method and the performances of the three scores were compared using the McNemar test [17].

3

Results

Three hundred and seventy-nine patients were included. Baseline characteristics of the population are summarized in Table 1.

Table 1.

Clinical, laboratory and histological aspects of the studied population.

Characteristic	N = 379
Age (years)	45.8 ± 13.3
Male	113 (30%)
Female	266 (70%)
Hypertension	225 (59%)
T2DM	189 (50%)
Dyslipidemia	301 (79%)
Metabolic syndrome	241 (88%)
BMI (kg/m^2)	35.3 ± 6.4
Platelet count (x10^9/L)	270.6 ± 72.4
AST (IU/mL)	37.26 ± 29.9
ALT (IU/mL)	51.70 ± 43.28
Glucose (mg/dL)	102.8 ± 31.42
Albumin (g/dL)	4.47 ± 0.38
HOMA-IR	5.8 ± 4.5
NFS score	−0.41 ± 1.3
Hepamet score	0.13 ± 0.18
FIB-4 score	1.04 ± 1.22
Fibrosis stage in liver biopsy (%)
F0: no fibrosis	172 (45%)
F1: perisinusoidal or periportal fibrosis	101 (27%)
F2: perisinusoidal and periportal fibrosis	29 (8%)
F3: septal or bridging fibrosis	62 (16%)
F4: cirrhosis	15 (4%)

Values are n (%) or mean ± SD. T2DM, Type 2 diabetes mellitus; BMI, body mass index; AST, aspartate aminotransferase; ALT, alanine aminotransferase; HOMA-IR, homeostatic model assessment for insulin resistance; NFS, NAFLD Fibrosis Score.

Patients with significant fibrosis (≥ F2) had different clinical and laboratory features from those of patients with F0 and F1, such as higher mean age, higher prevalence of T2DM and higher aminotransferase levels and lower platelet counts (Table 2).

Table 2.

Clinical and laboratory features according to the presence of significant fibrosis in liver biopsy.

	Fibrosis stage F < 2 (N = 273)	Fibrosis stage F ≥ 2 (N = 106)	P value
Age (mean ± SM)	44.1 ± 13.4	49.9 ± 12.2	0.0014*
Male (%)	78 (29)	35 (33)	0.4530^†
Female (%)	195 (71)	71 (67)
Hypertension (%)	166 (61)	59 (56)	0.4148^†
T2DM (%)	120 (44)	69 (65)	0.0002^†
Dyslipidemia (%)	222 (81)	79 (75)	0.1576^†
Metabolic syndrome (%)	167 (61)	74 (70)	0.1237^†
BMI (kg/m^2)	35.7 ± 5.9	34.3 ± 7.4	0.1212*
Platelet count (x10^9/L)	282.9 ± 69.2	238.9 ± 70.9	0.0153*
AST (IU/mL)	31.4 ± 22.1	52.4 ± 40.6	< 0.0001*
ALT (IU/mL)	45.5 ± 38.5	67.6 ± 50.6	< 0.0001*
Glucose (mg/dL)	99.4 ± 29.5	111.7 ± 34.6	0.8917*
Albumin (g/dL)	4.5 ± 0.4	4.4 ± 0.4	0.0001*
HOMA-IR	5.5 ± 4.2	6.8 ± 4.9	0.1288*
NFS	−0.52 ± 1.3	0.13 ± 1.4	< 0.0001*
Hepamet	0.08 ± 0.11	0.24 ± 0.25	< 0.0001*
FIB-4	0.80 ± 0.47	1.65 ± 2.07	0.0041*

Values are n (%) or mean ± SD. T2DM, Type 2 diabetes mellitus; BMI, body mass index; AST, aspartate aminotransferase; ALT, alanine aminotransferase; HOMA-IR, homeostatic model assessment for insulin resistance; NFS, NAFLD Fibrosis Score. * Paired T test; † Fisher's exact test.

ROC curves to detect significant fibrosis, advanced fibrosis and cirrhosis are shown in Fig. 1 for visual comparison, while the detailed test performances are displayed in Table 3.

Fig. 1.

ROC curves for significant fibrosis(A), advanced fibrosis (B) and Cirrhosis.

Table 3.

Area under the ROC curves of each test according to the stage of fibrosis.

	AUROC	Standard deviation	Two-sided p
Significant Fibrosis (F2-F4)
NFS	0.581	0.033
Hepamet	0.725	0.030	0.002*
FIB-4	0.744	0.029	0.659
Advanced Fibrosis (F3-F4)
NFS	0.646	0.036	0.267
Hepamet	0.691	0.036
FIB-4	0.703	0.036
Cirrhosis (F4)
NFS	0.737	0.058	0.153
Hepamet	0.809	0.057
FIB-4	0.873	0.033

AUROC, area under ROC curve; NFS, NAFLD Fibrosis Score. P calculated by Hanley and McNeil method. *Statistical significance.

We found higher Hepamet and FIB-4 AUROCs for significant fibrosis (0.73 and 0.74, respectively) than that of the NFS (0.58), with p < 0.05 (95% confidence interval), while the Hepamet and FIB-4 AUROCs for advanced fibrosis (0.69 and 0.70, respectively) were slightly higher but not significantly different than that of the NFS (0.65). There were no statistical differences among the Hepamet, FIB-4 and NFS AUROCs for cirrhosis.

In order to prevent spectrum bias, we calculated the DANA (Difference between advanced and non‐advanced fibrosis) [16], which had an homogenous distribution of fibrosis among our population, with a difference of 0.002 between the observed and adjusted AUROCs.

Table 4 shows the lower cutoffs for NFS, Hepamet and FIB-4 for significant fibrosis, advanced fibrosis and cirrhosis among our patients and their test performances. The optimal cutoff values are shown in Table 5.

Table 4.

Statistical performance of the lower (90% of sensitivity) cutoff values according to the stage of fibrosis.

	Calculated lower cutoff	Original lower cutoff	Specificity (%)	PPV (%)	NPV (%)
Significant Fibrosis (F2-F4)
NFS	−1.63		24.5	31.6	85.9
Hepamet	0.02		23.4	31.9	88.9
FIB-4	0.46		22.3	31	84.7
Advanced Fibrosis (F3-F4)
NFS	−1.38	−1.455	25.5	23.5	90.6
Hepamet	0.02	0.12	21.9	23.1	91.7
FIB-4	0.44	1.45	20.5	22.3	88.6
Cirrhosis (F4)
NFS	−0.99		32.4	5.4	99.2
Hepamet	0.03		34.3*	5.5	99.2*
FIB-4	0.92		61.0*	8.9	99.6*

PPV, positive predictive value; NPV, negative predictive value; NFS, NAFLD Fibrosis Score. *p < 0.05; *p < 0.0001 McNemar's Chi-squared Test with continuous correction.

Table 5.

Statistical performance of the higher cutoff values according to the stage of fibrosis.

	Calculated higher cutoff	Original higher cutoff	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Significant Fibrosis (F2-F4)
NFS	0.69		32.1	83.5	43.0	76.0
Hepamet	0.08		67.9*	65.6*	43.4*	84.0*
FIB-4	0.89		72.6*	66.7*	45.8*	86.3*
Advanced Fibrosis (F3-F4)
NFS	0.50	0.676	44.2	81.1	37.4	85.1
Hepamet	0.11	0.47	51.9	77.5	37.0	86.3
FIB-4	1.46	3.25	42.9	90.1*	52.4*	86.1
Cirrhosis (F4)
NFS	0.50		66.7	78.0*	11.1	98.3
Hepamet	0.30		60.0	89.3*	18.8	98.2
FIB-4	1.34		80.0*	81.6	15.2	99.0*

AUROC, area under ROC curve; PPV, positive predictive value; NPV, negative predictive value; NFS, NAFLD Fibrosis Score. *p < 0.0001 McNemar's Chi-squared Test with continuous correction.

For the lower cutoff, the performance criteria of the three scores were similar for significant and advanced fibrosis. However, FIB-4 was significantly better than Hepamet, and Hepamet was better than NFS for specificity and NPV to diagnose cirrhosis. Although the NPV of the two tests are numerically the same as in NFS and Hepamet for cirrhosis, Hepamet obtained better diagnostic accuracy for ruling out cirrhosis than NFS analyzed by the McNemar method.

With the upper threshold, NFS had lower sensitivity, specificity, PPV and NPV for significant fibrosis than either Hepamet or FIB-4, the latter two having similar performances. For advanced fibrosis, FIB-4 had higher PPV and specificity than the other two tests. FIB-4 also performed had higher NPV and sensitivity for cirrhosis, although Hepamet had higher specificity for this stage of fibrosis than did the other tests.

4

Discussion

Since previous papers on Hepamet, FIB-4 and NFS scores have focused on fibrosis stage ≥ F3 (with the limitation of a different fibrosis classification in the FIB-4 study) [5,7,12], more data are needed on the performance of these tests in assessing significant fibrosis and cirrhosis. In the present study, both Hepamet and FIB-4 showed appropriate AUROCs (≥ 0.70) for this aim, but the NFS score performed poorly compared to the other scores with significant and advanced fibrosis. Although the AUROCs were acceptable (≥ 0.70) for FIB-4 and Hepamet for significant and advanced fibrosis and for the three calculated scores for cirrhosis, it was not as high as those observed in the original papers (all >0.80) [5,7,12]. This may be due to differences among the studied populations. It is important to highlight that our cohort had a higher prevalence of T2DM (50% versus 37.8%) and higher mean BMI (35.3 ± 6.4 vs 31.7 ± 6.9 kg/m2) than cohorts included inthe paper by Ampuero et al. that validated Hepamet [12]. Peculiarities of Latin American populations may also play a role in determining the performance of non-invasive tests [18].

Defining thresholds was the main purpose of this study. Since classical Hepamet, FIB-4 and NFS cutoffs were determined by criteria [5,7,12] focused on advanced fibrosis, we chose to define our cutoffs to be able to compare the tests and find the most useful thresholds for our aims. Given that non-invasive fibrosis scores are more useful in ruling out than in diagnosing fibrosis [1,19,20], we decided not to focus on positive predictive values or specificity alone as criteria for the upper cutoff at the expense of having excessive false negatives. The Youden index was chosen as our upper threshold because it has balanced specificity and sensitivity and represents the maximum potential effectiveness of the test. The lower cutoff was determined as the point in the ROC curve with 90% of sensitivity to exclude liver fibrosis.

NFS has been widely used to detect liver fibrosis in patients with NAFLD [7,8,21]. Nevertheless, the NFS AUROCs were lower than those of the other tests for significant and advanced fibrosis. The original lower and upper NFS cutoffs are the points in ROC curves with 90% NPV and 90% PPV, respectively. Nevertheless, since prevalence can change both parameters, tests with these thresholds do not perform the same with all populations. In the case of NFS, applying the established cutoffs with our population with a 20% prevalence of advanced fibrosis should have resulted in 49% PPV and 94% NPV for the lower cutoff (< -1.455) and 88% PPV and 88% NPV for the upper cutoff (0.676). However, we found a surprisingly better NPV (85%) with our upper cutoff, which is slightly lower than the original one (0.5 versus 0.676) and a 23% PPV at our point of 91% NPV (cutoff of -1.38), which is much lower than the expected 49% for the NPV of 90% described in the paper (cutoff of -1.455).

The FIB-4 score was initially tested with patients with hepatitis C virus and Human Immunodeficiency Virus infection [5], but it has been validated in assessing fibrosis with patients with NAFLD [1,8]. Since the first paper, the lower cutoff established for advanced fibrosis (< 1.45) was the point in ROC curve with 90% NPV, while the upper cutoff (3.25) was the point with a specificity of 97%. In the present study, with the same prevalence of 20% of advanced fibrosis (despite differences in the staging system), we set one of the two cutoffs at 1.46 (all though by different methods), which had a negative predictive value of 86%. This is probably equivalent to the 90% obtained in the previous study, confirming that we had similar ROC curves. However, this was considered our upper cutoff, which highlights the different perspectives of the usefulness of non-invasive tests in clinical practice, since we want to exclude fibrosis, rather than identify it. These findings concur with those of previous studies that described lower cutoffs than standard thresholds [9,13]. We obtained an AUROC of 0.74, with a sensitivity and negative predictive value of 73% and 86%, respectively. Although higher thresholds have been reported, our FIB-4 cutoffs were useful in diagnosing the absence of advanced fibrosis and had NPVs similar to those in published studies [8,22].

In the present study, we also evaluated the Hepamet, recently developed non-invasive fibrosis score [12]. Using the Youden index, we obtained a similar cutoff to that reported by Ampuero et al. (0.11 vs 0.12) for diagnosing advanced fibrosis. However,with our threshold of 0.11, the sensitivity was only 52%, compared to 70.7% with the cutoff of 0.12 in the original paper, and a more comparable specificity of 77% (versus 75.5%). In the original paper, the second cutoff was defined as the point with 97% of specificity in order to better diagnose advanced fibrosis, which was not our aim. Therefore, our second thresholds are not comparable.

In accordance with our purpose, we found lower cutoffs in the present study than those found in previous studies.

5

Conclusion

The current study demonstrated that Hepamet and FIB-4 have better performances than NFS for significant and advanced fibrosis in Latin American NAFLD patients, while all three tests are useful for detecting cirrhosis. However, since they have such different criteria for their thresholds, it is necessary to determine a single way to calculate cutoffs in order to better interpret the results and apply them in clinical practice. Lower NFS, FIB-4 and Hepamet cutoffs (90% of sensitivity) may be useful in excluding both significant and advanced fibrosis. The outcomes and new proposed cutoff values are valid with independent cohorts.

Conflicts of interest

The authors report no potential conflict of interest relevant to this article.

Acknowledgements

Contribution from Marco Arrese was supported by the Chilean government through the Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT 1191145) and the Comisión Nacional de Investigación, Ciencia y Tecnología (CONICYT, AFB170005, CARE Chile UC).