The EncephalApp Stroop Test was developed to more easily diagnose minimal hepatic encephalopathy (MHE). A cut-off of >274.9sec (ONtime+OFFtime) reached a 78% sensitivity and 90% specificity in the validation study, but it has been poorly studied in Brazil. We aim to analyze the usefulness of this diagnostic method and to describe a cut-off value to screen MHE in Brazil.
MethodsIn this cross-sectional and single-center study, three positive psychometric tests defined the diagnosis of MHE as the gold standard. We evaluated gender, age, education, familiarity with smartphones, etiology of cirrhosis, Child-Pugh/MELD scores, and previous hepatic encephalopathy (HE). Healthy controls and patients without HE were compared for the task validation. The Chi-square and Mann-Whitney tests, logistic regression analysis, and ROC curves were used for statistical evaluation.
ResultsWe included 132 patients with cirrhosis (61% male) and 42 controls (62% male) around 51y. Sixty-three were diagnosed with MHE on psychometric tests and 23 had clinical HE. Viral hepatitis (38%) was the major etiology of cirrhosis. The median MELD was 10 and Child-Pugh A was more frequent (70%). There was no significant difference in test results between controls and patients without HE. There was also no influence of gender, age, education, and familiarity with smartphones in the test results. Child-Pugh A was associated with MHE (p=0.0106). A cut-off of >269.8sec (ONtime+OFFtime) had an 87% sensitivity and 77% specificity to detect MHE (p=0.002).
ConclusionThis is a valid and reliable tool for screening MHE. However, optimal cut-off values need to be validated locally.
Hepatic encephalopathy (HE) is a wide spectrum of nonspecific neurological and psychiatric features that commonly impair quality of life. It has been classified as either OVERT or COVERT, based on the severity of clinical presentation [1-3]. Overt HE is easily recognized on physical examination and includes symptoms graded II to IV according to the West Haven (WH) criteria [1]. On the other hand, covert HE encompasses two conditions: 1) WH grade I (characterized by impaired consciousness, altered sleep rhythm, euphoria, or anxiety); and 2) minimal HE (MHE) [1-3].
MHE is a preclinical stage, presented as slight cognitive deficits, reduced attention and psychomotor speed, in addition to impaired working memory and information processing [1]. Although barely noticeable, its prevalence in patients with cirrhosis ranges from 20% to 80% worldwide [1,4]. It influences daily function, leading to a higher risk of traffic violations, accidents, hospitalization, and death [5-7]. Unfortunately, it remains a challenge to identify affected patients. The Psychometric Hepatic Encephalopathy Score (PHES) is the most widely used diagnostic method; however, it takes a long time and is therefore not so suitable for routine use [8,9].
In order to make the diagnosis of MHE easier and faster, a smartphone app based on the Stroop Test was developed: the EncephalApp Strop Test (EncApp_ST) [9]. It evaluates psychomotor speed and cognitive flexibility by measuring the time required to correctly identify symbols and printed words in different colors. This is attractive and takes a few minutes to be applied. The task has two components: “OFF” and “ON” states, depending on the concordance or discordance of the colored stimuli. In the easier “OFF” state, the patient is subjected to red, green, or blue hashtags and must choose as quickly as possible by touching the matching color among multiple-choice buttons at the bottom of the screen. The color names are also randomized and not fixed to their respective positions (Figure 1A).
Screen of the EncephalApp Stroop Test (Brazilian Portuguese version). A) OFF mode: the symbols are in red and the patient must choose the button “Vermelho” (which means red). B) ON mode: the word “Azul” (which means blue) is written in red and the patient must touch the button showing the color name he sees (“Vermelho”), not the written word. White arrows show the correct answer on each mode. Notice that the position of the colors at the bottom changes randomly.
The “ON” state is more challenging because the color name is displayed in a discordant coloring and the patient is asked to touch the color of the text but not the written color; that is, the word “BLUE” is displayed in red and the correct response is red, not blue (Figure 1B). The results at the end of the EncApp_ST are: 1) OFFtime: total time (in seconds) for 5 correct runs in the “OFF” state; 2) OFFruns: number of attempts needed to complete 5 correct “OFF” runs; 3) ONtime: total time (in seconds) for 5 correct runs in the “ON” state; and 4) ONruns: number of attempts needed to complete 5 correct “ON” runs [9].
It has been reported that the sum of OFFtime and ONtime higher than 274.9 seconds had a 78% sensitivity and 90% specificity for the diagnosis of MHE [9]. However, some factors mainly related to the studied population may influence the outcomes. There is a lack of data on the usefulness of the EncApp_ST in Brazil. We aim to validate this diagnostic tool in our region and to describe cut-off values with a high sensitivity and specificity to select patients with MHE.
2Materials and methods2.1Study design and patient selectionThis is a cross-sectional and single-center study involving subjects with cirrhosis in an outpatient setting at the Hospital de Clínicas and at the Gastrocentro of the University of Campinas (Unicamp), located in the southeastern region of Brazil. They were invited to enroll in the study from March 2017 to February 2020.
Inclusion criteria: patients (aged 18–70 years) with liver cirrhosis diagnosed by 1) liver biopsy; OR 2) liver ultrasound with the morphology of cirrhosis plus at least one of the following findings: a) esophageal or gastric varices on digestive upper endoscopy; b) ascites; c) venous collateral vessels on ultrasound indicating portal hypertension; or d) a previous hepatic decompensation (ascites or digestive upper hemorrhage or even HE).
Exclusion criteria: unable to consent, use of psychoactive medications, hypnotic, sedatives (except antidepressants), alcohol abuse (>50g/day) [10], or other illicit drugs over the past 3 months, or the use of any amount in the last 24 hours, diagnosed with Daltonism or any degree of blindness, neuropsychiatric disorders (such as Parkinson's or Alzheimer's disease), or underwent interferon treatment during the past year. Patients using lactulose, ornithine aspartate, or antibiotics (except norfloxacin) were also excluded, as well as those with WH grade III/IV HE [1].
The inclusion of patients was aleatory, at the end of medical appointments, according to their availability. Patients were adjudicated as having clinical HE (WH grade I/II) [1] when they presented asterixis on physical examination OR at least two of the following findings: inversion of the sleep-wake cycle, reduced attention, or slower speech. Serious mental confusion, OR gross disorientation, OR intense muscular tenderness, OR bizarre behaviors—any of these findings set the patient as WH grade III/IV HE [1] (exclusion criteria).
Except for those diagnosed with clinical HE (WH grade I/II) [1], all other individuals underwent Mini-Mental State Examination (MMSE) and those with results under 25 were excluded [11,12]. We performed the Number-Connection Tests (NCT-A, NCT-B) and the Symbol-Digit Test (SDT) to all cirrhotic patients without clinical HE. Those with a performance out of two standard deviations based on previous reports in all three tests were defined as having MHE [8,13,14]. We recruited healthy people to compose a control group: family members of the patients, hospital staff, or patients with diseases in other systems, without neurological impairment, alcohol or drug abuse. They also underwent MMSE. We tried to recruit controls with age and education similar to the patients with cirrhosis.
The EncApp_ST was applied to all studied subjects (cirrhotic and controls). The version of the application used in the study was translated into Brazilian Portuguese by our staff and Dr Bajaj's group (Department of Gastroenterology of Virginia Commonwealth University, VA, USA). We applied two training rounds of each phase (OFF and ON) before starting the five valid rounds. All administrations were supervised by the main investigator (MCS). We analyzed OFFtime, ONtime, OFFruns, ONruns, ONtime + OFFtime (ON+OFF), and ONtime−OFFtime (ON−OFF). We evaluated age, gender, years of education, and ability in using smartphones. Among patients with cirrhosis, the etiology of liver disease, Child-Pugh score, and model for end-stage liver disease (MELD) score were also assessed, as well as the occurrence of previous episodes of HE at any time as described in patients’ medical records. The ability in using smartphones was classified as yes or no if the participant reported to use this device on a daily basis.
The study protocol was approved by the Ethics Committee of the University of Campinas (CAAE 50831615.1.0000.5404). The protocol was conducted in accordance with the ethical guidelines of the 2013 World Medical Association Declaration of Helsinki [15]. Informed consent was obtained from all participants.
2.2Statistical analysisExploratory data analysis was performed using summary measures (frequency, percentage, mean with standard deviation, and median with minimal and maximum values). The comparison between variables about groups (control × cirrhosis without HE) was assessed using the Chi-square and Mann-Whitney tests. The relationship between variables and HE was assessed through logistic regression analysis. To determine the cut-off points and the sensitivity and specificity data, receiver operating characteristic (ROC) curves were constructed. The significance level for this study was 5%. The Statistical Analysis System (SAS) for Windows software package, version 9.4 (SAS Institute Inc., 2002–2008, Cary, NC, USA) and R version 3.4.2 (The R Foundation for Statistical Computing, Vienna, Austria) were used for the statistical analyses conducted by biomedical statisticians from the Department of Biostatistics, University of Campinas (Unicamp), Campinas, Brazil.
3ResultsA total of 223 individuals were evaluated for study inclusion (179 with cirrhosis and 44 healthy controls). Among the patients with cirrhosis, 25 were excluded (4 with recent alcohol abuse, 1 on illicit drugs, 8 on neuropsychiatric medications, 4 on HE treatment, and 8 with insufficient performance on MMSE). We also excluded other 22 patients who had fewer than three altered psychometric tests (NCT-A, NCT-B, and SDT), as we could not be sure whether or not they had MHE. Another two healthy subjects were excluded due to unsatisfactory performance on MMSE. The flowchart of enrollment is shown in Figure 2. The study population was composed of 132 patients with cirrhosis (61.4% male) and 42 healthy controls (61.9% male), with a median age of 54 and 50.5 years, respectively. The population was homogenous regarding age, gender, years of education, and affinity in using smartphones (Table 1).
Characteristics of the 174 individuals among the groups: cirrhosis with clinical, minimal, without hepatic encephalopathy, and healthy controls.
HE: hepatic encephalopathy; y: years; min-max: minimal to maximum.
On physical examination, 23 patients (69.6% male) had clinical HE, and according to the results of the three psychometric tests, 63 (57.1% male) were diagnosed with MHE. The remaining 46 patients (63% male) had no HE (Table 1). Viral hepatitis (37.9%) and alcohol consumption (32.6%) were the most common etiologies of cirrhosis. Almost 70% of the patients had Child-Pugh score A, and the median MELD score was 10 (Table 2). In addition, almost 70% of the subjects with clinical HE had prior episodes of HE. Liver-related features of the 132 patients with cirrhosis are shown in Table 2.
Liver-related features of the patients with cirrhosis.
AIH: autoimmune hepatitis; HE: hepatic encephalopathy; MAFLD: metabolic dysfunction-associated fatty liver disease; min-max: minimal to maximum; MELD: model for end-stage liver disease.
There was no association of gender, age, years of education, and smartphone affinity with the occurrence of HE. However, prior HE and Child-Pugh score A were related to the diagnosis of MHE among 132 patients with cirrhosis (p=0.0037 and p=0.0106, respectively). Regarding clinical HE, its presence was associated with prior HE (p=0.0002), Child-Pugh score B/C (p=0.0199), and a higher MELD score (p=0.0035), as shown in Table 3.
The association of liver-related features with the diagnosis of minimal or clinical hepatic encephalopathy.
CI: confidence interval; MELD: model for end-stage liver disease; NS: not significant; OR: odds ratio.
The results of the EncApp_ST in the 174 studied subjects are shown in Table 4. In an attempt to validate the usefulness of the EncApp_ST in our sample, we compared the results between cirrhotic patients without HE versus healthy controls, and there was no significant difference among the app variables evaluated (Table 5). We also compared patients with or without prior HE in order to identify a possible influence on the test results and there was no significant difference. We performed the same analysis regarding years of education, which also showed no significant difference. These results are shown in Table 6.
Results of the EncephalApp Stroop Test among the groups.
HE: hepatic encephalopathy; min–max: minimal to maximum; sec: seconds.
Comparison between patients without hepatic encephalopathy versus healthy controls.
HE: hepatic encephalopathy; min–max: minimal to maximum; sec: seconds.
Results of the test regarding previous episodes of hepatic encephalopathy (HE) and years of education between patients with minimal HE and without HE (median and minimal to maximum).
HE: hepatic encephalopathy.
We compared the results of patients with MHE (n=63) versus those without HE (n=46). The results OFFtime (p=0.001), OFFruns (p=0.0412), ONtime (p=0.0049), and ON+OFF (p=0.002) were significantly different. Analyzing the result ON+OFF, a cut-off of >269.8 seconds had an area under the curve of 0.781 in patients with cirrhosis for the diagnosis of MHE with a sensitivity of 87.3% and specificity of 76.6% (Figure 3).
4Discussion
The optimal method for diagnosing MHE is still debated and no universal criteria have yet been established. In fact, none of the proposed tools can cover the complexity and the heterogeneity of cognitive impairment in MHE. Most of the studied methods are difficult to execute and not so feasible for routine use, mainly because they take a long time and depend on trained examiners [2]. Smartphones are more available than paper–pencil psychometric tests and this makes the EncApp_ST extremely easy to apply by any health professional or student. Zeng et al. (2019) reported that this tool is 38% time-saving compared to traditional tasks [3].
As demonstrated in other studies [3,9], previous episodes of HE were associated with a higher prevalence of current HE in our analysis. It is already known that this chronic complication of liver disease usually presents a higher risk of recurrence [1]. As expected, impaired liver function (assessed by Child-Pugh and MELD scores) was more associated with the occurrence of clinical HE. Interestingly, we found an association of MHE with Child-Pugh score A, since more than 82% of the patients with MHE had good liver function. This shows that the diagnosis of MHE should be considered even in patients with no other clinical manifestations of chronic liver disease. The median MELD score was also low, but we found no significance, possibly due to our small sample and we should also consider the application scope of MELD, which is generally assessed to predict mortality in patients with advanced liver disease.
In Brazil, the reported prevalence of MHE in patients with cirrhosis is around 50% [16,17]. Recently, a Brazilian study evaluated 82 patients with cirrhosis and described a 35.4% prevalence of covert HE [18]. The authors found a 96.6% sensitivity and 49.1% specificity of the EncApp_ST on the diagnosis of covert HE, but they referred to use parameters based on North American and German data. Gender and age did not influence the results, as in our study, but people with more years of education had a better performance than those with less access to education [18].
We found different results regarding years of education but the patients evaluated by Machado Junior et al. (2020) [18] had an average of 5 more years of education than ours. In the Bajaj et al. American study [9], the patients had an average of 7 more years of education than our subjects. In addition to years of education, other variables such as age, gender, ability to use electronic platforms, and etiology of cirrhosis did not influence the EncApp_ST results in our analysis.
These findings were different from those reported by Zheng et al. (2019) in a multicenter Chinese study, which demonstrated an influence of age, alcoholic hepatitis, experience with electronic platforms, and education duration on the results of the EncApp_ST [3]. There was no difference regarding the history of HE on the test results, and the authors consider that it may be justified by the extremely small sample size of patients with previous HE. Using the EncApp_ST, they found a 40.3% prevalence of covert HE, with an 86% sensitivity and 59% specificity. The low specificity rate was attributed to differences in race, ethnicity, cultural background, and spectrum of liver disease of the survey population [3].
A Korean study involving 220 hepatitis B-related liver cirrhosis and 376 healthy controls found a 20.6% prevalence of MHE [7]. This condition was associated with the female gender and the authors also described an influence of schooling, with a median of 10, 12, and 14 years of education in patients with MHE, without MHE, and healthy controls, respectively [7]. We have not found such association but our patients had an average of 7 years of education.
Despite the 47.7% prevalence of MHE among our patients with cirrhosis, this may not reflect the real burden of MHE in our setting because the recruitment of participants was aleatory and depended on the availability of the patient and the evaluator. We did not find different test results by comparing patients with or without previous episodes of HE as described by other researchers [9,19]. This may be due to a lower frequency of prior HE in our patients compared to other studied groups [19,20]. Furthermore, variation in cirrhosis severity between the studies may also have affected the results [3,9,19,20].
A multicenter cohort involving 437 patients found a sensitivity greater than 70% on diagnosing MHE [19]. A new analysis of the same population has demonstrated that combined testing decreases covert HE prevalence without improving the accuracy of overt HE prediction [20]. Testing with PHES or EncApp_ST alone, or a PHES plus EncApp_ST combination, is equivalent to diagnosing covert HE and predicting overt HE development [20]. In contrast, a multicenter Chinese survey showed that combined scores from the EncApp_ST, NCT-B, and SDT identified patients with covert HE with approximately 87% accuracy, and in a much shorter time than the standard PHES [21].
In our study, the PHES was performed as a gold standard method to select patients with MHE. We have locally validated the EncApp_ST by comparing the results of patients without HE versus healthy controls with similar demographic characteristics. There were no significant differences in test results between these groups. The task was effective on identifying MHE with a satisfactory sensitivity and specificity when the result ON+OFF was assessed, similar to the findings described by Bajaj et al. (2013) [9] (>269.8 seconds and >274.9 seconds, respectively). The EncApp_ST seems to have a great potential to be applied into a real-life setting. Interestingly, a study compared the test application on smartphones and tablets and the results were similar, which suggests that both devices are useful [7].
Several published studies had (to a greater or lesser extent) differences in methodology; variations on the severity of cirrhosis; the proportion of individuals with previous HE; whether or not they were using lactulose, ornithine aspartate, or antibiotics; affinity with electronic platforms; and a wide range years regarding education. The variability found in the test results is entirely acceptable and confirms the complexity of MHE. Furthermore, the diagnostic goal of the test was not uniform. Some tried to select patients with MHE [6,7,9] (as we did), while others tried to identify subjects with covert HE [18-20]. This can also make it difficult to compare the results among the studies.
By analyzing previous compiled results, perhaps if further studies do not include patients with advanced liver disease, prior HE, elderly, and poorly educated individuals, the diagnostic accuracy of the EncApp_ST will be greater [3,18]. A multicenter Japanese survey suggested different cut-off values according to patient age [22]. A prospective American study has shown that repeating the test at regular intervals can provide better results among patients with no history of covert HE [23]. Furthermore, changes in the test results after patients have undergone transjugular intrahepatic portosystemic shunt or treatment for metabolic disturbances have been reported [24]. These data show the usefulness of the EncApp_ST for monitoring patients over time, which could be extremely beneficial. Any effort to provide early diagnosis to these patients will never be in vain.
It seems important to realize that EncApp_ST results may suffer variations according to non-measurable factors (e.g., fatigue, stress, anxiety about obtaining a good test result, and other aspects) that could cause distraction during the exam [18]. We performed the test after a medical appointment and the tiredness of some patients may have impacted the results. These heterogeneities may be a limitation of our study, as well as the single-center study design and the small sample size. To avoid a prolonged time of evaluation, the diagnosis of MHE was based on only three psychometric tests instead of the five that are part of the PHES, which is commonly used in the studies [8]. Moreover, the set of clinical HE (WH grade I/II) was subjective and may have been influenced by the examiner. Even so, our results were similar to those described by Bajaj et al. [9]
5ConclusionTechnology has developed advanced diagnostic tools and the EncApp_ST has proven to be reliable and suitable for use in daily practice to diagnose MHE. Adopting the cut-off of >269.8 seconds on the result Ontime + OFFtime, the test had an 87.3% sensitivity and 76.6% specificity for the diagnosis of MHE in this study. Local validations are still needed in order to detect and exclude possible modifying factors and, subsequently, to find cut-off values with high accuracy to diagnose MHE.
FundingThis research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Authors´ contributionMCS: designed the study, collected and analyzed data, reviewed the literature and wrote the manuscript; FLPN PSA, LVP: collected data and reviewed the literature; RDG, TMLS, TSP: analyzed data and wrote the manuscript; LTM, MRI: reviewed the manuscript; DFCM: designed the study, analyzed data and reviewed the manuscript. All authors read and approved the final version of the manuscript.
We acknowledge Prof. Elza Cotrim Soares, Prof. Jazon Romilson de Souza Almeida for their contribution on the design of this study.
