The aim of the present study was to examine the diagnostic accuracy of screening tests in detecting cases requiring psychological intervention among patients referred for thoracic surgery.
MethodsEmotional distress was evaluated in 105 patients referred for thoracic surgery by means of a diagnostic psychological interview (criterion variable). The screening ability of the following methods was analyzed: the physician's opinion (Yes/No), Hospital Anxiety and Depression Scale (HADS), single-item interview: “Are you depressed?” (Depression Question, ADEP) (1–5) and the single-item interview: “Are you anxious?” (Anxiety Question, ANXQ) (1–5).
ResultsAccording to the clinical interview, 34% of the patients were clinical cases requiring psychological intervention. The total HADS (cut-off point of 10) showed a sensitivity=0.89, specificity=0.75 and AUC=0.883; the ADEP scale (>1) showed a sensitivity=0.79, specificity=0.74 and AUC=0.795; the ANXQ scale (>1) showed a sensitivity=0.78, specificity=0.41 and AUC=0.690; and the physician's opinion showed a sensitivity=0.47 and specificity=0.86.
ConclusionsA high percentage of patients referred for thoracic surgery required psychological intervention. The best instrument to identify those patients requiring psychological care, taking a psychological interview as the criterion variable, was the total HADS score. This test is brief, simple and well accepted by patients; it is easy to implement within a thoracic surgery service and has a good diagnostic accuracy.
El objetivo del presente estudio es examinar la precisión diagnóstica de los test de cribado utilizados para detectar la necesidad de atención psicológica en pacientes remitidos para cirugía torácica.
MétodosSe evaluó la presencia de malestar emocional en un total de 105 pacientes remitidos para cirugía torácica por medio de la entrevista clínica psicológica (variable criterio). Los métodos de cribado utilizados fueron: el juicio del médico (sí/no), la Escala Hospitalaria de Ansiedad y Depresión (HADS), la pregunta única para evaluar depresión «¿se siente usted deprimido?» (DEPQ) (1-5) y la pregunta única para medir ansiedad «¿se siente ansioso?» (AAANS) (1-5).
ResultadosDe acuerdo con la entrevista entrevista clínica psicológica, el 34% de los pacientes fueron casos clínicos que requerían intervención psicológica. La puntuación total de la escala HADS (punto de corte 10) mostró una sensibilidad del 0,89, especificidad del 0,75 y un ABC de 0,883; la ADEP (punto de corte 1) mostró una sensibilidad del 0,79, especificidad del 0,74 y ABC del 0,795; la AANS (>1) obtuvo una sensibilidad del 0,78, especificidad del 0,41 y ABC de 0,69 y el juicio del médico mostró una sensibilidad del 0,47 y especificidad del 0,86.
ConclusionesUn alto porcentaje de pacientes remitidos para cirugía torácica requirieron intervención psicológica. El mejor instrumento para identificar a aquellos pacientes con necesidades psicológicas, tomando la entrevista psicológica como variable criterio, fue la puntuación total del HADS. Este instrumento es simple y breve, bien aceptado por los pacientes, de fácil aplicación en un servicio de cirugía torácica y tiene buena capacidad diagnóstica.
Major surgery is associated with high levels of stress, depression and anxiety,1–3 which, in turn, predict an increase in post-surgical complications, and prolong hospitalization periods,4,5 as well as higher mortality rates,1,6,7 and greater deficits in the quality of life of patients.3
Therefore, it seems important to promptly identify those patients with high levels of emotional distress, even before they undergo surgery, using adequate screening methods, with the objective of offering psychological or psychiatric care to those patients who require it as soon as possible. The most common cause of referral to mental health services is the patients’ request for psychological help and the physician's judgment regarding the patients’ need for such psychological care. However, the patients’ desire for help shows a low correlation with the severity of the anxiety, depression or emotional distress symptoms they suffer.8,9 In many cases, physicians underestimate cases that require psychological care.9
Standardized clinical diagnostic interviews can be performed as part of the referral to psychological care procedure. The inconvenience of such interviews is that they are time consuming and costly and can be unpleasant for patients. Thus, their application within daily clinical practice, given the high demands of the hospital context, is very difficult. It would be much more efficient to use simple, concise emotional distress screening methods that are easily applied, and which present adequate reliability and validity, in order to detect those patients who need psychological treatment as early as possible.
Short screening procedures that use tests to measure anxiety, depression or emotional distress are the simplest, shortest and least intrusive way to identify people who need psychological care. Some of the most commonly used tests are The Hospital Anxiety and Depression Scale (HADS),10 or the single-item interviews, such as “are you depressed?”,11 or “are you anxious?”.12 However, to date, data on the validity of these screening tests in the case of pre-surgical patients are not available.
The objective of the present study is to determine the best screening instrument to identify those patients submitted to thoracic surgery that require psychological care, taking as criterion variable the psychological interview carried out by two psychologists. This interview assessed the patients’ level of emotional distress associated with the disease, understood as clinical symptoms of anxiety and depression. Moreover, it also assessed the presence of fears and concerns, the perceived social support and the coping resources with which patients respond to the situation.
MethodsA total of 133 patients referred for Thoracic Surgery at the San Carlos Clinical Hospital of Madrid were evaluated during the preoperative period, out of which 15 patients declined taking part in the study and 13 did not meet the inclusion criteria: being over 18 years of age, with good comprehension and expression in Spanish and with adequate levels of cognitive function (Mini-Mental score>26).13 Patients who – according to their medical and/or psychiatric diagnosis – had severe cognitive impairment or a severe psychiatric disorder were excluded from the sample. Thus, the final sample was composed of a total of 105 patients.
Variables and InstrumentsThe tests were chosen according to their simplicity and quick application, as well as for their adequate psychometrics properties. Their objective was to help the physician carrying out the intervention to detect emotional distress in a way that did not involve a time cost added to their routine care. Thus, these quick and effective instruments are:
Hospital Anxiety and Depression Scale (HADS) (Zigmond and Snaith, Spanish version by Caro and Ibáñez).10,14 It is a self-applied instrument, composed of 14 items. It evaluates the presence of symptoms of anxiety and depression, through 4 response options. Prior to completion, patients are asked to respond in relation to the symptoms they have suffered during the past week. High scores in its subscales – anxiety (HADS-A) and depression (HADS-D), of 7 items each – indicate a high frequency of these symptoms. The total scale score (HADS-T) provides a valid measure of emotional distress. Reviews on the diagnostic accuracy of HADS, based on diagnostic psychological interviews, have shown that its sensitivity is good-acceptable.15–17
The Single-item interview, which measures depressed mood through the question: “are you depressed?” (DEPQ) was developed by Chochinov et al.11 This question has a dichotomous response (Yes/No) and is complemented by 5 Likert-type response options (1–5). This can be a highly recommended measure due to the brevity of its application. Chochinov et al. reported values of sensitivity (SE) and specificity (SP) of 100%.12 The meta-analysis performed by Mitchell concluded that the SE of this test is 0.72 and SP is 0.83.17
The Single-item interview assessing anxiety “are you anxious?” (ANXQ), was developed by Davey et al.12 Patients answer on a Likert-type scale ranging from 1 to 5. Mitchell summarized research on single-item interviews about anxiety in relation to the HADS and reported a weighted SE of 0.79 and a weighted SP of 0.71.17
Medical judgment is issued by the physician after a medical consultation with the patient. Each physician completes a form in which he/she issues a judgment on whether the patient is susceptible of psychological attention or not; and if so, the reasons for the referral.
The need for psychological care has been determined through the comparison of each of these instruments with the criterion variable (specialized psychological interview), similar to that carried out by Bonachi et al.18 These interviews have been carried out by a psychologist, during 1h. At the end of the interview, the psychologist assessed whether the patient required psychological care or not, as indicated by at least one of the following variables evaluated throughout the interview:
- –
The presence and intensity of anxious-depressive symptoms, as well as fears and concerns related to the possible disease and other vital areas.
- –
The presence of current or previous psychopathology and the consumption of psychoactive drugs.
- –
The presence or absence of perceived family and/or social support.
- –
The presence of patient coping strategies, i.e.: both internal and external resources with which the patient can deal with the situation.
- –
The vital moment in which the patient finds him/herself, independently of that caused by the possible illness.
Each interview was recorded, and subsequently heard by another psychologist who independently evaluated, without knowing the interviewer's judgment, whether the patient was a clinical case or not. If there was a disagreement between the two psychologists, a discussion took place with a third clinical psychologist, issuing the final judgment.
Patients waiting to undergo thoracic surgery were informed by the physician during their consultation about the existing protocol (interview prior to surgery with a psychologist from the medical team). The patients met with the psychologist, where they were given information on the study and if they agreed to participate, they proceeded to sign the informed consent, guaranteeing the confidentiality of the data. It was then that the physician's assessment of the patient's need for psychological care was included in the study and the psychometric tests (HADS, DEPQ and ANXQ) were administered to the patient. Afterwards, and without knowledge of the results of the screening tests, the specialized psychological interview took place. At the end of this interview, a judgment was issued on whether that patient was a clinical case or not, depending on whether they required specialized psychological care. Subsequently, the inter-observer agreement took place, as noted above.
Statistical AnalysesThe frequencies and percentages of the categorical variables, as well as the means, standard deviations and the confidence intervals for the quantitative variables were calculated. Cohen's Kappa coefficient among the interviewers was calculated to verify the concordance between them.
Using the Kolmogorov–Smirnov test, the non-normality distribution of the studied variables was verified. Mann–Whitney U tests were performed, with the Bonferroni correction, in order to check for statistically significant differences between the scores of the HADS, DEPQ and ANXQ scales for the clinical and non-clinical cases.
In order to verify the diagnostic accuracy of the screening tests with respect to the interview, the Sensitivity (SE) and Specificity (SP) indices, the Positive Predictive Values (PPV) and the Negative Predictive Value (NPV) and the indexes recommended by Mitchell were calculated.19 Mitchell's recommended indexes were: The Clinical Positive Utility Index, (CUI+)=SE×PPV, which shows the usefulness of the test in detecting problem cases, and The Clinical Negative Utility Index (CUI−), used to determine the usefulness of the instrument when discarding the cases considered as non-clinical: (CUI−)=SP×NPV. In addition, the Overall Value Index for the (CUI+) and (CUI−) for each test was extracted. This is a single measure that determines the validity of each screening test. The qualitative value ranges for these indices are as follows: excellent: ≥0.81; good: 0.64–0.80; fair: 0.49–0.63; poor: 0.37–0.48; very poor<0.37 (see: http://www.psycho-oncology.info/cui.html). Those cut-off points that showed better levels of SE and SP were selected for each of the screening tests.
In addition, the diagnostic accuracy of each test was assessed using the Receiver Operating Characteristic (ROC). The Area Under the ROC Curve (AUC) indicated overall performance, with a higher AUC reflecting better performance (excellent: 1–0.90; good: 0.80–0.89; fair: 0.70–0.79; poor: <0.70). In those cases in which the AUC value was <0.70, or in those in which the Overall Value yielded a very poor level, the corresponding ROC curve is not presented.
ResultsThe sociodemographic and clinical data of the final sample of patients are presented in Table 1.
In the specialized psychological interview, thirty-six clinical cases (34.29%) and 69 non-clinical cases (65.7%) were detected. The Cohen's Kappa coefficient was 0.89 (excellent). The frequency of clinical cases with non-small cell lung cancer was 36.8% and for patients without cancer, 29.7%, without significant differences being detected in the χ2 test (P=.47).
Table 2 shows the means and standard deviations for each of the screening tests for both clinical and non-clinical cases, and overall. As can be observed, clearly significant differences were found between clinical and non-clinical cases.
Symptoms of Anxiety, Depression and Emotional Distress Measured by the HADS, ANSQ and DEPQ for Cases With/Without Need for Psychological Treatment According to the Interview (Mann–Whitney U Tests).
| Non-clinical Cases: 69 | Clinical Cases: 36 | P | Total (N=105) | |
|---|---|---|---|---|
| Mean/(SD) | Mean/(SD) | Mean/(SD) | ||
| HADS-A | 4.32 (3.19) | 8.64 (3.40) | P<.001 | 5.80 (3.84) |
| HADS-D | 2.70 (2.26) | 6.80 (2.68) | P<.001 | 4.11 (3.43) |
| HADS-T | 7.20 (4.87) | 15.44 (5.64) | P<.001 | 9.91 (6.51) |
| ANXQ | 1.75 (1.74) | 2.42 (2.00) | P<.001 | 1.98 (1.89) |
| DEPQ | 1.29 (0.52) | 2.17 (0.81) | P<.001 | 1.59 (1.76) |
HADS: Hospital Anxiety and Depression Scale; HADS-A: anxiety subscale of the Hospital Anxiety and Depression Scale; HADS-D: depression subscale of the Hospital Anxiety and Depression Scale; HADS-T: Total score of the Hospital Anxiety and Depression Scale; ANXQ: Anxiety Question; DEPQ: Depression Question.
The results of the diagnostic accuracy of each screening instrument for emotional distress are shown in Table 3. It can be observed that the medical referral had a very low SE (0.47) but a high SP (0.86). The HADS-T (cut-off point=10), presented high levels of SE (0.89) and moderate SP (0.75); in addition, the Overall Value was good (0.80). The subscale HADS-A presented its best cut-off point at 4 and showed a low SP (0.58). In the case of the HADS-D subscale, also with the best cut-off point at 4, discrete levels for SE (0.72) and high SP (0.81) were observed. The ANXQ with the cut-off point higher than 1, showed an acceptable level of SE (0.78), but its SP was very low (0.41). The DEPQ also had a cut-off point greater than 1, presenting an acceptable SE (0.79) and SP (0.74).
Diagnostic Accuracy of the Screening Tests.
| Test | Cut-off | SE (CI 95%) | SP (CI 95%) | PPV (CI 95%) | NPV (CI 95%) | (CUI+) (CI 95%) | (CUI−) (CI 95%) | O. Value |
|---|---|---|---|---|---|---|---|---|
| Medical referral | Yes/no | 47 (.31–.64) | 86 (.77–94) | 61 (.45–.81) | 76 (.66–.85) | .30 (.09–.51) | 67 (.57–.72) | 72 |
| HADS-A | >4 | 97 (.92–1) | 58 (.46–.70) | 55 (.43–.77) | .98 (.93–1) | .53 (.39–.68) | 57 (.46–.68) | 71 |
| HADS-D | >4 | .72 (.58–.87) | 81 (.72–.90) | 68 (.52–.82) | 85 (.76–.94) | 48 (.30–.66) | 69 (.76–.94) | 78 |
| HADS-T | >10 | 89 (.79–.99) | 75 (.65–.86) | 63 (.52–.79) | 93 (.86–1) | 58 (.43–.74) | 70 (.62–.78) | 80 |
| ANXQ | >1 | 78 (.64–.91) | 41 (.29–.52) | 41 (.29–.52) | 78 (.64–.91) | 32 (.15–.48) | 32 (.19–.44) | 53 |
| DEPQ | >1 | 79 (.64–.91) | 74 (.64–.84) | 61 (.47–.75) | 86 (.78–.95) | 47 (.30–.65) | 64 (.55–.73) | 75 |
HADS-A: Anxiety score of the Hospital Anxiety and Depression Scale; HADS-D: Depression score of the Hospital Anxiety and Depression Scale; HADS-T: Total score of the Hospital Anxiety and Depression Scale; ANXQ: Anxiety Question; DEPQ: Depression Question; SE: Sensibility; SP: Specificity; PPV: Positive Predictive Value; NPV: Negative Predictive Value; CUI+: Clinical Positive Utility Index; CUI−: Clinical Negative Utility Index; O. value: Overall Value.
The ROC for the HADS scale and the AUC values for each test are shown in Fig. 1. It is observed that the HADS-T scale obtained a good result (0.883). The HADS-D and HADS-A presented satisfactory results (0.837 and 0.831, respectively). The result of the HADS-T indicated that it is a good test.
The ROC and AUC value for the DEPQ are shown in Fig. 2. The DEPQ presented acceptable results, while the ANXQ showed the worst results. The ROC curve for the ANXQ scale is not shown as its AUC was too low (0.690) and its Overall Value was very poor (0.53).
In summary, the results showed that: (a) more than a third of the patients needed psychological attention; (b) the scores of the clinical cases in each of the diagnostic tests were very significantly higher than those of the non-clinical cases (P<.001); (c) the HADS-T proved to be a good screening test, and was the best; and (d) the HADS-A, HADS-D and DEPQ have shown to be satisfactory screening tests.
DiscussionThe prevalence of patients that require psychological treatment found in the present study is high. This may be due to the fact that the psychological evaluation was carried out around three weeks prior to surgery, thus all participants were facing an intense stressor (lung surgery). Moreover, the prevalence of patients with psychological care found in our studies is also consistent with previous results found in patients with non-small lung cancer.20–23 The similarity with these studies may be due to the fact that most of the sample of patients had a diagnosis of malignancy or potential malignancy (as the definitive diagnosis was obtained after surgery).
The levels of emotional distress assessed in the screening tests for clinical cases or problem cases were very high and differed significantly from those patients recognized as non-clinical cases, showing the discriminant validity of the present study.
The medical judgment showed poor sensitivity to adequately detect patients requiring psychological treatment, yet its SP was good. The results of the present study confirm previous findings that show that physicians underestimate the number of clinical cases with emotional distress. For this reason, medical judgment is not a satisfactory method for the psychological referral of cancer patients.8,9 Physicians’ attention toward the patient is mainly focused on diagnostic aspects and on medical therapeutic procedures. In short, patients need to show intense levels of emotional distress for a doctor to refer them to psychological care, however, this has the advantage of avoiding false positives.
The HADS-T self-report, with the cutoff point >10, presented a good SE, and the NPV for this variable was very high, indicating that the test yielded a low number of false positives (non-problem cases diagnosed by the test as clinical cases). In addition, the AUC for such test was good. The HADS-A (>4) variable was not a suitable tool for the detection of patients with emotional distress in the present study, as it presented a large number of false positive cases. This may be due to the fact that, at the time of the evaluation (a few weeks prior to surgery), the patient had high levels of anxiety, thus, the HADS-A tended to overestimate the need for psychological attention. The HADS-D (>4) subscale had an acceptable diagnostic accuracy but was lower than that shown by the HADS-T questionnaire (>10).
The results of HADS-T were similar to those reported by Schellekens et al. with patients with lung cancer. They used the SCID questionnaire in comparison, with the cut-off point of 15 for the HADS-T, finding an SE=0.75 and SP=0.87.23 In the review by Vordermaier and Millman, the best screening tool for emotional disorders was the HADS-T scale with a cut-off of 10 or 11 (SE=0.80 and SP=0.74).16 The results obtained by Mitchell's review regarding the diagnostic accuracy of the HADS questionnaire were: SE=0.62; SP=0.77.17
The single-item interview assessing Depression (DEPQ) showed the best results with cut-off point greater than 1. It had an AUC of less than 0.80 (acceptable), and adequate SE and SP, similar to data found in other studies.17 This is not enough to consider it as a satisfactory screening tool.
The single-item interview assessing Anxiety (ANXQ) (>1), obtained very deficient levels of SP and a discrete value for AUC. Overall, it is a poor test. Results are lower than those found by Mitchell's meta-analysis, that obtained a SE=0.72 and SP=0.83.17
One of the limitations of the study is related to the fact that the sample consists of different pathologies and types of surgery, both neoplasms with a diagnosis of malignancy or non-malignancy, as well as non-neoplasms; but the small number of participants does not allow the study of the differences in the diagnostic accuracy of the screening tests for each type of pathology. Another limitation is that, at the time of being evaluated, the patients did not have the confirmation of their diagnosis, whether it was oncological with signs of malignancy or benign. This uncertainty may explain the fact that patients had high levels of preoperative anxiety and, therefore, the number of people qualified as clinical cases by screening tests: ANXQ and HADS-A have been very high and SP is very low.
However, the main contribution of the present study is that it evaluates the total sample of patients at the same time of the disease process: three weeks prior to lung surgery. The present study is also the first to show how to detect the need for psychological intervention more efficiently in patients with pulmonary pathology in the moments prior to surgery, using an extensive clinical interview as criteria.
The best instrument for identifying patients submitted to thoracic surgery that require psychological care, taking as a criterion a specialized psychological interview, was the HADS-T (cut-off=10). A high percentage (34%) of patients submitted to thoracic surgery presented a high level of emotional distress, which indicated the need to detect them as soon as possible and refer them to psychological care. The HADS-T is a short screening test that is simple and well accepted by patients, easy to implement in a thoracic surgery service and has a good diagnostic accuracy, deeming it useful for adequate referral.
Conflicts of interestThe authors declare that they have no conflict of interests.
Please cite this article as: Cruzado JA, Martínez García V, Salas Gutiérrez V, Jarabo Sarceda JR, Fraile Olivero CA, Fernández Martín E, et al. Implementación de un programa de cribado del malestar emocional en un servicio de cirugía torácica. Cir Esp. 2019;97:275–281.
