Psychometric study of the Required Care Levels for People with Severe Mental Disorder Assessment Scale (ENAR-TMG)

Abstract

Introduction

People with severe mental disorder have significant difficulties in everyday life that involve the need for continued support. These needs are not easily measurable with the currently available tools. Therefore, a multidimensional scale that assesses the different levels of need for care is proposed, including a study of its psychometric properties.

Method

One-hundred and thirty-nine patients (58% men) with a severe mental disorder were assessed using the Required Care Levels for People with Severe Mental Disorder Assessment Scale (ENAR-TMG), the Camberwell Assessment of Need scale, and the Health of the Nation Outcome Scales. ENAR-TMG's psychometric features were examined by: (a) evaluating 2 sources of validity evidence (evidence based on internal structure and evidence based on relations to other variables), and (b) estimating the internal consistency, temporal stability, inter-rater reliability, and sensitivity to change of scores of the ENAR-TMG's subscales.

Results

Exploratory factor analyses revealed a one-factor structure for each of the theoretical dimensions of the scale, in which all but one showed a significant and positive correlation with the Camberwell Assessment of Need (range of r: 0.143–0.557) and Health of the Nation Outcome Scales (range of r: 0.241–0.474) scales. ENAR-TMG subscale scores showed acceptable internal consistency (range of ordinal α coefficients: 0.682–0.804), excellent test–retest (range of intraclass correlation coefficients: 0.889–0.999) and inter-rater reliabilities (range of intraclass correlation coefficients: 0.926–0.972), and satisfactory sensitivity to treatment-related changes (range of η²: 0.003–0.103).

Conclusions

The satisfactory psychometric behaviour of the ENAR-TMG makes the scale a promising tool to assess global functioning in people with a severe mental disorder.

Resumen

Introducción

Las personas con trastorno mental grave (TMG) presentan importantes dificultades en la vida cotidiana que conllevan la necesidad de una ayuda continuada. Estas necesidades no son fácilmente evaluables con los instrumentos actuales. Por ello se propone una escala multidimensional que evalúa los diferentes niveles de necesidad de atención, y se estudia el comportamiento psicométrico de dicho instrumento.

Método

Ciento treinta y nueve pacientes (58% hombres) con trastorno mental grave fueron evaluados con la Escala de valoración de los Niveles de Atención Requerida para personas con Trastorno Mental Grave (ENAR-TMG) y las versiones españolas del Camberwell Assessment of Needy la Health of the Nation Outcome Scales. Se examinó el comportamiento psicométrico de la ENAR-TMG: a) aportando evidencias de validez basadas en su estructura interna y en la relación con otras variables, y b) estimando la consistencia interna, la estabilidad temporal, la fiabilidad entre evaluadores y la sensibilidad al cambio de las puntuaciones de la ENAR-TMG.

Resultados

Con respecto a las evidencias de validez, cada una de las dimensiones teóricas del instrumento mostró una estructura unifactorial y, menos en un caso, una correlación significativa y positiva con las escalas Camberwell Assessment of Need (rango de r: 0,143-0,557) y Health of the Nation Outcome Scales (rango de r: 0,241-0,474). Asimismo, las puntuaciones de cada una de dichas dimensiones mostraron una consistencia interna aceptable (rango de coeficientes α ordinal: 0,682-0,804), unas excelentes fiabilidades test-retest (rango de coeficientes de correlación intraclase: 0,889-0,999) y entre evaluadores (rango de coeficientes de correlación intraclase: 0,926-0,972), y una sensibilidad al cambio satisfactoria (rango de η²: 0,003-0,103).

Conclusiones

El adecuado comportamiento psicométrico de la ENAR-TMG la convierte en un prometedor instrumento para la evaluación del estado y las necesidades de las personas con trastorno mental grave.

Introduction

People with severe mental disorder (SMD) have significant difficulties which increase their fragility in facing everyday events and these are associated with the need for continuous assistance in achieving the highest possible degree of personal autonomy.^1,2 Many people with SMD therefore find it difficult to independently cover their housing needs themselves, and to maintain decent accommodation in keeping with their requirements and wishes. Added to this are other factors generated by the disorder itself, which hinder or impede the development of the patient's functional skills in relation to aspects of daily life, such as personal hygiene, self-care, self-control or interpersonal relationships. These too may interfere with or limit the process required for searching, finding and securing accomodation.³

The lack of sufficient accommodation also contributes to a combination of negative consequences, such as undue usage of hospital care, the prolongation of hospital stays for psychosocial instead of clinical reasons or, failing this, continuous readmittance due to falls on returning to the home. Furthermore, because responsibility for care and support often falls to the families, situations of overburden and tension arise. Another negative consequence is the risk of marginalisation as people see themselves compelled to live on the streets.^4,5

In the Western social model, there are 3 basic aspects which confirm that a person is fully integrated: home, work and leisure. Within this trio, the home not only preserves privacy and is the starting point of integration for the person to feel they belong and to identify with their community, but for people with SMD, it is the platform from where they may develop their psychosocial rehabilitation goals.⁶ The resources to be implemented must be adapted to the different stages and needs for support which the person experiences, bearing in mind their socio-economic conditions, skills and the degree of autonomy they have in carrying out their everyday life activities.^7,8

Different studies confirm the relative lack of available resources to address the great complexity of patient profiles and their different levels of needs (based on their abilities to carry out everyday activities, how they function in society, their degree of psychopathological stability or the presence of specific general health problems).^9,10

Given the lack of specific assessment tools, our group developed and validated a new scale which: (a) would serve as a tool for measuring the need for care required by the users who were in homes for people with SMD, and (b) would also lead to the assessment of the evolution of these people: scale for the assessment of residential attention levels, for people with severe mental disorder (ENAR-CPB).¹¹ Prior to its design and development any tools whose purpose regarding variables and target population were similar to our own, and which were available in our geographical area of reference, were revised to see whether they could be used and how relevant they were. After reviewing fifteen scales, we observed that these tools separately assessed: (a) relevant constructs such as quality of life or social functioning (Camberwell Assessment of Needs questionnaire [CAN],^12,13 Everyday life skills profile,¹⁴ Quality of life questionnaire,¹⁵ among others), and (b) clinical status (Health of the Nation Outcome Scales [HoNOS],¹⁶ Short psychiatric assessment scale,¹⁷ Global activity assessment scale,¹⁸ among others). Furthermore, these tools were mostly overly long and had non specific response scales, which required a regularly fairly subjective assessment and which provided little precise information on the genuine difficulties of the residents. As a result, we decided to design and develop a specific scale. The validation study results show that the ENAR-CPB scale is a tool of measurement with a satisfactory psychometric pattern and presents the potential use of this tool not only for residents of homes for people with SMD, but also for anybody with SMD.¹¹

As a result of this, we decided to go a step further and partially modify the ENAR-CPB scale, keeping its structure in 6 subscales: (a) clinical dimension; (b) physical dimension; (c) everyday life activities; (d) autonomy and self-governance; (e) interpersonal relationships, and (f) personal well-being and social environment. Our aim was for it to be of use to the whole population with SMD to assess the required levels of attention beyond residential home care. The dual purpose of this study was therefore to: (a) develop a scale which assesses the degrees of need for attention in the population with SMD, and (b) examine the psychometric behaviour of this tool. The hypotheses put forward relating to this last point are that the theoretical subscales of the new tool will present a one-dimensional structure and a positive and statistically significant correlation with CAN and the HoNOS scale.

Method

Design

This was designed as an observational, single centred and prospective study.

Participants

The sample under study comprised 139 patients with SMD, with a mean age of 49.98 years (SD = 10.52; range: 24–85), mostly men (57.6%), mainly with a diagnosis of paranoid schizophrenia (30%), residual schizophrenia (25%), other schizophrenias (19%) or schizoaffective disorder (15%) (Table 1).

Table 1.

Distribution by patient diagnosis (n = 139).

Diagnosis	n	%
Paranoid schizophrenia	42	30.2
Residual schizophrenia	35	25.2
Other schizophrenias	26	18.7
Schizoaffective disorder	21	15.1
Affective disorders	8	5.8
Obsessive compulsive disorder	5	3.6
Other disorders	2	1.4

All patients, selected through convenience sampling, were users of the Mental Health Centre for Adults (MHCA) Dreta de l’Eixample (CPB-SSM), diagnosed through an unstructured clinical interview by their psychiatrists in accordance with the DSM-IV standards and with a diagnosis which was compatible with that of the SMD according to the Work Group of the Clinical Practice Guide of Psychosocial Interventions in Severe mental disorder.¹⁹ All patients were under pharmacological treatment established by their psychiatrists in the MHCA and the course of their condition had lasted for over 20 years. Furthermore, several of them were being treated in other departments or resources, such as the Community Rehabilitation Service, the Customised Service Programme and the Support for autonomy in the Home, or lived in homes for people with SMD.

Tools

Required care levels for people with severe mental disorder evaluation scale

This is an adaptation/modification of the ENAR-CPB¹¹ scale to be used in the whole population with SMD (not only in the case of patients cared for in homes for people with SMD). A detailed presentation of the design and development of the ENAR-CPB scale may be consulted in Lascorz et al.¹¹

For the development of the assessment scale of the required care levels for people with severe mental disorder (ENAR-TMG) an interdisciplinary team was formed (comprising a psychologist, a nurse and a social worker), who carried out a review of the ENAR-CPB¹¹ scale, partially changing some of the items to match the general population with SMD and replacing several others to incorporate aspects which had not been included. Specifically, the redaction of the items referring to care of personal space were modified (in the ENAR-CPB scale) to adapt to care of the home and domestic tasks (in the ENAR-TMG scale). Items relating to family relationships and occupation of time were rewritten so as not to refer to activities inside the care home, and finally the item referring to care of physical appearance was removed, and replaced by one which assessed the ability to ensure of a proper diet. As a result of the review an initial version was designed by maintaining the same structure as the original instrument, subdivided into 6 dimensions or sections: (a) clinical dimension; (b) physical dimension; (c) everyday life activities; (d) autonomy and self-governance; (e) interpersonal relationships, and (f) personal well-being and social environment). A brief pilot study was conducted with this first version of the ENAR-TMG with a small number of participants and the drafting of some of the items being partially modified. The final tool consisted of a total of 24 items formulated as shown (see Appendix 1), divided into the 6 before-mentioned sections (4 items for each dimension) to be evaluated and given a score by the interviewer using 5 possible response options (from lower [0] to greater [4] need for attention).

Camberwell assessment of needs questionnaire^12,13

The CAN scale leads to the assessment of needs in 22 basic areas: accommodation, diet, care of the home, personal care, daily activities, physical health, psychotic symptoms, information regarding condition and treatment, anxiety, self-assurance, trust in others, alcohol, drugs, company, relationships with partner, sexuality, care of children, basic education, telephone, transport, money and social aids. For each area the person is questioned about the existence of a current need, informal aid (friends and family) is assessed together with formal aid (health and social services) which they receive and the help they require.

Health of the Nation Outcome Scale¹⁶

The HoNOS scale is a clinical evaluation tool administered by a third party which comprises 12 scales or items covering 4 different areas: behavioural problems (aggressive and hyperactive behaviour, self-inflictive aggressive behaviour, use of psychoactive substances), impairment (cognitive, physical health), clinical problems (hallucinations and delirium, depression, other mental symptoms) and social problems (social relations, general functioning, accommodation problems, occupational problems.)

Procedure

The study, approved by the Research and Ethics Commission of the CPB-SSM, took place in the departments linked to the MHCA Dreta de l’Eixample of the city of Barcelona, during 2012 and 2013. The tools were completed, after the obtainment of informed consent, by professionals (psychologists, nurse, social worker or educator) who were the practitioners for each patient depending on the type of resource where they were treated (MHCA, Community Rehabilitation Service, Support programme for autonomy in one's home, Programme of customised or care home services). The patients did not receive any incentive for their participation in the study and they all agreed to participate.

Between the months of May and July 2012 an initial evaluation was made of the participants with the ENAR-TMG, CAN and HoNOS scales. In determining the reliability between evaluators of ENAR-TMG, 30 of the said patients were assessed independently by 2 professionals of the same department. Furthermore, to estimate the temporary stability of the ENAR-TMG scores, this questionnaire was completed again–by the same evaluator who had carried out the first evaluation–after 2 weeks in the case of one subgroup of 14 patients. Finally, to study sensitivity to the change of scores of the ENAR-TMG questionnaires, between the months of May and July 2013 (i.e. one year after the initial assessment), a subgroup of 125 patients were reassessed with this tool and with the HoNOS scale.

Data analysis

Prior to specifying the different analysis stages or phases and the statistical techniques or the calculators used in each of these phases we should point out that all the analyses performed within the framework of the item analysis process which characterised the psychometric quality of the instrument were a constant reference to the original theoretical dimensions and variables the developed tool was to include (see Lascorz et al.¹¹). Efforts were made to be as consistent and respectful as possible, despite being detrimental in some cases to the metric quality of the final tool.

All the items were initially subjected to a descriptive analysis, and for each the mean, variance, kurtosis and skewness indexes were calculated and the percentage of participants to whom each of the different response options were assigned.

The items of each domain or subscale were submitted to an exploratory factorial analysis (EFA). Since the items were ordinal and in the case of some of the subscales, the data varied from the multivariate normality,²⁰ on current recommendations,^21–23 we opted for EFA from the polychoric matrices and using the method of calculation of unweighted square minimals. The method used to determine the number of factors to retain was the optimised parallel analysis based on minimum range factorial analysis, a procedure which has been proven to be the best option in these cases.^24,25 Furthermore, for each subscale the adjustment to a single factorial structure was assessed using the adjusted goodness of fit index (GFI) and the residual mean square root index (RMSR).

To study the reliability of internal consistence of the scores of each subscales or factors, we calculated the respective α coefficients for ordinal data. For the study of homogeneity, the respective coefficients of corrected item-total correlations were calculated. Reliability was also assessed as temporary stability for each of the subscales using the estimation of the corresponding intraclass correlation coefficients between the 2 administrations (separated by 15 days) of the tool to a subgroup of 14 participants. Moreover, in one subgroup of 30 patients, reliability between evaluators was assessed for each of the subscales using the estimation of the coefficients of intraclass correlation.

To obtain proof of validity of the relationship with other variables, the corresponding Pearson correlation coefficients were calculated between each of the subscales of the ENAR-TMG and 2 tools which evaluated similar or related constructs (CAN and HoNOS scales).

Lastly, the sensitivity to change study of the scores of the ENAR-TMG subscales was performed using: (a) variance of repeated measurements analysis (each one of the 2 administrations of the tool separated by 12 months), and (b) calculation of the size of the effect of improvement or change in the score of the different subscales and subsequent comparison of these effect sizes with the change in overall score of the HoNOS scale.

FACTOR 9.3.1²⁶ and IBM SPSS version 22²⁷ programmes were used for data analysis

Results

Item analysis

Table 2 shows the mean and variance of the 24 items of the ENAR-TMG, as well as the corresponding skewness and kurtosis indexes and the distribution of patients according to the different response options. In a limited but not insignificant number of items (6), skewness and kurtosis coefficients surpassed the recommended interval of ±1. None of the items presents a variance equal to or below 0.250,²⁸ and only in the case of a single item (item 1) did 80% or more of the sample score with the same response option.²⁹

Table 2.

Mean, variance, skewness and kurtosis and distribution of the patients according to the responses, for the ENAR-TMG (n = 139) scale.

ENAR-TMG	Mean	Variance	Skewness	Kurtosis	Percentage of patients according to the response options
					0	1	2	3	4
Clinical dimension
Item 1	.338	.943	2.837	6.694	87.77	1.44	3.60	3.60	3.60
Item 2	.914	.942	0.887	0.207	42.45	31.65	19.42	5.04	1.44
Item 3	1.957	1.192	0.186	−0.972	5.76	36.69	21.58	28.06	7.91
Item 4	.835	1.016	1.272	1.218	47.48	31.65	13.67	4.32	2.88
Physical dimension
Item 5	1.590	1.666	0.376	−1.068	23.74	32.37	13.67	21.58	8.63
Item 6	1.151	.948	0.634	−.131	28.06	40.29	21.58	8.63	1.44
Item 7	1.554	1.657	0.430	−.880	25.90	27.34	22.30	14.39	10.07
Item 8	2.209	1.244	−0.013	−.736	5.76	21.58	33.81	23.74	15.11
Everyday activities
Item 9	1.201	.967	0.363	−.885	28.06	35.97	23.74	12.23	0
Item 10	1.770	1.127	0.036	−.676	12.95	27.34	33.81	21.58	4.32
Item 11	2.209	1.057	−0.628	−.115	9.35	10.79	35.25	38.85	5.76
Item 12	1.201	.952	0.709	.135	24.46	43.88	20.86	8.63	2.16
Autonomy and self-governance
Item 13	2.655	1.046	−.201	−1.073	0	16.55	25.90	33.09	24.46
Item 14	1.223	0.821	.362	−0.617	22.30	43.17	24.46	10.07	0
Item 15	1.446	1.686	.517	−0.832	30.94	25.18	21.58	12.95	9.35
Item 16	1.590	1.278	.347	−0.661	17.99	33.09	26.62	16.55	5.76
Interpersonal relationships
Item 17	1.24.	1.839	0.803	−.641	41.01	25.18	12.23	11.51	10.07
Item 18	1.928	1.952	−0.062	−1.324	23.02	17.99	16.55	28.06	14.39
Item 19	1.137	1.053	0.648	−.206	32.37	33.81	23.74	791	2.16
Item 20	1.561	1.009	0.324	−.469	13.67	38.13	29.50	1583	2.88
Personal well-being and integration into the social environment
Item 21	1.784	1.565	.149	−.924	19.42	22.30	29.50	17.99	10.79
Item 22	1.597	1.363	.337	−.662	20.14	28.78	29.50	14.39	7.19
Item 23	2.266	1.965	−.105	−1.393	10.79	28.06	12.95	20.14	28.06
Item 24	2.331	1.099	−.772	−.190	7.91	13.67	22.30	49.64	6.47

^aData expressed as percentages

Proof of validity based on the internal structure

Following confirmation of the appropriateness of the polychoric correlations matrix for factorisation in each subscale (in all cases the Kaiser–Meyer–Olkin sample adequacy measurement was acceptable [KMO range: .624–.747] and the Bartlett sphericity test was highly significant [p < .0001 in all cases]), we proceeded with the EFA of the items comprising each subscale.

The EFA of the different domains or subscales of the ENAR-TMG, performed with the non weighted least squares method, revealed that each of the said subscales presented a unifactorial structure in accordance with the optimised parallel analysis procedure. In keeping with these results, the unifactorial structure of each subscale showed appropriate adjustment to the data in accordance with the usually considered criteria (GFI > .95 and RMSR < .05 or RMSR < 1/√n⁻¹)^30–33: Clinical dimension, GFI = .99 and RMSR = .0554; Physical dimension, GFI = 1.00 and RMSR = .0482; Everyday activities, GFI = 1.00 and RMSR = .0420; Autonomy and self-governance, GFI = 1.00 and RMSR = .0122; Interpersonal relationships, GFI = 1.00 and RMSR = .0410; and Well-being and integration into the social environment, GFI = 1.00 and RMSR = .0369. All items of the different subscales presented factorial weightings (range: .362–.902; Table 3) above the usually required minimum of .32,³⁴ with these factorial weightings being considered very well (>.63)³⁴ in three quarters of the ENAR-TMG items.

Table 3.

Commonality, factorial weighting, corrected item-total correlation and ordinal α coefficient if the item for the ENAR-TMG scale is eliminated (n = 139).

ENAR-TMG	Commonality	Factorial weighting	Corrected item-total correlation ^*	Ordinal α coefficient if the item is eliminated
Clinical dimension
Item 1	.395	.629	.326	.623
Item 2	.239	.489	.299	.682
Item 3	.415	.644	.595	.617
Item 4	.439	.633	.442	.610
Physical dimensions
Item 5	.414	.644	.532	.755
Item 6	.704	.839	.708	.686
Item 7	.484	.696	.584	.736
Item 8	.371	.609	.565	.768
Everyday activities
Item 9	.415	.644	.556	.777
Item 10	.813	.902	.778	.689
Item 11	.410	.641	.592	.778
Item 12	.448	.670	.612	.768
Autonomy and self-governance
Item 13	.180	.424	.374	.776
Item 14	.538	.734	.599	.652
Item 15	.464	.681	.574	.672
Item 16	.612	.782	.631	.635
Interpersonal relationships
Item 17	.131	.362	.209	.711
Item 18	.243	.493	.330	.651
Item 19	.468	.684	.491	.569
Item 20	.745	.863	.616	.506
Personal well-being and integration into the social environment
Item 21	.807	.898	.703	.584
Item 22	.346	.588	.513	.697
Item 23	.440	.663	.572	.679
Item 24	.229	.479	.429	.742

*

p < .01 in all cases.

Internal consistency of ENAR-TMG scores

As we can see in Table 4, the scores of the different subscales presented mainly values of internal consistency (range of α ordinal coefficients: .604–.814) between acceptable and good.^28,35 In fact, it is only in the case of 2 items (items 13 and 17) that their elimination increased the internal consistency of the scores of the subscale in over .02 points³⁶ (Table 3). Moreover, all items of the different subscales–except the same item 17–presented values of corrected item-total correlation (Table 3) above the usual cut-off point of .30,³⁷ and these results were indicative that the scores of the different subscales presented appropriate homogeneity.

Table 4.

Internal consistency, temporary stability and reliability between evaluators of ENAR-TMG scores.

ENAR-TMG	Internal consistency (ordinal α coefficient) (n = 139)	Temporary stability (intraclass correlation coefficient and 95% confidence interval) (n = 14)	Reliability between evaluators (intraclass correlation coefficient and 95% confidence interval) (n = 30)
Clinical dimension	.698	.975 (.923–.992)	0.940 (0.873–0.971)
Physical dimension	.789	.979 (.935–.993)	0.926 (0.844–0.965)
Everyday activities	.804	.951 (.847–.984)	0.952 (0.900–0.977)
Autonomy and self-governance	.746	.87 (.958–.996)	0.944 (0.882–0.973)
Interpersonal relationships	.682	.889 (.654–.964)	0.932 (0.857–0.968)
Personal well-being and integration in the social environment	.739	.999 (.996–1.000)	0.972 (0.940–0.987)

Temporary stability of ENAR-TMG scores

The temporary stability or test-retest reliability of the scores of the different ENAR-TMG subscales, calculated from the intraclass correlation coefficients (range: .889–.999; Table 4), may be classified as excellent.³⁵

Reliability between evaluators

The reliability between evaluators when scoring on the ENAR-TMG questionnaire may be considered as excellent³⁵ in the case of each of their subscales or domains (range of coefficients of intraclass correlation: .926–.972; Table 4).

Proof of validity based on the relationship with other variables

Table 5 presents the correlations between the scores of the different subscales of the ENAR-TMG and those of 2 scales (CAN and HoNOS scales) which assess similar or related constructs, with statistically significant correlations being observed and with an effect size between slight and moderate³⁸ in all cases, except in the correlation of the score of the subscale clinical dimension with that of the CAN scale.

Table 5.

Proof of validity based on the relationship of the ENAR-TMG with other variables.

ENAR-TMG	CAN (n = 64)	HoNOS (n = 75)
Clinical dimension	.143	.466^**
Physical dimension	.557^**	.474^**
Everyday activities	.552^**	.432^**
Autonomy and self-governance	.489^**	.352^**
Interpersonal relationships	.422^**	.241^*
Personal well-being and integration into the social environment	.411^**	.298^**

Data expressed as coefficients of Pearson correlation.

*

p < .05.

**

p < .01.

Sensitivity to the change of scores of the ENAR-TMG

Finally, with regarded to sensitivity to change in the ENAR-TMG scores, Table 6 shows the means and standard deviations of each subscale at the beginning and end of a follow-up period of 12 months. The change or improvement was statistically significant in all subscales except the personal well-being and integration into the social environment. It is also of note that save in the case of this subscale, the change effect size found in the ENAR-TMG scores (range of η²: .032–.103), though minor,³⁸ is above that obtained in the case of the HoNOS (η² = .003) scale.

Table 6.

Sensitivity to change in ENAR-TMG scores (n = 125).

ENAR-TMG	Baseline score (mean and SD	Score after a year (mean and SD)	Significance (p value)	Size of the effect (η^2)
Clinical dimension	4.120 (2.864)	3.552 (2.513)	.001	0.091
Physical dimension	6.608 (3.735)	6.024 (3.620)	.003	0.070
Everyday activities	6.520 (3.330)	5.952 (3.369)	<0.001	0.098
Autonomy and self-governance	6.992 (3.388)	6.648 (3.387)	.044	0.032
Interpersonal relationships	5.936 (3.237)	5.320 (3.028)	<0.001	0.103
Personal well-being and integration into the social environment	8.104 (3.758)	7.984 (3.778)	.515	0.003

Discussion

The purpose of this study was to develop an adaptation of the ENAR-CPB to assess the levels of need for care for all people with severe mental disorders and to examine the psychometric behaviour of this tool. The final tool called ENAR-TMG, is a 24 item scale–4 for each of the 6 theoretical dimensions: (a) clinical dimension; (b) physical dimension; (c) daily life activities; (d) autonomy and self-governance; (e) interpersonal relations, and (f) personal well-being and integration in the social environment–the psychometric quality of which was satisfactory.

The results of the initial process of analysis of the items pointed towards the elimination of a single item which generated overlay unanimous and scarcely discriminative responses. However, the low number of items in each dimension or subscale was in favour of continuing to consider this item as a relevant indicator of its subscale. Proof of validity related to the internal structure of the ENAR-TMG, obtained through exploratory factorisation procedures, could be considered satisfactory at the discovery that each of its 6 theoretical dimensions or subscales presented a unifactorial structure. Reliability analysis as internal consistency and homogeneity shows acceptable results for the 6 domains of the ENAR-TMG. The fact that in none of the subscales is the ordinal α coefficient value in a very good or excellent range could be explained by the low number of items (4) comprising each subscale.^39,40 For its part, reliability between evaluators for each of the 6 areas of evaluation may be classified as excellent. Beyond the quality of the tool the extensive knowledge on the status and situation of the patient by the evaluators may have contributed to this result. Furthermore, the test–retest reliability shows that the scores of the 6 ENAR-TMG subscales also present excellent temporary stability. With regard to the evaluation of the level of association with similar constructs, which were operational through the CAN and HoNOS scales, congruent correlations were obtained with them. Thus, with the logical exception of the clinical dimension, correlations of the ENAR-TMG subscales with the overall clinical scores from the other 2 tools applied are of higher magnitude (approximately 10%–15% more) in the case of the CAN scale, i.e. less comparatively focalised with the HoNO in clinical aspects. These differentiated results support evidence of the validity of using ENAR-TMG based on its relationship with other variables. Finally, sensitivity to change of scores of practically all of its subscales maybe considered as satisfactory, as there were statistically significant differences, albeit of small size, in 5 of the 6 subscales comparing 2 administrations of this tool which war separated by one year. In the area that concerns us, a small effect size does not prevent the classification of sensitivity to change as being satisfactory if we consider that: (a) this is a sample mainly made up of people with psychotic disorders of a chronic nature and (b) for the majority of these people no clinical and/or social improvements of great magnitude are expected. In any case, the size of the effect of the change in the only subscale of the ENAR-TMG where no statistically significant change was observed o (personal well-being and integration into the social environment) is identical in size to the effect of change, which is equally statistically insignificant, observed in the HoNOS scale. The latter is a widely used tool for measuring global functioning in any different geographical areas.^41,42

The results obtained in our study should be interpreted in the light of certain characteristics which could limit the generalisation of these result to other samples of patients with SMD. Among these characteristics the non random selection process of the participants is of note, together with its single centre aspect–centre here being understood as the closely linked different departments with a single mental health centre–and the exhaustive level of information available from the respective references/informing agents on the status and situation of the assessed patients. Furthermore, the total sample size impeded making disaggregated analysis by gender,⁴³ and the difference of the sample size in some analysis could have been used as a selection bias. Finally, sample size in the test–retest reliability analysis could be considered insufficient when compared with normal standards.⁴⁴

These limitations pinpoint the need for further studies in more extensive and diverse patient and evaluator samples in order to verify the results obtained in this initial study on the ENAR-TMG. Notwithstanding, the study's results are satisfactory with regard to the psychometric behaviour of this tool. To conclude, the satisfactory psychometric behaviour of the ENAR-TMG makes the scale a promising tool with which to the assess status and needs of people with severe mental disorder.

Ethical disclosures

Protection of people and animals

The authors declare that no experiments have been performed on humans or animals for this research.

Data confidentiality

The authors declare that no patient data appear in this article.

Right to privacy and informed consent

The authors declare that no patient data appear in this article

Conflict of interests

There was no conflict of interests in this study.