Reliability of the Mental Health Nurse Competency Assessment Tool (ECOEnfSM): a pilot study

Abstract

Aim

To evaluate the reliability of the ECOEnfSM scale as a tool to assess the professional competencies of Mental Health Nurse Practitioners (MHNP) in their clinical practice.

Method

A pilot study, observational, descriptive and cross-sectional, focuses on MHNP who have completed their Specialized Health Training Program (SHTP) in Spain. The data were collected by general and collaborators mentors of the Multiprofessional Teaching Units (MTUs). The Mental Health Nursing Competency Assessment Tool (ECOEnfSM) was used, which consists of three subscales and eight Competence Units (CU). A reliability and validity analysis were conducted (Cronbach's alpha and Spearman's correlation coefficient).

Results

The Rotation Assessment subscales showed excellent reliability (r > 0.90) with high and very high correlations (r > 0.6) in all UCs with high levels of significance (p < 0.01). The Annual Assessment subscale showed good reliability (r > 0.80) with a medium and very high correlation (r > 0.4) with high levels of significance (p < 0.01). All UCs showed a good to excellent consistency (r > 0.80). The "Home Visits" assessment criterion showed heterogeneity of data due to there are few MTUs that fully develop it.

Conclusion

The ECOEnfSM scale showed very high reliability in MHNP during their training program in Spain. The ECOEnfSM is considered the only objective tool in Spain to assess the professional competencies in this population.

Resumen

Objetivo

Evaluar la fiabilidad de la escala ECOEnfSM para la evaluación de competencias de los Enfermeros Interno Residentes (EIR) de Salud Mental en su práctica clínica.

Método

Estudio piloto, observacional, descriptivo y transversal, dirigido hacia el EIR de Salud Mental cursando la Formación Sanitaria Especializada (FSE) en España que haya completado su programa formativo. Los datos fueron recogidos por tutores generales y colaboradores de las Unidades Docentes Multiprofesionales (UDM) de Salud Mental. Se utilizó la escala de Evaluación de Competencias Enfermeras en Salud Mental (ECOEnfSM), compuesta por tres subescalas y ocho Unidades de Competencia (UC). Se realizó un análisis de fiabilidad y validez (α de Cronbach y correlación Spearman).

Resultados

Las subescalas de Evaluación de Rotación obtuvieron una fiabilidad excelente (r > 0,90) con niveles de correlación altos y muy altos (r > 0,6) entre todas sus UC con niveles altamente significativos (p < 0,01). La subescala de Evaluación Anual obtuvo una fiabilidad buena (r > 0,80) con niveles medios y muy altos de correlación (r > 0,4) altamente significativos (p < 0,01). Todas las UC obtuvieron una consistencia de buena a excelente (r > 0,80). El criterio “Visita domiciliaria” sufrió gran disparidad en sus datos ya que existen pocas UDM que la desarrollen por completo.

Conclusiones

La escala ECOEnfSM obtuvo una fiabilidad muy alta en EIR de Salud Mental durante su programa de residencia en España. La escala ECOEnfSM se considera la única herramienta objetiva en España para evaluar las competencias profesionales de esta población.

What is known?

In Spain, the assessment of specialised health care training courses does not differentiate between different professions or specialties. We only found one specific competency assessment scale for mental health nursing but its use is not widespread.

What does it contribute?

This is a validated and specific tool for the evaluation of competencies of mental health nurses. adapted to their clinical practice. to thus be able to undertake continuous, annual assessment following the official Spanish evaluation guidelines.

Introduction

Nurses specialising in mental health are the basis of care for the mentally ill and are defined as "the health care professionals who provide specialised mental health care through the provision of nursing care”.¹

Their training has experienced major ups and downs, affected by cultural and political aspects in Spain throughout history, from Psychiatric Nurse to Mental Health Nurse Practitioner. The way of acquiring this qualification has also evolved, as has the assessment of the professional skills to be developed.²

There is official documentation on the professional competencies that Mental Health Nurse Practitioners must acquire through their training course, which details the criteria to be met and the activities to be undertaken, as well as the pace of the assessment (Order SPI/1356/2011).¹

However, the figure in charge of the assessment of the mental health nurse practitioner (MHNP) does not benefit from any unified type of assessment, with different criteria being applied, depending on the particular Multidisciplinary Teaching Units (MTUs) in mental health in Spain.

They have only basic guidelines for running competency assessments provided by the Ministry of Health. However, these official documents are not very complete, too generic, leaving room for the subjectivity of the evaluator and, above all, are addressed equally to all the professional disciplines that are included within Specialised Health Training (FSE): Biology, Nursing, Pharmacy, Physics, Medicine, Psychology or Chemistry.³

Due to this great limitation, Rivera develops a scale that is able to evaluate these professional competencies acquired by Mental Health EIRs, termed the Assessment of Nursing Competencies in Mental Health Scale (ECOEnfSM).⁴

Currently, there are numerous studies that measure professional competencies but are focussed on Psychiatric Nursing students.^5,6 Hartley et al focus on the development of training courses aimed at nurses working in Psychiatry, which improve their knowledge and attitudes.⁷ However, it is difficult to find evidence in the population of mental health nurses.

On the other hand, new technologies have favoured new ways of acquiring skills. Piot et al and others compile studies focussed on learning with high-fidelity simulations, virtual reality, mannequins or voice simulations, which improve the knowledge, skills and attitudes of nurses and psychiatry students.^8–10 However, this progress is detrimental to experiential learning through clinical practice.

Hawsawi et al and Dickens et al highlight studies for the acquisition of competencies for Mental Health Nurses in areas such as physical health (obesity, diabetes, toxic substances??, sexual health, etc.), recovery, suicide or pharmacological treatments. These courses favour learning but they focus on very specific topics within clinical practice, do not cover the totality of the functions of a Mental Health Nurse and are often brief.^11–13

Piot et al argue that it is necessary to standardise educational quality criteria and structure curricula.⁸ On the other hand, Immonem et al argue that it is necessary to use reliable and valid instruments, as the tutors stress the importance of having clear assessment criteria and further training on assessment.⁵

There are other assessment scales focussed on nursing students and Paediatric or Medical-Surgical Nursing,^14–17 but the ECOEnfSM scale stands out as the only specific tool able to evaluate Mental Health Nursing staff in their clinical practice in a complete, objective, and standardised way, based on official documents.^1,3,4

However, its use has not currently spread to hospitals and Mental Health UDMs in Spain, so the ECOEnfSM scale has not been validated in clinical practice. From this derives the objective of the present study, which was to evaluate the reliability of the ECOEnfSM scale as a tool for assessing the professional competencies of Resident Mental Health Nurses (EIRs) in their clinical practice.

Method

Design

A pilot, observational, descriptive and cross-sectional study was conducted.

Population and scope

Resident Nurse Interns (EIR) taking the Specialised Health Training Programme me (SHTP) in Mental Health Nursing at one of the Multiprofessional Teaching Units (MTUs) for mental health in Spain.

Only those residents who fully complied with the 2-year residency training course were admitted to the study. There were no further exclusion criteria.

Variables

The Assessment of Nursing Competencies on the ECOEnfSM mental health scale was used, which assesses the professional competencies of mental health nurse practitioners during their SHTP training. This is a heteroadministered, validated scale (Content Validity Index = 0.96, Cronbach’s α = 0.968), developed by Rivera (2021),⁴ based on the ECOEnf scale (Gómez del Pulgar, 2011), where the competencies of nursing graduates are evaluated.¹⁷

This is composed of 3 subscales and 8 headings or competence units (CUs). Each CU is composed of assessment criteria and level descriptors (basic, intermediate and advanced levels). The ECOEnfSM scale has a total of 62 assessment criteria and 186 level descriptors (Supplementary Material 1).

It is subdivided into three subscales:

• -

  Rotation Assessment: this is an assessment after having completed the practical training for each of the health care devices or rotations on the SHTP programme me. This is subdivided into generic rotations, coupled with rotation at the mental health centres. According to the Mental Health Nurse Practitioner course,¹ the training itinerary and its duration must adhere to the following guidelines:

Health care unit or rotation
First year
Long-Stay Rehabilitation Units	2 months
Acute Patient Hospitalisation Unit	6 months
Other units: Addictions, Day Hospital, Refractory Psychosis Unit, Dual Pathology Unit, Eating Disorders Unit, Psychosomatic Disorders Unit	3 months
Second year
Mid-stay unit	4 months
Community Rehabilitation Programmes
Mental Health Centre	5 months
Child and Adolescent Mental Health Programmes or Services	2 months
Source: Order SPI/1356/2011, of 11^th May, which approved and published the training programme me for the specialty of Mental Health Nursing. Spanish Official State Gazette, number 123 (24th May 2011).^1

• • o

    Generic Rotation Assessment: all specific CUs are evaluated (CU1-Assessment and diagnosis, CU2-Planning, CU3-Intervention and CU4-Assessment) except for the assessment criteria of "Home Visit", together with a cross/disciplinary CU (CU8-Communication and interpersonal relationships).

  • o

    Rotation Assessment in Mental Health Centres (MHCs): all specific CUs are evaluated (CU1-Assessment and diagnosis, CU2-Planning, CU3-Intervention and CU4-Evaluation) including the assessment criteria of "Home Visit", together with a cross-sectional CU (CU8-Communication and interpersonal relationship).

• -

  Annual Assessment: evaluation undertaken in the SHTP training year after having completed all the Rotation Assessments for that same year. The rest of the cross-sectional CUs (CU5-Research, CU6-Clinical Management and CU7-Teaching) are evaluated.

Passing the assessment criteria will depend on the year of residency of the mental health hospital practitioners (Supplementary Material 2). The final assessment ranges from 0 to 10 points ((number of criteria passed/total CU criteria) x 10), provided that the minimum and essential number of assessment criteria has been approved. Subsequently, the weighting percentages are applied (Supplementary Material 3).

Data collection

The data was collected by the residents' general tutors and the collaborating tutors for each mental health nursing practitioner (MHNP) rotation in each mental health multiprofessional unit (MTU) in Spain through the website developed ‘ad-hoc’ (https://ecoenfsm.es/). Various meetings were held, as well as communications at scientific events, dissemination by messaging and advertising on social networks to contact the tutors participating. Once contacted, they were assigned a personal "Tutor Code" to access registration on the website and fill out the assessment headings, thus ensuring truthful data collection. Non-probabilistic convenience sampling was undertaken.

The assessments were done at the end of each rotation and on an annual basis. The data collection process spanned from October 2022 to April 2024, obtaining a total of 64 assessments.

Data was collected to demonstrate the reliability of the scale, according to a study using the ECOEnf scale by Gómez del Pulgar, where 66 participants were needed to answer the same objective.¹⁸

Data analysis

SPSS v.23.0 was used for a reliability analysis (internal consistency) applying Cronbach's α statistic to measure intraobserver reliability, according to George and Mallery (2003)⁹: excellent at α between 0.9 and 0.95, good at α > 0.8, acceptable at α > 0.7, questionable at α > 0.6, poor at α > 0.5 and unacceptable at α < 0.5. A non-parametric correlational analysis was run using Spearman's rho coefficient to measure validity and correlation between the assessment criteria and the competence unit, as well as between the CUs, according to Kuckartz et al (2013)²⁰: scores > 0.7 were considered to be very highly correlated, > 0.5 high correlation, > 0.3 medium correlation, > 0.1 low correlation and < 0.1 no correlation.

Ethical considerations

The express authorisation of the residents and tutors in charge of evaluating the MHNPs in the mental health units was requested prior to undertaking the assessment, thus signing the informed consent. The data collected was treated anonymously and confidentially. For this purpose, each participant was assigned an alphanumeric record and personal data was anonymised, making it impossible to consult the professionals who undertook the study and thus the association of these participants with the results. This study has been accepted by the CEIC (Clinical Research Ethics Committee) at the San Carlos Clinical Hospital (nº 22/289-E).

Results

A total of 64 assessments were obtained, of which 67.2% were rotational (n = 43) and 32.8% annual (n = 21). Within the rotational assessments (n = 43), 12 were specific to mental health centres and the rest were generic rotations. The assessments were. undertaken by 17 tutors and 39 MHNPs were assessed (Table 1).

Table 1.

Summary of the sociodemographic characteristics of the sample

n = 39	M (DE)
Age	27.3 (4.8)

n = 39	n (%)
Sex
Female	37 (94.8)
Male	2 (5.2)
Autonomous Community
Madrid	38 (97.4)
Other	1 (2.6)
Multiprofessional Mental Health Teaching Unit
Hospital Universitario General Gregorio Marañón	8 (20.5)
Instituto Psiquiátrico José Germain	7 (18.0)
Hospital Clínico San Carlos	6 (15.4)
Hospital Universitario de Fuenlabrada	5 (12.8)
Hospital Universitario Doce de Octubre	4 (10.2)
Hospital Universitario Infanta Cristina	3 (7.7)
Other	6 (15.4)
Year of residency
1st year MHNP	25 (64.1)
2nd year MHNP	14 (35.9)

M(SD) = Mean (Standard Deviation)

Internal consistency for the ECOEnfSM scale good to excellent was shown in all CUs (r > 0.80) with a 95% CI (Table 2).

Table 2.

Reliability of the subscale competence units on the ECOEnfSM scale

	α de Cronbach	Standardised Cronbach's α		CCI 95% CI	CCI p value
Generic Rotation Assessment (n = 43)
CU1	.89	.90		.838 – .936	<.000*
CU2	.88	.88		.822 – .931	<.000*
CU3	.87	.87		.800 – .922	<.000*
CU4	.89	.89		.833 – .934	<.000*
CU8	.92	.92		.880 – .952	<.000*
TOTAL	.93	.94	.899 – .961		<.000*
Rotation Assessment in Mental Health Centre (MHC) (n = 21)
CU1	.93	.93		.848 – .976	<.000*
CU2	.87	.86		.706 – .955	<.000*
CU3	.89	.89		.756 – .963	<.000*
CU4	.95	.95		.898 – .984	<.000*
CU8	.91	.91		.806 – .970	<.000*
TOTAL	.90	.90	.762 – .966		<.000*
Annual Assessment (n = 21)
CU5	.93	.94		.875 – .968	<.000*
CU6	.91	.91		.843 – .960	<.000*
CU7	.90	.90		.812 – .952	<.000*
TOTAL	.84	.87		.666 – .929	<.000*

*

Significance level for 95% CI. CU: competence unit; 95%CI ICC: 95% confidence interval of the intraclass correlation coefficient; ICC p-value: level of significance of the intraclass correlation coefficient.

The Generic Rotation Assessment subscales and evaluations at MMCs obtained excellent reliability (r > 0.90) and the Annual Evaluation subscale had good reliability (r > 0.80) (Table 2).

When running the Shapiro-Wilk test, CUs did not resemble normal distribution (Z = 0.59-0.83, p < 0.05), so Spearman's rho correlation was applied (Table 3).

Table 3.

Correlation of the items with their competence unit (ECOEnfSM scale)

	Spearman`s Rho	p value
CU1 – ASSESSMENT AND DIAGNOSIS (n = 43)
Clinical interview	.59	≤.001*
Identification of crisis factors	.50	.001*
Assessment by mental health patterns	.41	.007*
Nursing diagnosis	.49	.001*
Support Group	.27	.083
Social and health resources	.38	.013*
Transculturalism	.47	.001*
Biological treatments	.40	.008*
Home visit (n = 12)	.34	.276
CU2 – PLANNING (n = 43)
Health care Planning	.42	.005*
Adequacy of the plan	.57	≤.001*
Use of protocols	.29	.056
Risk planning	.42	.006*
Nursing taxonomies	.28	.072
Resource Planning	.36	.019*
Home visit (n = 12)	.37	.242
CU3 – INTERVENTION (n = 43)
Individual intervention	.56	≤.001*
Family intervention	.71	≤.001*
Group intervention	.57	≤.001*
Crisis intervention	.44	.003*
Biological treatments	.39	.009*
Home visit (n = 12)	.48	.114
Referral at discharge	.67	≤.001*
CU4 – EVALUATION (n = 43)
Individual intervention	.55	≤.001*
Family intervention	.50	.001*
Group intervention	.34	.026
Crisis intervention	.59	≤.001*
Mental Health Nursing Care	.32	.035*
Biological treatments	.50	.001*
Home visit (n = 12)	.74	.006*
Course and Protocol Assessment	.59	≤.001*
CU5 – RESEARCH (n = 21)
Observant and reflective attitude	.38	.085
Attitude of learning and improvement	.48	.027
Questions reality	.20	.380
Draws up protocols	.02	.915
Scientific papers	.79	.734
The Research Question	.59	.801
Information search	.32	.152
Dissemination of results	.39	.082
CU6 – CLINICAL MANAGEMENT (n = 21)
Process management	.16	.481
Leadership	.05	.822
Improvement in quality (care work)	-.14	.542
Improvement in quality (Mental Health Services)	.14	.533
Measurement of Quality	.12	.616
Information Systems	.10	.657
Health care policies	.31	.177
CU7 – TEACHING (n = 21)
Reflective attitude on legal and ethical importance	.69	.001*
Assesses needs for EFH** interventions	.62	.003*
Plans interventions	.28	.226
Conducts individual and group interventions	.58	.006*
Education for Health Courses	.52	.014*
Self-instruction	.45	.038*
Attitude of continuous improvement	.66	.001*
Teaching activities	.67	.001*
CU8 – COMMUNICATION AND INTERPERSONAL RELATIONSHIPS (n = 43)
Active listening	0.54	≤.001*
Verbal expression	0.54	≤.001*
Communication of Support Material and IR	0.60	≤.001*
Interpersonal relationship	0.67	≤.001*
Multicultural context	0.55	≤.001*
Conflict resolution (patients, family and team)	0.36	.018*
ICT Management	0.50	.001*
Interprofessional relationship	0.40	.008*

*

significance level for 95%CI. CU: competence unit.

**

EFH: Education for Health.

CU1 (assessment and diagnosis) obtained medium-high correlation levels for all criteria (p < 0.01) except for the "Support groups" criterion, which obtained a low correlation (r = 0.27, p = 0.08) and the "Home visit" criterion with average correlation, but without significance (r = 0.34, p = 0.27).

CU2 (planning) obtained a significantly medium-high correlation strength for all criteria (p < 0.05) except for "Use of protocols" (r = 0.29, p = 0.05), "Nursing taxonomies" (r = 0.28, p = 0.07) and "Home visit" (r = 0.37, p = 0.24).

The CU3 (intervention) obtained a medium-very high correlation for all items (p < 0.01) except for "Home visit" (r = 0.48, p = 0.11).

The CU4 (assessment) obtained a medium-very high correlation for all items.

For CU5 (research) the correlation ranged between absence of correlation and very high correlation, which was not-significant (p > 0.05).

Absence of correlation, low correlation or inverse correlation (p > 0.05) was obtained in the assessment criteria for CU6 (clinical management).

CU7 (teaching) obtained medium-high correlations for all their items (p < 0.05), except for "Intervention planning" where weak correlation was noticeable (r = 0.28, p > 0.22).

CU8 (communication and interpersonal relations) obtained a medium-high correlation in all its criteria (p < 0.01).

The rotation assessment subscale (generic and mental health centres) obtained high and very high levels of correlation (r > 0.6), as did the annual assessment subscale with average and very high levels (r > 0.4, p < 0.01) between all its CUs with highly significant levels (p < 0.01), as shown in Table 4.

Table 4.

Correlation of the competence units in the ECOEnfSM subscales.

				Spearman’s
Rotation Assessment (n = 43)
	CU1	CU2	CU3	CU4	CU8
CU1	-	.89*	.80*	.77*	.78*
CU2	-	-	.66*	.68*	.84*
CU3	-	-	-	.82*	.77*
CU4	-	-	-	-	.80*
CU8	-	-	-	-	-
Annual Evaluation (n = 21)
	CU5	CU6	CU7
CU5	-	.79*	.71*
CU6	-	-	.47*
CU7	-	-	-

*

significance level for 95% CI. CU: competence unit.

Discussion

The data from the study indicate high levels of reliability, with good to excellent internal consistency (r > 0.80) in all subscales and CUs in the clinical practice of Resident Nurse Practitioners in Mental Health (MHNPs).

Rivera (2021) demonstrated the reliability of the ECOEnfSM scale with content validation by a group of experts in mental health nursing at the Regional Office for the Coordination of Mental Health and Addictions, based on the training course for the specialty of Mental Health Nursing¹; and with very high internal consistency (Content Validity Index = 0.96, α = 0.968).⁴

The ECOEnfSM scale is based on the ECOEnf scale, which also demonstrated high levels of internal consistency (Content Validity Index = 0.85, α = 0.986),¹⁶ intraobserver agreement (r = 0.98) interobserver agreement (r = 0.66) and correlation between items (r > 0.50, p > 0.05 ) to assess the nursing competencies of nursing graduates in the university population.^16–18

Specific CUs (CU1-Assessment and Diagnosis, CU2-Planning, CU3-Intervention, and CU4-Assessment) correlated very well. However, the same was not the case for the cross-sectional CUs as, although they demonstrated acceptable correlation, the data obtained showed very dissimilar results between the items. The research competence unit (CU5) stood out due to significant variability (very high correlations or absence of correlation) and, above all, the clinical management competence unit (CU6) recorded either inverse levels, or zero or poor correlation.

This data is consistent with clinical experience, where we find Mental Health MTUs with a major interest in training MHNPs in research and others who consider it a simple procedure to approve the SHTP courses. On the other hand, it seems obvious from the results that management competencies are not skills that are currently developed on training courses, perhaps due to lack of time, lack of interest in MHNPs or MTUs, due to prioritising care competencies or because of the complexity of management functions.

The Rotation Assessment subscale obtained good levels of correlation between their CUs, since these are attitudes, competencies, and knowledge very frequently developed in all MTUs and are easy to evaluate.

However, the Annual Assessment subscale obtained a lower correlation between their CUs because these are less developed competencies or are considered less important, since they are not often used in daily clinical practice. The sample size of the Annual Assessment reflected this fact, since less participation was achieved than in the rest.

In fact, data as disparate as low correlations or lack of correlations, and even inverse correlations in CUs (CU5-Research, CU6-Clinical Management), can be explained by lower representativeness, great variability between hospitals and difficult assessment. However, CU7-Teaching and CU8-Communication and interpersonal relations were cross-cutting competencies with excellent data.

Rivera also highlighted the cross-cutting CUs as those with the lowest level of reliability, albeit considered excellent in his study, in relation to specific CUs.⁴

Similarly, the assessment criterion of "Home Visit" also showed low representativeness and disparity in its data, as there is little health care rotation in the current training programme me with community care, where hospital care predominates (inpatient and residential units, day hospitals, etc.), few MTUs that prioritise or have this portfolio of services developed completely and/or short rotations (<2 months) where there is no time to develop home visit skills. Similarly, we find very high (r = 0,74, p < 0.01) and very low (r = 0,34, p = 0,27) levels of correlation at the same time for this criterion in the different CUs. Researchers committed to reviewing these weak correlations as a process of continuous improvement, possibly due to an insufficient sample.¹⁸

At the international level, there are numerous tools for assessing competencies, although many have been developed ad-hoc by researchers to respond to their research objective, but without having gone through rigorous validation processes.^12,21

Other authors use validated tools assessing very specific competencies such as knowledge and attitudes, it being difficult to find assessment of other more specific competencies. In addition, these are usually topics of classic care by mental health nurses, such as the management of physical health (Physical Health Attitude Scale for Mental Health Nurses -PHASe-)²² or containment (Attitudes towards Containment Measures Questionnaire -ACMQ-),^11,12,23 without taking into account other more complex, novel and vital skills for this population, such as the need for research, attitudes or interpersonal or interprofessional communication, included in the training programme me for mental health nurses.¹

Dickens et al. use tools to assess these nursing competencies in mental health in specific pathologies such as borderline personality disorders (Personality Disorders Knowledge, Attitudes and Skills Questionnaire-PD-KASQ-,²⁴Attitude to Personality Disorder Questionnaire -APDQ-²⁵) or psychotic disorders.^11,12 However, only Rivera et al. offer a competency assessment tool for the care of all disorders treated by the Mental Health Nurse.⁴

Sheikhbahaeddinzadeh et al. analysed the composition and psychometric properties of competency assessment tools in psychiatric nurses²⁶: the Psychiatric Nursing Performance Appraisal Instrument (PsychNPAI),²⁷ the Clinical competency in mental health nursing students^2,8 the Clinical Competency of Mental Health Nursing (CCMHN),²⁹ the Case management competence scale,³⁰ the Mental Health Objective Structured Clinical Examination (MHOSCE)³¹ and the Questionnaire to assess nursing competencies for the care of people with psychiatric disabilities in a hospital environment.³²

However, these tools were focussed on nurses working in Psychiatry or mental health nursing students but none were found to be focussed on the clinical competencies of the postgraduate Psychiatric Nurse.²⁶

Therefore, the ECOEnfSM scale stands out for being the only objective and complete procedure for assessing mental health nurse practitioners, this being essential for professional development and as a theoretical basis of nursing knowledge. In addition, it enables the competencies developed in the training programme me to be evidenced,¹ as well as justifying a pass or fail on the SHTP course in the face of official bodies such as the Spanish Ministry of Health.

Among the limitations of this research, are, firstly, the fact that this is a pilot study, secondly, the lack of stability of the data due to the lack of a test-retest, given the difficulties of clinical practice, and thirdly, the possible need to undertake prior training in the procedure aimed at tutors as evaluators of mental health nursing practitioners.

Proposals for improvement are put forward, such as increasing the sample size and including other statistics that measure validity or reliability. We propose undertaking concordance studies with two evaluators and/or studies to evaluate criteria validity by comparing these with the final grade in the current assessment of the Ministry of Health.³

According to Gómez del Pulgar: "Competency assessment is a global concept and should not be based on a single instrument; the subjectivity of the same can be reduced by combining instruments and methods”. The ECOEnfSM scale thus contributes to continuous and objective assessment.¹⁸

In conclusion, we would like to point out that the ECOEnfSM scale obtained very high reliability in Resident Nurses in Mental Health in Spain, this being considered the only valid and objective tool to assess the professional competencies of this population so far.

Funding

This research has not received any specific support from any agencies from the public sector, commercial sector or non-profit organisations.

Conflict of interest

The authors declare that there is no conflict of interest.

Uncited references

[19,28,33].

Acknowledgements

To all the study participants: tutors, residents, colleagues and friends. Especially the Mental Health Nursing Association and the Advisory Committee on Health Care within the Regional Office for Mental Health and Addictions for the Community of Madrid.