Analysis instruments for the performance of Advanced Practice Nursing

Abstract

Advanced Practice Nursing has been a reality in the international context for several decades and recently new nursing profiles have been developed in Spain as well that follow this model. The consolidation of these advanced practice roles has also led to of the creation of tools that attempt to define and evaluate their functions. This study aims to identify and explore the existing instruments that enable the domains of Advanced Practice Nursing to be defined. A review of existing international questionnaires and instruments was undertaken, including an analysis of the design process, the domains/dimensions defined, the main results and an exploration of clinimetric properties. Seven studies were analysed but not all proved to be valid, stable or reliable tools. One included tool was able to differentiate between the functions of the general nurse and the advanced practice nurse by the level of activities undertaken within the five domains described. These tools are necessary to evaluate the scope of advanced practice in new nursing roles that correspond to other international models of competencies and practice domains.

Resumen

La Enfermería de Práctica Avanzada es una realidad en el contexto internacional desde hace ya varias décadas y, en los últimos años, también en España se están desarrollando nuevos perfiles que siguen este modelo. La consolidación de estos roles avanzados también ha propiciado la aparición de herramientas que intentan definir y evaluar sus funciones. Este estudio tiene como objetivo identificar y explorar los instrumentos existentes que permitan definir los dominios de la Enfermería de Práctica Avanzada. Se ha realizado una revisión de los cuestionarios e instrumentos existentes a nivel internacional, incluyendo un análisis del proceso de diseño, los dominios/dimensiones definidas, los principales resultados y la exploración de propiedades clinimétricas. Se han analizado 7 estudios, aunque no todos demostraron ser herramientas válidas, estables o fiables. Un instrumento incluido fue capaz de diferenciar entre las funciones de la enfermera de cuidados generales y la de práctica avanzada por el nivel de las actividades emprendidas en los 5 dominios descritos. Estas herramientas son necesarias para evaluar el alcance de los nuevos roles de la práctica avanzada en Enfermería que corresponden con otros modelos internacionales de competencias y dominios de práctica.

Introduction

Scientific-healthcare development and socio-demographic changes in the population require changes in nursing practice to increase excellence and evidence-based care in a sustainable manner. Many countries began to develop the role of advanced practice nursing (APN) at the end of the 20th century, when they arose as the result of the need to contain costs, improve access to care, reduce waiting time, offer a response to the most vulnerable people and maintain health among specific groups.¹ The role of APN covers common functions which generally characterise these profiles. The International Council of Nurses defines the advanced practice nurse as “a graduate nurse with expert knowledge, skills in complex decision making and the necessary clinical competence for advanced practice, with a master/doctorate level of training.²” In the last decade, the role of the APN has become a generalised trend and led to great national and international interest^3,4 in uniting criteria and obtaining consensus on the competence framework and the definition of characteristics under a globalising and unique concept. Although consensus exists regarding ANP knowledge and skills in practice, the concept implies a great variability of nursing profiles, with different competences, defined by the model Strong⁵ or the model of Hamric et al.⁶ They are accepted within the nursing profession, where the main difference in the professional profile between the APN and general care nurses is the APN's ability to accept more complex cases and manage them with greater independent, judgement and responsibility.

Academic and professional development in Spain

In Spain, university level training for nurses has undergone a major transformation during recent years, from being at university diploma level to becoming a university training on all academic levels of graduate, master and doctorate.⁷ Professionally, notable advances have also been made, among which the development of specialist nurses stands out,⁸ and implies the existence of two categories of professionals in our country: general care nurses and specialist nurses.⁹ Several autonomous communities have also created professional degree systems and legal standards have also appeared which cover the exercise of new skills in the prescription of medication and healthcare products,¹⁰ in addition to the Royal Decree which regulates the advanced accreditation diplomas¹¹ under the heading of continuous professional training.

In recent years there has also been a notable increase in the number of studies on advanced practice in our environment. National studies assess different initiatives such as case management,¹² emergency nursing consultation for minor conditions in primary care,¹³ advanced clinical nursing in the Basque Country,¹⁴ and manuals which include the requirements for professional accreditation, such as the oncological APN in Andalusia.¹⁵ A national study also exists on consensus of APN competence¹⁶ or studies on the need for introduction of these profiles justified by accessibility and continuity of health attention in chronic diseases.¹⁷ Similarly, just as there are nursing specialties and other profiles described in the literature, we also find innovative profiles in nursing which offer a portfolio of expert services, specific to the area of healthcare. These nursing profiles develop competent practice of greater complexity and autonomy in performing their tasks than a general care nurse. In general these nursing profiles are distinguishable because they offer a different portfolio of services and they are usually integrated into professional career models or into assessment of performance in their institutions and they could coincide with the description of advanced practice activities on an international level.

Definition of roles and profiles

The definition of the range of activities of advanced practice is essential for its implementation, development and sustainability long term.¹⁸ Internationally, the implementation of new roles of advanced practice has been slow and heterogeneous due to its complexity. Despite the important contribution and benefits which these nurses offer to health services, greater comprehension of functions and activities exercised is needed. To do this tools need to be identified which help to define the range of activities inherent in the existing APN. The aim of this study is to identify and explore the instruments designed to analyse APN competence through the analysis of the activities and functions defined in countries where this role has already been fully established. It also suggests the need to assess the advanced practices in our environment of existing nursing profiles or emerging nursing profiles in Spain which may or may not coincide with the activities described and defined in other countries.

Description and analysis of the tools

After analysing the main national and international data bases in Health Sciences and the consensus documents designed by the main professional organisms and collectives, 7 studies were selected for data extraction and exploration: Brady and Neal,¹⁹ Willens et al., ²⁰ Mick and Ackerman,²¹ Brown and Waybrant,²² Chang et al.^23,24 and Fry et al.²⁵. Table 1 summarises the main characteristics of each instrument.

Table 1.

Summary of the studies included.

	Brown and Waybrant, 1987	Brady and Neal, 2000	Mick and Ackerman, 2000	Willens et al., 2010	Chang et al., 2010	Fry et al., 2013
Conceptual model	Conceptual model of the National Organisation of Nurse Practitioners Faculties (U.S.A):	Conceptual model of the National Association of Paediatric Nurses and the National Organisation of Nurse Practitioners Faculties (U.S.A)	Strong Model of Advanced Practice (U.S.A.)	American Nurses Credentialing Centre (ANCC) Pain Management (U.S.A.)	Modified-Strong Model of Advanced Practice (Australia)	New South Wales Health Department guidelines (Australia)
Population	Nurse practitioners graduated to master level from the Pacific Northwest University	Paediatric nurse practitioners	Specialist nurses (clinical nurse specialists) and advanced practice nurses in critical care (Advanced Nurse Practitioners)	Nurses specialised in pain management	Nurses and midwives randomly selected from a national database	Consultant nurses
Method	Anonymous Questionnaire via post which includes: (a) demographic data; (b) Coordination activities inventory, and (c) Health promotion inventory	Anonymous Questionnaire via post	Anonymous Questionnaire via post which includes: self-assessment instrument of APN role definition	Anonymous Questionnaire via email and post	State survey via post with 3 sections: (a) demographic data; (b) IDRAPN^a modified, and (c) time dedicated to general practice domains	Review of the literature, on-line questionnaire and description of existing job functions
Variables	Time dedicated to each activity	Years of experience, working environment, geographical region, advocate authority and sub-speciality of practice	Self-assessment of the experience in practice domains and assessment of the practice domains and assessment of tasks relating to the role	Pain management activities related to the consequences, frequencies and expectations of the performance of activities	Frequency of advanced practice activities in keeping with nursing level	Characteristics and responsibilities of an advanced practice role - consultant nursing
Clinical impact	Scale permitting the exploration of the component of the nursing practice (nurse practitioners)	Process to assess several items passed from data of activity in primary care, level or responsibilities and type of pathologies managed by paediatric nurses	Scale permitting the differentiation of 2 specialist nurse roles (clinical nurse specialists) and nursing practitioner in critical care	Scale permitting the measurement of the specific activities and responsibilities of nurses specialising in pain management	It is possible to measure advanced practice activities which enable the differentiation between practice and advanced nursing practice	Can be used to measure the scope and responsibilities of the role of the consultant nurse. Does not determine the individual skill or differentiate between specialities

a

Instrument defining the role of the advanced practice nursing.

Brady and Neal¹⁹ carried out a descriptive exploratory study aimed at identifying the responsibilities and functions of the APN in paediatrics. To do this they designed a questionnaire based on the conceptual model of the National Association of American Paediatric Nurses and the National Organisation of Nurse Practitioner Faculties of the United States. The tool was a questionnaire with socio-demographic variables and a Likert type scale divided into 3 sections, which contained work activities (175 items), specific clinical problem management (303 items) and professional role responsibilities (21 items). Apparent validity was initially assessed by a panel of 8 experts and later through a pilot study of 30 participants with a response rate of 67%. The final instrument was distributed to a random sample of 994 participants, out of whom a total of 337 responses were obtained. The study assessed the frequency and importance of each item and the scale in general. Mean scores for each of the dominions were analysed with demographic variables of the study population, years of experience, working environment, geographical region, prescriptive authority and practice sub-specialities, thus determining the functions and responsibilities of the sample within the study context.

In their study, Willens et al.²⁰ conducted an analysis of the job for the American Society of Pain Management Nursing. The study comprised 3 phases in which 6 dominions of performance were developed for APN performance in pain management. The instrument included 70 activities or items divided into 6 practice areas: (1) assessment, follow-up and pain assessment; (2) pharmacological pain management; (3) non-pharmacological pain management; (4) therapeutic communication and consultancy; (5) education of the patient and family, and (6) collaboration and organisation activities. The apparent content of the study was assessed using an independent panel of experts and a pilot study. A descriptive, exploratory study was conducted, which obtained a final sample of 585 certified advanced practice nurses in pain management. Results were sufficient to validate the performance domains identified by a panel of experts. An internal consistence measurement was used (Cronback alpha) using the classifications of consequence, frequency and participant performance. Frequency reliability was 0.97 and performance expectation was 0.95 in the described tool.

The study proposal of Mick and Ackerman²¹ was to differentiate between the specialist nursing and advanced practice in critical care nursing roles. They conducted a descriptive study to this end which obtained 18 valid responses. The authors adapted the “Strong” model of advanced practice to develop the descriptors of the advanced practice functions in critical care and designed an instruments validated by experts to differentiate between the two clinical practice roles. The instrument classifies individual knowledge using a visual analogue scale which ranges between 1and 10 (from apprentice to expert) in 5 practice domains: direct comprehensive care, support to systems, education, research and publication and professional leadership. The 42 activities described in each of the domains were measured using self-assessment of the experience in the practice domains and evaluation of the tasks relating to the role on a Likert type scale, where 1 was “unimportant” and 4 “very important”. The function marking instrument describes the components of the APN role in daily practice.

Brown and Waybrant²² also carried out a descriptive, exploratory study, which analyses the practice components of the nurse practitioners. A convenience sample of 146 nurse practitioners trained in the specific master course of the Pacific Northwest University. The instrument included demographic data, an inventory of coordination activities and an inventory of activities related to health promotion. The instrument designed comprises 27 items distributed into 3 practice domains: health promotion, education to the patient and counselling, which assesses the time dedicated to each domain using a Likert type scale. Content validity and appearance was made, as with previous studies, through a panel of experts and a pilot study. The instrument obtained a score of 0.90 in the Cronbach alpha coefficient for the inventory of coordination activities, similarly to the inventory of health promotion activities.

Chang et al.^23,24 established a group of experts to revise the activities of advanced practice of the tool created by Mick and Ackerman²¹ using the Delphi technique with a panel of experts. During the study 5 activities were modified and consensus was obtained for the validation of the content of the modified tool. The main changes in the instruments were the elimination of an activity and the designing of another 5 activities. In a second publication²⁴ the authors established the validity of constructing the tool using an exploratory factorial analysis (EFA), and in a third publication Gardner et al.^26,27 assessed the internal validity of the same tool. In the study the statistical potential for EFA was low and obtained an item ratio of 16:1, indicating a precision level of between 60% and 70%. The construction validity of the tool was proven with their own values higher than 1, which represent 70% of the total variance. Five elements were named in accordance with the 5 domains of advanced practice of the original instrument and multiple lineal regression analysis was carried out where the factors which influence the advance practice nursing were examined in each of the domains described in the tool. The global validity rate of the questionnaire was also established derived from the percentage of answers using an applied Likert scale which measures the time each nurse dedicates to each of the domains. The percentages in accordance with the described domains were of between 100% and 87.5%. The EFA of the 5 practice domains ranged between the coefficients of 0.83 and 0.95, whilst the general Cronbach alpha coefficient of the tool was 0.94. The results of the tool allowed advance practice activities to be measured and to differentiate between general care nurses and the APN included in the study.

Lastly, the objective of the study carried out by Fry et al.²⁵ was to measure the performance of the consultant nurses, on the specific APN level, in Australia, in relation to the practice domains. The authors used existing literature, an on-line survey and professional degree descriptions to generate the practice domains. They conducted semi-structured interviews and a pilot study to test out the validity of the tool. The tool was designed with 50 items assigned to 5 practice domains with a hierarchical scale of10 points within each domain. 56 consultant nurses participated in the study and the characteristics and responsibilities of these nurses in relation to the clinical service and consultancy, leadership, research, education and management and planning of clinical services with respect to their professional level were analysed.

Discussion

It is important to analyse in depth each of the chosen questionnaires and instruments included in the validity and reliability tests (Table 2). There are common characteristics² to the advanced practice nurse's role although each professional profile, work post and location have their singularities. The studies included in this review demonstrate the interest in creating instruments which are capable of analysing the APN domains in highly specific settings. Only Chang et al.^23,24 and Fry et al.²⁵ included different specialities aimed at analysing common practices and being able to differentiate the role of advanced practice of general care.

Table 2.

Analysis of the clinimetric results and properties of the instruments.

	Brown and Waybrant, 1987	Brady and Neal, 2000	Mick and Ackerman, 2000	Willens et al., 2010	Chang et al., 2010	Fry et al., 2013
Number of items	27	499	42	70	41	50
Domains	(1) Health promotion (2) Patient education (3) Counselling	(1) Activities focused on health care (2) Specific clinical problems (3) Professional responsibilities	(1) Direct integral care (2) Interprofessional support systems (3) Education (4) Research (5) Publications and professional leadership	(1) Assessment, monitoring and pain assessment (2) Pharmacological pain management (3) Non pharmacological pain management (4) Therapeutic communication and counselling (5) Education to the patient and family members (6) Collaborative and organisational activities	(1) Direct comprehensive care (2) Interprofessional support systems (3) Education (4) Research (5) Publications and professional leadership	(1) Clinical service and consultancy (2) Leadership (3) Research (4) Education (5) Management and planning of clinical services
Content validity	Expert panel and pilot test	Expert panel	Expert panel	Expert panel and pilot test	Expert panel Global index of 0.75 or above	Semi-structured interviews and pilot test
Appearance validity	Expert panel and pilot test	Expert panel and pilot test	–	Expert panel and pilot test	Expert Panel and pilot test	Semi-structured interviews and pilot test
Construct validity	–	–	–	–	EFA 5 elements with loads of ≥400 of 40 of the 41 activities of advanced practice which were similar to the 5 domains of advanced practice described Precision level of 60–70%	–
Validity of criteria	–	–	–	Assessment of action of each of the domains and activities (frequency, consequence and expectations of activities)	Yes	Assessment of action of each of the domains and level of nursing (professional level)
Reliability-internal consistency	Cronbach's alpha coefficient of 0.90 for the coordination activities inventory scale Cronbach's alpha coefficient of 0.90 for the Health promotion inventory	–	–	Cronbach's alpha coefficient of 0.95 in the performance expectation Cronbach's alpha coefficient of 0.97 in the frequency of scale Cronbach's alpha coefficient of 0.99 in the assessment of consequences	Cronbach's alpha coefficient of 0.94 for the 5 elements found in the EFA ranged from 0.83 to 0.95	–
Reliability-temporary stability/test–retest	–	–	–	–	–	–

The same level of evidence²⁵ is appreciated among the main characteristics of the 7 studies analysed, with the exception of Fry et al.²⁵ that uses the synthesis of qualitative data. As may be observed, 2 of the studies included are designed as pilot studies, which may have the possibility of making inexact predictions and informing of imprecise results on initially obtained data bases. The characteristics of the population in each of the studies are convenience and specific in the context in which they are designed. Different profiles were included in the studies, as for example: nurse practitioners,²¹ paediatric nurses,¹⁸ specialist nurses in pain managment¹⁹ or intensive care nurses.¹⁹ Two studies included APN of different clinical areas^23,24 and one instruments was designed to include advanced practice roles regardless of the speciality, job function or professional level.²³ In this study different nurses from different sectors participated (primary, tertiary and intensive) with different professional categories. The definition of intervention in each one of the studies was made using anonymous questionnaires with similar objectives, intended to study the frequency, time, experience, characteristics of the activities and the APN domains. The majority of them were based on clinical guides established by professional associations or the corresponding health department, except for 2 studies which were based on the Strong Model of Advanced Practice⁵ conceptual model. Mick and Ackerman²¹ and Chang et al.²³ stated the combination of domains and activities described in the instruments in relation to the skills and domains of advanced practice described in the literature.^5,6

With regard to clinimetric properties all scales analysed the properties of the instruments with multiple items. Using the panel of experts or semi-structured interviews the studies made it possible to obtain a judgement on the values of the content validity of each instrument. All the studies tried to establish the content validity of the instruments studied, 5 analysed the validity of appearance,^{18,19,21,23,24} one the construct validity,²¹ 3 the validity of criteria^19,21,22 and 3 analysed the reliability^18,21,23 of the instruments. With the exception of the Chang et al.²³ and the Gardner et al.²⁶ study, the other studies referred to expert consensus in the design of the tools although they did not report on the validity rates or the instrument construct validity.^5,6 The lack of references to these indexes minimises the evidence of assessment of each domain and the association with other validated measurements of the same construction in the instruments included. In Fry et al.,²⁵ A review of the previous literature was made and semi-structured interviews were conducted, in contrast with Chang et al.,²² where the content of the instruments on the domains and activities of the APN was validated through a Delphi study.

The validity of appearance was also established in the majority of instruments through a pilot study after the panel of experts. Willens et al.²⁰ assessed the action of each of the described domains and activities. This established the capacity of the instrument to carry out the tasks for which it was designed and proved the specific use of the questionnaire for the study aim put forward. The values of frequency, consequences and expectations of tasks between the participants established the validity criteria of the questionnaire for the specialist nurses in pain management The study by Chang et al.²² was the only one to carry out an EFA to establish the construct validity of the instrument. In this sense, and given the difference between countries, sometimes difficulties occurred in the contextualisation and interpretation of results, particularly when they mentioned performance in the different areas of clinical specialisation, where depending on the country, there could be general, specialist or APN nurses. First, through a Delphi study, the authors developed an instrument for the APN definition, in which a consensus of informed opinions were obtained. The percentages of agreement in the different Delphi rounds confirmed that the instrument could be useful in reducing the confusion which existed on the definition of the APN role.

With regard to internal reliability and consistency of the questionnaires included, 3 studies calculated Cronbach's alpha coefficient as above 0.90, which reflects the level in which the elements in the scales are associated to one another. These studies are those of Brown and Waybrant,²² Willens et al.²⁰ and Chang et al.²² The stability test-retest of the instruments was not determined in the studies included, and no reference was made to the absence of errors of measurement, or what is the same, the stability of the scores of an instrument over time.

The findings from the Chang et al.²³ study are the only ones to show the different activities carried out by the APN and prove major statistical differences in accordance with the professional level and score of the total activities in all APN environments. The differences between the different practice domains of nurses who exercise advanced practice and those who do not, mean that differentiation made be made between the different levels of professional practice and allows us to define and differentiate the role of the APN from the role of the general care nurses within an international framework using a valid and reliable questionnaire.

To sum up, all the instruments included have their own limitations both in the development of studies and in the interpretation of results to establish validity and reliability. The most extensive instruments by Brady and Neal¹⁹ and Willens et al.,²⁰ which comprise between 499 and 70 items, respectively, could be a limitation when applying them in practice. Furthermore, 4^19–21 of the studies include was designed for a specific sample of the APN population, and results may not be generalised. The specific activities of the APN in paediatrics, pain, nurse practitioners and critical care are not possible to translate into the activities and general domains of the APN in keeping with the studies described. Fry et al.²⁵ included consultancy nurses in a tertiary level hospital, and although this study assesses the domains of practice of different specialities it remains specific at a certain level in the study context. None of the studies included established temporary stability.

Conclusions

The set of function domains inherent to APN and their complexity as described in the questionnaires included in this paper are consistent with other international competency models and advance practice domains. This study describes the validity of content of the tools included; however, not all proved to be valid, reliable or to have temporal stability. We identified only one instrument as capable of differentiating advanced practice activities from general nursing care activities. There is an ever-increasing need, both nationally and internationally, to distinguish advanced practice roles from general care roles. Therefore using questionnaires to analyse the activities currently undertaken in our setting could contribute to the definition and analysis of APN functions. While various APN profiles and some advanced practice roles have been defined, such as nurse prescribing to cover the needs of the current health system, APN has not yet been fully developed or legally recognised in Spain. There is a need for reliable instruments, based on those currently available and adapted to our context, to provide greater clarity on the role of APN and to contribute to its correct implementation, development and sustainability within the health system.

Conflict of interests

The authors have no conflict of interests to declare.

Acknowledgements

We would like to thank the technical Committee of the Nursing Management for its support and commitment to the study and the development of Advanced Nursing Practice in the Hospital Clínic of Barcelona.