Integrating wellbeing and social accountability into medical professionalism self-assessment: A mixed methods approach

Abstract

Background

Medical professionalism is an essential characteristic of excellent physicians and in most need of training. It should involve areas not included in traditional frameworks, such as physicians’ self-care and social accountability. This study aims to develop a comprehensive self-assessment instrument to improve medical professionalism training in medical students.

Methods

Mixed-methods design that integrated a scoping review, a qualitative framework method, and a cross-sectional design. The qualitative phase included focus groups with 19 medical students and 9 in-depth interviews with clinician educators. The quantitative phase included descriptive and exploratory factor analysis of 112 interviews to fifth-year medical students at the Catholic University in Chile.

Results

In the scoping review, 290 articles were screened. The Professionalism Mini-Evaluation Exercise (P-MEX) instrument was selected as the reference instrument. The qualitative phase generated 188 open codes and 468 quotes, which were analyzed using the Atlas-ti 8.0 software®. Three levels of professional assessment emerged. The micro level included self-reflection and self-care skills; the meso level included doctor–patient relationship, team interaction, and management skills; and the macro level included social accountability skills. The new adapted instrument (P-MEX-LA) contained 30 items and achieved high reliability and internal consistency (Omega 0.86; Cronbach’s alpha 0.82). Self-care and social accountability skills were the areas with the lowest level of achievement by students.

Discussion

A Latin American self-assessment version of the P-MEX instrument (P-MEX-LA) that included self-care and social accountability skills achieved high content validity, internal consistency, and reliability. It could contribute to improving medical professionalism training among medical students.

Resumen

Antecedentes

El profesionalismo medico es una de las características básicas de excelencia médica y debiera integrar áreas no incluidas en definiciones tradicionales tales como autocuidado y compromiso social. El objetivo de este estudio fue desarrollar un instrumento para mejorar la formación en profesionalismo en estudiantes de medicina.

Métodos

Diseño metodológico mixto que integró una revisión de alcance, un diseño cualitativo de marco analítico y un diseño transversal. La fase cualitativa, incluyó grupos focales con 19 estudiantes de medicina y nueve entrevistas en profundidad con educadores clínicos. La fase cuantitativa incluyó un análisis descriptivo y factorial exploratorio basado en 112 entrevistas a estudiantes de quinto año de medicina de la Universidad Católica de Chile.

Resultados

En la revisión de alcance, 290 artículos fueron tamizados. El instrumento Professionalism Mini-Evaluation Exercise (P-MEX) fue seleccionado como el de referencia para ser adaptado y evaluado. La fase cualitativa generó 188 códigos abiertos y 468 citas que fueron analizadas usando el software Atlas ti 8.0®. y de la cual surgieron tres niveles de valoración. El micro nivel agrupó habilidades de autorreflexión y autocuidado, el meso nivel habilidades relacionales de médico-paciente, trabajo en equipo y gestión y, el macro nivel, habilidades de compromiso social. El nuevo instrumento adaptado (P-MEX-LA) alcanzó una alta confiabilidad y consistencia interna (Omega 0.86; Alfa de Chronbach 0.82). Las áreas de menor desarrollo en estudiantes fueron autocuidado y compromiso social.

Discusión

El instrumento P-MEX-LA incluye áreas emergentes de profesionalismo médico y presenta alta validez de contenido, confiabilidad y consistencia interna.

Background

Medical professionalism is considered 1 of the 3 key competencies of excellent physicians.¹ The aim of learning medical professionalism is to ensure the development of professional identity among medical students.² Medical professionalism provides physicians with competencies to address stressful scenarios, promote wellbeing and dignity for their patients, and contribute to reducing health disparities in clinical practice. In the process of exercising medical professionalism, physicians develop their professional identities and reshape their medical practices.

Medical professionalism and professional identity are formative processes, and self-assessment of physicians’ own practices is a key component of the learning process. Self-assessment is an important strategy to improve medical competencies³ and is also an essential strategy to facilitate the transition of medical students to clinical practice.⁴ In a study on self-assessment needs perceived by medical students in their final year across 35 European medical schools, professionalism was 1 of the 3 areas that students perceived as in need of most improvement.⁵

Medical professionalism is an evolving concept with a variety of definitions across cultures. Ethical principles related to patients’ respect, and interpersonal dimensions in the context of the doctor–patient relationship, management practices, and interaction with team members are the main attributes included in most professionalism assessment frameworks.⁶ These attributes were also included in a definition of medical professionalism developed by an expert panel in Latin America.⁷

Medical professionalism and professional identity are closely related concepts. Teaching medical professionalism is considered an essential strategy to support medical students as they develop their own professional identity.^2,6,8 Personal, professional, and social dimensions interact in the process of identity formation of medical students, and teaching medical professionalism is a means to facilitate this process. The relevance of socialization, i.e., the interaction of medical students with peers, health team members, and local communities at large, has been increasingly recognized as an important driver in the personal and professional dimensions of professional identity.⁸ On the other hand, the development of self-care abilities is increasingly considered an essential competence to transit through the process of becoming a physician.⁸ However, these dimensions have not been consistently included in current professionalism frameworks. The Lancet editorial (2021)⁹ emphasized that modern professionalism should move from a traditional paternalistic approach to a broader and more integrative approach that includes dimensions such as physicians’ wellbeing, self-care, equity, and social justice. The emerging evidence on the high prevalence of emotional distress and burnout among physicians¹⁰ and the impact of social health disparities in clinical practice¹¹ supports the need to include self-care and social accountability among medical professionalism competencies.

This study used a mixed-methods approach to adapt and test a self-assessment instrument to improve medical professionalism skills and professionalism identity in medical students during their clinical phase of training. The aim of this study was to collect evidence of validity and develop a comprehensive, and practical self-assessment instrument that could assist students and clinician educators to improve medical professionalism training in highly demanding clinical settings.

Methods

This study used a mixed-methods approach that consisted of 3 phases. The first phase included, a comprehensive and a systematic search of the literature to select an instrument for self-assessing medical professionalism. The second phase consisted of a qualitative component based on the framework method¹² that was used to critically analyze the instrument selected for medical professionalism self-assessment and for redefining dimensions and items to be included in the new adapted version. Finally, a quantitative component was applied to test the instrument using a cross-sectional design.

The systematic literature search was conducted in English and Spanish and included 3 databases as well as a hand search of article references and books. The databases included in the search were PubMed, Lilacs, and Epistemonikos. Lilacs is the most important and largest health science database in Latin America and the Caribbean region and contains journal articles, regional reports, and government documents. Epistemonikos is a collaborative, multilingual database of health evidence. It is the largest source of systematic reviews relevant for health decision-making and a large source of other types of scientific evidence. The search included publications in the databases through June 2020. The key terms used in English and Spanish were professionalism (profesionalismo), medical professionalism (profesionalismo médico), medical students (estudiantes de medicina), assessment (evaluación), instruments (instrumentos), and questionnaires (cuestionarios). The MESH terms used were ((“professionalism”[Mesh])) AND “Students, Medical”[Mesh]; AND “instruments”[Mesh]; (“questionnaire”[Mesh]) AND “assessment”[Mesh]. Articles that provided information on the target population, psychometric properties, instrument length, specific content, and cultural diversity were prioritized for selection and analysis. Written authorization for using the selected scale was requested after the reference article was selected.

A framework method approach was used in the qualitative component of our study. This method has been increasingly used in educational research¹³ and was selected due to its capacity to analyze data based on a predefined set of themes and categories such as those contained in the preselected instrument.¹² The selection criteria for focus group participants considered that they were undergraduate students in their clinical training phase at the School of Medicine Universidad Católica de Chile. The criteria for selecting participants for the in-depth interviews, considered that they were clinician educators with 5 or more years of teaching experience from general surgical and non-surgical specialties. The final sample included clinician educators from family medicine, internal medicine, emergency medicine, anesthesiology, obstetrics/gynecology, and pediatrics. The focus group and individual interviews scheme was based on a semi-structured 8-item questionnaire containing open-ended questions. In the first section of the interview scheme, the concepts of professionalism and professional identity were explored. In the second section, participants were asked to comment on and assign a value to each dimension and item of the self-assessment instrument selected in the first phase of the study and to add or change new dimensions or items. In the third section, participants reported on the best strategies for teaching and learning medical professionalism based on the previously discussed dimensions.

A framework method approach was used to analyze the information gathered. The method was selected for its capacity to analyze data based on a predefined set of themes and categories (framework), such as those contained in the preselected instrument.¹² The analysis was based on the full transcription of the audio-recorded interviews and focus groups. Atlas-ti 8.0 software® was used for the analysis. It included a sequential process that started with open and axial coding, followed by the identification of nodes (key concepts) that were grouped into wider categories (dimensions). The program also provided information on resource density and conceptual density.

In the quantitative phase of the study, the adapted instrument was tested. The data were analyzed using the R software. Fifth-year medical students in clinical phase of training at the Catholic University School of Medicine in Santiago, Chile, were asked to respond to the adapted self-assessment professionalism instrument. Students had to read and provide informed consent before participating. They were informed that the questionnaire was designed as a self-assessment instrument and part of a formative strategy for improving medical professionalism. Therefore, there was expected variability and room for improvement in the achievement of the different dimensions of the instrument. Students were asked to rate their perception of achievement on each dimension using a scale from 0 to 4 (0: Not applicable; 1: very low; 2: low; 3 good; 4: excellent). A descriptive and psychometric analysis was conducted to determine the reliability and internal consistency of the adapted scale using the exploratory factor analysis technique.

The study was reviewed and approved by the Ethics Committee of the Pontificia Universidad Católica de Chile (ID 200302009). All participants from the qualitative and quantitative components of the study reviewed and signed an informed consent form before participating. The data were anonymized prior to its analysis. No patient data appear in the article.

Results

Fig. 1 summarizes the phases of the study. In the first phase, a total of 290 articles were found. None of the articles found reported a Latin American questionnaire designed to assess or self-assess medical professionalism. A total of 42 articles were fully reviewed, and 5 were included in the final analysis. The Professional Self Identity Questionnaire,¹⁴ the Professionalism Assessment Scale,¹⁵ and the Arnold Scale¹⁶ reported limitations in their content validity, reliability, or cross-cultural validity.¹⁷ The Penn State University College of Medicine Scale was tested with Colombian students.¹⁸ A high percentage (57%) of the participants in that study were in their first semester of medical school with no exposure to clinical practice, and the authors found that the scale demonstrated important differences when applied to Latin American medical students. The Professionalism Mini-Evaluation Exercise (P-MEX)¹⁹ was originally designed as a practical instrument to help exercise and improve medical professionalism. It has been tested with medical students and residents in a variety of cultural settings.²⁰ In a systematic review conducted by Kwan et al. (2018),²⁰ the authors found that the P-MEX scored the highest in terms of content and structural validity. It was originally developed as an observer-based instrument; however, it has also been used as a self-assessment instrument for medical students.²¹ Given the psychometric consistency of the P-MEX’s validity across different settings and cultures, it was selected for use as a reference in the next phases of the study. Written authorization from the authors was obtained to use the P-MEX instrument in this study. The original P-MEX instrument underwent a meticulous translation and back-translation process, ensuring its accuracy in both Spanish and English. Written consent was obtained from the authors for using the instrument in this study. (Written consent to use the instrument in this study was provided by Cruess R, Centre for Medical Education, McGill University, by email on August 24, 2020).

Fig. 1.

Phases of the study.

In the qualitative phase, 3 focus groups with a total of 19 medical students in their final stage of clinical training were recruited. The average time for each group was 65 min (range: 61–68 min). In addition, 9 in-depth interviews with clinical educators were conducted with an average time of 40 min (range: 38–46 min). The analysis generated 188 open codes and 468 quotes. An inductive approach following the framework methods was conducted. Axial codification generated 2 main categories: (1) the interaction between professional identity and medical professionalism and (2) medical professionalism self-assessment skills.

The majority of clinician educators perceived professional identity as an umbrella for professionalism. Professional identity was more related to “being” and professionalism was more related to “doing.” Professional identity was perceived as being developed from a number of sources beyond medical practice such as cultural background, family and friend relationships, and personal traits. Professionalism, on the other hand, was perceived as a way of practice or as an expression of professional identity. The majority of students perceived professional identity as a “personal imprinting” of medical practice, as a trait that differentiated their practice from others. They perceived professionalism as a more standard concept or the “way we all should” practice. In contrast, professional identity was related to the uniqueness of medical practice. Clinician educators and students agreed that both characteristics are dynamic and developed during medical practice and that they evolve and interact with each other. They also agreed that formative self-assessment was an essential strategy for developing professional identity and exercising professionalism.

When analyzing the P-MEX instrument for professionalism self-assessment, nearly all clinician educators and medical students agreed on the relevance of the dimensions included in the original version; however, the majority expressed the relevance of including wellbeing and self-care as key personal dimensions. Likewise, the majority expressed the importance of including a dimension related to social accountability and social justice and instead to include a broad contextual perspective that captures the relevance of social determinants of health in clinical practice. Fig. 2 presents the 3 main levels for assessing medical professionalism identified by the participants in the qualitative phase of the study. They identified a micro level, which grouped personal practices of reflective and self-care skills, and a meso level, which integrated interpersonal practices of doctor–patient relationships, interactions with other team members, and interactions with the system or organization. The macro level was associated with the interactions of physicians with society and expressed the responsibility of trying to achieve equity by reducing unjust inequalities in medical practice. Medical students identified 5 components to be included in the self-care subcategory of the medical professionalism assessment: identification of boundaries to their medical and academic responsibilities, early recognition of burn-out symptoms, ability to save time for self-care and wellbeing practices, request for help in situations of anxiety, and learning of relaxation techniques. Three components were highlighted in the social accountability subcategory: appropriate use of limited health resources, identification of social determinants that could affect clinical care, and identification of options for addressing social disparities in clinical care. Most clinician educators reinforced the inclusion of these components and shaped them in the iterative process of the in-depth interviews. In this process, the items of the scale were refined and translated into skills. The final scale to be tested included 30 items with 2 new dimensions: self-care and social accountability skills.

Fig. 2.

Levels and attributes of medical professionalism.

In the quantitative phase of the study, 112 of 124 (90.3%) fifth-year students completed the adapted P-MEX questionnaire. Their mean age was 24 years (standard deviation 2.7 years) and 48% of the participants were female. No item was rated at level zero (not applicable). Fig. 3 shows the distribution of their self-related skills on medical professionalism. Students’ perceptions of achievement were higher than good on average in all aspects measured. About two-thirds of the participants rated their meso level skills (i.e., doctor–patient relationship, team member interaction, and management skills) as “excellent.” However, only about half of them rated their micro level skills as “excellent.” In this area, 40% of them considered their self-care skills low or very low. In the same line, only 42% of medical students rated their social accountability skills as “excellent.”

Fig. 3.

Self-assessment skills levels reported by medical students (n = 112).

For the exploratory factor analysis, a Likert-type scale from 1 to 4 was applied. To determine the number of factors to be extracted, statistical criteria such as the Kayser criterion, scree plot analysis, and parallel analysis were considered, as well as the exploration of different solutions suggested by those criteria (3- and 4-factor solutions). In the 4-factor solution, only 1 item was clearly defined in the fourth factor, and this explained only 3% of the common variance, so it was discarded. We decided that the structure that could be at the base of the scale is 3 factors, which explained 27% of the variance. Table 1 shows the final P-MEX-modified scale with the means, standard deviations, and reliability estimators at the dimensional and global levels, and the configuration matrix with varimax rotation. For the global consistency and reliability of the P-MEX modified scale, the Cronbach’s estimate was 0.82 and the Omega estimate was 0.86.

Table 1.

Levels, dimensions, and items of the adapted P-MEX instrument.

	Mean	SD	[F1]	[F2]	[F3]
Micro Level
Reflective skills:
1. Demonstrates awarnesss of limitations	3.46	0.63	0.16	0.24	0.33
2. Admits errors or omissions	3.57	0.55	0.12	0.09	0.48
3. Requests feedback	3.06	0.79	0.44	0.17	0.09
4. Accepts feedback	3.80	0.42	0.25	0.03	0.38
Selfcare skills
5. Recognize burn-out symptoms	3.15	0.87	0.12	0.58	−0.01
6. Identifies boundaries in professional and academic activities	2.99	0.82	0.17	0.56	0.09
7. Requests help in situations of anxiety or exhaustion	2.29	1.01	−0.02	0.54	0.07
8. Saves time for self-care and recreation activities	2.93	0.90	−0.01	0.60	0.05
9. Uses relaxation techniques in situations of stress or anxiety	2.54	0.99	−0.10	0.73	−0.09
Meso Level
Doctor–patient relationship skills
10. Listens actively to patients	3.74	0.48	0.40	0.06	0.41
11. Shows interest in patients as persons	3.80	0.40	0.25	0.04	0.35
12. Recognize patient’s needs	3.50	0.52	0.49	0.07	0.08
13. Tries to respond to patient’s needs	3.78	0.44	0.59	0.04	0.20
14. Ensures continuity in patient’s care	3.44	0.83	0.33	0.05	0.17
15. Maintains appropriate boundaries with patients and colleagues	3.67	0.57	0.14	0.13	0.50
Management skills
16. Is punctual	3.67	0.59	0.09	0.12	0.04
17. Completes tasks in reliable fashion	3.68	0.59	0.38	0.30	0.20
18. Maintains appropriate personal appearence	3.88	0.33	0.09	−0.10	0.26
19. Addressed own gaps in knowledge and skills	3.40	0.66	0.51	0.22	0.24
20. Maintains patient’s confidentiality	3.77	0.50	0.32	0.02	0.48
21. Respects rules and procedures of the system	3.82	0.49	0.22	0.08	0.59
Interprofessional skills
22. Demonstrates respect for to colleagues and health team members	3.95	0.23	−0.09	0.01	0.69
23. Is available to support other colleagues or health team members	3.76	0.47	0.56	−0.01	0.29
24. Avoids derogatory language	3.82	0.45	−0.19	0.03	0.60
25. Maintains composture in difficult situations	3.57	0.64	0.20	0.23	0.31
26. Demonstrates skills for collaborative work with colleagues and health team members	3.66	0.49	0.46	−0-06	0.21
Macro Level
Social accountability skills
27. Advocates on behalf of patients when addressing limitations in the health-care system	3.45	0.82	0.44	0.16	0.14
28. Uses health resources appropriately	3.46	0.68	0.21	0.20	0.17
29. Identifies social determinants that affect clinical care	3.48	0.58	0.43	−0.03	0.04
30. Explores options to address social inequities in clinical care	3.15	0.90	0.41	0.10	−0.02
Eigenvalues			4.91	1.93	1.48
% variance by dimension			16	6
% variance global scale				27
Omega ω by dimension			0.78	0.74	0.74
Cronbach’s alpha global scale	0.82
Omega ω global scale	0.86

Discussion

This study provides relevant information on new key dimensions to be considered when promoting and assessing professional identity and medical professionalism among medical students. In addition to the original dimensions, 2 new key dimensions emerged: self-care as a strategy for improving wellbeing and social accountability to fulfill the social contract of medicine in a world of increasing health disparities. In the scoping review of professionalism assessment tools conducted by Ong et al. (2020),⁶ the authors reviewed 138 articles and found that interpersonal behaviors were highly predominant over individual and organizational dimensions. Self-care skills were not included in the tools reviewed, and social accountability skills were marginally considered. A Lancet editorial emphasized the importance of considering self-care, equity, and social justice as part of modern professionalism.⁹ The adapted P-MEX instrument tested in this study incorporated self-care and social accountability skills. The descriptive analysis of the information showed that these 2 areas were the least rated by students, and therefore needed the most improvement. The self-assessment instrument showed high internal consistency and reliability, and it is one of the few medical professionalism instruments tested in and adapted for Latin America.

Physician wellbeing has emerged as an important concern given the high prevalence of burnout, mental health problems, and suicidal behaviors among physicians.¹⁰ In a systematic review conducted by Dewa et al. (2017),²² the authors found an association between burnout and lower quality and safety of care. Similar findings were found when studying the association between resident burnout and patient safety.²³ The quality of the studies included in both reviews was moderate, but the authors highlighted the need to develop early identification processes and interventions at the physician’s formative level. In a meta-analysis published by Hathaisaard et al. (2022),²⁴ the authors selected 6 high-quality studies and found that mindfulness-based interventions could significantly reduce subjective stress among medical students. The authors emphasized the relevance of having early systematic detection of stress symptoms among students.

In our study, self-care skills were one of the areas with the lowest level of achievement among medical students. Only 27% of students rated their self-care skills as excellent. The components perceived by students with the lowest skills in this area were their ability to request help in situations of anxiety and exhaustion and their competence in using relaxation techniques in situations of stress or anxiety. This information is relevant and emphasizes the need to develop these competencies in teaching medical professionalism to improve future physicians’ well-being

The social role of the medical profession was highlighted very early in the Charter on Medical Professionalism.²⁵ The Latin American Panel on Social Accountability also emphasized the relevance of including social accountability training skills at the undergraduate medical level.²⁶ Screening and early intervention on social determinants of health have been identified as essential strategies to improve social accountability and medical fairness in clinical settings. Identification of social and cultural factors such as low health literacy levels or cultural beliefs can be essential to improve clinical fairness in underserved populations with chronic diseases such as diabetes, coronary artery disease, or mental health conditions. Not considering these factors in the clinical interaction could increase health disparities.²⁷ In a study conducted by Schickedanz et al. (2019),¹¹ the authors found that only 41% of clinicians expressed confidence in their skills to address social needs, and only 23% of them practiced social screening. Similar results were reported by Garg et al. (2019),²⁸ who studied social screening practices among pediatricians in the United States.

Our study showed that only 40% of medical students rated their social accountability skills as excellent. Including self-assessment of social accountability skills as part of undergraduate medical training could improve opportunities for learning these skills early on.

This study has some limitations that are important to address. First, it was conducted in a medical school in Latin America with an extensive history in medical education and therefore, students and clinician educators might have a different perspective of medical professionalism that might not represent other academic centers. However, the extensive review of instruments and the selection of one with high cross-cultural testing assures that the dimensions explored are highly accepted in different contexts. Self-care and social accountability are consistent with the importance reported of these dimensions in medical education across cultures, and the addition of these dimensions reported in the current study might be relevant in other clinical settings. The sample size is another potential limitation of the quantitative phase. Although our sample size could be appropriate for exploratory factorial analysis according to several recommendations,^29–31 our study used a convenience sample and applied an exploratory approach. Future studies could replicate the proposed structure with a larger sample size and apply confirmatory factor analysis.

Another limitation is the adaptation of an originally observed-based designed instrument to use as a self-assessed instrument. However, the selection process of the instrument considered that the P-MEX instrument had been previously used and tested as a self-assessment tool.¹⁹ Additionally, this process also included a large and systematic qualitative component with the participation of medical students and clinician educators to ensure content validity. The subsequent application and analysis of the instrument tested the internal consistency and reliability of the instrument to assure the validity of the instrument.

In conclusion, this study provided evidence of validity of an adapted assessment instrument with high internal consistency, and reliability to self-assess medical professionalism in medical students during their clinical phase of training. The adapted instrument includes micro-, meso-, and macro level skills, and it incorporates 2 new dimensions: self-care and social accountability skills. The instrument can be an effective tool for guiding students in their medical professionalism and professional identity formation.

Source funding

Novus-Triada University Research Academic Grant (Universidad Católica de Chile; TEC de Monterrey, Mexico; Universidad de los Andes, Colombia).

Ethical Approval

This study was reviewed and approved by the Ethics Committee of the Pontificia Universidad Católica de Chile (ID 200302009). All participants reviewed and signed and informed consent form previous to the participation in the study.