A competency-based Internal Medicine course in an emergency education setting

Abstract

Introduction

At the beginning of 2020, the coronavirus SARS-CoV-2 arrived, triggering the COVID-19 pandemic that affected every aspect of daily life including global education systems. This dire public health situation across the planet forced the accelerated implementation of emergency remote teaching at all education levels. Educational institutions and hospitals adopted virtual classes, webinars, tutorials, and virtual libraries to continue academic activities. Other useful tools researchers used were standardizing patients and clinical simulation. The central objective of this study was to analyze how an Internal Medicine competency-based course, with a theoretical and a practical block, responded to an emergency education environment from June to December of 2020.

Material and methods

Students and teachers completed written evaluations, the students completed the DREEM questionnaire and teachers recorded their personal insights. The authors conducted focus groups and analyzed transcripts of the virtual activities.

Results

All 60 students (100%) participated in the study. The students’ average score of the DREEM questionnaire on education environment was 177.8/200, and the students’ average teaching evaluation was 6.38/7. All students passed the final exam. The authors also categorized the favorable and unfavorable findings from both the theoretical and the practical blocks of the course on three dimensions: organization, pedagogy, and technology.

Conclusion

The results indicate that the students perceived an excellent educational environment. Several points for improvement emerged such as teacher training in computer technologies, the socialization of learning objectives, and the feedback that students receive from teachers to achieve the proposed learning objectives.

Resumen

Introducción

a principios de 2020, llegó el coronavirus SARS-CoV-2, que desencadenó la pandemia de COVID-19 afectando todos los aspectos de la vida cotidiana, incluidos los sistemas educativos mundiales. Esta grave situación de salud pública en todo el planeta obligó a la implementación acelerada de la enseñanza remota de emergencia en todos los niveles educativos. Las instituciones educativas y los hospitales adoptaron clases virtuales, seminarios web, tutoriales y bibliotecas virtuales para continuar con las actividades académicas. Otras herramientas útiles que utilizaron los investigadores fueron los pacientes estandarizados y la simulación clínica. El objetivo central de este estudio fue analizar cómo un curso de Medicina Interna basado en competencias, con un bloque teórico y otro práctico, respondió a un ambiente de educación de emergencia de junio a diciembre de 2020.

Materiales y métodos

los estudiantes y los maestros completaron evaluaciones escritas, los estudiantes completaron el cuestionario DREEM y los maestros grabaron opiniones personales. Los autores realizaron grupos focales y analizaron las transcripciones de las actividades virtuales.

Resultados

todos los 60 estudiantes (100%) participaron en el estudio. La puntuación media de los alumnos en el cuestionario DREEM de clima educativo fue de 177,8/200, y el promedio de valoración docente fue de 6,38/7. Todos los alumnos aprobaron el examen final. Los autores también categorizaron los hallazgos favorables y desfavorables de los bloques teórico y práctico del curso en 3 dimensiones: organización, pedagogía y tecnología.

Conclusión

los resultados indican que los estudiantes percibieron un excelente ambiente de aprendizaje. Surgieron varios puntos de mejora como la formación de los docentes en tecnologías informáticas, la socialización de los objetivos de aprendizaje y la retroalimentación que reciben los estudiantes por parte de los docentes para alcanzar los objetivos de aprendizaje propuestos.

Introduction

Medical education involves several sectors such as health, science, technology, and education, and given its complexity, it must be supplemented by basic, social, human, and clinical sciences.¹ However, in 2020, an additional challenge presented itself: the novel coronavirus SARS-CoV-2 and the ensuing COVID-19 pandemic, which among its effects on every other area of life suddenly required teaching during an emergency.

Worldwide, in medical education, countries made a range of changes to their curricula to meet pandemic conditions,^2,3 for instance withdrawing students from face-to-face clinical assignments and moving to short-term remote learning⁴. Other institutions adopted alternating rotating groups where one group is in the hospital and the other is at home for simulation, research, and independent work⁵. Researchers also identified changes in university evaluation practices,^6,7 including of graduates, intend to generating more medical personnel available to attend to the pandemic⁸. All these changes highlighted the lack of teacher training in remote education and virtual strategies, which created stress for teachers that served as barriers for many to teach effectively in these formats.⁹

In Colombia, some institutions decided to extend the vacation period, and others joined the initiative to migrate to distance learning supported by digital technologies, including alternating in-class groups to avoid exposing all students. Teachers incorporated other platforms such as instructional videos and podcasts and created activity guides for work at home. Instructors have also used didactic strategies such as the inverted classroom with remote synchronous sessions, and in medical school in particular, surgical simulation with remote tutoring has even been considered.^10–12 During this emergency, the term emergency remote teaching (ERT) emerged to describe this unplanned shift in learning.¹³

The emergency in ERT implied that the measures were temporary and that institutions would return to the previous methods once the crisis passed.¹⁴ The aim of ERT was to counteract the effects caused by the spatial distance between the student and the learning resources.¹⁵ Another term that gained traction during the pandemic is e-learning, defined as the use of computer technology to deliver training, including technology-supported learning, whether online, offline, or both. Internet-based learning, computer-based learning, virtual classrooms, and digital collaboration represent different types of e-learning that teachers have also used during the pandemic.^16–19

However, despite multiple efforts to ensure education during the pandemic, studies have shown that forms of ERT have had significant negative effects on students’ clinical learning experiences.^4,19 It is necessary for the relevant stakeholders to review and adapt current teaching and learning methods to the current conditions. The question that arises with this unprecedented experience is how leaders and instructors adapted the teaching of a competency-based Internal Medicine course, to the COVID-19 emergency.

Material and methods

The Internal Medicine course, has two components: theoretical and practical, lasting 20 weeks at 40 hours per week. The course is taught by 16 specialists from areas related to Internal Medicine and aimed at sixth-semester medical. Fig. 1. The two course components are presented in virtual and face-to-face formats. To reduce the exposure and transmission of the virus, it was decided to work with alternation, in which students are divided into two groups (virtual and face-to-face) and every 10 days they change groups.

Fig. 1.

Structure of the internal medicine course.

Data collection

We used a mixed-methods research approach in this descriptive, nonexperimental study, collecting the study data from multiple sources. We conducted focus groups with both teachers and students using a guide we developed. We recorded the virtual activities and the teacher's personal insights. We also collected the students’ written final evaluations, the written teacher’s evaluation and administered the DREEM questionnaire to the students, all of them at the end of the academic period. The students underwent three evaluations during the course, each session lasting 1 hour and 30 minutes and comprising 50 questions; the questions were multiple-choice clinical cases with a single correct answer rated from 1 to 5, where a score of 3 is considered approved. The university teacher’s evaluation questionnaire consists of 22 questions with numeric response options from 1 to 7, with 1 being the lowest score; in addition, there is an open comment section. The items evaluated are general development of the course, what happens in the classes, evaluation activities, and instructor’s teaching strategies implemented. For the students, we administered the Dundee Ready Education Environment Measure (DREEM), a Spanish validated questionnaire. It consists of 50 items grouped into five subscales and rated on five-point Likert-type scales, and the scores are categorized as follows: 0–50, very poor educational environment; 51–100, environment has many problems; 101–150, environment is more positive than negative; 151–200 excellent educational environment.²⁰

Data processing and analysis

Given that we did a mixed approach, we initially outlined the data processing steps for the quantitative section. For the DREEM questionnaire, we calculated the average scores and standard deviation for each domain. Next, we determined the coefficient of variation (between agree and disagree) to assess the extent of response variability within each domain. Furthermore, items with 50% or more responses indicating “agreement” or “strong agreement” were classified as positive results. Using this information, we identified items with positive and negative results enabling us to determine the domains with the highest and lowest scores, respectively. Regarding the written evaluation of the 60 students, we calculated the average grades obtained and determined the percentage of students who scored above 4. Similarly, for the written teacher’s evaluation we computed the average score for each item and identified items with positive and negative results to determine the categories with the highest and lowest scores respectively.

Secondly, we describe the data processing steps for the qualitative section. To analyze the data derived from non-participant observation, we transcribed the information gathered from teacher’s personal insights and focus groups. Additionally, an observation grid was used to document recorded virtual activities (Annex 1). The information was then organized to define three analysis dimensions: Organization, pedagogy and technology. Subsequently, the data was coded based on these analysis dimensions for both theoretical and practical components using the ATLAS.ti program (Fig. 2).

Fig. 2.

Methods.

Results

The course we studied had a total of 60 students, average age 20 years, and 100% participated in the study. Most were from a high socioeconomic level, none worked, and most lived in Cali (94%). None of the students were married, had children, or had disabilities.

Quantitative findings

Of the total 60 students, 55 students completed the DREEM questionnaire. The mean total score for the five domains was 177.8, putting the students’ ratings in the range of excellent educational environment. The domains of learning and teaching obtained the highest scores, with averages of 42.76 and 40.82 and coefficient of variations of 14.4% and 12.5%, respectively. Conversely, the social domain received the lowest score, with an average of 23.02 and a coefficient of variation of 18.7%. (Table 1) Based on the above findings, the items that made the greatest contribution to the highest and lowest scored domains are presented. Within the learning domain, the items with the highest level of agreement were: "I am encouraged to participate in classes" and "Teaching is well focused" both with agreement rates of 58.2%. Following closely was the item "I am clear about the learning objectives of my courses” with an agreement rate of 56.4%. Regarding the social domain, the items that contributed negatively were "I rarely get bored in the courses I am taking" with 63.6% and "there is a good support system for students who suffer from stress" with 67.3% agreement rates. Despite being the domain with the lowest scores, the average score indicated a "not so bad social environment".

Table 1.

Descriptive summary by domain or sub-scale.

Domain	Average of total sums	Standar deviation	Variation coefficient	Agreement level
Academic	30.98 (3.87)	3.87	12.5%	25%
Learning	42.76 (6.14)	6.14	14.4%	42%
Atmosphere	40.24 (7.20)	7.2	17.9%	8%
Teachers	40.82 (5.09)	5.09	12.5%	36%
Social	23.02 (4.31)	4.31	18.7%	14%
Total	177.82 (22.49)	22.49	12.6%

All students completed the teacher’s evaluation for a 100% response rate. The average teachers rating was 6.38 points out of a maximum of 7. In the open comments, the willingness of teachers to resolve doubts and affability stood out as positive, and students’ negative comments were related to difficulties with internet connectivity. The student’s categories identified as most needing improvement were general course development and evaluation, probably related to the lack of interaction and feedback of the teachers, consistent with earlier findings.^24,25

For the written student evaluations, all students approved each exam, which likely reflected the alignment of the methodology and the evaluation strategy (inductive method of clinical cases). Nearly all, 93.3%, of the students obtained a final score greater than 4, with an average score of 4.3 of 5 and deviation of 0.36. The evaluations were based on course competencies required by Icesi University both global (critical thinking) and specific (diagnosis of health situations and actions in health situations).

Qualitative findings

For the qualitative analysis, three dimensions of analysis were used, both in the theoretical and practical components. At the level of the theoretical implications, on the organization dimension, the University held virtual, synchronous classes, and the students identified a number of strengths. First, the objectives of the course aligned with the professional profile of Icesi University. The courses trained students on technological tools and had a technological infrastructure that supported the educational processes. Students’ main proposed improvement on the organization dimension was making more meetings available for feedback between teachers, which serve as input for reflection and development of contingency plans in case of future emergencies.

Strength’s students identified on the pedagogy dimension were the inductive case method of teaching and the fact that the student’s evaluations aligned with the general objective of the classes; instructors chose the case method so that students could learn based on real-life experiences and situations. Regarding improvements, students commented on the teacher's lack of training in pedagogy and didactics; students proposed that with better training, teachers would know how to propose different strategies in technology-mediated learning.

On the technology dimension, students mostly identified lack of connectivity issues or issues with access to materials on digital platforms as the primary strength of this dimension. Despite this strength, the students believed that teachers were not adequately trained to operate the digital platforms, that efforts were needed to increase teachers’ professionalism, the use of technological tools to optimize participation, and interest; to reduce distractions; and to encourage meaningful learning through training courses, video tutorials, and teachers guides. Another aspect to improve was that some teachers and students do not turn on the camera during virtual classes; students found that this decreased contact and interest in the class and affected the teacher-student relationships.

Practical implications

The face-to-face practice blocks lasted three days on average, and the strategies used were practice with the patient and clinical simulation (standardized patient and I-human virtual platform). On the organization dimension of the practice block, the main strength students identified was scheduled access to a patient without exposure to the disease. Students also observed that the atmosphere was pleasant, they could freely ask questions, and their comments were encouraged. The coursework fostered the development of students' patient evaluation skills and diagnostic analysis, and the teachers effectively managed the activity. The main area for improvement was the lack of interaction between students. Among the strengths of the virtual simulation was that the students themselves could program their access and time, but the teachers cited as an area for improvement that it was not possible to know whether all students were participating.

On the pedagogy dimension of the practice block, the students identified the following as strengths: they could acquire clinical abilities and skills in simulated scenarios without putting patients at risk and they played active roles. They also mentioned that the online questionnaire for the evaluation after each rotation was effective. As areas for improvement, the students felt the questions did not adequately assess the entire group and disapproved of the fact that the I-Human platform had used non-Colombian guides to develop the questions.

Among the strengths on the technology dimension, the students appreciated that in the standardized patient, after the synchronous virtual meetings, workshops and materials such as infographics and Word documents were made available. For improvements, students identified the need to incorporate more digital technologies that take advantage of the virtual space in addition to promoting activities to improve interaction with teachers and peers. Regarding the I-human platform, among the strengths was that it allowed students to acquire clinical skills in scenarios similar to real ones, without putting patients or students at risk. Among the aspects to improve, students cited difficulties with connectivity and the fact that the platform yielded automated results with no explanations.

According to the students, the strengths of the practice block on the organization dimension were the face-to-face spaces for patient examination and the small groups that allowed for almost personalized teaching. Students also found that the sessions had been adequately planned. The most important aspect to improve was that teachers were using class time to matters related to their own medical practice. The main aspect to improve lies in achieving better time management in the teacher's dedication to their students, as they are operating in a real clinical environment where it is challenging to refrain from attending to their professional duties.

Among the strengths on the pedagogy dimension were the didactic strategies used: clinical cases, applied technologies, and real patients. The student approved the practice section of the course if more than 80% of the teachers rated the student as qualified. Important aspects to improve were the short durations of the clinical rounds and the small number of patients seen in the week.

Discussion

Regarding the analysis conducted using the DREEM questionnaire, the social domain may have been affected by various factors. These include a decrease in face-to-face interactions with both teachers and peers, as well as the overall impact of emergency conditions and measures on student’s lives. These findings align with the results presented by Ríos,²⁵ who observed low scores in social self-perception among students in Paraguay, Venezuela, and Colombia during their second to fourth years of study. Similar outcomes were also noted by researchers investigating students at the School of Medicine of the Industrial University of Santánder.²² Despite these challenges, it is noteworthy that the summarized scores obtained in our study were consistent with previous findings in Colombia and, in fact, higher than scores reported in studies conducted before the pandemic.^21–24 From these results, it can be inferred that a competency-based curriculum has the potential to provide enjoyable learning experiences. Additionally, implementing changes in learning methodologies, such as smaller rotation groups and increased communication with teachers, can contribute to a more satisfactory online learning experience compared to the traditional teaching model. Universities are increasingly aware that it is not enough to be a good doctor to be a good professor of medicine and that continuous training is required not only in pedagogy but also in technological tools.²⁶

For the teacher’s evaluation, this received a positive rating. However, regarding the overall course development, there is a need for improvement, specifically in terms of increased interaction between students and teachers, which was hindered by the pandemic. In some cases, this interaction was negatively impacted by connectivity issues. As for the student’s written evaluation All students passed the exam, the majority with an average grade above 4, which is positive and demonstrates that both the activities and resources were aligned with the set objectives.

On the technology dimension, students cited as a strength that the teachers distributed learning materials such as infographics and medical records for reference outside of class. The primary improvement the students suggested was incorporating more technology resources in the sessions. The education needs that emerged from this analysis will be the basis for developing contingency plans for future emergencies. Regarding the organizational dimensions of an Internal Medicine course, faculty need to be prepared, cooperative, and transdisciplinary to facilitate building an academic community that allows continuous monitoring and evaluation of the learning environment. In response, teachers need training on the use of technological tools in a pedagogical and creative way to optimize students’ cooperative participation and interest, reduce distractions, and promote meaningful learning.

It is important to highlight that some teachers are still reluctant to shift to competency-based education, partly because some do not have university teaching training and others consider that they are already poorly recompensed for their work without new roles. The directors of Icesi University must encourage teaching from not only a financial but also an academic point of view, awarding spirit awards to teachers the students pick and awarding scholarships to stimulate teachers training and research in medical education. Students also have some responsibility for their learning processes, and instructors should at all times be cultivating in students the capacity to be autonomous.

In our experience with this study, we identified points to consider that helped us adapt a competency-based methodology to remote education. First, both formative and summative assessment made it possible to address the education process as a whole rather than focus only on outcomes. However, in the analysis of results, we identified a predominance of summative over formative assessments, primarily in the theoretical component of the course rather than the practical. Thereby, there is an urgent need for versatile mechanisms of assessment focused on training that allow for comprehensive evaluation rather than a basis solely on accreditation.

Second, we determined that the evaluations aligned with the course learning objectives, and third, we found that some technological aids such as tutorials helped stimulate autonomy and active learning. Finally, the university psychology service’s coping workshops promoted the students’ skills at living among each other during the crucial time of social isolation and mandatory quarantine. We found it a significant challenge to adapt competency-based medical education to a situation that makes face-to-face classes impossible, but we believe that our efforts make a good start toward a positive change.

Finally, the pandemic created restrictions that led educational institutions to seek innovative strategies, which are still in place as we return to face-to-face learning. One such example is the reduction of class sizes during rotation and the alternation between theoretical and practical components each week. These changes were implemented during the pandemic to address capacity issues in shared spaces. However, their implementation revealed advantages and improvements in the students' learning process, such as personalized and timely feedback, as well as increased interaction between teachers and students in practical settings with patients. Similarly, this occurred with the evaluation process of the practical component, where an online questionnaire was used to avoid in-person meetings of teachers for assessment, resulting in constant, timely, and higher-quality feedback by providing explicit details about each student's performance.

The Instructions for Authors and the ethical responsibilities have been taken into account and, all the signing authors fulfil the authorship requirements, and all have declared the not existence of any conflict of interests.

All the authors mentioned in the manuscript have agreed for authorship, read and approved the manuscript, and given consent for submission and subsequent publication of the manuscript.

The manuscript in part or in full has not been submitted or published anywhere, and will not be submitted elsewhere until the editorial process is completed.

This study did not needed consent by the Icesi University ethical approval committee because it was considered a no risk investigation, we did not make any intervention or modification in any variables (biological, physiological, psychological or social) of the students who participated in the study, but we did obtain written informed consent from the participants of the study and their information was anonymous.

The authors wish to thank the students who were part of this study for their willingness, effort and help and extend their sincere gratitude to Icesi University and Fundación Valle del Lili for their support during the execution of this study.

The authors received no financial support for the research, authorship, and/or publication of this article.