Effect of the Extended Inverted Classroom on clinical simulation for the resuscitation of trauma patients: Pilot study of student perceptions of learning

Abstract

Introduction

The inverted classroom is an interactive teaching model with promising results in surgery. Evidence of this model involving components of clinical simulation is scant.

Objective

To present a model of “Extended Inverted Classroom” (EIC), involving low and medium fidelity simulation, and to describe its effect on student perceptions of learning.

Methods

An EIC was designed for teaching medical students the clinical skills for the management and resuscitation of trauma patients. The pre- and post-interventional perceptions of learning were assessed using the “Flipped Classroom Perception Instrument” validated in the Spanish language. The Mann–Whitney test and the t-test (p < .05) were used for comparisons and Cohen's d was used for calculating the effect size of the intervention. Secondary outcomes were academic performance and student satisfaction.

Results

A total of 75 students participated in the study. Global perception of learning was 3.98 ± 0.58 (1.5–5.0) (pre-test) versus 4.24 ± 0.64 (1.38–5.0) (post-test) (p < .05). The effect size was d = 0.42 (95% CI 0.094–0.75). High academic achievement and student satisfaction were identified.

Conclusions

The EIC model involving clinical simulation for the resuscitation of trauma patients has a positive effect on student perceptions of learning in the short term. Further studies are required to evaluate this model in other scenarios of resuscitation, critical care and emergencies, as well as to measure their effect on long-term learning.

Resumen

Introducción

El aula invertida es un modelo de enseñanza interactiva con resultados promisorios en cirugía. La evidencia de este modelo involucrando componentes de simulación clínica es insuficiente.

Objetivo

Presentar un modelo de “aula invertida extendida” (AIE), que incorpora simulación de baja y mediana fidelidad, y describir su efecto sobre las percepciones de aprendizaje estudiantil.

Métodos

Se diseñó una AIE para la enseñanza de habilidades clínicas para el manejo y resucitación del paciente traumatizado dirigida a estudiantes de medicina. Las percepciones de aprendizaje pre y post-intervención fueron evaluadas mediante el instrumento “Flipped Classroom Perception Instrument”, el cual fue validado en idioma español. Las comparaciones se realizaron con pruebas de Mann-Whitney y t-test (p < .05) y el tamaño del efecto de la intervención se calculó con la prueba d Cohen. Como desenlaces secundarios se calcularon el rendimiento académico y la satisfacción estudiantil.

Resultados

Un total de 75 estudiantes participaron en el estudio. La percepción global de aprendizaje fue 3.98 ± .58 (1.5-5.0) (pre-test) versus 4.24 ± .64 (1.38-5.0) (post-test) (p < .05). El tamaño del efecto fue d = .42 (IC95% .094-.75). Se identificó alto rendimiento académico y satisfacción estudiantil.

Conclusiones

El modelo de AIE involucrando simulación clínica para el manejo del paciente traumatizado tiene un efecto positivo sobre las percepciones de aprendizaje estudiantil en el corto plazo. Se requieren nuevos estudios para evaluar este modelo en otros escenarios de resucitación, cuidado crítico y emergencias, así como mediciones de su efecto en el aprendizaje de largo plazo.

Introduction

Trauma and noncommunicable diseases continue to be the leading causes of death in Colombia.¹ Greater efforts are required on the part of the state at different levels in order to reduce disease burden, as well as to optimise education processes and to develop standardised protocols.^2,3 In terms of the latter, the Advanced Trauma Life Support program (ATLS) of the American College of Surgeons has contributed to the development of improved short- and medium-term knowledge and skills among healthcare professionals, but its impact on the reduction of morbidity, mortality and disability is still unknown.^4,5 The problem is even worse considering that, although protocols like ATLS already exists, curricula offered in medical schools is still poor, particularly and paradoxically in countries like Colombia were the prevalence of traumatic injury is high.⁶ To compound this situation, these curricula are developed mainly under an instructional design in which information is conveyed in a unidirectional way. This design is implemented in master classes where the theoretical principles of ATLS are emphasised over the development of practical skills. A fragmented model in which the theorist focuses on the ability to memorise instead of analysing and evaluating, and in which the theory is disconnected from practice, results in lower or even absent long-term knowledge retention when put to the test in real situations of daily practice.⁷ An integrated instructional design which avoids fragmentation, promotes interaction and uses simulation of varying degrees of fidelity, may help solve these challenges.⁸ However, the evidence available is limited.

The inverted classroom is a way to provide students with an interactive format of instructional design. Using this strategy, the traditional components of the class are inverted and made available to the students for independent study in the form of videos, documents and other materials, usually available on-line. That way, the traditional time of the class is used for interactive teaching based on case discussions. Case discussions may promote superior analytical and evaluative cognitive skills. However, one of the limitations of this classic approach of the inverted classroom is how to integrate practical skills with the use of clinical simulation. Applied to resuscitation, simulation offers the possibility of promoting clinical and non-technical skills such as leadership, teamwork and communication that result in patient safety.⁹ Some recent experiences have incorporated simulation into the inverted classrooms with promising results, especially in surgery, emergency medicine and resuscitation.^10–13 However, these studies have not assessed student perceptions of learning. Neither have they assessed long-term knowledge retention and application. This study contributes to fill the former of these gaps. For this purpose, we have given the name of “Extended Inverted Classroom” (EIC) to an interactive teaching/learning model that incorporates not only out-of-classroom activities and conventional classroom activities of the inverted classroom, but also “extends the classroom” to controlled scenarios in the simulation laboratory. The objective of this pilot study is to measure learning perceptions of undergraduate students regarding this model developed for the care of adult trauma patients.

Materials and methods

Design and participants

This is a pre-experimental pre-test and post-test study designed to measure the effect of a training course in the management of trauma patients, developed through an EIC model, on student perceptions about learning. During the months of February and March 2017, 75 medical students in their fourth year of medical training in a private Colombian university were invited to participate. The Medical Education Committee (Medical School, Universidad de la Sabana) approved the ethical components of the study. All the participants gave verbal consent for inclusion. The authors ensured anonymity and confidentiality of all the information about the participants.

Intervention: Extended Inverted Classroom (EIC) design

The course on the management of trauma patients was designed between August and December 2016 under a 4C/ID comprehensive conceptual approach (four components of the instructional design) including learning task design, support information, procedural information, and practical tasks.¹⁴ These components were developed in accordance with the 9th edition of ATLS.¹⁵ In general, learning tasks were stated as Expected Learning Results (ELR) and, methodologically, the instructional design was planned through an EIC teaching/learning model implemented over several phases during a five-week period (Fig. 1).

Fig. 1.

Inverted classroom model extended to clinical simulation.

Source: Authors.

EIC phases

The first phase consists of self-learning activities outside the classroom (two weeks). Learning tasks focus on individual review and study of the support information (videos and literature references) following self-study checklists. These materials are lodged in an institutional on-line platform to support independent work, where a discussion forum is enabled.

Later, during the second phase, learning activities are developed in the classroom. For these activities, students are provided with relevant procedural information and practical simulation guidelines. Initially, this phase includes “in-class” low-fidelity simulation tasks developed in 6-h sessions during which students review eight clinical cases working interactively in small groups with a facilitator. Later, each case is analysed in a plenary session and synthesised in conceptual maps. Finally, classroom activities are extended to medium-fidelity simulations “in the laboratory” in order to reinforce learning tasks and provide familiarity and authenticity in the care of trauma patients (using models, mannequins and images). Practical tasks are carried out in 6-h work stations focused on primary assessment and resuscitation, airway management, helmet removal and cervical spine immobilisation, management of chest injuries, shock management and venous access, systematic approach to the interpretation of basic chest, neck, pelvis X-ray images and head CT in the trauma patient, and insertion of urinary and nasogastric catheters.

Consistent with the ELRs, the third phase (last week of the course) focuses on student performance evaluation with summation and education purposes at a “know how” and “show how” level. To this end, students take a written multiple choice test centred on clinical case analyses, as well as a test based on the Objective Structured Clinical Examination (OSCE) methodology designed to assess practical skills. For the latter, different scenarios with clinical problems are presented to the students who are evaluated by independent observers in several stations, using global scales and checklists. The evaluation ends with a bidirectional feedback process between instructors and students. Finally, the instructional design includes a quality assurance process which spans the entire course.

End points, tools and data collection

The primary endpoint was the impact of EIC on student perception of learning, assessed by means of the Flipped Classroom Perception Instrument (FCPI),¹⁶ built on previously designed tools,^17,18 translated into Spanish and adapted for the purpose of this research. The tool consists of eight questions rated on a Likert scale (1 = totally disagree; 5 = totally agree), distributed in two sub-scales (“pre-class” activities: items 1–3; “in-class” activities: items 4–8). The internal consistency of the questionnaire (Cronbach's α) in its original version is 0.81.¹⁶ The secondary endpoints were academic performance and student satisfaction with the evaluation. Performance was determined on the basis of the scores obtained in the theoretical and practical OSCE-type test (see statistical analysis). Student satisfaction was assessed using a generic institutional questionnaire previously designed to assess relevance, consistency, and contribution of the evaluation to student learning. The questionnaire consists of 5 questions rated on a Likert scale (1 = poor; 5 = excellent).

A printed questionnaire was prepared for data collection. After conducting a pilot test, the tool was administered to the students two weeks before the course in a voluntary, anonymous and confidential way. Post-intervention evaluation was performed two weeks after the end of the course. Considering the duration of the intervention, pre- and post-test evaluations were done with a nine-week difference. Student satisfaction with the evaluation was measured at the end of the practical test. The results were tabulated and stored in a database.

Statistical analysis

• 1.

  Construct validity and questionnaire reliability

An exploratory factor analysis (EFA) was conducted for this purpose. Sample adequacy was determined using the Kaiser–Meyer–Olkin (KMO) criterion (satisfactory if >0.80).¹⁹ The EFA was conducted using the Maximum Plausibility (MP) method in order to derive the values for each individual factor (eigenvalues) and their proportion of total variance. Once the number of factors was determined, an oblique rotation (promax) was performed. The proposed criterion to retain an item of the corresponding factor was >±0.30.²⁰ The internal consistency of the scale was determined using Cronbach's alpha coefficient (adequate if >0.70).²¹

• 2.

  Pre- and post-intervention comparisons

Means, standard deviations (SD) and medians for the questionnaire were calculated for this purpose, as well as the sub-scales and items in the two time periods. Comparisons between the pre- and post-intervention results were done in accordance with the distribution symmetry, using the student t test and the Mann–Whitney test (significant if p < .05). Cohen's d was calculated to indicate effect size among the means for each item, sub-scales and overall scores (95% confidence interval). Reference criteria for interpreting effect size were: small effect d = ±0.20, medium effect d = ±0.50 and large effect d = ±0.80.²²

• 3.

  Academic performance and student satisfaction

Academic performance was calculated as the average of the scores obtained in the written test and the OSCE (range 0–5), with a cutoff point for a passing grade of 3.0. Academic performance was divided into quartiles (from the 25% of students with the lowest results, to the 25% of the students with the highest results). Finally, descriptive statistics (means, SD, 95% CI and ranges) were calculated for the items related to measurements of student satisfaction with the evaluation. The statistical analysis was performed using the Stata software package version 14 (Stata Corp., USA).

Results

For the pre-test, the response rate was 93.3% (70 students) and it was 100% for the post-test and the satisfaction assessment (75 students). The mean age of the participants was 21.92 ± 1.45 years (60% females).

• 1.

  Construct validity and tool reliability

The analysis covered the total number of observations, including pre- and post-intervention observations (n = 145). The value of the KMO test was 0.84. The EFA allowed to identify two factors that explained 83% of the total variance (values of 3.01 and 1.99, respectively). Following oblique rotation, the factorial solution allowed to identify a structure similar to that of the original version in which the factorial loads for Factor-1 were taken by items 1–3, and those of Factor-2 by items 4–8. Factorial loads are illustrated in Table 1. In this research, Factor-1 was called “out-of-classroom activities,” and Factor-2 was called “classroom activities and simulation lab.” The internal consistency of the scale (Cronbach's α) was 0.85.

• 2.

  Pre- and post-intervention comparisons

Table 1.

Factorial loads after EFA and oblique rotation.

Items	Factor 1	Factor 2
Item 1	0.8809	0.0003
Item 2	0.4176	0.3945
Item 3	0.7071	0.0599
Item 4	0.1893	0.6763
Item 5	0.0070	0.6462
Item 6	0.0320	0.7854
Item 7	−0.0101	0.8299
Item 8	−0.0346	0.4103

EFA: exploratory factor analysis.

Source: Authors.

For overall learning perception, means, SD and range were 3.98 ± .58 (1.5–5.0) (pre-test) versus 4.24 ± .64 (1.38–5.0) (post-test) (p < .05). Effect size was d = 0.42 (95% CI 0.094–0.75). For learning perceptions regarding “out-of-classroom activities” (sub-scale) the values were 3.90 ± .72 (2.0–5.0) (pre-test) versus 4.27 ± .80 (1.0–5.0) (post-test) (p < .05). Effect size was d = 0.48 (95% CI 0.15–0.81). Finally, for learning perceptions regarding “classroom and simulation lab activities,” the values were 4.02 ± 0.64 (1.0–5.0) (pre-test) versus 4.22 ± .63 (1.6–5.0) (post-test) (p < .05). Effect size was d = 0.31 (95% CI – 0.013–0.64). These comparisons are shown in Fig. 2. Table 2 shows descriptive statistics and effect size for every item in the questionnaire.

• 3.

  Academic performance and student satisfaction

Fig. 2.

Comparisons of student perceptions before and after the intervention.

Source: Authors.

Table 2.

Pre- and post-intervention comparisons, statistical differences and effect size.

Items, sub-scales and overall score		Pre-intervention (n = 70)			Post-intervention (n = 75)			p	Effect size
		Mean	SD	Mediana	M	SD	Median		d	95% CI
Item 1	ONLINE modules improve my learning	3.57	1.12	4	4.10	.93	4	.0008	.51	.18–.84
Item 2	Learning of the key content before classroom and simulation lab activities improve my learning	3.95	.85	4	4.34	.79	4	.00045	.47	.14–.80
Item 3	ONLINE modules combined with knowledge application improve my learning	4.21	.74	4	4.36	.92	5	.05	.17	−.14–.50
Total Sub-scale 1		3.90	.72	4	4.27	.80	4.33	.002^**	.48	.15–.39
Item 4	Interactive and applied activities in the classroom and the simulation lab improve my learning	4.34	.84	5	4.50	.74	5	.12	.20	−.12–.52
Item 5	I participate and engage in classroom and simulation lab activities	3.60	.93	4	3.93	.93	4	.06	.35	.02–.68
Item 6	Application of the main content in class and in the simulation lab improves my learning	4.34	.69	4	4.28	.72	4	1	−.85	−.41–.24
Item 7	Classroom discussion of the main content improves my learning	4.14	.80	4	4.36	.81	4	.05	.27	−.054–.60
Item 8	Group work improves my learning	3.68	1.07	4	4.02	.92	4	.002^**	.34	.012–.66
Total Sub-scale 2		4.02	.64	4	4.22	.63	4.20	.008	.31	−.013–.64
Global		3.98	.58	4	4.24	.64	4.31	.0005	.42	.094–.75

Sub-scale 1: items 1–3; Sub-scale 2: items 4–8. SD: standard deviation; CI: confidence interval.

**

Statistical differences calculated using the t-test. The remaining differences were calculated using the Mann–Whitney test.

Source: Authors.

Average academic performance on the basis of the written test and the OSCE was 3.95 ± 0.47 (2.6–4.9). Student distribution by academic performance quartiles (Q) was: Q1: <3.6 (25.33% of the students); Q2: 3.7–3.9 (25.33%); Q3: 4.0–4.3 (26.67%); Q4: >4.4 (22.67%). All of the student satisfaction items regarding evaluation in terms of relevance, consistency and contribution to overall training were higher than 4.0 (Table 3).

Table 3.

Student satisfaction with the evaluation (n = 75).

Item	Mean	SD	95% CI
Topics evaluated/discussed were relevant	4.51	0.08	4.33–4.69
Consistency of the objectives proposed in the evaluation	4.21	0.10	4.00–4.43
Relevance and currency of the topic evaluated/discussed	4.33	0.09	4.15–4.52
Contribution to overall student/participant training	4.47	0.09	4.28–4.65
Actual examples and experiences used during the evaluation	4.14	0.12	3.89–4.40

SD: standard deviation; CI: confidence interval.

Source: Authors.

Discussion

The findings of this study indicate that, in medical students, the EIC model designed for the management of trauma patients, had a positive effect on global student perceptions regarding learning and out-of-classroom and in-classroom components. There was also evidence of adequate academic performance and high student satisfaction as a result of the relevance and consistency between the topics included in the curriculum and performance evaluation.

These findings may be explained by the instructional model design that incorporates low and medium fidelity simulation components in an aggregate fashion into a curriculum centred on clearly defined learning outcomes for the level of performance of the general practitioner. The available literature reports the benefits of this type of curricular integration, as a longitudinal design that shows consistency and relevance with clinical competencies.^10–13,23 In essence, the whole purpose of the inverted classroom methodology is to increase student engagement with the content, optimise teacher-student contact time, and improve learning.^24,25 However, this rationale requires empirical evidence. Our study provides information in several directions, in particular the way in which students perceive the effects on learning of such a design. One of the key elements has to do with the aspects behind self-regulated independent learning outside the classroom. After completion of the intervention, all global perceptions regarding out-of-classroom activities reflected significant changes that help explain the moderate effect of this type of design on motivation and independent study. Likewise, the size of the effect on classroom activities showed that the incentive towards interactive and collaborative work, as well as the application of the theoretical content to practical situations by means of discussion and clinical simulation as a result of the EIC model, showed a positive change after the intervention. These positive effects were complemented with a student satisfaction assessment which showed consistency and relevance of the intended objective, between what was done and what they were evaluated on. These components have been recognised as key for promoting self-determination in healthcare education.²⁶

This study has several strengths and limitations. Strengths include an education intervention design supported by a valid methodology for instructional design that integrates theory and practice, and which also incorporates clinical simulation with a degree of fidelity consistent with the learner's level of competency. Moreover, adaptation of measurement tools (FCPI) by means of statistical procedures (EFA) provides validity and reliability to the construct. However, this is a pilot study conducted in a single institution, which may limit the ability to generalise the outcomes. Likewise, the study did not include a control group to compare with the outcomes of other teaching strategies. It would be important to conduct other research studies of experimental and quasi-experimental design to help fill these gaps.

This study has practical implications. On the one hand, it provides evidence on an interactive instructional design that may lead to a paradigm shift in traditional medical education. In this regard, it has implications for faculty development. Another practical implication has to do with the use of technology in education as a means to strengthen, through interactive teaching, the most important component in our opinion, which is classroom teaching. Finally, it represents a way to connect clinical simulation with the theoretical component of the curriculum, incorporating simulation of varying degrees of fidelity both into the learning process as well as into the evaluation. We also believe that this study opens the possibility for new questions and future research into different areas. How to connect the EIC model with real scenarios (high fidelity)? How to transfer EIC to practical rotations in the hospitals? What is the effect of the EIC model on long-term learning (sequential measurement throughout time), in order to assess the ability to retain knowledge in the long run? What is the relationship between the EIC model and performance at the top tier of Miller's pyramid (do) in practical situations and in contact with dummy patients and real patients later on? These questions require studies with different methodological designs and theoretical approaches.

To conclude, we have presented the design of an EIC model involving simulation scenarios for the management of trauma patients, which is perceived positively by the students in terms of learning, has practical implications, and opens new inroads for future research.

Ethical disclosures

Protection of human and animal subjects

The authors declare that no experiments were performed on humans or animals for this study.

Confidentiality of data

The authors declare that they have followed the protocols of their work center on the publication of patient data.

Right to privacy and informed consent

The authors declare that no patient data appear in this article.

Funding

Medical School – Universidad de la Sabana, Colombia.

Conflict of interest

None reported.