“Learning to learn” in the Extended Inverted Classroom: An evaluation of the effects of interactive teaching on knowledge and cognitive regulation in medical students

Abstract

Introduction

The evidence regarding the effects of the Inverted Classroom on students' metacognitive skills is limited. This study evaluates these effects on student perceptions about knowledge and cognitive regulation during a surgical course.

Methods

We measured student perceptions before and after a conventional Inverted Classroom in surgery using the Metacognitive Awareness Inventory. We evaluated the mean differences between the scores of the two measurements using Student’s t-test (p < 0.05), and the size of the effect on knowledge and cognitive regulation using Cohen’s d test (95%CI).

Results

A total of 158 students (pre-intervention) and 155 students (post-intervention) were included in the analysis. Significant differences were found (p < 0.05) in, as well as positive effects on, the students’ abilities to know their own learning (declarative knowledge) (d = 0.24; 95%CI, 0.02−0.47), to use learning strategies (procedural knowledge) (d = 0.19; 95%CI, –0.02 to 0.41) and to understand when and why to use these strategies (conditional knowledge) (d = 0.20; 95%CI, –0.01 to 0.42), in addition to the abilities to plan (d = 0.31; 95%CI, 0.09−0.53) and to evaluate their learning (d = 0.31; 95%CI, 0.08−0.53).

Conclusions

The Inverted Classroom has positive effects on metacognitive skills according to students’ perceptions. Further studies are required that compare these effects with other methodologies for teaching in the classroom and the workplace.

Resumen

Introducción

La evidencia sobre los efectos del Aula Invertida en las habilidades metacognitivas de los estudiantes es escasa. El presente estudio evalúa estos efectos en las percepciones estudiantiles acerca del conocimiento y la regulación cognitiva durante un curso de cirugía.

Métodos

Las percepciones estudiantiles se midieron antes y después de un Aula Invertida convencional en cirugía, utilizando el Metacognitive Awareness Inventory. Las diferencias de medias entre las puntuaciones de las 2 mediciones se evaluaron mediante la prueba de la t de Student (p < 0,05) y el tamaño del efecto en el conocimiento y la regulación cognitiva, mediante la prueba de la d de Cohen (IC95%).

Resultados

Se incluyó en el análisis a 158 (grupo preintervención) y 155 estudiantes (posintervención). Se encontraron diferencias significativas (p < 0,05) y efectos positivos en las habilidades estudiantiles para conocer el propio aprendizaje (conocimiento declarativo) (d = 0,24; IC95%, 0,02-0,47), emplear estrategias de aprendizaje (conocimiento procedimental) (d = 0,19; IC95%, –0,02 a 0,41) y entender cuándo y por qué utilizar dichas estrategias (conocimiento condicional) (d = 0,20; IC95%, –0,01 a 0,42). Además, sobre las capacidades para planificar (d = 0,31; IC95%, 0,09-0,53) y evaluar su aprendizaje (d = 0,31; IC95%, 0,08-0,53).

Conclusiones

El Aula Invertida tiene efectos positivos en las habilidades metacognitivas según las percepciones estudiantiles. Se requieren nuevos estudios que comparen estos efectos con otras metodologías de enseñanza en el aula y el sitio de trabajo.

Introduction

Metacognition has implications in the field of learning and instruction, and today it is a central topic in educational research.¹ It represents a superior cognitive ability to understand one's own abilities to "learn to learn", which involve knowledge, monitoring and cognitive control.^2,3 The first represents the way in which information is acquired and skills and strategies are developed to address specific tasks. Knowledge includes, for example, speed, analysis, and correct use of scientific information to solve a problem.² Monitoring and control refers to how the individual monitors and regulates his or her level of knowledge and identifies his or her own deficiencies, and is involved in self-regulation of learning, clinical judgement, and academic outcomes.^3–6 Taken together, metacognitive skills influence critical thinking and decision making.^7,8

From an instructional point of view, it is assumed that active teaching methodologies promote metacognitive skills and the performance of medical students.^3,9–12 Of these methodologies, the Flipped Classroom is a resource with increasing and promising use. Through exercises aimed at problem solving, case discussion and cooperative reflection in interactive virtual and in-person learning environments (in the conventional classroom or in its variants extended to the simulation laboratory), this methodology aims to stimulate higher cognitive abilities, different from memorising or remembering information.^13–15

However, although the benefit of the Flipped Classroom is evident in various educational outcomes such as learning climate, engagement, student motivation, and collaborative learning,^16–22 the evidence supporting its relationship with the abilities of medical students to know, monitor and control their ability to "learn to learn" is scarce. Likewise, its effects compared to other traditional teaching methodologies (e.g., lectures) and other workplace-specific (in the clinical context at the bedside and in the community) warrant further study.^19–22 These evaluations are justified because they can provide empirical evidence about the effect of the Flipped Classroom on learning. Furthermore, they contribute to rationalising and enhancing its use and scope, especially when used in conjunction with traditional methodologies and teaching approaches for deliberative practice in the clinical context.^18,23 Considering the first of these limitations, the purpose of the present study is to evaluate the effect of the Extended Flipped Classroom on student perceptions of their metacognitive skills, providing empirical evidence that helps fill this knowledge gap.

Material and methods

Study design and participants

The study was carried out at the Faculty of Medicine of the Universidad de la Sabana (Colombia), which has a traditional curriculum in Medicine. A cross-sectional study was designed before and after the intervention to measure the effects of the Flipped Classroom on student self-perceptions related to knowledge and regulation of learning. One hundred and seventy-five fourth-year students, distributed over the two academic periods of 2017, were invited to evaluate their metacognitive skills in a voluntary, anonymous and confidential manner.

Intervention

During each academic period, students studied the General Surgery course in an 18-session Extended Flipped Classroom that incorporates a virtual environment, a conventional classroom environment for interactive discussion of 4–6 clinical cases per session, and a medium-fidelity clinical simulation environment for learning practical skills.¹⁵ Throughout the academic period, students attend clinical rotations at different institutions affiliated with the programme.

Instruments and information collection

Student self-perceptions of their metacognitive abilities were assessed using the Spanish version of the Metacognitive Awareness Inventory (MAI).^4,24 Although there are multiple instruments for the evaluation of metacognitive skills, the MAI was chosen due to its wide diffusion for the evaluation of learning achievement, as well as its relevance in the field of educational research that allows generalising and comparing the results of various contexts.^4,25 Through 52 items, the questionnaire evaluates two large domains (knowledge and regulation of cognition). The knowledge domain is composed of three metacognitive categories (declarative, procedural and conditional knowledge) and the regulation domain, in turn, of five categories (planning, organisation, monitoring, debugging and evaluation). The definitions of each of these categories are found in Table 1. Each item is rated on a Likert scale (from 1, completely disagree, to 5, completely agree). In its English version, the instrument demonstrates high reliability (Cronbach's alpha = 0.90),²⁴ just like its Spanish version (α = 0.94).⁴

Table 1.

Definition of the domains and categories of the Metacognitive Awareness Inventory (MAI), descriptive statistics, differences and effects between the two measurement times.

Domain	Category	Items	Definitions	Time 1 (n = 158)		Time 2 (n = 155)		p	Size of the effect, Cohen's d test (95% CI)
				M	SD	M	SD
Knowledge of cognition	Declarative knowledge	5, 10, 12, 16, 17, 20	Knowledge that a subject has of his or her learning, his or her skills and the use of his or her cognitive abilities	3.65	0.82	3.83	0.61	0.01	0.24 (0.02−0.47)
	Procedural knowledge	3, 14, 27, 33	Knowledge that a subject has about the use of their learning strategies	3.91	0.63	4.02	0.48	0.04	0.19 (–0.02 to 0.41)
	Conditional knowledge	15, 18, 26, 29, 35	Knowledge that a subject has about when and why to use learning strategies	3.83	0.86	3.98	0.57	0.03	0.20 (–0.01 to 0.42)
Regulation of cognition	Planning	4, 6, 8, 22, 23, 42, 45	Planning by the subject of study times, setting learning goals and selecting resources	3.52	0.85	3.77	0.72	0.02	0.31 (0.09−0.53)
	Organisation	9, 13, 30, 31, 37, 39, 41, 43, 47, 48	Process carried out by the subject that allows him or her to organise activities around learning	3.97	0.78	4.00	0.62	0.3	0.04 (–0.17 to 0.26)
	Monitoring	1, 2, 11, 21, 28, 34, 49	Supervision exercised by the subject of the learning process during the development of tasks	3.78	0.77	3.87	0.65	0.13	0.12 (–0.09 to 0.34)
	Debugging	5, 40, 44, 51, 52	Process carried out by the subject and that allows him or her to identify weaknesses in learning and adjust strategies to improve his or her performance	4.06	0.65	4.04	0.53	0.6	–0.03 (–0.25 to 0.18)
	Evaluation	7, 19, 24, 36, 38, 50	Analysis by the subject of the effectiveness of the implemented strategies	3.50	0.84	3.74	0.70	0.003	0.31 (0.08−0.53)

95% CI: 95% confidence interval; SD: standard deviation; M: median.

Statistically significant difference if p < 0.05. Guide to interpretation of effect size: small effect, d = ±0.20; medium effect, d = ±0.50, and large effect, d = ±0.80.²⁷

The definitions are taken from the work of Huertas Bustos et al.⁴ after validating the instrument in Colombian students.

Information collection was carried out at two times (t1 and t2) digitally. The pre-intervention evaluation (t1) was carried out on the first day of each course and the post-intervention (t2) on the last day. The principal investigators described verbally and in digital format the scope of the study and the subsequent use of the information. As this was anonymous information, informed consent was not obtained from the participants. Procedures were used to guarantee the protection of information included in databases.

Statistical analysis

The measurement of descriptive statistics of all demographic variables allowed us to find out the general characteristics of the study population. Likewise, a preliminary evaluation of the reliability of the MAI was made with the empirical data using Cronbach's alpha coefficient (adequate if α > 0.70).²⁶ Then the mean ± standard deviation (range) of each of the domains was calculated and categories of the MAI at the two measurement times (before and after the intervention). The total score of the questionnaire and that of each of its sub-scales at the two times were compared using the Student t-test (significant if p < 0.05). The size of the effect of the intervention between t1 and t2 was evaluated using Cohen's d test with its 95% confidence interval (95% CI) for each domain and category. The following criteria were used for the interpretation of the d test: small effect, d = ±0.20; medium effect, d = ±0.50, and large effect, d = ±0.80.²⁷

Results

A total of 158 participants self-assessed their metacognitive skills at t1 (response rate, 90.28%) and 155 at t2 (response rate, 86.85%). The average age of the participants was 21.41 ± 1.42 (18–27) years. Sixty-five point six percent were women. No significant differences were identified in the demographic variables of the participants between the two academic periods. The reliability of the MAI resulted in α = 0.92 at t1 and α = 0.97 at t2.

The MAI scores ranged from 3.52 ± 0.085 (planning category) to 4.02 ± 0.51 (debugging category) in the measurement at t1 and from 3.74 ± 0.70 (evaluation category) to 4.04 ± 0.53 (1–4) (debugging category) at t2 (Table 1). Comparisons between measurements before and after the intervention demonstrated statistically significant differences in all categories of the knowledge domain (declarative, procedural and conditional), as well as in two of the five categories of the cognition regulation domain (planning and evaluation) (p < 0.05). The size of the effect of the intervention in these categories between t1 and t2 was moderate for planning (d = 0.31; 95% CI, 0.09−0.53) and evaluation (d = 0.31; 95% CI, 0.08−0.53) and small for knowledge categories (d = ±0.20). The MAI scores, statistical differences, and d test values are shown in Table 1.

Discussion

In this study, the effect of the Extended Flipped Classroom on the metacognitive skills of surgery students was evaluated. The results demonstrated positive effects of the intervention on their ability to know their own learning (declarative knowledge), use learning strategies (procedural knowledge), and understand when and why to use these strategies (conditional knowledge). In addition, positive effects were identified on the skills to plan and evaluate their learning. However, it is worth mentioning that, although the findings were statistically significant, the sizes of the effect were small for most domains and the 95% CIs of the effects were at both cut-off points.

These results have several explanations. Preliminary evidence indicates that active learning strategies, such as the Flipped Classroom, promote critical thinking and deep understanding of concepts¹⁹ while helping students regulate and direct their learning.²⁸ In our experience, students access virtual learning environments to review and evaluate different theoretical content and educational materials independently, in order to prepare the work in conventional in-person classroom sessions. These sessions, very interactive and in small work groups, allow them to discuss clinical cases with the accompaniment of a tutor. We believe that the positive effect found in this study on declarative, procedural and conditional knowledge and planning abilities is due to the fact that this type of methodology can promote autonomy, avoid the paternalism typical of traditional teaching strategies, place the student in real contexts, enable the practical application and transfer of theory and reduce the exclusive dependence on memory for decision making, in addition to encouraging students to create individual study plans and evaluate and reflect on the knowledge acquired through participation in discussions and clinical simulation sessions. Previous studies support these observations.^3–6,29

However, we recognise that active learning strategies, although they promote metacognitive thinking, do not generate metacognition by themselves.³⁰ A student-centred instructional strategy engages the student and holds him or her accountable for what he or she learns. However, assuming an active role in the learning process is not a guarantee of meaningful and deep learning, since this is only achieved with active cognition tasks promoted by a tutor through the support and scaffolding necessary to stimulate processing, organisation and consolidation of knowledge.³¹ In previous studies it has been observed that the most important phase of a Flipped Classroom session is the in-person discussion and co-operative interaction.³² These observations reflect that, although the Flipped Classroom promotes behaviours related to self-regulation, recognition of initial knowledge, planning and monitoring of learning, the student needs explicit support in these tasks.³³ The facilitating activity must include, therefore, concrete actions that stimulate the student not only to remember information, but also to reflect through permanent and timely feedback. It is essential that the tutor explains to students the importance and value of the learning process, and not only the importance of remembering content, and helps them to actively reflect.²⁹

In this context, although the intervention had a positive effect in all domains of knowledge, the size of the effect was small. Although the Flipped Classroom facilitated the development of these skills, it is still necessary to include activities that encourage the student to reflect more on the knowledge they have about what they know, the learning strategies they use, and when to use these strategies. We emphasise that our intervention is immersed in a traditional curriculum focused on master classes throughout the degree in medicine. The isolated inclusion of the Flipped Classroom in surgery within this type of curriculum may explain the magnitude of the effect found.

This study has strengths and weaknesses. We believe that it involves a standardised intervention (both in its virtual and in-person components), the use of valid measurement instruments and a representative sample of participants. However, the short duration of the intervention (20 weeks) is recognised. Metacognitive skills develop gradually and cumulatively over time, in relation to the student's exposure to this type of learning environment. In particular, the metacognitive effect of the Flipped Classroom is not long-lasting, and although students have the ability to develop incremental metacognitive skills, they must be constantly exposed to these strategies for them to truly be integrated.¹⁹ As already noted, the participants in this study were exposed to traditional teaching strategies during the first three years of their studies. In this regard, this situation opens the door to conducting new research that compares the effect of different teaching strategies on the metacognitive processes of knowledge and regulation, as well as the combined effects of these interventions.

The relationship between the effect of various teaching strategies on student metacognitive skills is a fundamental aspect in the evaluation of teaching quality and offers implications for practice. Different authors maintain the importance of teaching metacognition to students, of explaining to them how they should orient themselves towards raising questions and of reflecting during the planning, monitoring and evaluation of the different moments of learning a complete course or a subject.^3,34,35 The practical implication of this study also lies in the planning of the instructional design in the Flipped Classroom, focused especially on cooperative learning in in-person sessions. We believe that these observations highlight the importance of not concentrating efforts exclusively on the use of technological platforms and virtual environments, but rather on enriching the interaction between students and teachers in low and medium fidelity simulation situations (in the case of the Extended Flipped Classroom).

In conclusion, the Flipped Classroom has positive effects on the declarative, procedural and conditional knowledge perceived by surgery students, as well as on the skills to plan and evaluate their learning. These results have practical implications and offer opportunities for further studies.

Funding

Faculty of Medicine, Universidad de la Sabana, Chía, Colombia.

Conflicts of interest

None.