The impact of the learning environment sudden shifts on students’ performance in the context of the COVID-19 pandemic

Abstract

Introduction

This study aims to determine the effect of sudden changes in learning environments on students’ performance, in the context of the COVID-19 pandemic lockdown. We present an analysis of the kinesiology program, focusing on the learning modality changes through the years, and its impact on students’ performance.

Methods

We analyzed three periods over five years. During the Pre-pandemic period (2018-2019), classes had been taught in-person, during the pandemic (2020-2021) classes had been taught online, and during end of lockdown (2022) classes had return to in-person modality. In addition, we also examined the academic performance outcomes by gender during the three periods.

Results

We found that the academic performance significantly increased in all cohort of career, increasing the average grade from 4.7 ± 0.08 (2018 to 2019, in-person) to 5.15 ± 0.07 during the pandemic period, from 2020 to 2021, when online modality was utilized. Furthermore, when returning to in-person classes in 2022, the academic performance reduced significantly to 4.6 ± 0.17. We also found that gender did not have an influence on academic performance in any of the learning environments presented. However, during clinical internships, we found that gender had a significantly effect on academic performance.

Conclusion

Based on these results, we conclude that the sudden shift from in-person learning to online learning modality helped improved the learning performance of student, reflecting those results on better students’ performance scores that could be associated with the enhanced efficient use of time.

Resumen

Introducción

Este estudio tiene como objetivo determinar el efecto de los cambios repentinos en los entornos de aprendizaje sobre el rendimiento de los estudiantes, en el contexto del confinamiento por la pandemia del COVID-19. Presentamos un análisis del programa de la carrera de kinesiología, enfocándonos en los cambios de modalidad de aprendizaje a través de los años, y su impacto en el rendimiento de los estudiantes.

Métodos

Se analizaron tres períodos a lo largo de cinco años. Durante el periodo Pre-pandemia (2018-2019) las clases se habían impartido de forma presencial, durante la pandemia (2020-2021) las clases se habían impartido online y durante el fin del confinamiento (2022) las clases habían vuelto a la modalidad presencial. Además, también examinamos los resultados del rendimiento académico por género durante los tres períodos.

Resultados

Se encontró que el rendimiento académico aumentó significativamente en toda la cohorte de la carrera, aumentando la calificación promedio de 4.74 ± 0.08 (2018 a 2019, presencial) a 5.15 ± 0.07 durante el período de pandemia, de 2020 a 2021, cuando la modalidad en línea fue utilizada. Además, al regresar a las clases presenciales en 2022, el rendimiento académico se redujo significativamente a 4,6 ± 0,17. También encontramos que el género no influyó en el rendimiento académico en ninguno de los entornos de aprendizaje presentados. Sin embargo, durante las pasantías clínicas, encontramos que el género tuvo un efecto significativo en el rendimiento académico.

Conclusión

Con base en estos resultados, concluimos que el cambio repentino de la modalidad de aprendizaje en persona a la modalidad de aprendizaje en línea ayudó a mejorar el rendimiento de aprendizaje de los estudiantes, lo que refleja esos resultados en mejores puntajes de rendimiento de los estudiantes que podrían estar asociados con un mayor uso eficiente del tiempo.

Introduction

Traditional teaching corresponds to the most used teaching-learning methodology in health sciences, which focuses principally on expository teaching, and autonomous learning. In addition to theoretical learning, practical training also plays an important role in this teaching.¹ However, academic instructors are currently encouraged to implement activities that allow the participatory construction of a student-centered learning mechanism,² which is an extremely time-consuming process. Nevertheless, there are adverse condition that can force the change from in-person to online learning, such as the COVID-19 pandemic.^3,4 In an effort to curb its spread, on 11 March 2020 the World Health Organization (WHO) declared COVID-19 a pandemic. The first case of SARC-CoV-2 infection in Chile was confirmed on March 3, 2020. On March 16, 2020, when Chile had 156 confirmed cases of SARC-CoV-2 infection,⁵ the Chilean government decided to close all educational institutions.

The Chilean government decided that all levels of education, from elementary school to university, should implement an online instruction-modality.⁶ The initial decision of the Chilean government was to suspend in-person learning in Chilean educational institutions for 2 weeks, which was later extended until further notice. University education abruptly changed to on-line learning, disrupting traditional instruction, especially the health sciences education. In university programs in this area, only theoretical lessons can be performed via on-line learning. Practical lessons, such as teaching laboratories or student training at hospitals and clinics, were suspended during the first semester of the year 2020 (March to July), while only a few students could perform their clinical practice in tele-rehabilitation mode, a mode never used before. Practical activities in clinics, laboratories and hospitals in Chile were able to resume their operations in August 2020 and have not been interrupted since then.

Although in-person and online learning share aspect in communication, collaboration and interaction. There are several differences among these forms of teaching. First, traditional learning is based on teacher-centered learning that emphasizes the transmission of knowledge associated with listening.⁷ Second, the resources of traditional learning correspond to books, blackboards and reading form, which are considered methods of a passive nature that promote poor student attention.^8,9

On the other hand, online learning advocated for student centeredness, looking for the students be active participants in the learning process. The digital resources of online learning correspond to interactive component, internet, computers, mobile phones, etc that allows students to receive information, search for information and share information amongst themselves. In other words, modern resources enhance the active participation of students,¹⁰ suggesting that the development use of educational technology (videos, smart phones, learning management systems and social media sites) raises quality learning on online environment as compared to in-person environment.¹¹

In Chile, specifically in higher education, the digital gap is mainly caused by poor internet access due to geographical distance and less is caused by lack of electronic equipment. In addition, the gap between those who had and those who do not had access to computers was quickly resolved. Our university provided students with free laptops and Wi-Fi access to use at home so as not to limit the viability of online learning. After 2 years of an online learning environment (year 2020 and 2021), on 02 March 2022 the government confirms the return to mandatory in-person environment for students.⁶ Therefore, the main question we want to answer is whether the forced change in the learning environment from in-person to online and the subsequent return to in-person affected the academic performance of kinesiology students.

Methodology

Participants

A longitudinal non-experimental design was used to evaluate the performance of student of kinesiology career from University of Viña del Mar (UVM), Chile to compare the academic performance of the entire study program between the years 2018–2019 (pre-pandemic), dictated in-person environment; between the years 2020–2021, dictated in online environment and its return to in-person mode during the year 2022. The final grades for each course served as the comparative factor in assessing performance differences between online and in-person environment. The study was carried out in 2 stages: in the first stage, the performance of in-person classes (pre-pandemic period) was compared to performance of online classes (during the pandemic). In the second stage, the performance during the return to in-person classes (end of lockdown) was included. The bachelor's degree in kinesiology takes five years to finish, which consists of two (2) semesters each. Each semester consists of five or six courses. To graduate, the student must approve 46 courses (300 credits) in total. The academic performance was analyzed by learning modality, compared by year. 100% of the courses of the study program were included, but electives and courses taken for a second time were excluded.

Teaching methods and type of examinations

Due of the high stakes related to teacher quality to teach in an unknown modality. All teachers were instructed to have the skills necessary to construct and manage the online classroom efficiently and to can use technology in meaningful ways. The online classes were carried out through the blackboard platform synchronously, in order to generate feedback with the students. In addition, the classes were saved for later use. The evaluations were carried out on the same platform. The evaluations carried out during the pandemic consisted of multiple-choice tests, using question banks. The tests followed the following distribution of questions: 50% low complexity, 30% medium difficulty, 20% greater difficulty. The complexity of the questions was designed according to Bloom's taxonomy. Low level complexity questions corresponded to information recognition or recall questions. The questions of medium complexity required mental processes linked to the association/comparison of key concepts, and the questions of high complexity, corresponded to questions linked to processes of interpretation and application. In addition, development tests, clinical cases, use of forums and creation of video capsules, etc. were used.

Data collection procedures

This study was approved by UVM ethics committee and by director of the school in health sciences prior to accessed to the database. The database contains information about: (1) academic performance by gender and (2) progress of career plan of study by semester. It should be noted that during the last year of academic training, a professional internship is performed in different healthcare centers, so students during clinical internship have not been affected by the pandemic or are affected by changes in the learning environment. Therefore, we used this data as internal control.

The grading scale in Chile ranges from 1.0 to 7.0, where a grade equal to or greater than 4.0 is required to pass and a grade lower than 4.0 to fail.¹² Generally, in higher education, about 80% of the passing grades are in the 4.5 - 5.4 range. A grade between 4.0 to 4.9 points in considered “sufficient”, while a grade exceeding 5.0 points is generally considered a "good" grade, and a grade between 5.5 to 5.9 point is considered a very good grade. Exceeding 6.0 is considered "high academic excellence".

Statistical analyses

All statistical analyses were performed using GraphPad Prism 8.0.1 program (GraphPad Software., San Diego, USA). All variables (number of students by cohort; academic performance by gender or environments learning) were analyzed for normal distribution using Shapiro-Wilk test, whereas the homoscedasticity was analyzed using the Levene test. The characteristics of kinesiology students such as numbers of students, gender and age were presented using means, standard deviations, frequency, and percentages. The changes in performance over the 5-year duration of the kinesiology program was analyzed using a two-way repeated measures ANOVA [environments (in-person, online) × years (1, 2, 3, 4, 5)] followed by Bonferroni post-hoc comparison tests. For the comparison between the academic performance score during the in-person and online environments, was analyzed using Mann-Whitney U test. The changes in performance by gender during the 5-year duration of the kinesiology program was analyzed using repeated measures ANOVA followed by Bonferroni post-hoc comparison tests. The differences between average performance during learning strategy by gender was analyzed using 2 × 2 factorial ANOVA followed by Bonferroni post-hoc comparison tests. The comparison between academic performance by gender during clinical internship was analyzed using t-test. The comparison between academic performance by gender during different learning environments was analyzed using 2 × 3 factorial ANOVA [Factor 1 (gender: male, female) × Factor 2 (learning environment)] followed by Bonferroni post-hoc. All data were presented as mean ± standard deviation (SD), and the difference was considered significant when p < 0.05.

Results

Table 1 shows the number of students per year, by cohort (5 cohorts per academic year), showing a gradual increase in the number of students from 2018 to 2022, linked to the male gender. On average, male represent 56% of the student population while female represent 44%. The age ranges between 18 to 41 years throughout the bachelor’s degree in kinesiology, with a median between 22 to 24 years. In addition, the average academic performance shows that during online modality, the academic performance is significantly higher than during the in-person modality (2018 and 2019) and during the return to in-person modality (2022).

Table 1.

Demographics table.

	2018	2019	2020	2021	2022
Number of students	205	224	234	248	244
Male	107 (52,2%)	132 (58,9%)	131 (56%)	141 (57.1%)	144 (59%)
Female	98 (47,8%)	92 (41,1%)	103 (44%)	107 (42.9%)	100 (41%)
Age
Mean ± SD	23,02 ± 2.4	22.21 ± 2.9	23.15 ± 2.8	21.83 ± 2.9	22.3 ± 3.1
Range	18-29	18-32	18-41	18-31	18-36
Median	23	22	24	22	22
Mean academic performance
Male	4.7 ± 0.7	4.6 ± 0.7	5.2 ± 0.6*	5.0 ± 0.6*	4.4 ± 0.7
Female	4.9 ± 0.8	4.7 ± 0.8	5.1 ± 0.7*	5.2 ± 0.6*	4.7 ± 0.7

Each data point represents the average by cohort.

Student performance by type of learning environments

The two-way mixed model ANOVA showed that the academic performance score significantly increased (F(1,19) = 42.88, p < 0.0001) during the pandemic period (2020 and 2021), with online classes (Fig. 1A and B). There was a main effect of years (each year of career program) and interaction between years and learning environments, whereas the post-hoc test showed that during the first, second and third year, the academic performance under online learning environment significantly increases the academic performance compared to the in-person learning environment (F(4,76) = 6.85, p < 0.0001) (Fig. 1A). In addition, during in-person learning (pre-pandemic period), a significant increase in performance is observed as the study program of the career progresses, finding significantly differences between the performance of the first years compared to the performance of the fourth and fifth year (p < 0.001). Conversely, during the online environment (during the pandemic), an increase in academic performance was not observed throughout the development of the career program (Fig. 1A). Fig. 1B shows the academic performance average during all years of the career program (same data as in Fig. 1A). The online environment (during the pandemic) significantly increased the average academic performance compared to in-person environment (Mann-Whitney U test, p = 0.0002) (Fig. 1B).

Fig. 1.

Effects of learning environment on academic performance. A. Online learning scenario significantly increased the academic performance compared to in-person scenarios during first, second and third year. B. The average academic performance was significantly higher in online learning compared to in-person learning (In-person: 4,7 ± 0,08; Online: 5,1 ± 0,07). Data represent means ± SD. Asterisk (*) indicates significant differences.

Academic performance by gender and by learning environment type

The two-way mixed model ANOVA showed that the academic performance score significantly increased during the pandemic period, with online classes in both male and female compared to in-person environment respectively (Fig. 2A and B). Fig. 2A shows that during the first, second and third year of the development of the career program in male, a significant increase in academic performance is observed under the online environment compared to the in-person modality (F(4,59) = 44.05, p < 0.0001) (Fig. 2A). In the case of the female gender, there was a main effect of learning environmental on academic performance where, online classes increased significantly the academic performance compared to in-person environment (F(1,19) = 32.43, p < 0.0001) (Fig. 2B). There was a significant effect of years (each year of career program) on academic performance (F(4,46) = 6.13, p < 0.001), showing that during the sophomore and junior years of the development of the career program, a significant increase in academic performance is observed under the online environment compared to the in-person modality (Fig. 2B). We found that during the online environment increased the academic performance (Fig. 1B). Therefore, we want to elucidate if during the learning environment there are influences of gender, therefore we compared the academic performance by gender under both learning environment. The 2 × 2 factorial ANOVA [Factor 1 (gender: male, female) × Factor 2 (learning environment)] followed by Bonferroni post-hoc showed a main effect of learning environment on academic performance (F(1,332) = 30.17, p < 0.0001) (Fig. 2C). Conversely, gender did not affect the academic performance in any learning environment (F(1,332) = 0.50, p > 0.05) (Fig. 2C).

Fig. 2.

Academic performance by gender and learning environment. A. Male academic performance increased significantly during online instruction period compared to in-person environment (p < 0.0001). B. Female academic performance increased significantly during online instruction period compared to in-person environment. C. During the online instruction period, the academic performance is significantly higher compared to in-person period. However, there are no significant differences in academic performance by gender in both learning environments (p > 0.05). Data are represented as means ± SD. Asterisk (*) indicates significant differences while ns: non-significant differences.

We compared the academic performance during last year of academic training (professional internship) and we found that female student obtains a significant increase on academic performance compared to male student (Male: 5.4 ± 0.05; Female: 5.7 ± 0.04, t-test, p < 0.05) (Fig. 3A). In the other hand, we analyzed using 2 × 3 factorial ANOVA followed by Bonferroni post-hoc the academic performance during clinical internship. There was a main effect of both gender and learning environment. During online environments, male performance is significantly reduced compared to female performance (F(1,30) = 10.83, p < 0.001) (Fig. 3B). The return to in-person environment reduced significant the academic performance of compared to online modality (F(2,30) = 5.82, p < 0.001) (Fig. 3B).

Fig. 3.

Academic performance by gender, during clinical internship. A. Academic performance in students during clinical practices from 2018 to 2022. Female academic performance is significantly higher compared to male academic performance (Male: 5.48 ± 0.05, Female: 5,7 ± 0.04, p < 0.001). B. Academic performance by gender in different learning environments during clinical internship. Female performance is higher compared to male in the online environments. (Pre-pandemic in-person: Male: 5.46 ± 0.06, Female: 5,62 ± 0.06, p > 0.05; Online: Male: 5.50 ± 0.08, Female: 5,81 ± 0.03, p < 0.05; Return to in-person classes: Male 5,2 ± 0.03, Female: 5,45 ± 0.15 p > 0.05). The academic performance decreased significantly in the return to in-person environment compared to online environment. Data represent means ± SD. Note: Asterisk (*) indicates significant differences while ns: non-significant differences.

Effect of return to in-person environment learning on academic performance

The two-way repeated-measures ANOVA showed that learning environments significantly increased the academic performance compared to both in-person and return to in-person environments (F(2, 46) = 5.135), p < 0.001) (Fig. 4A). In addition, among the in-person environments (pre-pandemic period and end of lockdown), we found significant differences in the fourth year of the career. One way ANOVA showed a main effect of learning environments on academic performance (F(2, 185) = 5.135), p < 0.0001) (Fig. 4B), where the post-hoc test showed that online environment increased significantly the academic performance compared to both in-person and return to in-person environments (In-person: 4.68 ± 0.7; Online: 5.1 ± 0.6; Return to in-person: 4.66 ± 0.7) (Fig. 4B). In the other hand, we analyzed using 2 × 3 factorial ANOVA [Factor 1 (gender: male, female) × Factor 2 (learning environment)] the academic performance during different learning environments and we found that online environments increased significantly the academic performance compared to both in-person and return to in-person environments (F(2, 370) = 17.69), p < 0.0001) (Fig. 4C). Conversely, gender did not affect the academic performance in any learning environment (Fig. 4C).

Fig. 4.

Effects of return to in-person environment and academic performance. A. The academic performance significantly increased in an online environment compared to an in-person environment. However, the academic performance decreased when returning to an in-person environment. B. The average academic performance increased significantly in an online environment compared to in-person environment. In addition, the average academic performance decrease significantly due to return to in-person environment. C. Academic performance during transient changes on learning environments. We did not fund significantly differences on academic performance by gender on both learning environments. However, we found that the academic performance increased significantly in an online environment compared to both, pre- and post-pandemic in-person environments (p < 0.0001). The academic performance decreased significantly in the return to in-person period. Data represent means ± SD. Note: Asterisk (*) indicates significant differences while ns: non-significant differences.

Discussion

The aim of this study was to determine the effects of shifts in learning environment on academic performance, in the context of the COVID-19 confinement. We present the results of a series of analyses of all cohorts of the Kinesiology career program, comparing academic performance by learning modality, from years 2018 to 2022. During the period 2018 to 2019 learning was made on in-person mode, whereas between 2020-2021, the education has been forced to teach online classes, and during the year 2022 it has returned to in-person classes. First, we investigated whether student academic performance is related to learning environments. We found that the average academic performance enhances significantly in the online environments (average academic performance in-person: 4.72 ± 0.08 vs. online: 5.14 ± 0.07). Thus, the results show that there is a significant positive effect of online modality on students´ performance during the COVID-19 pandemic confinement. This finding correlates with (13) that online learning increased the students’ performance. In support of this idea, several studies shown that online modality (synchronous and asynchronous) provides more advantages for learning.^13–15 Indeed, asynchronous courses present an advantage to no-traditional students, such as parents and working professionals, who need a flexible schedule to pursue their education.¹⁴

3 Reasons why students prefer online classes

Flexibility: According to the characterization survey, twenty-five to thirty-five percent (25-35%) of the kinesiology students at our university, at the time of entering the degree, performs some type of paid work (data not shown). Therefore, it is not surprising that the students’ preference is online classes when offered as an alternative.^16,17

Support: On the one hand, while the students in-person mode must listen to the teacher and quickly take notes for later study, the students that missed classes in-person depend on the notes of their peers to prepare for evaluations. On the other hand, online modality, the classes are given in synchronously, which are recorded for asynchronous review, as necessary.¹⁸ Therefore, it is not surprising that students prefer recorded classes to improve the effectiveness of learning and that flexibility and convenience of online classes makes it attractive option.¹⁸ However, the disadvantage of asynchronous classes is that peer-focused activities such as feedback are lost compared to students in synchronous settings.¹⁹

Time management: Online classes allows saved on travel time, reducing distraction like compulsive participation in conversions, and waste of time in finding a proper place to study alone and that the improvement in performance could be due to enhanced utility of time.²⁰ This in turn could be to COVID-19 confinement changed students’ learning strategies to a more continuous habit, improving their learning performance as described by Gonzalez and colleagues.¹³

Other studies have not found significant differences in performance between online and traditional classroom students with respect to modality²¹ or gender.²² On the other hand, other studies have shown that students still prefer classroom classes over online classes due to many problems they face when taking online classes, such as lack of motivation, understanding of the material, decrease in communication levels between the students and their instructors and their feeling of isolation caused by online classes.²³

We found that students´ performance increased during the COVID-19 confinement. However, the reasons for such an improvement are unknown. The change of modality from in-person to online, requires changes on: (1) the new learning methodology and (2) The new assessment process.¹³ In addition to better performance in all the years of the career program, the students also achieved significant improvements in their performance even in fifth-year courses, which have not been affected by the confinement (clinical internship). These findings reveal that the new assessment process cannot be the reason for the improvement in students' performance because the learners also achieved better performance when the format of the assessment did not change (Fig. 3A). For these reasons, we suggest that the new learning methodology is the main reason for the change in students' performance during the confinement. In the one hand, before the emergence of COVID-19, the instructors routine included the use of the blackboard classroom platform, a virtual classes platform, which use was extended throughout the confinement period. This instructors’ competence was helpful for launching online classes and integrate advancing their technological skills in their teaching sessions, which benefit students.²⁴ In the other hand, studies by Almahasees and collaborators,¹⁴ establish that faculty preferred traditional teaching, in-person, rather than online classes, because in-person instruction allows to applied active learning strategy with immediate feedback to discuss and have lively guidance for your students.²⁵ Furthermore, most of the faculty members indicated that online classes’ preparation entails more effort to ensure students’ interaction online.

Gender is found to be one of the important factors that has a correlation with the academic performance. However, the results are mixed. While no differences exist in some studies, others showed significant differences.²⁶ This study found no significant difference in the academic performance in terms of gender in any learning environment (Fig. 4C). This finding correlates with^22,27 that there was no significant difference between the academic performance of male and female students. However, we found significant differences in academic performance in terms of gender during clinical internship and during online modality (Fig. 3). The students of kinesiology career will be healthcare professionals involved in the recovery of functions of patients with disabilities as a consequence of chronic diseases or accidents.²⁸ During the clinical internship, the students had recurrent interactions with patients and their family,²⁹ therefore having soft skills is crucial, including active listening, conflict resolution and mediation, time management, ability to complete activities multitask, ability to achieve goals and follow guidelines, make optimal decisions, take initiative, motivate others, problem solving, being able to advise, explain and convince, public speaking, translating and giving instructions.³⁰ Our premise is that the significant difference in terms of gender on academic performance during the clinical internship could be due to a different set of soft skills by gender, but further research is needed to understand this correlation.

The limitations of this research included to one of the most challenging aspects of online education is the students' assessment due to various cheating behaviors.³¹ The students could be the option to copy each other, to use digital resources such as the Internet, mobile phones that allow them to receive information and share it among themselves. Therefore, the increase on performance may be influenced by cheating behaviors. However, in an effort to prevent interaction between students during online examinations, in the multiple-choice tests, question banks were generated to avoid the similarity of tests between students and were presented randomly. Unfortunately, on other assessments, such as developmental tests, clinical cases could exist an increase in performance per copy among students.

We conclude that there is a real and measurable improvement in the students’ learning performance during COVID-19 confinement period that demonstrated the correlation between online learning and higher academic performance. We suggest that online learning allows a better use of time and an increase in peer-focused activities. However, the sudden reinstation of in-person learning could have harm in the short term the advantages obtained from online learning period. Future work will include the study of a hybrid methodology in the context of Kinesiology education.

Funding

The author received no financial support for the research. However, the publication of this article was supported by funding from Escuela de Ciencias de la Salud, Universidad Viña del Mar, Chile.

Author contributions

MP, GUC contributed to the conception and design of study and contributed the acquisition and analysis of the data. MP wrote the manuscript. PT contributed to proofread the final version of the manuscript. All authors contributed to the article and approved the submitted version.

Data availability statement

The datasets generated for this study are available on request to the corresponding author.

Ethics statement

The studies involving human participants were reviewed and approved by ethics committee of the Universidad Viña del Mar, Chile. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Acknowledgements

We would like to thank the team for their collaboration in designing the study and for discussing obtained results