The SARS-CoV-2 pandemic has affected training opportunities for healthcare professionals partly because face to face courses were cancelled. This study analyzes the results of participation and satisfaction of the AEC Virtual Classroom sessions during the first year.
MethodsThe AEC Virtual Classroom includes a combined format of weekly Webinar broadcast live that can be viewed on a delayed basis in a virtual platform. In this study, the results in its first year are evaluated considering the number of live participants, the delayed views and the global reach; as well as the results of the satisfaction survey in each of the sessions (0–10).
ResultsFrom 16/04/2020 to 15/04/2021, 50 sessions of the Virtual Classroom AEC were held. The average scope of the sessions was 509 ± 288 views with a range between 196 and 149. At the times of highest incidence of cases during the pandemic, a decrease in live participants was observed 275 ± 135 vs. 391 ± 233 (P = 0.032). The mean score on the format was 8.46 ± 0.31/10. The best-scored sessions were those of the subject related to coloproctology with a statistically significant difference in the mean score 8.79 ± 0.42 vs. 8.39 ± 0.27 (P = 0.035). 90% of users considered the sessions useful. 97.2% of respondents believe that the sessions should be maintained after the pandemic.
ConclusionsThe AEC Virtual Classroom has had very good results in the first year, proving to be a useful surgical teaching tool that will foreseeably survive once the pandemic is over.
La pandemia de COVID-19 ha afectado a las oportunidades de formación de los profesionales sanitarios, en parte porque se anularon muchos cursos presenciales. En este estudio se analizan los resultados de participación y satisfacción de las sesiones del Aula Virtual AEC durante su primer año.
MétodosEl Aula Virtual AEC incluye un formato combinado de seminarios semanales emitidos en directo que pueden ser visionados en diferido. En este estudio se evalúan los resultados en sus primeros 12 meses, considerando el número de participantes en directo, el número de visualizaciones en diferido y el alcance global, así como los resultados de la encuesta de satisfacción realizada en cada una de las sesiones (1-10).
ResultadosDesde el 16 de abril de 2020 hasta el 15 de abril de 2021 se realizaron 50 sesiones del Aula Virtual AEC. El alcance medio de las sesiones ha sido de 509 ± 288 visualizaciones con un rango entre 196 y 1490. En los picos de la pandemia se observó un descenso de los participantes en directo: 275 ± 135 vs. 391 ± 233 (p = 0,032) La puntuación media sobre el formato fue 8,46 ± 0,31/10. Las sesiones mejor puntuadas fueron las de temática relacionada con coloproctología con una diferencia estadísticamente significativa en la puntuación media 8,79 ± 0,42 vs. 8,39 ± 0,27 (p = 0,035). Un 90,76% de usuarios consideraron las sesiones útiles. Un 97,2% consideraban que deben mantenerse tras la pandemia.
ConclusionesEl Aula Virtual AEC ha tenido muy buenos resultados en los primeros 12 meses de desarrollo, resultando ser una herramienta útil de docencia quirúrgica que previsiblemente sobrevivirá a la época de pandemia.
The COVID-19 pandemic has had a significant psychosocial impact and has greatly affected training opportunities for all healthcare professionals1–3. Learning is essential and provides doctors with the knowledge and skills to care for their patients in conditions of maximum safety and efficiency. It is well known that surgical training requires a theoretical component, which can be acquired through personal study using books, scientific literature published in journals, or audiovisual material, combined with attendance at courses and conferences, and a practical part that is acquired in courses in the experimental operating room or with simulators and clinical surgical practice with initial mentoring. In 2020, the Spanish Association of Surgeons (Asociación Española de Cirujanos, AEC) had to cancel more than 150 in-person courses that were aimed at residents and junior surgeons, covering a variety of topics (emergency surgery, care for polytrauma patients, abdominal wall surgery, bariatric surgery, breast pathology, endoscopy, esophagogastric surgery, proctology, hepatobiliary-pancreatic surgery and transplantations, mesenchymal tumors, sarcomas and endocrine surgery), representing a loss of training opportunities for surgical residents. The impact of the COVID 19 pandemic on the training of specialists in surgery has been variable and depends in part on the year of residency or surgical practice, but there is no doubt that surgical training opportunities for surgical specialists have been significantly limited during the pandemic4–7. This situation led to the search for viable training alternatives to reduce this negative impact8,9. The AEC Virtual Classroom emerged in April 2020 as a teaching alternative in times of health crisis in an attempt to partially substitute the canceled in-person courses. Online teaching or e-learning is not new; the digital transformation of medical education has been progressive, and the number of remote courses and master programs available was already significant before the SARS-Cov-2 pandemic. However, learning practical surgical skills in a virtual format is challenging10–14. This study analyzes the participation and satisfaction results of the AEC Virtual Classroom sessions during its first year, as well as the lessons learned by the organizers in order to improve virtual surgical teaching projects in the future.
MethodsThe AEC Virtual Classroom is a teaching project developed during the health crisis caused by the SARS-Cov-2 pandemic. It includes a combined format of weekly one-hour seminars broadcast live, with the option to ask/answer questions or hold a discussion at the end of the session. These seminars are recorded, edited, and subsequently hosted on the AEC website for later viewing. These sessions are also linked to a series of new educational videos that have been incorporated in the AEC Video Atlas related with the topic of the seminar, which complement the training objectives.
This study evaluates the results of the sessions held in the AEC Virtual Classroom in its first 12 months, analyzing the number of live participants, number of deferred views and global outreach, as well as the results of the satisfaction survey carried out during each of the sessions. Given that specific sessions were created for residents and others were aimed at all the members of the AEC, the results of the sessions have been compared according to their target audience. The results are also analyzed based on the format of the sessions, whether it be a lecture or seminar type, in addition to assessing the impact based on the selected topic.
Taking into consideration the evolution of the pandemic, we also compared the scope of the sessions during the peaks of the highest incidence of COVID-19 cases and the highest hospital patient load versus the weeks with the lowest incidence of cases. We also analyzed the 2 stages of the AEC Virtual Classroom, each being 6 consecutive months.
After the seminars, participants were asked to complete a satisfaction survey of 4 standardized questions. Each of the questions was scored on 5 points and then transformed into a score from 1 to 10 (increasingly positive) for the purposes of statistical analysis. In the first stage, a fifth question was added about continuing the sessions after the pandemic.
The lessons learned by the organizers of the AEC Virtual Classroom are subjectively described herein with the purpose of improving virtual teaching sessions and establishing recommendations for the future.
Statistical analysisFor the statistical analysis, we used the SPSS 21.0 statistical package. Mean and standard deviation were used to describe the quantitative variables and the percentages in the quantitative variables. Parametric and non-parametric tests were performed to compare proportions and means. P < 0.05 was considered statistically significant.
ResultsFrom April 16, 2020 to April 15, 2021, 50 sessions of the AEC Virtual Classroom were held: 46 in a standard format of 3–4 lectures with a subsequent discussion, and 4 in seminar format. Out of this total, 39 were developed primarily for MIR residents and 11 for all AEC members, with an external sponsor. The sessions addressed a wide variety of topics selected by the different divisions of the AEC, including in the second semester a series of 8 training sessions developed by the breast pathology division.
In terms of the scope of the sessions, a mean of 342 ± 204 live participants were registered per session (range 131–1102 viewers connected). When we added the live connections and the deferred views of the recorded sessions, the average outreach of the AEC Virtual Classroom sessions was 509 ± 288 views (range: 196–1490).
At the peaks of the pandemic, we observed fewer live participants (275 ± 135 vs. 391 ± 233; P = 0.032), as well as fewer deferred views. Thus, the global outreach of the sessions held at the peaks of greatest activity during the COVID-19 pandemic was 392 ± 229 vs. 594 ± 300 (P = 0.013) in the period with the lowest incidence of cases during the pandemic.
Furthermore, when we analyzed attendance of the first 6 months versus the last 6 months (regardless of the peaks of the pandemic), we observed that more participants connected live in the second stage of the AEC Virtual Classroom than in the first stage (407 ± 196 vs. 282 ± 197 (P = 0.03).
The sessions with the greatest scope (direct + deferred) were about coloproctology, with 864 ± 351 vs. 442 ± 221 connected viewers (P = 0.002).
A 4-question satisfaction survey was planned for all the sessions, and it was completed in its entirety in 45 sessions. In 2 sessions, the survey could not be carried out due to technical problems, and the survey was not complete in another 3. Results are shown in Table 1.
Results of the satisfaction survey of the AEC Virtual Classroom.
| Did you like the format of the session? (1–10) | Did you find the topic interesting and appropriate? (1–10) | Did the lecturer use virtual media adequately? (1−10) | Was the session useful? (0%–100%) | |
|---|---|---|---|---|
| Total | 8.46 ± 0.31 | 8.42 ± 0.28 | 8.41 ± 0.5 | 90.76 ± 5.9% |
| Sessions created for residents vs. all the members of the AEC | 8.45 ± 0.3 vs. 8.49 ± 0.34 (P = 0.72) | 8.43 ± 023 vs. 8.41 ± 0.39 (P = 0.87) | 8.44 ± 0.49 vs. 8.31 ± 0.47 (P = 0.45) | 90.97 ± 4.7 vs. 90.09 ± 9.1% (P = 0.67) |
| Sessions about coloproctology | 8.79 ± 0.42 vs. 8.39 ± 0.27 (P = 0.035)* | 8.58 ± 0.22 vs. 8.39 ± 0.27 (P = 0.07) | 8.6 ± 0.35 vs. 8.37 ± 0.51(P = 0.22) | 92.25 ± 8.2 vs. 90.4 ± 5.4 (P = 0.44) |
| First stage (6 months) vs. second stage (6 months) | 8.54 ± 0.36 vs. 8.36 ± 0.19 (P = 0.041)* | 8.4 ± 0.32 vs. 8.44 ± 0.23 (P = 0.57) | 8.49 ± 0.40 vs. 8.33 ± 0.55 (P = 0.25) | 92.64 ± 4.7 vs. 88.9 ± 6.4 (P = 0.037)* |
| Pandemic peaks vs. no peaks | 8.33 ± 0.18 vs. 8.54 ± 0.34 (P = 0.008)* | 8.32 ± 0.29 vs. 8.50 ± 0.23 (P = 0.029)* | 8.4 ± 0.40 vs. 8.42 ± 0.55 (P = 0.90) | 9016 ± 6.2 vs. 91.19 ± 5.8 (9 = 0.572 |
Results of question 1: “Did you like the format of the session?” The overall average score obtained was 8.46 ± 0.31 considering all the sessions. No significant differences were found when we compared the sessions addressed to residents or to all the members of the AEC, or when we compared sessions in seminar or lecture format. The sessions from the first 6 months scored better than those of the second 6 months (8.54 ± 0.36 vs. 8.36 ± 0.19; P < 0.041). The sessions with the highest scores were about coloproctology, with a statistically significant difference versus the other sessions in the mean score (8.79 ± 0.42 vs. 8.39 ± 0.27; P = 0.035). The sessions held at peak times of the highest incidence of COVID-19+ cases of the pandemic had worse scores (8.33 ± 0.18 vs. 8.54 ± 0.34; P = 0.008)
Results of question 2: “Do you think that the topics were interesting and appropriate?” The overall mean score of all sessions was 8.42 ± 0.28. No statistically significant differences were found between the sessions directed at residents or all the members of the AEC, nor when the sessions were in a lecture versus seminar format, nor when comparing different topics, nor between the 2 stages of the AEC Virtual Classroom. At the peaks of the pandemic, the overall score was lower (8.32 ± 0.29 vs. 8.50 ± 0.23; P = 0.029).
Results of question 3: “Do you think that the speakers have used virtual media appropriately to convey the most important concepts?” The overall mean score was 8.41 ± 0.5 considering all the sessions, with no statistically significant differences found between the sessions aimed at residents versus all the members of the AEC, nor when they included sessions in lecture versus seminar format, nor among the different topics, nor between the 2 stages of the AEC Virtual Classroom, nor in relation to the peaks of the pandemic.
Results of question 4: “Do you think that the session could be useful for your usual clinical practice?” An average of 90.76% ± 5.9% of users considered the sessions useful. A higher percentage of participants considered the sessions useful in the first 6 months of the AEC Virtual Classroom (92.64 ± 4.7 vs. 88.9 ± 6.4; P = 0.037).
Results of question 5: “Should the Virtual Classroom be maintained after the pandemic?” 97.2% of users thought that the AEC Virtual Classroom format should be maintained as a complementary activity once in-person courses could be resumed. This question was only included in the satisfaction survey during the first trimester, after which it was no longer considered necessary.
At least one of the project coordinators has been present in all the AEC Virtual Classroom sessions. The subjective impressions of the 2 coordinators are collected in Table 2: lessons learned.
Lessons learned in the AEC Virtual Classroom: recommendations to improve virtual training sessions.
| Moderators | Lecturers | Lectures |
|---|---|---|
| Punctuality at the start of the session is crucial so as not to lose viewers and transmit professionalism. | Sit in a quiet place with good lighting and a good internet connection. | Do not use too many animated transitions between slides. |
| Moderators should stimulate the participation of the viewers by means of comments or comments in the chat, and one of the moderators should oversee the chat and the questions of the participants. | Use speakers with a microphone, which should be kept off while not speaking. | If videos are used, they should not be large files and should be preferably inserted in the slides themselves. Test the videos before the session to ensure correct transmission. |
| Time should be distributed so that there is always sufficient time for discussion and to respond to the viewers’ questions. | Rehearse the session to be sure you know how to use the technology and platform utilized, while sharing your slides and videos without any problems. | Involve the live viewers by asking multiple choice questions during the session. |
| Schedule a previous meeting with the lecturers to confirm the assigned times, the session plan, and the tools to be used (surveys, questions for the viewers, videos). | Avoid virtual backgrounds; it is preferable to use a neutral background (like a white wall or similar); if using a virtual background, it is better to use a Chroma greenscreen for a better result. | Limit your talk to the time allotted by the organizers. |
| Make the seminar more participative by asking at least one multiple-answer question before introducing the lecturer to see the participants’ opinion and understanding of the topic. | Combine the colors of your clothes according to the background (for instance, do not wear white against an all-white background) so that the contrast is correct. | |
| Wear proper attire. | Wear proper attire. | |
| Confirm with the technicians that the viewers’ microphones are off. |
The AEC Virtual Classroom has turned out to be a useful tool for teaching surgery that will foreseeably survive the pandemic era, given that 97% of the participants have expressed their wish for this activity to continue, even when it is possible to resume in-person courses. Having expert surgeons share their knowledge ‘in the comfort of your own home’ has been very attractive and has been manifested by the significant scope of the AEC Virtual Classroom sessions, both in live participation and in deferred viewings, reaching over 1400 views in certain cases. This scope is far superior to any traditional in-person courses.
More than 90% of the participants have found these sessions useful, which can be considered a great success given the limitations of the virtual format. Other published experiences of teaching activities in this webinar-type format conducted during the pandemic have also been satisfactory, and it seems that this format could continue to be useful once it is made compatible with in-person courses. What is clear is that this format had been underused, because the platforms to conduct these on-line seminars were available previously. The combination of the pandemic together with the cancellation of in-person courses led many scientific societies to accelerate the implementation of these new platforms to develop training programs6,8,15. Many technological changes that the pandemic has abruptly imposed on the healthcare system may be positive in the intermediate term, including increasingly varied technological options for teaching, consolidating virtual events and tele-teaching. A new term has even appeared to describe the new residents as ‘techno-omnivores’, referring to their great ability to assimilate new technologies and virtual platforms applied to telemedicine and teaching16.
However, the great proliferation of virtual events since the first months of 2020 has caused a certain amount of fatigue in the participants, running the risk of saturating the format. We observed this in our analysis when we compared the results of the first 6-month stage of the AEC Virtual Classroom with the second stage, finding a small decrease in the satisfaction survey scores in the second stage. Despite this fact, the number of live participants increased in the second stage, with an average increase of more than 120 participants per session compared to the first stage, reaching an average of more than 400 participants. We attribute this increase in participants in part to a consolidation of the format and schedule, but also to a greater outreach to surgeons from Latin American countries.
The topic of the sessions influences the number of live participants, global outreach, and satisfaction survey scores. The sessions with the highest participation and the best score in terms of format were those about coloproctology, but other sessions about transversal issues, such as infections or new technologies, also had a large influx of participants. In general, participant registrations for the seminars were lower when the topic discussed was more specific and less prevalent.
The workload of health professionals during the pandemic and the anxiety caused by the peak influx of COVID-19 patients has undoubtedly influenced their motivation to learn and is having a very important psychological impact on hospital staff around the world1. Another interesting factor we observed when analyzing the results was how the number of participants significantly decreased in both the live sessions and recorded videos during the weeks with the highest global incidence of COVID-19 cases, and how this also negatively influenced the results of the satisfaction survey.
In conclusion, the AEC Virtual Classroom project, which includes live weekly virtual seminars and an associated video platform, has had very good results over the first 12 months of development. It has also been a good teaching alternative during the COVID-19 health crisis, and this project will very likely survive the pandemic given the good results of the satisfaction survey and the growing number of participants. Nevertheless, it is interesting to bear in mind that the great proliferation of webinars is saturating the format, and modifications may be necessary in the future so that it continues to be a useful tool that is well valued by participants.
FundingThis study has received no specific funding.
Conflict of interestsThe authors have no conflict of interests to declare.
