Evaluation of Surgical Skills in Medical Students Using a Virtual Simulator

Abstract

Introduction

The use of virtual simulators together with hospital practicals during undergraduate training can improve basic surgical skills, and may even be an important motivating focal point. The aim of this work is to determine the endoscopy skill level and its learning curve in three stages within the Medical degree (second, fourth, and sixth year) using a virtual simulator.

Material and methods

The Lapsim^® simulator with 7 basic modular exercises was used. Twenty-four students (8 second, 8 fourth, and 8 sixth year) were subjected to a training program (basic test plus 3 sessions), and an examination at 30 days. A control group (n=24) from each year was subjected to 2 examinations with an interval of 30 days.

Results

All the groups improved significantly after training, and this was maintained in all the exercises after a “wash out” period of 30 days. All the groups achieved lower baseline scores in the “Cutting” and “Lifting and grasping” exercises, considered of intermediate level than any of the others. However, after the training process, they were the only exercises that the sixth year students demonstrated that they were clearly superior to the rest of the years: “Cutting” (94.5%±11% vs 81%±7%) (P<.05), “Lifting and grasping” (88.1%±13% vs 68%±11%) (P<.001).

Conclusions

The virtual simulator appears to be of greater use for those students in higher years with more clinical experience, there already being a discrimination process during this period. Surgical skills during this phase should be reinforced by means of new teaching challenges, such as simulation programs in the framework of skills laboratories.

Resumen

Introducción

Durante el pregrado, la utilización de los simuladores virtuales junto con las prácticas hospitalarias puede mejorar las aptitudes quirúrgicas básicas, e incluso ser un eje motivacional importante. El objetivo del trabajo es determinar el grado de habilidad endoscópica y su curva en 3 etapas dentro del grado de Medicina (segundo, cuarto y sexto curso) mediante un simulador virtual.

Material y métodos

Se ha utilizado el simulador Lapsim^® con 7 ejercicios del módulo básico. Se somete a 24 alumnos (8 de segundo, 8 de cuarto y 8 de sexto curso) a un programa de entrenamiento (prueba basal más 3 sesiones) y a un examen a los 30 días. Un grupo control de cada curso (n = 24) ha sido sometido a 2 exámenes con un intervalo de 30 días.

Resultados

Todos los grupos mejoran significativamente después del entrenamiento y se mantienen después del periodo de lavado de 30 días, y en todos los ejercicios. Los ejercicios «Cutting» y «Lifting and grasping», considerados de nivel intermedio, destacan sobre los demás por el bajo score basal que tienen todos los grupos. Después del proceso de entrenamiento, en cambio, son los únicos ejercicios en los que los alumnos de sexto se muestran claramente superiores respecto a los demás cursos: «Cutting» (94,5% ± 11 vs 81% ± 7) (p < 0,05), «Lifting and grasping» (88,1% ± 13 vs 68% ± 11) (p < 0,001).

Conclusiones

La utilidad del simulador virtual parece mayor para aquellos alumnos de cursos superiores con una mayor experiencia clínica, existiendo un proceso de discriminación ya durante este periodo. Las competencias quirúrgicas durante esta fase deberían reforzarse mediante nuevas apuestas docentes como los programas de simulación en el marco de los laboratorios de habilidades.

Introduction

Over the years, surgical teaching has had to adapt to technological advances, mainly by the development of endoscopic surgery. This type of surgery requires following a progressive learning curve, in order to provide the surgeon with sufficient technical experience.¹ In this process of acquiring skills, virtual simulators have been gradually implemented in the curricular itinerary and have been shown to be a great help. The main advantages of virtual simulators are avoiding patient injury and reducing the costs associated with the use of cadaver and live animals, following the principles of reduction, substitution and refinement. They also provide the possibility of repeating surgical procedures as many times as necessary to learn them correctly, while offering revisualization of the procedures performed.² There are different models, manufacturers and prices that can adapt to departmental budgets. Likewise, there is the possibility of extended modules after basic skills are acquired (cholecystectomy, hemicolectomy, etc.).

Virtual simulators are included in the framework of “skills laboratories” where simulations are also done with mannequins and diverse models that may even be personal designs. In addition, there are low-cost closed-circuit TV simulation models that have been shown to be useful in developing basic skills.³ The use of simulators can be applied at different levels: during specialty training, the post-graduate period or even as maintenance training for experienced professionals, although these are not implemented in a mandatory fashion.⁴ Their application during medical school is not a part of the curricular itinerary as many Spanish medical schools do not have skills laboratories. This has repercussion on the quality of teaching, both in the field of medicine as well as in surgery. Updating teaching resources should be a priority in student education and training. The joint use of virtual simulators together with practical hospital experience can improve the basic surgical aptitudes of students and help them acquire correct tridimensional orientation in the endoscopic two-dimensional plane.⁵ Furthermore, it may even be an important motivational element during the learning stage.

The general objective of this study is to determine the degree of endoscopic skills of medical students at 3 different stages within the 6-year undergraduate period (second year, fourth year and sixth year) by means of a virtual simulator and after a training period.

Materials and Methods

Participants

A total of 48 participants were included in the study, all of whom were medical students at the Medicine and Health Science School of the Rovira i Virgili University in Tarragona, Spain. There were 16 second-year participants, 16 fourth-year and 16 sixth-year students. Each of the 3 groups was divided into a training program subgroup (n=8) and a control subgroup (n=8).

Inclusion criteria for study participants: It was required for sixth-year students to have completed their 9-week hospital rotation in the General and Digestive Surgery Department at the Sant Joan de Reus University Hospital (Tarragona, Spain). The fourth-year students needed to have completed the 3-week rotation at the same hospital, in addition to the surgical skills laboratory training module belonging to our Surgery Department. The second-year students were required to have passed the Surgical Anatomy course (first cycle) in which students first come into contact with surgery (one operative session plus one session in the surgical skills laboratory). In the skills laboratory, none of the students had had any experience with surgical endoscopic simulation; only basic suture and knot-tying skills were developed. This avoided possible bias in the results in spite of the acquisition of basic skills.

Material

The LapSim^® simulator (Surgical Science Ltd., Göteburg, Sweden) uses a non-haptic hardware interface that includes 2 articulated supports with sensors and corresponding hand controls for each hand. At the same time, a computer with Windows Xp Pro Operating System (Pentium IV processor, 250 Gb ram) and a 17-inch flat screen was used to process and visualize the exercises. The software used was the 2009 version of LapSim Basic Skills (Surgical Science Ltd., Göteburg, Sweden).

Exercises

Seven exercises were chosen in order of difficulty according to the LapSim Basic Skills 2009 computer program. Each was explained and shown to the participants by an instructor who tutored the sessions. Each session was 40 min long, which was the time limit established for performing the 7 exercises. The default design of each exercise was determined by the software, with no added adjustments such as changes in the camera angle or zoom. The overall score (%) for each exercise was determined based on tissue damage, maximum damage, estimated blood loss, time, angled or straight navigation, etc.

The selected exercises included:

• -

  Camera navigation: This is a basic exercise in which the participant should locate and focus on 5 red balls in a randomly appearing virtual field. The correct execution of the exercise turns the balls green.

• -

  Instrument navigation: Here, each hand corresponds with an endoscopic instrument whose tip is a different color (red and green). Ten colored balls appear in random locations, and each should be touched with the hand corresponding with each color.

• -

  Coordination: This combines the camera and instruments. Ten randomly appearing red balls should be found, touched and transported to a target that appears instantaneously.

• -

  Grasping: An intermediate-level exercise that involves grasping 6 longitudinal bands or appendices and transporting them to an identifier or target. Each hand corresponds with a color that is randomly defined by the band on the screen.

• -

  Cutting: This entails grasping a segment or “vessel”, extending it and preparing it with the right hand to be cut with the ultrasonic scalpel (left hand and pedal). The cut segment should be placed in the corresponding identification area that appears on the screen. The complete exercise includes 3 vessels.

• -

  Lifting and gasping: In this exercise, a cube is lifted to reveal a curved needle, which is then grasped and placed in the identification area. There are a total of 7 cubes that appear randomly, and the grasping and placement of the needle should be done with both hands according to the color of the tip of the forceps.

• -

  Fine dissection: This is considered a more difficult exercise. A conglomeration of vessels appears, which should be extended for correct visualization; the thinner vessels are then cut with the harmonic scalpel (right hand and pedal) without cutting the larger vessels.

Design

In the intervention/training groups (second, fourth and sixth years, n=24), an initial baseline test was completed of the entire battery of exercises. Ten days later, a standardized training program of 3 sessions (one session per week) was begun. Afterwards, and after a 30-day washout period, a final test was done to determine the effects of the training program.

In the control groups (second, fourth and sixths years, n=24), the first baseline test was completed of the entire battery of exercises. After the same 30-day washout period, a final test was done to determine the effects in the training group.

Analysis

To compare the means of the scores in the same group or between groups, we used the Wilcoxon and Mann–Whitney U-tests (for paired and unpaired data, respectively). Global analysis was done with the SPSS-PC 17.0 computer program. A P value ≤.05 was considered statistically significant.

Results

All the student groups improved significantly after the training process and this result was maintained over a 30-day washout period and in the entire battery of exercises. The evolution of the means through the learning process by student year and per exercise is demonstrated in Figs. 1 and 2 and in Table 1.

Fig. 1.

Evolution of the training by student year in the Camera navigation, Instrument navigation and Coordination and grasping exercises.

Fig. 2.

Evolution of the training by student year in the Cutting, Lifting and grasping and Fine dissection exercises.

Table 1.

Mean and Standard Deviation of the Scores (%) of the Baseline and Final Tests, by Exercises.

	Baseline test	Final test
Camera navigation	80.3±12	95.0±7^*
Instrument navigation	83.6±9	99.9±1^*
Coordination	63±15	79.5±8^*
Grasping	70.6±13	91.4±8^**
Cutting	60.3±8	85.2±15^**
Lifting and grasping	59.3±9	72.7±13^**
Fine dissection	83.3±11	96.3±10^*

SD: standard deviation.

*

P<.05.

**

P<.001.

Contrarily, the students in the control group (second, fourth and sixth years) did not significantly improve their scores (%) after the 30-day washout period, as can be observed in Table 2. The difference between the score of the final exam of the control subjects and the final exam of the training group is significant for all the exercises. In the fourth- and sixth-year control subjects, there is a possible memory effect for the easiest exercises (Camera navigation and Instrument navigation) (Table 3).

Table 2.

Differences of the Final Mean Scores (%) of the Exercises in the Control and Training Groups.

	Control groups, pre- and post-washout (n.s.)	Training groups, final test
Camera navigation	70.6±15—78.2±11	95.0±7^*
Instrument navigation	84.2±12—91±13	99.9±1^*
Coordination	62.8±14—67.1±12	79.5±8^*
Grasping	69.9±13—73.1±10	91.4±8^**
Cutting	54.9±14—56.5±10	85.2±15^**
Lifting and grasping	56.3±10—58.6±11	72.7±13^**
Fine dissection	87±11—85.4±13	96.3±10^*

*

P<.05.

**

P<.001.

Table 3.

Differences Observed in the Scores (%) Between the Different Groups Pre- and Post-washout Period, According to Student Years and Exercises.

	Camera Navigation	Instrument Navigation	Coordination	Grasping	Cutting	Lifting and Grasping	Fine Dissection
2nd year control, pre-	77.1±8	90.7±10	57.2±11	66.1±10	55.5±9	60.3±13	88±15
2nd year control, post-	74.3±9	85.2±12	61.1±8	68±9	54.7±11	59.1±11	84.6±12
2nd year training, pre-	76.1±10	77±9	53.2±10	66.1±12	55.5±12	60.3±15	77±14
2nd year training, post	97.5±8^*	98.3± 8^**	81.3±9^**	87.7±11^**	81.3±9^**	76.3±10^*	98.3±9^**
4th year control, pre-	60.6±11	78.3±12	65.6±9	75±10	53.2±12	54.5±13	80.2±13
4th year control, post-	77.1±10^*	95.3±11^*	72.6±10	79.6±15	55±10	57.1±11	86.3±15
4th year training, pre-	79.3±10	87.5±9	70.2±8	76.5±9	61.1±10	55.7±9	87.5±10
4th year training, post-	94.2±8^*	97.2±8^*	81.1±7^*	94.1±9^*	80.5±11^*	64±8^*	97.2±9^*
6th year control, pre-	72.5±11	83.6±10	61.7±16	68.6±13	56.1±14	54.3±12	90.5±16
6th year control, post-	83.3±10^*	92.7±8^*	67.5±14	71.8±11	59.1±13	59.6±10	76.3±14
6th year training, pre-	86.3±8	85.7±10	65.1±12	75.1±10	65.1±12	62.6±12	85.7±13
6th year training, post-	93.5±8^*	94.2±7^*	76.1±9^*	92.5±11^*	94.5±11^*	88.1±13^*	94.2±10^*

*

P<.05.

**

P<.001.

In the Cutting and Lifting and grasping tests, the sixth-year students obtained a significantly higher score than the other students after the washout period: Lifting and grasping (88.1±13 vs 62.6±121) (P<.001) and Cutting (94.5±11 vs 65.1±12) (P<.05).

The second-year students had inferior skills than the fourth- and sixth-year students on the baseline test and throughout the entire training period. In spite of this, in the final exam their scores were at the same level as the rest of the students (except in the Cutting and Lifting and grasping exercises, where the sixth-year students had higher scores).

In the least difficult exercises (Camera navigation and Instrument navigation), the baseline scores were higher than those for more difficult exercises (Cutting and Lifting and grasping): 80.3±12 and 83.6±9 vs 60.3±8 and 59.3±9, respectively; P<.001.

Discussion

In order to demonstrate their validity and teaching potential, virtual simulators should be able to discriminate between the different skill levels of their users. This has been demonstrated at the professional level, among resident physicians and professionals with extensive laparoscopic experience.⁶ Even so, the clinical transferability of these skills has not been demonstrated in all the simulators on the market, and there is need for improvement of the so-called “predictive validity”, which is only achieved with standardized programs and cutting-edge software.⁷

Few studies in the literature include medical students in virtual training in order to define their abilities. Its usefulness at this level has been demonstrated not only with regard to technical skills, but also in reducing stress levels when the transfer is made to real surgical scenarios.⁸ In our case, we can say that the virtual simulator is able to discriminate between students in the sample, especially between those in their second and sixth years. In the baseline tests for all the exercises, except in the Lifting and grasping test, the second-year students obtained significantly lower scores despite the fact that after the training phases these levels were comparable to those of the other students. The superiority of the fourth- and sixth-year students at baseline is an indicator of the usefulness of standardized practical training within the study plan. This training, together with the skills laboratory training, has a positive effect on students who, once their medical school education is completed, will have superior overall skills than those students whose curricular itinerary has included little practical training.

The 3-week training program of this study produces a significant improvement in all the student groups. Taking into account this fact and the demonstrated importance of clinical practice, its standardized application in the curricular itinerary of residents should be a priority teaching objective in hospital centers that are accredited for training specialists.⁹ Subsequently, the evaluation of technical skills with virtual simulators could be considered a valid, objective method in hiring processes, together with personal interviews and professional experience.¹⁰

In the Cutting and Lifting and grasping exercises (considered to be of intermediate difficulty), it was striking to see the low baseline scores of all the analyzed groups. After the training process, however, these are the only exercises in which the sixth-year students are seen to be clearly superior to the other students. This is similar to data found in the literature at the post-graduate level where, after training, surgeons with greater laparoscopic experience obtain higher scores.¹¹ This fact once again reinforces the importance of clinical practice in the exponential acquisition of surgical skills when various teaching methods are combined. On the other hand, the exercise considered as being the most difficult (Fine dissection) obtained a significantly higher score both before and after the training period. At this point, we should also mention that during this study most students indicated that the most difficult exercise in their opinion was Lifting and grasping. This fact is supported by other papers in which the exercise used to evaluate surgeons’ skill is none other than Lifting and grasping.¹¹

In the less difficult exercises of our study (Camera navigation and instrument navigation), there has been a possible memory effect in the 30-day follow-up in the fourth- and sixth-year groups. This did not happen in the other exercises or in the case of the second-year students, in whom the 30-day period was sufficient to validate the design. No participant had had contact with a virtual simulator before taking part in the experiment, although they had had contact with the skills laboratory to work on basic skills such as suturing or knotting. There are no values that refer to years of medical school, but a longer period could be considered for evaluating elemental exercises. When evaluating competencies among groups of surgeons, washout periods can even be extended to more than 6 months.¹² However, when reviewing LapSim^® validity studies, it is observed that this type of simulator is much more effective in inexperienced subpopulations, which demonstrates its applicability and efficacy for use in medical schools.¹³

Lastly, during the exercise process and follow-up of the participating students, we have observed an important motivational component for learning surgical skills in most participants. Currently, part of the competency teaching in surgery follows a cross-curricular process that may endanger surgical teaching as its own entity.¹⁴ Surgical skills laboratories, with the advent of specific curricular programs using simulators, can also be motivational for the acquisition of technical skills.¹⁵

Medical school students who participated in a training program with a virtual simulator have significantly improved their basic endoscopic skills. This training is effective, and the acquired skills are not lost after a rest period. The usefulness of the virtual simulator seems to be higher for students in more advanced years who have greater clinical experience. There is already a discrimination process seen during this period. Surgical abilities during this educational phase should be reinforced with new teaching techniques such as simulation programs in the skills laboratories. There are many factors that promote simulation in the educational setting, such as those that help students acquire standardized learning in order to achieve specific competencies and avoid irresponsible actions in real patients. Competency-based learning in the new curriculum should also be evaluated and, to do so, simulators could play a fundamental role in exams.¹⁶ This means that instructors and professors should be capacitated in this field, defining teaching outlines and learning objectives for each year of medical school,¹⁷ although there are studies where the figure of the instructor is considered unnecessary and unrelated to the results obtained.¹⁸ Even so, the simulation program should be designed and evaluated by trained instructors. Finally, it should be mentioned that the standardized application of a training program during the residency period would be highly effective and efficient for acquiring the knowledge necessary to complement residents’ training as specialists, which has been demonstrated by recent studies.^19,20

Conflict of Interests

The authors declare having no conflict of interests.

Acknowledgements

We would like to thank our surgical residents and the medical students at the Rovira i Virgili University for their disinterested, voluntary participation in this study.