Development and validation of a low-cost laparoscopic simulation box

Abstract

Introduction

The acquisition of laparoscopic technique skills in an operating room is conditioned by the expertise of the tutor and the number of training interventions by the trainee. For students and surgeons to use a laparoscopic simulator to train their skills, it must be validated beforehand.

Methods

A laparoscopic simulator box was designed, along with 6 interchangeable training games. The simulator was validated by a group of 19 experts, physicians with an experience from at least 100 laparoscopic surgeries, and 20 students of 4^th to 6^th grades of medical school (non-experts). To evaluate its construct validity, time-to-completion and the number of successfully completed games were assessed. We used 11 and 9-item questionnaires to gather information on content and face validity respectively. In both questionnaires, answers were collected through Likert-type scales, scored from 1 to 5.

Results

The group of experts required less time and successfully completed more games than the group of non-experts (p < 0.01). The group of non-experts gave a score ≥ 4 points on each of the questions regarding the content validity of the tool, however, the experts rated with a significant lower mean score the need for the simulator to learn the surgical technique (3.68 points; p < 0.01). Regarding the face validity, all items were graded with a score ≥ 4 points except for the question relating to the spatial realism (3.82 points).

Conclusion

The laparoscopy simulation box and the games were valid means for training surgeons and medical students to develop the skills required for the laparoscopic technique.

Resumen

Introducción

La adquisición de las habilidades propias de la técnica laparoscópica en quirófano está condicionada por la experiencia del tutor y el número de intervenciones que realiza el aprendiz. Para que los estudiantes y cirujanos en formación utilicen un simulador laparoscópico para entrenar sus habilidades es importante que este esté previamente validado.

Métodos

Se diseñó una caja de simulación laparoscópica, junto con 6 juegos de entrenamiento intercambiables. El simulador fue validado por un grupo de 19 expertos, médicos con experiencia de al menos 100 cirugías laparoscópicas y 20 estudiantes de 4º a 6º del Grado en Medicina (no expertos). Para evaluar su validez de constructo, se evaluó el tiempo hasta la finalización y el número de juegos completados con éxito. Se utilizaron cuestionarios de 11 y 9 ítems para recopilar información sobre la validez de contenido y la validez aparente, respectivamente. En ambos cuestionarios, las respuestas se recogieron con escalas tipo Likert de 1 a 5.

Resultados

El grupo de expertos necesitó menos tiempo y completó con éxito más juegos que el grupo de no expertos (p < 0.01). El grupo de no expertos puntuó con ≥ 4 puntos cada una de las preguntas relativas a la validez del contenido, sin embargo, el grupo de experto dio una puntuación significativamente menor la necesidad del simulador para aprender la técnica laparoscópica (3,68 puntos; p < 0,01). En cuanto a la validez aparente, todos los ítems fueron calificados con una puntuación ≥ 4 puntos excepto el relativo al realismo espacial (3,82 puntos).

Conclusión

La caja de simulación de laparoscopia y los juegos que desarrollamos son válidos para que cirujanos y estudiantes de medicina desarrollen las habilidades necesarias para la cirugía laparoscópica.

Introduction

The laparoscopy is a minimally invasive surgical technique in which various surgical instruments and materials are introduced into the operative field through tiny incisions, in addition to a camera system that broadcasts a video of the procedure, in two dimensions, onto a screen. The laparoscopic technique was designed to fulfill the need for reducing the risk of complications during and after surgical interventions, while speeding-up the recovery of the patient¹. Nevertheless, the laparoscopic surgery is not exempt from risks, particularly for novel techniques and if the surgeon is unexperienced². Additionally, compared to open surgery, laparoscopy is harder to learn³.

Traditionally, when surgical training was based on the principles of “observe, help, and then operate”, the acquisition of surgical skills relied on the expertise of the tutor and the number of interventions performed by the apprentice. Currently, surgical trainees can begin their formation process and acquire relevant surgical skills before entering the operating room through simulators. It is known that simulation provides with a safe environment in which to repeat as many attempts as needed to achieve a degree of expertise that was previously acquired through years of experience^4–6.

To carry out a laparoscopy safely, unique psychomotor skills are required from the surgeon. The common obstacles of the technique include adapting to the shift from three-dimensional vision to only two dimensions, increased complexity of two-handed dexterity, handling long instruments, which entail an increase in distal tremor and the need of dealing with the fulcrum effect, and the loss of tactile feedback⁷.

The most widespread simulation methods to learn the laparoscopic technique include laparoscopic simulation boxes (endotrainers), cadaver models, and virtual simulators⁸, with endotrainers being the most accessible ones for medical schools. Compared to virtual simulators, laparoscopy simulation boxes have the clear advantage of utilizing actual laparoscopic instruments, such as cameras and monitors, which offer realistic tactile feedback and depth of view^9–11. Various types of endotrainers exist based on their degree of fidelity, however lower fidelity models can be as efficient to acquire the necessary skills as those of greater fidelity¹².

The aim of this work was to validate a low-cost laparoscopy simulation box along with six interchangeable games of increasing difficulty levels. For this goal, we adhered to the following concept of validity: “the property of being true, sound, and in accordance with reality”, which is essential in the evaluation of a simulator¹³. The following three aspects of validity were considered: construct validity, content validity, and face validity. Construct validity reflects the ability of a simulator to provide feedback on its intended goal, that is, being able to discriminate between individuals with different levels of proficiency^13,14. Content validity evaluates the usefulness of a simulator as a training tool, and face validity evaluates the subjective realism of the simulator¹⁵.

Material and Methods

• a)

  The laparoscopy simulation box

The box was designed and constructed at the Laboratory of Medical Engineering of the Faculty of Medicine at the University of Navarra, considering the existing simulators and including the following improvements: 1) an analog camera with a focus feature, 2) a camera support system with 4 degrees of freedom, 3) a light source provided by the camera support system, 4) an enclosed simulator, i.e. all external light sources should be blocked, and 5) the design of the box and the orifices for the trocars should be anatomically correct¹⁶.

The box was designed with a curvature on its anterior side that emulates the appearance of the abdomen, and a flat posterior surface for support (Fig. 1). The simulator box (external dimensions: 48 × 33 x 21 cm) was comprised of three main structures: the base, a lid, and a stem. Once open various exercises could be installed inside the box (game dimensions: 19 cm of diameter x 3.5 cm height). On the exterior surface of the box, a total of 7 orifices were made in addition to 2 more located on the base of the box. Each of these orifices were partially sealed with a flexible disc with an X-shaped slit on its surface through which 5 mm x 10 mm trocars could be introduced. The base of the box had an additional orifice for the stem, located on the exterior of the box. This stem was connected to the camera and lighting system of the simulator, through which the focus of the camera could be controlled.

Fig. 1.

The laparoscopy simulation box.

The cost of the development and implementation of our simulation box was approximately 215 euro (138 euro for plastic material, 19 euro for hardware and 58 euro for electronics), which is well below most of the simulators available on the market whose price can vary depending on their functionalities and equipment between 900 and 3000 euro.

• b)

  The games

The games were designed, 3D printed, and assembled in a similar fashion to the simulator box in the Laboratory of Medical Engineering of the Faculty of Medicine at the University of Navarra. Each game was a short cylindrical structure that could be fitted and secured to the base of the box through a twist-lock mechanism (Fig. 2).

Fig. 2.

Set of 6 games designed to develop the necessary skills to carry out the laparoscopic technique. From left to right, and top to bottom: b1. Ring game, b2. Peg game, b3. Loop game, b4. Sorting puzzle I, b5. Sorting puzzle II, and b6. Suture game.

The following 6 games were designed and described in increasing degree difficulty:

b1. Ring exercise: The activity consisted in placing plastic circular rings around upward oriented pegs, placed throughout the surface of the structure.

b2. Peg exercise: The goal of this exercise was to place wooden pegs upright inside small orifices located on the surface of the structure.

b3. Loop exercise: The exercises’ goal was to insert a suture string through the loop of various loop screws on the surface of the structure.

b4. Sorting puzzle I: For this exercise each trainee had to adequately sort various plastic figures to their complimentary slots on the surface of the structure.

b5. Sorting puzzle II: Similar to the previous exercise, various plastic figures had to be sorted in their corresponding slots.

b6. Suture exercise: The trainee had to perform suture knots on a silicone base of various multiaxial patterns.

The average cost of the developed games was around 10 euro, also lower than the average market price.

Validation Study

For its validation, a group of 39 participants were included: 19 experts and 20 non-experts. The group of experts consisted of trained general surgeons, gynecologists, and urologists from the University of Navarra Clinic, who had performed at least 100 laparoscopic interventions¹⁷. In contrast, the group of non-experts consisted of medical students in their 4^th to 6^th year of medical training at the Faculty of Medicine of the University of Navarra; none of them had previously performed a laparoscopic surgery.

Successful completion was defined individually for each exercise as follows: placing 2 rings under 5 minutes for the ring game, inserting two pegs under 5 minutes for the peg game, inserting the suture strand trough two loops under 5 minutes for the loop game, placing two figures under 5 minutes in either of the sorting puzzles for the games with puzzles, and correctly performing a suture stitch under 10 minutes for the suture game. Additionally, the exercise had to be completed in a single attempt to be considered successful.

The validation study was carried out over the course of two days in the Medical Simulation Center of the University of Navarra, under similar conditions for both experts and non-experts. Each participant was briefed on the characteristics of the simulator and was given a limited amount of time to familiarize themselves with the equipment prior to performing all the trials. The games were to be carried out in the pre-established sequence, in increasing degree of difficulty beginning with exercise b1 and ending with exercise b6. As established in the study protocol, an external examiner was present and supervised each trial. The examiner oversaw the following tasks: keeping track of time for each attempt, communicating to the participant if he/she had reached the maximum permitted time for a trial, and determining whether the game was successfully completed.

To determine the construct validity of the simulator, the time to successfully compete a game and the overall evaluation of the examiner were considered. After completing the activity, all participants completed a questionnaire with 11 questions to assess the content validity of the simulator. Each question had Likert-type scale responses with a score ranging from 1 to 5. In addition, the group of experts answered a questionnaire to evaluate the face validity of the simulator. This questionnaire consisted of 9 questions with Likert-type scale responses that also ranged from 1 to 5.

Ethics approval

This study was approved by the Board of the Medical School of the University of Navarra.

Statistical Analyses

Data are presented as means and standard deviations (SD). Mean scores of the content validity questionnaire, mean time to completion for each exercise, and the number of successfully completed games were compared between the group of experts and non-experts; the Mann Whitney U-test was used to determine the statistical significance of each comparison. All statistical analyses were carried out using STATA version 15.0. P-values were analyzed as two-tailed, and the significance level was predefined to a value of p < 0.05.

Results

The laparoscopy simulation box we developed has significant improvements over previous simulators as shown in Table 1.

Table 1.

Comparison of the characteristics of the box-simulator developed at the Medical Engineering Laboratory of the University of Navarra and the simulators available on the market.

Company	Simulator Model	Camera DOF	Rounded Operating Space	No external Light	Multiple Trocar Insertion Sites	Illumination Originating from Camera Tip	Camera focus
3-Dmed	T5 Series Large	4	×	×	✓	×	×
3-Dmed	T12-HD	4	✓	✓	✓	×	×
Medical-X	Lap-X Box	0	×	×	×	×	×
Simulab	Laptrainer with Simuvision	3	×	×	✓	×	×
eoSurgical	eoSim SurgTrac Core	0	×	×	×	×	×
Laparo	Advance Ver. Training Station	4	✓	×	✓	×	×
Inovus	Pyxus HD move	4	×	×	×	×	×
kroton	Kroton medium laparoscopic trainer	4	✓	×	✓	×	×
Medical Engineering Laboratory, University of Navarra	Designed box-simulator	4	✓	✓	✓	✓	✓

DOF: Degrees of freedom.

Construct Validity

As shown in Fig. 3, the mean time to completion for each of the games was significantly higher amongst the group of non-experts. In line with this, the sum of completion times was significantly higher (p < 0.001) in the group of non-experts (38.50 minutes [SD:5.4]) than in the group of experts (6.21 minutes [SD: 2.02]) as well.

Fig. 3.

Box plot of the mean time-to-completion of each game. Results from experts (N = 19) and non-experts (N = 20).

Each of the expert participants successfully completed all 6 exercises, whereas the group of non-experts completed a mean of 4.8 exercises (p < 0.001). Only two participants from the group of non-experts successfully completed exercise b6, the suture exercise, within the time frame limit.

Content Validity

Table 2 contains the mean scores of experts and non-experts for each of the questions in the content validity questionnaire. In both groups, all questions had a mean score ≥ 4, except for the question on the need of the simulator to evaluate the training of the laparoscopic technique (mean score from experts: 3.68 [SD:1.00], mean score from non-experts: 4.55 [SD:0.69]).

Table 2.

Mean (SD) of each item evaluating the content validity of the simulator. Results from experts (N = 19) and non-experts (N = 20).

ITEM	Experts	Non-experts	p
The use of the simulator increases user’s level of confidence in order to eventually perform this technique on patients.	4.68 (0.58)	4.55 (0.83)	0.85
The use of the simulator is helpful in the learning process of the user.	5.00 (0)	4.90 (0.31)	0.32
Training with the simulator can help the user improve their two-dimensional spatial perception.	4.94 (0.23)	4.80 (0.41)	0.34
Training with the simulator helps the user familiarize himself/herself to the use of a single light source.	4.58 (0.69)	4.65 (0.59)	0.83
Training with the simulator helps the user adapt himself/herself to the lack of tactile feedback experienced in laparoscopic surgeries.	4.36 (0.90)	4.65 (0.67)	0.54
The simulator is USEFUL to learn the laparoscopic technique.	4.68 (0.67)	4.70 (0.47)	0.69
The simulator is NEEDED to learn the laparoscopic technique.	4.58 (0.69)	4.45 (0.76)	0.57
The simulator is USEFUL to monitor the learning of the laparoscopic technique.	4.37 (0.83)	4.80 (0.41)	0.16
The simulator is NEEDED to monitor the learning of the laparoscopic technique.	4.00 (1.05)	4.45 (0.76)	0.34
The simulator is USEFUL to evaluate the training of the laparoscopic technique.	4.21 (0.79)	4.75 (0.44)	0.04
The simulator is NEEDED to evaluate the training of the laparoscopic technique.	3.68 (1.00)	4.55 (0.69)	0.01

The questions with the highest overall scores were: “The use of the simulator is helpful in the learning process of the user” (5.0 [SD:0.0] and 4.9 [SD:0.31] for the group of experts and non-experts respectively), and “Training with the simulator can help the user improve their two-dimensional spatial perception” (4.94 [SD:0.23] and 4.80 [SD:0.41] for the group of experts and non-experts respectively).

Face Validity

Except for the question on the realism of the spatial orientation of the simulator, all questions on the face validity of the simulator had a mean score ≥ 4 (Table 3). The highest valued features of the simulator by the group of experts were the live-transmission of all actions onto the screen (4.79 [SD: 0.42]), its ability to improve two-handed dexterity (4.89 [SD:0.32]), and being able to train hand-eye coordination (4.74 [SD: 0.65]).

Table 3.

Mean (SD) of each item evaluating the face validity of the simulator. Results from experts (N = 19).

ITEM	Score
The spatial orientation is realistic	3.82 (0.95)
The simulator allows for an adequate illumination of the games	4.05 (0.91)
The simulator displays the exercises onto the screen with adequate contrast	4.00 (0.88)
The simulator displays a good quality image.	4.00 (0.94)
The simulator offers enough entry orifices, and these are correctly positioned.	4.16 (0.83)
The simulator is provided with a camera that allows adequate control and mobility.	4.16 (0.90)
The simulator helps monitor actions in real-time.	4.79 (0.42)
The simulator helps improve two-handed dexterity.	4.89 (0.32)
The simulator allows for the improvement of eye-hand coordination.	4.74 (0.65)

Discussion

In this report, our aim was to design and validate a low-cost endotrainter-type simulator along with six activities to develop the skills needed for a laparoscopic surgery. The results show that our simulator has sufficient construct, content, and face validity to train medical students and first year residents of general surgery in improving 2D spatial perception, developing of two-handed dexterity, and using specialized instruments, all of which are critical skills for this surgical technique.

Currently, various laparoscopy simulators exist, however, our design exclusively includes among its features, an analog camera mounted on a base that allows four degrees of freedom, and a light source that is attached to the camera support system. Additional benefits of this simulator are its enclosed workspace, which blocks all external light sources; the general design of the box, which resembles a human abdomen; and the anatomically correct entrance orifices for the trocars. Two of the games used to validate our simulator have been previously described in similar simulating programs (Fundamentals of Laparoscopy Surgery ™ (FLS) https://www.flsprogram.org/), these were loop game (b1), which is comparable to their “Peg transfer” exercise, and suture game (b6), which is comparable to their “suture with intracorporeal knot” exercise.

Evidence of the construct validity of our simulator are the clear differences between experts and non-experts in the time required to complete each game. Similar to other validation studies¹⁸, the suture game (b6) showed the greatest difficulty, and in our case, it was completed by only two non-expert participants. This result contrasts with the fact that all the non-expert participants had previously studied and practiced surgical stitching techniques in the Medical Simulation Center during their medical degree training, and highlights the added difficulty of performing this, and similar procedures, with a two-dimensional visual perception. In agreement with prior validation studies¹⁹, we believe that the simulator described in this report could be of interest for medical residents, as well as for undergraduate medical students who wish to improve these skills.

Regarding the content validity of our simulator and similar to prior studies, both experts and non-experts considered the endotrainer as beneficial for the acquisition of the necessary skills to perform the laparoscopic technique^7,20,21. This surgical procedure requires a particular set of skills, that are different to those required for an open surgery. Each of the 6 games described in this study allow the trainee to train his two-dimensional vision, adapt himself with a single light source, and familiarize to the lack of tactile perception. Therefore, we can conclude that the simulator we are presenting is a valid tool to develop the abilities required for a laparoscopic surgery.

Consistent with other studies, the items relating to the need of the simulator to evaluate the training of the laparoscopy technique had the lowest scores in the questionnaire of content validity²². In a hospital environment, medical specialists are not evaluated as medical students are in medical schools. It is through their personal success rate and the opinion of colleges and supervisors how trained specialists identify their weaknesses. This way of assessing learning contrasts with that of medical students, which are more used to punitive evaluation systems. As such, it is comprehensible that experts do not consider the simulator necessary to evaluate one trainee’s learning level.

The face validity of the simulator presented in this study is upheld by the score given by the experts, in particular for the items “The simulator helps improve two-handed dexterity” (4.89 [SD: 0.32]), “The simulator helps monitor actions in real-time” (4.79 [SD: 0.42]), and “The simulator allows for the improvement of eye-hand coordination” (4.74 [SD: 0.65]). Overall, these items relate to fundamental skills required from a specialist when performing a laparoscopy surgery.

Interestingly, only one question on the questionnaire of face validity had a mean score below 4 points (3.82 [SD: 0.95]); the one regarding the realism of the orientation of the simulator box. This aspect in particular is difficult to assess and validate, as performing an actual laparoscopy is vastly different from performing a series of exercises in a simulator setting. The next lowest scored items were those related to the lighting (4.05 [SD: 0.91]), image contrast (4.00 [SD: 0.88]), and image quality of the simulator (4.00 [SD: 0.95]), but those scores cannot be considered low. The selection of a camera for the simulator was particularly difficult to implement, aside from being easy to adapt and install in the simulator. Image quality is one of the most difficult aspects to implement in a simulator, since it is necessary to have a camera that fits the dimensional constraints of the simulator and includes a specific set of features that correctly emulate the characteristics of a laparoscopy, all within a reasonable cost-effectiveness ratio.

We have identified additional aspects that could be improved in following versions of the simulator. The circular disks covering the orifices of the simulator should be made of a more flexible plastic material to allow for greater movement of the trocars and surgical instruments. Regarding the support system for each game, it could be improved by designing a more simple and robust locking system and by increasing the angle between the base and the cylindrical structure from a 6 ° to a 10 ° angle. For the sorting puzzle II (b5), providing an image of a solved puzzle could improve the execution of the task, and for the suture game (b6), the silicone base should be covered with an anti-adhesive material. Besides, that game could be completed by adding a stitch on a single axis (X-axis or Y-axis). Finally, the sequence of the exercises should be reorganized: The Peg game (b2) should be performed after the sorting puzzle II (b5), as the mean completion time of the former was longer than that required for exercises b1 through b5.

In conclusion, we can confirm that the laparoscopy simulation box and the associated games designed by the Laboratory of Medical Engineering of the Faculty of Medicine at the University of Navarra are valid tools to train medical residents and undergraduate medical students in the use of instruments and performance of movements typical of the laparoscopic technique. Additionally, follow-up studies should be conducted to describe the impact of this simulator on the learning curve of surgery residents and to establish an estimate of training hours needed to achieve an expert level of proficiency.

Author contribution

ML, SF and AFA designed and developed the laparoscopic simulation box. BG, ND and FJP designed and developed the validation study. NM-C reviewed the statistical analyses and wrote the first draft of the manuscript. All the authors critically review the manuscript and approved its final version.

Funding sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors report no proprietary or commercial interest in any product mentioned or concept discussed in this article.