Usefulness of an Optical Tracking System in Laparoscopic Surgery for Motor Skills Assessment

Abstract

Introduction

The objective of this study is to assess the usefulness of an evaluation system of surgical skills based on motion analysis of laparoscopic instruments.

Method

This system consists of a physical laparoscopic simulator and a tracking and assessment system of technical skills in laparoscopy. Six surgeons with intermediate experience (between 1 and 50 laparoscopic surgeries) and 5 experienced surgeons (more than 50 laparoscopic surgeries) took part in this study. All participants were right-handed. The subjects performed 3 repetitions of a cutting task on synthetic tissue with the right hand, dissection of a gastric serous layer, and a suturing task in the dissection previously done. Objective metrics such as time, path length, speed of movements, acceleration and motion smoothness were analyzed for the instruments of each hand.

Results

In the cutting task, experienced surgeons show less acceleration (P=.014) and a smoother motion (P=.023) using the scissors. Regarding the dissection activity, experienced surgeons need less time (P=.006) and less length with both instruments (P=.006 for dissector and P=.01 for scissors). In the suturing task, experienced surgeons require less time (P=.037) and distance traveled (P=.041) by the dissector.

Conclusions

This study shows the usefulness of the evaluation system for the cutting, dissecting, and suturing tasks. It represents a significant step in the development of advanced systems for training and assessment of surgical skills in laparoscopic surgery.

Resumen

Introducción

En este trabajo se estudia la utilidad de un sistema de evaluación de destrezas quirúrgicas basado en el análisis de los movimientos del instrumental laparoscópico.

Método

El sistema consta de un simulador físico laparoscópico y un sistema de seguimiento y evaluación de habilidades técnicas quirúrgicas. En el estudio han participado 6 cirujanos con experiencia intermedia (entre 1 y 50 intervenciones laparoscópicas) y 5 cirujanos expertos (más de 50 intervenciones laparoscópicas), todos ellos con la mano derecha como dominante. Cada sujeto realizó 3 repeticiones de una tarea de corte con la mano derecha en tejido sintético, una disección de la serosa gástrica y una sutura en la disección realizada. Para cada ejercicio se analizaron los parámetros de tiempo, distancia recorrida, velocidad, aceleración y suavidad de movimientos para los instrumentos de ambas manos.

Resultados

En la tarea de corte, los cirujanos expertos muestran menor aceleración (p = 0,014) y mayor suavidad en los movimientos (p = 0,023) en el uso de la tijera. Respecto a la actividad de disección, los cirujanos expertos requieren menos tiempo (p = 0,006) y recorren menos distancia con ambos instrumentos (p = 0,006 para disector y p = 0,01 para tijera). En la tarea de sutura, los cirujanos expertos presentan menor tiempo de ejecución que los cirujanos de nivel intermedio (p = 0,037) y recorren menos distancia con el disector (p = 0,041).

Conclusiones

El sistema de evaluación se mostró útil en las tareas de corte, disección y sutura, y constituye un progreso en el desarrollo de sistemas avanzados de entrenamiento y evaluación de destrezas quirúrgicas laparoscópicas.

Introduction

Laparoscopic surgery is a surgical discipline with high technical requirements for the surgeon, including the use of new instruments,¹ the lack of tactile feedback, the reverse movement effect (known as the “fulcrum effect”), and loss of sense of depth. This poses a major challenge for the surgeon, i.e., acquiring new motor skills to be trained in performing safe surgical procedures.^2,3

It would be useful to know the psychomotor skills of surgeons undergoing training as an essential part in the assessment of surgical competence. For this reason, authors, surgeons and associations increasingly demand the development of assessment tools to certify surgeons as technically competent.^3–5

To meet this need, we have developed several training systems and surgical assessment to avoid endangering the patient, favoring their use outside the clinical environment, in addition to objectivity and automation of the surgeon assessment processes. Mainly, these systems are classified into 2 large groups: virtual and physical systems.⁶ In this paper, we will focus on physical training and surgical assessment systems because of their greater realism in the use of instruments, and in the interaction with the work environment.^7,8 Currently, there are available systems with several features and applications, including commercial and academic. However, there is no system that is universally extended or recommended as an objective tool for assessing surgical skills.³

Analyzing instrument movements is an effective method to objectively assess the surgeon's psychomotor skills.^6,9,10 During the learning task, the subject performs more efficient movements when he/she masters this task.¹¹ Several technologies have been used to record this information (electromagnetic, mechanical, optical) to track instruments during the assessment activity.^12–14

Various authors have analyzed diverse parameters based on the movement of the laparoscopic instruments to carry out an objective assessment of the surgeon's technical skills.^6,9 These parameters can predict the level of experience of the surgeon as accurately as using certified scales.¹⁵ To use these assessment measures, in order to differentiate between surgeons with different levels of experience, it is necessary to establish which of these are critical to evaluate each task.

The main objective of this study is to analyze the usefulness of a system to assess psychomotor skills in laparoscopic surgery, based on laparoscopic instrument motion analysis. This study was conducted in 2 surgeon groups with different experience in laparoscopic surgery performing a set of basic training tasks using a simulator, including synthetic tissue cutting, gastric serosa dissection, and organic tissue suture.

Method

The Simulator

The system consists of 2 modules: a physical simulator for laparoscopic skill training (SIMULAP^®, CCMIJU, Cáceres, Spain)^7,16 and a surgical technical skills tracking and assessment system (Fig. 1). The following items were used for the assessment tasks: laparoscopic shears (Endo Shears, Auto Suture, Covidien, Mansfield, MA, USA), a laparoscopic dissector (Richard Wolf GmbH, Knittlingen, Germany) and a laparoscopic needle-holder (Karl Storz GmbH & Co. KG, Tuttlingen, Germany).

Fig. 1.

Surgical skills training and assessment system: (a) Tracking system. (b) Laparoscopic instrumentation with artificial tracking marks. (c) Training simulator.

Laparoscopic Instrumentation Tracking System

This system consists of a third generation optical device (MicronTracker^®, 3Hx60, Claron Technology Inc., Toronto, Canada), which has been adapted for use with actual laparoscopic instruments. The system uses artificial marks identified in the visible spectrum. A support has been designed for each instrument used; supports come with 3 artificial marks providing a front and a side view of each instrument, to improve their tracking range (Fig. 2). The support's design is lightweight and sturdy, and does not interfere with the normal use of the instrument. Prior to data analysis, a Kalman smoothing filter was applied¹⁷ to the recorded signal of each instrument.

Fig. 2.

Detailed design and location of artificial marks in each type of instrument.

Subjects

The study involved 6 surgeons with intermediate experience (between one and 50 laparoscopic procedures), and 5 experienced surgeons (more than 50 laparoscopic procedures), all right-hand dominant. Lacking standard criteria to define surgical expertise, the classification value has been defined by the number of surgical interventions. Specifically, the intermediate surgeon group had performed a mean of 17.6 laparoscopic procedures (ranging from 7 to 25 interventions) and expert surgeons, a mean of 85 interventions (ranging from 50 to over 150 interventions). At the beginning of the activity, all participants completed a questionnaire on demographic and surgical experience.

Tasks and Assessment Parameters

Each subject performed 3 basic training laparoscopic tasks, with different degrees of complexity and no time limit for completion. First, a cutting task was performed on a synthetic fabric platen, following a set pattern and using the dominant hand. Next, a dissection of a porcine cadaver stomach serosa was conducted along a 3 cm incision, ensuring muscle layer separation. Finally, the dissection was sutured. The suturing task was performed in 4 phases: needle passage, one double knot suture, and 2 single knot sutures in opposite directions. All study conditions were standardized for all subjects. For cutting and dissection practice, a dissector was used with the left hand and shears with the right. For the suture practice, a dissector was used with the left hand and a needle-holder with the right. Subjects followed the task via a monitor that showed them the video from the camera inside the simulator.

For the assessment of the technical skills of each subject, a series of objective metrics^6,18 were calculated based on the use of laparoscopic instruments for each task (Table 1). 3 repetitions for each task were performed and used to provide the mean value for each assessment parameter for the left hand and right tool.

Table 1.

Description of the Assessment Parameters Analyzed.

Metrics	Definition	Value
Time (s)	Time to perform the task	T=tf−t0
Traveled distance (mm)	Length of the trajectory described by the tip of the instrument while performing the task	∫t=0Tdr→(t)dt2dt
Speed (mm/s)	Instrument position variation index with respect to time during the course of the task	1T∫t=0Tdrxdt2+drydt2+drzdt2
Acceleration (mm/s^2)	Instrument speed variation index with respect to time during the course of the task	1T∫t=0Td2rxdt22+d2rydt22+d2rzdt22
Movement smoothness (mm/s^3)	Changes in instrument accelerating causing sudden movements during the task	1T∫t=0Td3rxdt32+d3rydt32+d3rzdt32

Statistical Analysis

The comparative study between the assessment parameters was conducted between the 2 surgeon groups and for each task. Statistical analysis was performed using SPSS software (version 15.0, SPSS Inc., Chicago, IL, USA) and the U Mann–Whitney test was applied to compare the statistical differences between the two groups, using a significance value of P<.05.

Results

During the synthetic fabric cutting, significant differences were obtained in mean acceleration and mean smoothness parameters for movement in using laparoscopic shears (Table 2).

Table 2.

Comparison of the Motor Skills Assessment Parameters During the Synthetic Fabric Cutting Task.

Variable	Intermediate experience		Experts		P
	Mean value	CI	Mean value	CI
Time	100.7	73.5–127.9	90.0	80.5–99.6	.56
RH traveled distance	6882.1	4167.9–9596.5	5310.4	4787.6–5833.2	.298
LH traveled distance	2826.1	2508.8–3143.3	2308.8	1911.2–2706.4	.145
Max speed RH	590.4	352.9–827.9	555.4	381.0–729.9	.783
Max speed LH	234.2	192.3–276.0	217.8	141.5–294.2	.057
RH mean speed	67.4	59.6–75.3	60.2	56.0–64.5	.314
LH mean speed	29.5	24.0–35.0	26.3	21.9–30.8	.408
Max acceleration RH	4310.7	2689.6–5931.9	3321.9	2526.5–4117.2	.121
Max acceleration LH	2108.6	1723.2–2494.0	1841.2	1405.4–2277.1	.22
RH mean acceleration	438.1	397.0–479.3	367.6	354.5–380.8	.014^*
LH mean acceleration	208.6	163.1–254.2	174.1	146.2–202.1	.232
Max movement smoothness RH	29 037.4	18 424.2–39 650.6	27 701.8	22 369.5–33 034.1	.211
Max movement smoothness LH	13 632.7	11 315.8–15 949.6	12 761.5	9739.8–15 783.2	.44
RH mean movement smoothness	2285.5	2072.7–2498.4	1913.5	1829.9–1997.1	.023^*
LH mean movement smoothness	1191.4	950.2–1432.6	1016.3	855.7–1176.9	.121

CI: confidence interval at 95%; RH: right hand; LH: left hand.

*

P<.05.

With respect to dissection practice, time and distance values for both instruments are clearly lower for the group of expert surgeons (Table 3). However, for the organic tissue suturing practice, other than the time to perform the practice, only the traveled distance by the dissector shows significant differences between the two groups; both were lower in the case of the expert group (Table 4).

Table 3.

Comparison of the Assessment Parameters of Motor Skills for the Organic Tissue Dissection Task.

Variable	Intermediate experience		Expert		P
	Mean value	CI	Mean value	CI
Time	179.5	144.5–214.6	76.2	59.4–93.0	.006^*
RH traveled distance	8185.6	6872.7–9498.5	3712.0	2767.3–4656.7	.006^*
LH traveled distance	6628.8	5253.2–8004.5	3120.7	2088.5–4152.8	.01^*
Max speed RH	502.4	409.7–595.2	382.6	218.8–546.3	.308
Max speed LH	281.0	227.6–334.5	311.8	220.3–403.2	.497
RH mean speed	46.7	44.0–49.4	48.9	45.7–52.2	.396
LH mean speed	37.4	35.7–39.1	41.4	33.3–49.4	.174
Max acceleration RH	3165.9	2691.9–3639.8	2685.2	2252.7–3117.7	.308
Max acceleration LH	2248.9	1994.2–2503.7	2171.9	2012.9–2331.0	.734
RH mean acceleration	294.3	281.8–306.8	313.7	272.2–355.3	.168
LH mean acceleration	248.1	234.0–262.3	285.3	217.3–353.4	.153
Max movement smoothness RH	18 249.3	14 982.0–21 516.6	18 302.3	14 670.8–21 933.9	.865
Max movement smoothness LH	15 400.7	13 249.2–17 552.2	14 319.9	13 484.8–15 155.1	.497
RH mean movement smoothness	1558.8	1497.6–1620.0	1678.2	1475.9–1880.5	.234
LH mean movement smoothness	1341.7	1258.7–1424.8	1571.0	1192.5–1949.4	.182

CI: confidence interval at 95%; RH: right hand; LH: left hand.

*

P<.05.

Table 4.

Comparison of Motor Skills Assessment Parameters for the Organic Tissue Suturing Task.

Variable	Intermediate experience		Expert		P
	Mean value	CI	Mean value	CI
Time	204.5	162.9–246.0	135.8	97.4–174.3	.037^*
RH traveled distance	7465.7	5833.4–9098.0	6304.0	4188.8–8419.3	.522
LH traveled distance	8457.7	6556.7–10 358.6	5303.4	4436.6–6170.2	.041^*
Max speed RH	535.3	435.0–635.6	368.4	272.9–463.8	.055
Max speed LH	306.9	238.6–375.2	299.4	249.2–349.6	.873
RH mean speed	38.8	33.9–43.8	45.7	41.9–49.5	.078
LH mean speed	41.6	38.4–44.8	41.8	36.4–47.2	.873
Max acceleration RH	3144.2	2602.7–3685.6	2594.4	2050.9–3137.8	.262
Max acceleration LH	2577.0	2185.2–2968.7	2488.7	2109.7–2867.7	.631
RH mean acceleration	244.7	206.2–283.2	289.7	273.2–306.3	.15
LH mean acceleration	281.8	255.7–307.9	277.2	243.2–311.3	.749
Max movement smoothness RH	18 693.2	15 997.3–21 389.2	14 473.6	11 667.8–17 279.4	.109
Max movement smoothness LH	17 479.4	14 291.6–20 667.3	15 715.7	13 544.3–17 887.2	.423
RH mean movement smoothness	1306.8	1100.7–1512.8	1525.9	1447.1–1604.7	.2
LH mean movement smoothness	1515.7	1390.5–1640.8	1483.2	1313.0–1653.5	.631

CI: confidence interval at 95%; RH: right hand; LH: left hand.

*

P<.05.

Considering the learning curves for the suture practice (Fig. 3), intermediate level surgeons show a significant decrease in task execution time and in the traveled distance by the two instruments. However, they show an increase in maximum speed and acceleration in using the needle-holder, and the maximum movement smoothness for both types of surgical instruments. Skilled surgeons reflect a decrease in maximum acceleration and movement smoothness in using the needle-holder. Both groups show an increase in mean acceleration and smoothness of movement in using the dissector.

Fig. 3.

Learning curves for motor metrics analyzed during the suture task. Results of the intermediate and expert level surgeon group with respect to right hand (RH) and left hand (LH) instrument.

Discussion

The growing need to develop new objective assessment methods for laparoscopic surgical skills has led to the development of several assessment systems.^19,20 However, this task requires more automatic procedures, such as those based on the analysis of instrument movements. To develop these types of systems, either for use in image-guided surgery,²¹ robotic surgery²² or for skills assessment, various tracking technologies have been used, such as electromagnetic,¹³ mechanical,²³ optical²⁴ or video based systems.^12,14

This paper proposes a new design for implementing a third generation optical tracking system for training and use in the objective assessment of psychomotor skills in laparoscopic surgery. We analyzed the usefulness of the system in 3 basic training practices. Thus, we can determine whether the system is able to differentiate between surgeons with intermediate and expert level experience. The results obtained in previous studies, with respect to the accuracy of the tracking system used, reflect values suitable for use as a tool for instrument movement analysis, although improved accuracy is required for use in image-guided surgery systems.²⁵ The system introduced provides a unique solution with respect to the objective and automatic assessment of surgical skills, and does not interfere with the instrument's ergonomics.

With respect to the results obtained in this study, no assessment parameter showed significant differences in the 3 tasks assessed. This leads us to think that it is not possible to accurately determine the level of expertise of the surgeon with a single assessment measure for the 3 tasks assessed.¹⁰ Chmarra et al., also established in previous studies that it would be advisable to assess psychomotor skills of the surgeon with at least 2 assessment parameters.⁸

The effectiveness of the system in assessing the cutting task has been demonstrated in terms of the mean movement acceleration and smoothness metrics in using the laparoscopic shears. For the same practice, Pellen et al., apart from movement smoothness, obtained significant differences with respect to the time and the mean traveled distance by both instruments.²⁶ With respect to this study, we consider that factors such as previous experience in basic laparoscopic tasks of both surgeon groups and the simplicity of the analyzed task meant that no more differentiating parameters occurred between the groups.

Skilled surgeons require less than half the time required by surgeons with intermediate experience to complete the dissection task. In addition, the expert surgeon group shows a higher mean speed in using both instruments. However, this group also shows expeditious movements in using both types of instruments. This may be because experience implies faster movements, and sometimes, with less delicacy, compared to possible uncertainty at particular times in the intermediate level surgeon group.

For the suture task, statistically significant differences were obtained in the time and traveled distance parameters by the dissector. These results coincide with those obtained from the work by Yamaguchi et al.²⁷ for surgeons with different experience levels. Van Sickle et al.²⁸ analyzed the needle passage using a latex pattern, a task that produced significant differences in the execution time, mean traveled distance and mean smooth movement of both instruments for novice and expert surgeon groups.

The suture task has been assessed previously by an electromagnetic device attached to the palm of the surgeon, which has proven useful in the assessment of skills with respect to execution time and number of moves metrics.^29,30 In previous studies,¹³ we used the same device but, in this case, to analyze the movement of instruments during a laparoscopic suturing task conducted by novice, intermediate and expert surgeons. The suture task was divided into 4 subtasks, as we have mentioned in this study, and analyzed individually. The results for the surgeon group with intermediate experience and experts showed significant differences in the traveled distance by the needle-holder during the double knotted subtask. However, no significant differences were obtained with respect to the use of the dissector, as described in the results from the work by Yamaguchi et al.,²⁷ Van Sickle et al.²⁸ or in this study. Consequently, this leads us to think that, besides the execution time of the task, the traveled distance by the instruments is a factor in most studies examining laparoscopic suturing, and that extending the study sample would be appropriate to define more precisely what instrument (dissector, needle-holder or both) is crucial in differentiating the level of technical skill among surgeons for this task.

If we analyze the evolution of both groups during the suture task, we see definite improvement of the surgeons with intermediate experience with respect to the execution time and the traveled distance by the two instruments. Expert surgeons’ movements are faster when using the needle-holder; this seems to be clearly related to the skill level for this task. However, both groups have a similar rate of movement for the use of the dissector. The 2 groups show negative progression in the increase of the mean acceleration and mean movement smoothness in using the dissector, resulting in an increase in sudden changes of speed and acceleration in their movements. Moreover, skilled surgeons display smooth acceleration and higher mean movements in using the needle-holder. As has happened in the dissection practice, it may be that experience encourages faster movements, and sometimes, less delicate movements.

This first approach allows us to have a system that is able to discern between levels of experience. However, if we carry out a full assessment of surgical competence, apart from the surgical skills of a surgeon, we must take into account other factors, such as knowledge and readiness against possible complications, among others. This is part of a future challenge facing the development of advanced systems for assessing surgical skills in the work we have begun.

In conclusion, this study shows the usefulness of the system introduced for objective assessment of laparoscopic psychomotor skills with respect to the 3 analyzed tasks, i.e., cutting into inorganic tissue, dissection and suturing organic tissue gastric serosa for surgeons with intermediate and expert level experience. The ultimate objective is to transfer these learned and assessed skills to the real surgical environment. Future work plans include increasing the study sample, testing subjects without previous experience, as well as increasing the number of parameters to evaluate.

Financing

This work was financed in part by the Department of Employment, Enterprise and Innovation of the Government of Extremadura and the European Social Fund.

Conflict of Interest

The authors declare having no conflict of interest.