To describe, validate, and evaluate the effectiveness of an in-house-developed mannequin versus standard commercial mannequins for emergency medicine training among undergraduate medical students.
Materials and methodsFirstly, we developed a 3-in-1 model consisting of 3 modules (sampling techniques, fluid resuscitation and IV drug administration, and minor surgery). Forty participants of 2nd-year medical students were enrolled. Each student completed three scenarios through the objective structured clinical examination (OSCE) simulation assessment. The questionnaire was used to rate the mannequins' overall quality, usability, comparability, cost efficiency, and realism.
ResultsAlthough students using an in-house-developed mannequin had similar OSCE scores to a standard mannequin, students rated an in-house-developed mannequin easier to use. On the other hand, most students agreed that the standard commercial mannequin was more realistic than a substitute one. The costs of the materials needed for mannequin fabrication were less than 100 USD, and it was functionally comparable.
ConclusionAn in-house-developed mannequin was well accepted by students and teachers and could be used to deliver and assess clinical skills for medical students effectively at low-cost.
describir, validar y evaluar la efectividad de un maniquí desarrollado internamente versus maniquíes comerciales estándar para la capacitación en medicina de emergencia entre estudiantes de medicina de pregrado.
Materiales y métodosen primer lugar, desarrollamos un modelo 3 en uno que consta de 3 módulos (técnicas de muestreo, reanimación con líquidos y administración de fármacos por vía intravenosa y cirugía menor). Se inscribieron 40 participantes de estudiantes de medicina de segundo año. Cada estudiante completó 3 escenarios a través de la evaluación de simulación del examen clínico estructurado objetivo (OSCE). El cuestionario se utilizó para calificar la calidad general, la facilidad de uso, la comparabilidad, la rentabilidad y el realismo de los maniquíes.
Resultadosaunque los estudiantes que usaron un maniquí desarrollado internamente obtuvieron puntajes OSCE similares a los de un maniquí estándar, los estudiantes calificaron un maniquí desarrollado internamente como más fácil de usar. Por otro lado, la mayoría de los estudiantes estuvo de acuerdo en que el maniquí comercial estándar era más realista que uno sustituto. Los costos de los materiales necesarios para la fabricación del maniquí eran inferiores a 100 USD y eran funcionalmente comparables.
Conclusiónun maniquí desarrollado internamente fue bien aceptado por estudiantes y profesores y podría usarse para brindar y evaluar habilidades clínicas para estudiantes de medicina de manera efectiva a bajo costo.
Medical doctors require sufficient clinical competencies in medical emergencies. Theoretically, the most effective way to attain clinical skills is through a direct encounter with the patients. However, critical cases are associated with medicolegal problems, making it impossible for training purposes.1 Recently, patient and mannequin-based simulations have been proven to be an effective method for teaching in medical schools,2 although the degree of realism has not been entirely accepted.3 There is rapid growth of various commercially available mannequins in the market, allowing students to familiarize themselves with different types of clinical procedures, such as venipuncture, resuscitation, and intubation.
Although it has been shown that the use of mannequins improves students' performance, clinical comprehension, and ethical practice, commercial mannequins and its associated technology are often expensive and require extensive maintenance. Hence, these mannequins may either not be available in many schools, particularly in low and middle-income countries, or the schools must selectively decide on several mandatory models.4 Although a previous study demonstrated that there is no significant difference in terms of students' performance between the use of high and low-fidelity mannequins,5 there is still a lack of scientific evidence regarding the comparability of an in-house-developed mannequin in terms of learning success and realism. Hence, our study aims to describe the development, validation of an in-house-developed (substitute) mannequin, and to investigate the learning outcomes of medical emergency training in undergraduate medical students using an in-house-developed (3D-printed) mannequin versus standard commercial mannequins.
Materials and methodsDevelopment of an in-house mannequinAn in-house developed mannequin was developed according to 3 OSCE modules, 1) sampling techniques, 2) fluid resuscitation and IV drug administration, and 3) minor surgery. Briefly, the model was 3D-printed onto four parts and then assembled. The transparent tube was connected to a water container containing red-stained water (as blood), a catheter (as vein or artery), water aerator equipped with a potentiometer and adaptor. Melted sealant glue was used to fill the holes in the model (as transparent skin), while polyvinyl alcohol (PVA) cloth was implemented as the skin for suturing technique. The model of an in-house developed mannequin and all materials used to develop the mannequin are depicted in Fig. 1 and Table 1.
Material used for mannequin fabrication.
| No | Mannequin part | Material | Cost (in USD) |
|---|---|---|---|
| 1 | Hand | 3D-printed model | 47.20 |
| 2 | Vein and artery | 18 Fr Catheter (a total of 4) | 2.70 |
| 3 | Transparent skin | Sealant glue gun | 6.08 |
| 4 | Skin | PVA cloth | 1.35 |
| 5 | Transparent tube | Flexible PVC tube, 1 m | 0.67 |
| 6 | Reservoir | Empty plastic container (recycled) | 0.00 |
| 7 | Pressure control |
| 13.152.3618.88 |
| Total | 92.74 |
This study is an experimental research project conducted with a total of 40 participants, all of whom are 2nd-year undergraduate medical students. The study was performed according to the ethical principles of the Helsinki Declaration (Ethical Principles for Medical Research Involving Human Subjects) and was approved by the medical ethic committee.
EvaluationStudents were asked to rate the mannequins' overall quality, usability, and comparability on a 5-point Likert scale and open questions. The usability of the mannequins was compared in each module. In addition, the teachers were asked and interviewed regarding their perceptions toward the development of a new mannequin for teaching and examination purposes. The detailed questionnaires are presented in Supplementary Tables 1 and 2.
OSCE simulation assessmentThe OSCE simulation consisted of 3 scenarios, one dealing with a patient requiring arterial blood gas analysis (ABG) due to a metabolic disorder (Scenario 1), one patient suffering from hypovolemic shock (Scenario 2), and the third one involving a patient suffering skin lacerations due to a motor vehicle accident (Scenario 3). Assessment requires all participants to perform each scenario using both in-house (A) and standard (B) mannequins. The standard mannequin was initially used, followed by the in-house mannequin on a rotating basis. The participants had a 10-minute time frame to complete each task. Students' performance were evaluated using a well-defined checklist assessing their competencies.
Statistical analysisThe questionnaires were developed using Google form, and the data were then extracted in Excel format. The scores given for in-house and standard mannequins-based OSCE scenarios were compared using the 2-tailed Wilcoxon signed-rank test. A paired t-test was used to compare the OSCE simulation score in each scenario. All the graphs and statistical analyses were performed using the SPSS Ver. 25 and Graphpad Prism 9 for Mac. The Likert figure was developed using an online Likert free tool provided by likertplot.com.6 The free comments were analyzed according to the principles of qualitative content analysis.
ResultsResults of the OSCE simulationAlthough the score of students using an in-house developed mannequin (A) tended to be lower than a standard commercial mannequin (B) in scenario 1 (83.21 ± 8.89 vs. 84.50 ± 9.47), no statistical difference was observed (p = 0.540, Fig. 2A). Similarly, no significant differences regarding the results between the two groups were found, either in scenario 2 (76.48 ± 7.14 vs. 75.00 ± 6.37, p = 0.325, Fig. 2B) and scenario 3 (80.48 ± 10.61 vs. 79.63 ± 8.17, p = 0.685, Fig. 2C). Teachers agreed that the use of an in-house developed mannequins had similar results in achieving the learning objective (LO) of each module (Q1, Supplementary Table 1), which was supported by the statement, "although the appearance were not as realistic as a standard mannequin, I believed that this substitute mannequin had a similar function and is useful for students".
Evaluation of mannequins
Unexpectedly, our data showed that the quality of mannequin A was significantly higher than mannequin B (Fig. 3). This is possibly related to the general statement collected from Q9 (see Supplementary Table 2), "mannequin A was more practical and convenient because it contained three different clinical procedures in a single model". Students agreed that mannequin A was significantly easier to use compared to mannequin B in all scenarios (sampling technique, fluid resuscitation, and minor surgery technique, Fig. 3). Indeed, students thought that "the vein and artery were more visible in mannequin A than B; therefore, it was easier to perform blood collection or fluid resuscitation" (Q9, Supplementary Table 2). Although most students acknowledged that mannequin A and B were comparable (Fig. 3), it was worth noting that students felt mannequin A needed more improvement, marked by students' testimony stating "the appearance of mannequin A seems unrealistic compared to B"; "mannequin B feels like holding real patients" (Q10, Supplementary Table 2).
Cost analysis
A standard commercial mannequin used in the present study costs approximately USD 3108.63 for fluid resuscitation or arterial blood sampling training (Kyoto Kagaku) and USD 1013.68 for strap-on limb skins suture training (Kyoto Kagaku). Usually, we performed teaching/training in small groups and OSCE cycle in parallel, and thus a total of at least four mannequins are required. In contrast, an in-house developed mannequin containing 3 in 1 of the above models only cost USD 92.74 (Table 1), which costs 78 times less than the standard mannequin. In addition, another point to consider was that the maintenance and disposable materials in mannequin A were more affordable than in mannequin B. Our interview conducted with 24 lecturers (Q2, Supplementary Table 1) confirmed agreement among all participants indicating that a substitute mannequin was cost-effective, quoting, "the appearance of mannequin A might be not perfect, but it seems to me that it was functional, low-cost, and versatile to achieve minimum competencies in emergency medicine".
DiscussionThe implementation of mannequins as teaching media is becoming more important in medical school to introduce and obtain clinical skills for students.7 Although there is a variety of commercially available mannequins in the market, the effectiveness of these models in the teaching process and curricula remains unclear. In this present study, students' performance, effectiveness, perceived realism, and cost-analysis derived from both an in-house developed (substitute) and standard commercial mannequin in emergency medicine training for undergraduate medical students were analyzed. Most students believed that a standard mannequin was more realistic than the substitute one. This is possibly because the substitute mannequin did not reflect the color and elasticity of human skin. However, despite the less realistic appearance, students' performance did not differ between the two groups. In fact, the substitute mannequin outperformed the standard mannequin in terms of usability, implying that a substitute mannequin is just as effective as a standard mannequin in attaining minimum clinical skills competencies. Despite the positive outcomes observed, it is important to acknowledge that the use of less realistic mannequins may discourage students from fully engaging with real clinical cases and hinder their learning experience. Numerous studies have demonstrated that utilizing specific simulation scenarios based on standardized patients (SPs) leads to improved performance and a heightened sense of realistic experience for the students.2,8 Therefore, further improvement and evaluation are warranted in the future.
Currently, there is limited data regarding the comparison between substitute and standard mannequins on teaching success. Similar to our findings, Al-Wassia et al.9 demonstrated that a low-fidelity mannequin did not affect the students' scores and technical skill performance. Moreover, although high-fidelity mannequins may improve students' confidence, no evidence indicates that students who used high-fidelity mannequins increase their performance more than those who used low-fidelity mannequins.5 Indeed, our data suggest that a substitute mannequin was comparable with a commercial mannequin in terms of functionality. Hence, the choice of teaching and evaluation media should be assessed by their suitability according to the LO.
The costs of the materials needed to develop a substitute mannequin were less than 100 USD, being seventy times less expensive than the price of three commercial mannequins. It should be noted that similar to the standard mannequins, spare parts, such as a catheter, sealant glue gun, and PVA cloth, are still required. Nonetheless, the price of spare parts in the substitute mannequin is relatively lower than in a commercial one. Thus, our new substitute mannequin is economically affordable, sustainable, functionally comparable, enables the initiation of several skills related to emergency medicine for every medical student, and can be implemented in any medical school around the world.
There are several limitations identified in the current study. Firstly, during OSCE simulation, students may have prior knowledge on how to handle the commercial medium, which could have influenced the results of the study. Secondly, it was observed that the level of difficulty associated with the substitute mannequin was comparatively lower than that of the commercial one. This difference in complexity could potentially influence the student's perception of the ease of use. Thirdly, our developed mannequin is far away from reality. Therefore, further modification and improvement are necessary. Finally, it should be noted that the investigators conducting this study also serve as lecturers for the students. This dual role may potentially introduce a bias or affect the accuracy of the responses provided by the students. Altogether, we conclude that the in-house-developed, low-cost, multi-module mannequin was well accepted by students and teachers and could be used as an alternative to commercial mannequins in teaching and assessing the students' skills and competencies in emergency medicine-related topics, particularly in low- and medium-resources countries.
Ethical approvalThe study was performed according to the ethical principles of the Helsinki Declaration (Ethical Principles for Medical Research Involving Human Subjects) and was approved by the medical ethic committee.
Author contributionsZU and FN conceived and designed the research.
ZU, FN, and AH designed and developed the mannequin.
ZU processed and analyzed the data.
CI, AA, and ZU collected the data.
ZU and GS wrote and revised the manuscript.
ZU is the main contributor in this manuscript.
FundingThis research did not receive any specific grant from funding agencies in the public, commercial, or not for profit sectors.
Supplementary material
