Systematic reviews and meta-analyses are critical in Evidence-Based Medicine, representing the highest level of scientific evidence. Despite the inclusion of Research Methodology and Statistics in medical curricula, the publication rate among residents remains low. This study focuses on residents in the Medical Genetics specialty in Mexico City, illustrating challenges faced in a developing country.
MethodsUsing both qualitative and quantitative methods, from the perspective of the Curriculum Action Research movement, the study surveyed 25 residents from eight hospitals in Mexico City. The information was collected using the proposed evaluation instrument, which was modified for validation.
ResultsIn this study, we showed that 64% had published work, but only 12.5% had completed systematic reviews. A significant proportion perceived not receiving formal training in Research Methodology (4%) or Statistics (52%). Key barriers identified were lack of time (56%), insufficient research experience (46%), cost or lack of resources (44%), and the administrative procedures required being very complicated (44%). The study developed and validated a questionnaire to assess knowledge, which demonstrated good reliability and construct validity with a Cronbach’s alpha of 0.85, a Kaiser-Meyer-Olkin (KMO) measure of 0.505 and a significant Bartlett's test of sphericity (p<0.0001).
ConclusionsThe findings highlight substantial gaps in practical research skills and suggest that the current curriculum inadequately integrates Research Methodology and Statistics. This underscores the need for more effective incorporation of these subjects into training programs to enhance residents' research capabilities. Addressing these gaps could improve publication rates and overall research output among medical residents.
Las revisiones sistemáticas y los meta-análisis son fundamentales en la Medicina Basada en Evidencia, representando el nivel más alto de evidencia científica. A pesar de la inclusión de la Metodología de la Investigación y la Estadística en los programas académicos de Medicina, la tasa de publicación entre los residentes sigue siendo baja. Este estudio se enfoca en los residentes de la especialidad de Genética Médica en la Ciudad de México, ilustrando los desafíos enfrentados en un país en desarrollo.
MétodosUtilizando métodos cualitativos y cuantitativos, desde la perspectiva del movimiento de Investigación-Acción Curricular, el estudio encuestó a 25 residentes de ocho hospitales en la Ciudad de México. La información se recopiló utilizando el instrumento de evaluación propuesto, el cual fue modificado para su validación.
ResultadosEn este estudio mostramos que el 64% de los estudiantes de especialidad había publicado trabajos, pero solo el 12.5% había realizado revisiones sistemáticas. Una proporción significativa percibió no haber recibido formación formal en Metodología de la Investigación (4%) ni en Estadística (52%). Las principales barreras identificadas fueron la falta de tiempo (56%), la experiencia insuficiente en investigación (46%), el costo o la falta de recursos (44%) y que los procedimientos administrativos requeridos eran muy complicados (44%). El estudio desarrolló y validó un cuestionario para evaluar el conocimiento en esta área, el cual demostró una buena fiabilidad y validez de constructo, con un alfa de Cronbach de 0.85, una medida de Kaiser-Meyer-Olkin (KMO) de 0.505 y una prueba de esfericidad de Bartlett significativa (p<0.0001).
ConclusionesLos hallazgos destacan brechas importantes en las habilidades prácticas en investigación y sugieren que el currículo actual integra de manera insuficiente la Metodología de la Investigación y la Estadística. Esto subraya la necesidad de una incorporación más efectiva de estos temas en los programas de formación para mejorar las capacidades de investigación de los residentes. Abordar estas brechas podría mejorar las tasas de publicación y la producción general en investigación entre los residentes médicos.
The imperative of publishing is becoming increasingly prominent among medical specialty residents. In the United States, this pressure starts during medical school, where a robust publication record is often seen as a key factor in securing residency positions. Notably, candidates are now expected to have an average of 6.2 research outputs to increase their competitiveness.1 Systematic reviews and meta-analyses offer the highest level of scientific evidence within Evidence-Based Medicine. Therefore, it is crucial that medical specialty training programs provide trainees with the essential tools and skills to conduct these advanced research methods effectively.2
The Medical Genetics specialty in Mexico City is accredited by the National Autonomous University of Mexico (UNAM) and is conducted through eight prestigious institutions: Hospital General de México Dr. Eduardo Liceaga, Hospital Infantil de México Federico Gómez, National Medical Center Siglo XXI, National Medical Center La Raza, National Medical Center 20 de Noviembre, Salvador Zubirán National Institute of Medical Sciences and Nutrition, National Institute of Pediatrics, and Luis Guillermo Ibarra Ibarra National Rehabilitation Institute.
The Mexican Association of Human Genetics (MAHG) hosts a series of presentations, both oral and poster, as part of its annual National Congress of Human Genetics (NGHG). This platform gives medical residents an invaluable opportunity to showcase their research, fostering wider dissemination of their work. Over the past eight years (2016–2023), a total of 1405 dissemination works were presented, of which only eleven were systematic reviews, accounting for approximately 0.78% of the total. In contrast, case reports or case series emerged as the most frequently presented type of work.3–10
The Medical Genetics specialization program, accredited by UNAM, integrates a Research Seminar from the first year and throughout the entire three-year duration of the specialization. According to the program, this Seminar will be allocated two hours per week, totaling 5% of the resident's overall training hours by the end of the program. The Seminar encompasses comprehensive coverage of Research Methodology and Statistics.11 This approach to curriculum design is referred to as vertical integration, where a subject is taught in isolation with continuity over time. However, many experts argue that horizontal integration—where topics from one subject are interwoven with those from other subjects within the same timeframe—may be a more effective strategy. Ideally, a mixed approach, often termed spiral integration, which combines both vertical and horizontal elements, would offer the most comprehensive educational framework.12 Several models of integration have been proposed, including co-teaching support and comprehensive mentoring. All these models are grounded in a thorough analysis of the needs and opportunities for improvement identified in the target students.13–16
MethodologyThis exploratory study, employing both qualitative and quantitative methods, aimed to develop a tool for identifying knowledge gaps among residents in the Medical Genetics specialty that hinder their ability to conduct systematic reviews or meta-analyses. The research question was What are the knowledge breaks faced by residents in the Medical Genetics specialty in Mexico City regarding the methodology and statistics necessary for conducting systematic reviews or meta-analyses? It was carried out from the perspective of the Curriculum Action Research movement as a cross-sectional study for the 2022–2023 academic year, it involved prospective data collection via an asynchronous, self-administered questionnaire.17 The study targeted first-, second-, and third-year residents across eight hospitals in Mexico City where the Medical Genetics specialty is offered. A total of 65 resident physicians were invited to participate.
The study protocol adhered rigorously to the Mexican Official Standard NOM-012-SSA3–2012 and the Federal Law on the Protection of Personal Data Held by Private Parties, which represent the current national regulations. Data collection was conducted using non-probabilistic convenience sampling over approximately three months through the Google Forms application. After securing informed consent, which included a detailed explanation of the study's objectives, participants completed a measurement instrument comprising three sections. The first section gathered information on their academic background, experience in methodology and statistics, and perceived limitations or barriers to producing scientific publications.
The subsequent two sections of the questionnaire were designed based on an extensive literature review, ensuring content validity. The general variable, “knowledge required for conducting a systematic review/meta-analysis,” was evaluated through two dimensions. The “Methodological” dimension included ten items that assessed both theoretical and practical knowledge in research methodology essential for performing systematic reviews or meta-analyses. The “Statistical” dimension comprised ten items that evaluated the theoretical and practical knowledge in statistics necessary for these tasks. The instrument comprised closed-ended questions with four response options, including an “I am not sure” option for students to indicate their awareness of their training needs. To assess the reliability of the questionnaire, internal consistency was measured using Cronbach's alpha in a pilot group of seven students. The overall Cronbach's alpha coefficient was 0.86, with the Methodology section scoring 0.77 and the Statistics section scoring 0.71. These results indicate that the instrument demonstrated sufficient reliability for its intended application.
Statistical analysis of the results was conducted using descriptive statistics with Excel, XLSTAT, and STATSTM2.0 software. To assess the construct validity of the proposed instrument, an exploratory factor analysis was performed via Principal Component Analysis (PCA). This analysis followed Bartlett's Test of Sphericity to confirm data factorability and the Kaiser-Meyer-Olkin (KMO) measure to evaluate sampling adequacy. Subsequently, we analyzed the dimensions that explained the greatest variance (>50%) and the items with the highest factorial loadings (>0.4) to finalize the adaptation of the questionnaire. This study aims to offer a descriptive overview of the knowledge possessed by Medical Genetics residents in Mexico City regarding systematic reviews and meta-analyses. Additionally, we seek to validate and present an instrument (Supplementary Material 1) for measuring this knowledge, which can be extrapolated and applied to other medical specialties or related fields.
FindingsA total of 25 students (38.4%) agreed to complete the survey, yielding a confidence level of 80% with a maximum acceptable error of 10%. Of the respondents, 13 (52%) identified as female and 12 (48%) as male, with a mean age of 29 years (range 26–34). Representation was achieved from each of the eight hospitals in Mexico City that offer the Medical Genetics specialty.
Regarding the academic profile of the sample, six individuals (24%) reported having completed another postgraduate course prior to their specialization. Sixteen participants (64%) had produced scientific or academic publications, with case reports being the most common type (75%). Only two individuals (12.5%) conducted a systematic review. Nineteen participants (76%) had presented work (either oral presentations or posters) at an NGHG. Notably, only one participant (4%) indicated that they had not received any training in Research Methodology during their high school, undergraduate, or Medical Genetics specialization studies. Additionally, thirteen participants (52%) reported not having received any training in Statistics (either Descriptive or Inferential) throughout their high school, undergraduate, or specialization education.
The primary perceived limitation for publishing during residency was a lack of time (56%), followed by insufficient research experience (46%), costs or lack of resources (44%), and complicated administrative procedures (44%). Other noted constraints included competing priorities such as study, work, or family responsibilities (36%) and the absence of a competent mentor (32%). No participants cited personal disinterest as a limitation, and only one individual (4%) considered the endeavor not useful.
The overall average score on the twenty-item questionnaire was seven correct answers (range 0–18). Within the Research Methodology section (10 items), the average number of correct answers was 3.8, with an average of 1.8 errors and 4.4 responses indicating uncertainty. In the Statistics section (10 items), the average number of correct answers was 3.2, with an average of 2.4 errors and 4.4 uncertain responses (see Fig. 1).
The overall Cronbach's alpha for the complete questionnaire was 0.85. Bartlett's test of sphericity was significant (p < 0.0001), leading to an item analysis through Principal Component Analysis (PCA). This analysis identified three dimensions that accounted for 53.79% of the variability, each with an eigenvalue greater than 1.5. Items with factor loadings below 0.4, those loading on multiple dimensions, or those with lower loadings in dimensions with a high number of items were removed. Consequently, questions 2, 12, 16, 19, and 20 were excluded, resulting in a refined questionnaire with fifteen items (Supplementary Material 1). The final version comprised three dimensions —two with two items each and one with eleven items— achieving a Kaiser-Meyer-Olkin (KMO) measure of 0.505, a significant Bartlett's test of sphericity (p < 0.0001), and maintaining a Cronbach's alpha of 0.85.
DiscussionThe students' perceptions in the sample do not align with the academic program of the Medical Genetics specialization. While nearly all participants reported having attended Research Methodology (with only one student noting a lack of such courses), only 48% indicated they had received training in Statistics. This discrepancy may be attributed to variations in how the academic program is implemented across different hospitals. However, a more comprehensive study is needed to clarify this issue. In the sample analyzed, just over half (64%) of the students reported having at least one publication, and 76% had presented at least one academic work at an NGHG. Notably, only two participants conducted a systematic review. The most common type of publication, due to its practicality and ease of preparation, was the case report, which accounted for 75% of the publications in our sample. Although case reports contribute to the expansion of scientific knowledge, their evidence contribution is generally considered lower compared to systematic reviews and expert opinions.
Several studies have assessed the key limitations faced by residents in conducting publications or research. Chan et al. reported that among Internal Medicine residents in Singapore, 85% identified lack of time as the primary constraint. Other significant limitations, with over 50% agreement, included insufficient skills in statistical analysis, limited research experience, and inadequate training in Research Methodology.18 Dahn et al. evaluated perceived barriers among Canadian Radiation Oncology residents, revealing that 40% strongly agreed that their clinical workload was excessively high. Additionally, 31% cited a lack of protected time for research activities, 29% reported interference from competing academic responsibilities, and 23% noted insufficient experience in developing research proposals or training in medical manuscript writing.19
In Latin America, there are few studies on this topic. Merino et al. investigated a sample of students from the Residency Programs at the Faculty of Medicine, Pontifical Catholic University of Chile. Consistent with previous research, they identified a lack of protected time as the primary perceived barrier. Additionally, they found that competing commitments, such as other studies, family, and work, further exacerbated this issue. Contrary to earlier studies, Merino et al. also highlighted the impact of costs and the absence of dedicated funding for research activities as significant obstacles.20
In this study, the primary perceived limitation for publishing during residency was a lack of time, as reported by 56% of participants. This issue appears to be a common challenge among residents globally. The second most frequent barrier was insufficient research experience, identified by 46% of respondents, which aligns with findings from residents in other countries. Additionally, 44% cited the costs or lack of funding and the complexity of administrative procedures as significant obstacles. While these administrative challenges may be particularly pronounced in Latin America, further research is needed to confirm this hypothesis.
To evaluate the knowledge required for conducting systematic reviews/meta-analyses—one of the identified barriers—a questionnaire was developed, initially comprising two sections with 10 items each, as outlined in the Methodology section. The results indicate that students demonstrate a reasonable awareness of their knowledge gaps, with 44% in both areas acknowledging uncertainty about questions they do not know. There were no significant differences in the average number of correct answers between Research Methodology and Statistics (38% versus 32%) or in the number of errors (18% versus 24%) (see Fig. 1). The study revealed a modest proficiency in Research Methodology compared to Statistics, likely reflecting a greater perception of having covered more content in the former area.
A Principal Component Analysis (PCA), a widely utilized method for construct validation with numerous precedents in the literature,21–23 identified three factors that accounted for 53.79% of the variability, encompassing a total of fifteen items. By analyzing the identified items, we observed that the first domain primarily encompassed theoretical definitions and concepts, leading us to rename it as “Theoretical-Conceptual Aspects.” The second domain, which included eleven items, focused on tools and practical exercises for graph interpretation, and was thus renamed “Practical Aspects.” The final domain addressed the correct application of effect models (random effects model versus fixed effects model) based on heterogeneity in meta-analyses and was therefore renamed “Effect Measurement.”
The revised final questionnaire is proposed as a tool for more precisely identifying areas of strength and weakness among students in a specialty regarding their ability to conduct systematic reviews and meta-analyses. With a Kaiser-Meyer-Olkin (KMO) measure of 0.505, a significant Bartlett's test of sphericity (p < 0.0001), and a Cronbach's alpha of 0.85, this instrument reveals that the domain of “Theoretical-Conceptual Aspects” is the strongest area in this sample, with an average of 66% correct answers. Conversely, the “Practical Aspects” domain showed an average of 32% correct answers and 44% “I'm not sure” responses, indicating a need for further reinforcement. The “Effect Measurement” domain exhibited the most significant knowledge gap, with only 14% correct answers and 62% “I'm not sure” responses, highlighting this as the area with the greatest need for improvement (see Fig. 2).
These results underscore a concerning inefficiency in the vertical integration of Research Methodology and Statistics within the Medical Genetics specialty curriculum. This integration is perceived as an isolated subject, disconnected from the day-to-day clinical activities. Consequently, the evaluations for this area are likely to be predominantly theoretical. A more effective approach would involve horizontal integration, embedding these topics throughout all subjects and daily activities. Kern's six-step approach, which has been successfully employed in medical education to integrate new skills and educational products, could serve as a model for enhancing the functional integration of Research Methodology and Statistics into the curriculum.24 The six steps of this approach are identifying the problem and assessing general needs, evaluating specific needs, setting goals and objectives, developing educational strategies, implementing the plan, and conducting evaluation and feedback.25
This paper has addressed the initial two steps of the approach by identifying the primary barriers perceived by students in conducting systematic reviews and meta-analyses and highlighting the inadequate integration of these areas within the academic program. The subsequent objective should be to achieve a more horizontal integration of this knowledge, with a particular emphasis on practical application. While numerous examples of educational strategies, as well as methods for implementation, evaluation, and feedback aimed at this goal, are available in the literature, they are beyond the scope of this publication.14–16
ConclusionIn this study, we identified lack of time and insufficient experience or knowledge in research and statistical analysis as the primary perceived barriers to publication among Medical Genetics specialty students, reflecting similar findings in other global studies. However, the issue of limited resources, which was highlighted in our study and one other conducted in Chile, maybe a distinctive challenge in the Latin American context. Furthermore, despite the inclusion of a Research Seminar in the Medical Genetics specialization curriculum, our findings indicate that the integration of research and statistical methodologies are inadequate. We propose a validated questionnaire designed to assess the knowledge required for conducting systematic reviews and meta-analyses, which could be applied across various medical specialties.
The evaluation of knowledge using the developed questionnaire reveals that, while there is a solid grasp of theoretical concepts, there is a notable deficiency in practical training, particularly in critical areas such as effect measurement. This highlights the need for a comprehensive reform in how research skills are integrated into the curriculum for the Medical Genetics specialty—and likely other medical specialties as well. Adopting a more horizontal and contextualized integration of Research Methodology and Statistics into clinical training, guided by Kern's six-step approach, could address these deficiencies effectively. Implementing educational strategies that target these gaps and applying a structured learning framework could significantly enhance residents' preparation and boost the production of high-impact research in the medical field.
Declaration of generative AI and AI-assisted technologies in the writing processDuring the preparation of this work, the author(s) used ChatGPT to improve the readability and language of the manuscript. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.
To my colleagues in the Medical Genetics specialty. Because their valuable participation was the central theme of this project. We would also like to thank Alvaro Taype-Rondan, MD, who gave his permission to use the images, graphs, and tables that are part of the questionnaire in the supplementary material.
This work was part of the Master's Thesis in Education by Dr. Luis León and was partially presented under the title “Academic characterization of residents of the Medical Genetics specialty of Mexico City on systematic reviews” within the framework of the Interprofessional Meeting of Education in Health Sciences, as part of the EPPENS Interprofessionalism International Congress, from August 9 to 13, 2023 at the Faculty of Medicine of the National Autonomous University of Mexico.
Supplementary material.
