Quality medical education can only be achieved if it is evaluated using a valid instrument. To evaluate user satisfaction, here, that of undergraduate interns, a multimodal ad hoc questionnaire was validated to rate professors, the program content, the didactic material and the facilities for three different generations of medics. The results showed consistency and reliability in 14 cases (α Cronbach of 0.863 with Bartlet test of X2 102.8, p=0.001). By contrasting the changes before and after the internship, a statistically significant change could only be perceived in relation to the content (p=0.04), material (p=0.04) and facilities (p=0.0001) in the 2011 generation. Generally speaking, the global ratings for the three different generations were good or excellent.
La educación médica de calidad solo puede ser gestionada si se evalua con un instrumento validado. Para evaluar la satisfacción del usuario, en este caso el médico interno de pregrado, se valido en tres generaciones un cuestionario ad-hoc multimodal para calificar a los profesores, al contenido del programa, al material de enseñanza y a las instalaciones en tres diferentes generaciones de médicos. Los resultados mostraron consistencia y fiabilidad en 14 reactivos (Alfa de Cronbach de 0.863, previa prueba de Bartlet X2 102.8, p=0.001). Al contrastar los cambios entre antes y después del internado se apreció un cambio de percepción de los médicos internos estadísticamente significativo solo en lo referente al contenido (p=0.04), material (p=0.04) y a las instalaciones (p=0.0001) en la generación 2011. En general las calificaciones globales para tres diferentes generaciones fueron entre buenas y excelentes.
Medical education has become a subject of great social interest, as has the provision of quality health services.1–3 These services are dependent on several factors, including the disposition of human resources and the organizational aspects involved in their execution. Another determining aspect is the competence of the health professionals (general and specialist medics) who play a fundamental role in the process of teaching medical students.4–6
In recent decades, medical education has become a task shared by superior educational institutions and health service institutions. The latter have become an important part of the development of undergraduate and postgraduate physicians.7–11
Within these educational models, we must consider the proposal de la Orden, which, since 1997, has developed a systemic model of medical education in which the quality of education is associated with the coherence between its different components.12 In this model, it is necessary to evaluate the hospital components that are involved in the process of education. Hence, in 2007, the Director of Education of the General Hospital of Mexico (HGM) designed a questionnaire to evaluate the satisfaction of the undergraduate interns during their internship.
The present work has two fundamental objectives: to validate a questionnaire that will evaluate the satisfaction of undergraduate interns in three different generations and to apply the same questionnaire to another three generations of undergraduate interns and compare their perceptions at the beginning and end of the year of their internship.
Material and methodsAn evaluation questionnaire concerning the multidimensional satisfaction of the undergraduate medic was designed using an ad hoc scale with ratings of excellent, good, average and poor (Fig. 1). The questionnaire was composed of four sections, in which the professors, the program content, the didactic material and the facilities of the HGM dedicated to education were evaluated. Aspects that were considered in the design of the questionnaire included the number and composition of the items, the content of the questions, and the definition and order of the items. To prevent a learning bias, an ordinal scale was included, and the answers were polytomously codified. The consistency of the questionnaire was measured using Cronbach's α coefficient.
The present study first comprised a stage of validation, which included all undergraduate interns who were assigned to the HGM and were in their fifth year of a bachelor's degree in surgery at the Superior School of Medicine from Instituto Politécnico Nacional (IPN), one of the three medical schools within Universidad Nacional Autónoma de México (UNAM) or the Health Sciences School (Anáhuac University). The questionnaire was administered in a single session that was held in the Aquilino Villanueva Auditorium of the HGM, where instructions were first given for its completion. The maximum time for its completion was 30min, and all uncertainties about its completion were answered. The second phase consisted of a hypothesis proof to establish whether any significant differences existed between the score obtained using this questionnaire and those of undergraduate interns from 2011, 2012 and 2013.
The information was collected in a database for subsequent analysis using the program Statistical Package for the Social Sciences (SPSS 2.2). The analysis and interpretation of the results was performed using a Xi2 proof.
ResultsThis poll was validated by being applied to 414 undergraduate interns for a period of three consecutive years, i.e., 2008, 2009 and 2010 (130, 141 and 143 students per year, respectively). In the demographic table, the distribution by year, gender, age and prior school can be observed. Initially, the questionnaire contained 27 questions, but by 2010, it had been shortened to 21. A liability of scale evaluation was performed, and an α Cronbach index of 0.863 was obtained for 14 points (prior with Bartlet test of X2 102.8, p=0.001). In Table 1, the mean of the qualifications for the three repetitions can be observed.
Demographic data of three generation of students who were underwent to validation of survey. UNAM-CU is campus University City of UNAM, UNAM-Zaragoza is campus Zaragoza and campus Iztacala both of UNAM.
| Generation (n) | Age (years) | Gender | Number/Eschool |
|---|---|---|---|
| 2008 (130) | 21–27 | 87 Fem, 43 Male | UNAM-CU 74UNAM-Zaragoza 6UNAM-Iztacala 4IPN 45Anahuac 1 |
| 2009 (141) | 20–25 | 92 Fem, 49 Male | UNAM-CU 64UNAM-Zaragoza 10UNAM-Iztacala 8IPN 45Anahuac 8La Salle 6 |
| 2010 (143) | 21–32 | 89 Fem, 54 Male | UNAM-CU 66UNAM-Zaragoza 8UNAM-Iztacala 12IPN 43Anahuac 8La Salle 6 |
Once the questionnaire was validated, a test of hypothesis constancy was performed over the next three years. The evaluation was performed regarding the period of pre- and post-internship on the four aforementioned topics (professors, program content, didactic material and facilities). For every answer in every rubric, a value of 10 points was assigned to augment the scale of the statistical analysis. The pre-internship questionnaire was included in the enrolment paperwork and before the end of the first bimonthly rotation such that this measurement could be denominated as “user expectative”. The second evaluation occurred during the last two months before the end of the internship, and this stage could be considered the scale of satisfaction with the product that was obtained after the internship.
In Table 2, we can observe the global situation for the three years of evaluation in both the pre- and post-internship questionnaires in each of the rubrics (professors, program content, didactic material and facilities), and we compare the means of the scores according to the performance ranges by using a T-test for related samples. In the analysis, it becomes evident that only one significant change occurs, in the category of modification toward excellence. We can interpret this result as an improvement of the users’ expectancies with the center in each of the categories over the three years. It is also interesting to observe that the expectancy of the quality of the professors did not change, which indicates that it is highly probable that the students chose this center specifically for its docent quality and thus found more than what they were looking for over the year. Despite the increasing number of students who consider the facilities to be average or poor in each year, these changes are not statistically significant.
Global comparison of changing in score of survey between pre and post Internet Course (2011, 2012, 2013).
| Generation(n) | Professorμ (EE) | Contentμ (EE) | Materialμ (EE) | Installationμ (EE) |
|---|---|---|---|---|
| 2011 (130) | 182.0 (64.3) | 231.3 (49.2) | 356.7 (78.4) | 2433 (3.3) |
| p | 0.106 | 0.04 | 0.04 | 0.0001 |
| 2012 (130) | 113.3(214.8) | 16.7 (118.9) | 143.3(207.4) | 33.3(127.7) |
| p | 0.65 | 0.90 | 0.56 | 0.82 |
| 2013 (130) | 26.67(17.6) | 46.7(21.8) | 43.3(24.0) | 40.0(20.8) |
| p | 0.27 | 0.17 | 0.07 | 0.19 |
Bold values indicate significant changes.
In the general analysis, given that there were no great changes in the expectancies in any other categories in the scale, we can interpret that the center exceeds or at least fulfills all of the students’ expectancies. The perception of between 80 and 90% of the interns ranged from good to excellent, and only 10% responded with ratings of average or poor (Fig. 2).
Discusion
The evaluation of the user satisfaction, here, of undergraduate students in the fifth year of their career, allowed us to validate the questionnaire by applying it from 2008 to 2010. Once it was validated, we could evaluate the students’ satisfaction with the professors, program content, didactic material and facilities. Despite generally obtaining very satisfactory results for the institution, as the analyzed results suggest, the facilities received negative scores from the post-internship students of 2011 and 2012; these differences are not statistically significant. It is evident that the evaluation process is dynamic and that the institution must remain vigilant in performing evaluations to enrich the educative research and that can be used for the continuous improvement of all of the processes involved in managing institutional quality.
The quality of an educational institution is achieved by improving the real knowledge of how the institution functions, which is done by continuously sampling the achievement of the learning goals. The purpose is to introduce adequate and timely modifications to improve students’ performance and decrease scholastic failure. One key aspect to improving scholastic performance is the process of teaching and learning as well as the atmosphere both in the classroom and in the institution in which they develop. The aspects favored by the organization and the general environment of the educational center must be regularly monitored to prevent failures before they can occur.13
The purpose of the educative model of the hospital and its evaluation is to verify and analyze the work performed by the Director of Education to elaborate plans and strategies for developing the educative and assistance services that it provides to society. The potential of evaluation to improve the quality of education is of great importance in that it enables the identification of the positive and negative aspects with which the program operates and the different elements that contribute to the achievement of the education objectives, thereby guiding us to the improvement of the practice.
Conflict of interestThe authors declare that they have no conflict of interests.