Do primary health centres and hospitals contribute equally towards achievement of the transversal clinical competencies of medical students? Performance on the Objective Structured Clinical Examination (OSCE) in competency acquisition

Abstract

Objectives

The adaptation of the educational programmes of European faculties of medicine to the European Higher Education Area guidelines has focused curricula design on competence acquisition. Competencies are defined as the achievements of a predetermined level of efficacy in real-world scenarios. Our objective was to assess whether performance on a common competence evaluation test, the Objective Structured Clinical Examination (OSCE), resulted in different scores for second-year students after a practical medical training course took place in a primary health centre (PHC) or in a hospital.

Design

A descriptive study was conducted during the 2010–2014 academic year of the OSCE test scores obtained by all second-year students.

Location

Faculty of Medicine at the University of Lleida (Catalonia, Spain).

Main measurements

We performed a correlation analysis between students who completed their practical medical training at the PHC and hospitals utilising Student's t-test for comparison of means.

Results

423 students who completed internships at the PHC and at hospitals obtained OSCE mean scores of 7.32 (SD; IC) (0.82; 7.18–7.47) points and 7.17 (0.83; 6.07–7.26) points, respectively (p = 0.07).

Conclusions

Second-year medical students acquired similar competency levels in the two analysed training scenarios. The two areas both serve their teaching purpose.

Resumen

Objetivo

La adaptación de los programas educativos de las facultades de Medicina europeas a las directrices del Espacio Europeo de Educación Superior se ha centrado en los planes de estudio mediante la adquisición de competencias. Las competencias se definen como los logros de un nivel predeterminado de eficacia en escenarios del mundo real. Nuestro objetivo fue evaluar si el rendimiento en una prueba de evaluación de competencias transversales, mediante un Examen Clínico Objetivo Estructurado (ECOE, dio lugar a diferentes resultados de los estudiantes de segundo año después de un curso de formación médica práctica que se llevó a cabo en la atención primaria o en un hospital.

Diseño

Se realizó un estudio descriptivo de los resultados de las pruebas de ECOE obtenidos por todos los estudiantes de segundo año.

Emplazamiento

Facultad de Medicina de la Universidad de Lleida (Cataluña, España) durante 2010–2014.

Mediciones principales

Se realizó un análisis de correlación entre los estudiantes que completaron su formación médica práctica en la atención primaria de salud y hospitales utilizando la prueba t de Student para la comparación de medias.

Resultados

423 estudiantes que completaron sus prácticas en la atención primaria de salud y en los hospitales de la OSCE obtuvieron puntuaciones medias de 7,32 (DE; IC) (0,82; 7,18–7,47) puntos y 7,17 (0,83; 6.07–07.26) puntos respectivamente (p = 0,07).

Conclusiones

Los estudiantes de medicina adquieren niveles de aprendizaje de competencias similares en los dos escenarios de formación analizadas. Las dos áreas docentes cumplen su objetivo de enseñanza.

Introduction

The effective practice of medicine requires a holistic approach to health and disease management. Following the recommendations of the White Paper on medical education,¹ this broad perspective of medicine was addressed in the curricula of most faculties of medicine in Spain, during their process of adaptation to the European Higher Education Area (EHEA), also known as the “Bologna Process”.² Thus, an integrated vision of pathophysiology, psyche and an environmental biopsychosocial model,³ provides upcoming physicians with new opportunities to cope with their future patients’ health problems.

The EHEA was constructed to improve the quality and international competitiveness of higher education in Europe and to increase the mobility and employment levels of university graduates as laid down in the Schengen Agreement. The adaptation of the Spanish Faculties of Medicine educational programmes to the EHEA has focused curricula design on competence acquisition.⁴ Because competence acquisition involves attaining a predetermined level of efficacy in real scenarios,⁵ it has become necessary for students to acquire more realistic healthcare experiences, as in all other graduate degrees and their respective professional areas.^6–8

Until recent years, undergraduate medical training in most faculties of medicine merely complemented the curriculum because only knowledge was evaluated. Practical medical training was usually not mandatory, and training objectives were neither clearly stated nor focused on the acquisition of competencies related to the practice of medicine. Students simply accompanied a physician during attending rounds at the hospital or during primary care consultations, and the physician shared the patient's most relevant clinical data with the student. This method of learning through observation and imitation was inherently limited because the role model and the teacher were one and the same. In addition, traditional medical practice was passive and lead to poor skill consolidation in an imprecise, heterogeneous and unpredictable way.

However, medical educational planning should be based on defined competence objectives, as stated in the EHEA documents and the Spanish Ministry of Education.^9,10 Thus, competence acquisition began to gain significant relevance and became fundamental to the task of providing an adequate accreditation of medical student learning.^11,12

General practitioners treat a wide range of medical conditions and have the skills to develop this competency model. This enables them to motivate and stimulate students in the techniques of self-learning amidst the progression from “knowing” to “know-how” and “doing”.¹³ However, curricular changes advocated in the Bologna process that include adding specific specialised competencies are still possible in Spain.¹⁴ In our degree in medicine, a large part of cross-curricular competencies can be addressed throughout the learning process from the perspectives of different disciplines. In our subject, Primary Care, we evaluated acquired skills using an Objective Structured Assessment (OSCE) (introduced by Harden et al.¹⁵ in 1975) shared with two other subjects (Clinical Examination and Assisted Training-I).

The curriculum of the Faculty of Medicine of Lleida has designated that during the second half of the second year, students must take three compulsory subjects, Primary Care, Clinical Examination and Assisted Training-I (AT-I), each with 6 ECTS credits.¹⁶ The first two courses are preferably attendance classes, while the subject of AT-I is completely practical and involves a rotation of one month in primary care or medical services. The main objectives of these courses are to begin the acquisition of basic skills in history taking, physical examination, and report writing and developing clinical judgement (Table 1).

Table 1.

Subjects during the second semester of the second year of the medical degree curriculum with their respective competencies, according to the current Spanish legislation.

Assisted Training I (6 ECTS)

28 Interpretation of normal clinical analysis.

31 General physical examination.

115 To have the ability to make a full medical history focused on the patient and their pathologies and to understand its meaning.

116 Physical examination of apparatus and systems, as well as psychopathological examination, and their interpretation.

40 To be able to write medical-legal documents.

45 To recognise the need to maintain professional competence.

46 To be able to address the professional practice respecting patient's autonomy, belief and culture.

51 Planning, timing, and assessment health programmes.

53 Assessment of the health care quality and patient's security strategies.

58 To know economic and social implications related to medical intervention considering the criteria of efficiency and effectiveness.

75 To know the aspects of communication with the patients, their relatives and social environment: Models of clinical relationship, survey, verbal and non-verbal communication and interference.

76 To deliver the bad news.

77 To write histories, reports, instructions and other records, understandably to the patients, their relatives and other professionals.

78 To make a public presentation, written and verbal, of scientific works and/or professional reports.

113 Family and community medicine: patient environment, promotion of health within the family and community.

157 To write correctly medical prescriptions which are consistent with the status of the patient in accordance with the legal requirements.

Clinical Examination (6 ECTS)

28 Interpretation of normal clinical analysis.

117 To appreciate clinical parameters’ modifications at different ages.

119 To establish an action plan tailored to patients’ needs and their familiar and social environments, consistent with the signs and symptoms of the patient.

122 To know the indications from biochemical, haematological, immunological, microbiological, anatomopathological and image analysis.

After finishing the internship, students are evaluated by their supervisors using an individualised report (30% of final score). Additionally, a year-end OSCE test is taken to evaluate the specific competencies addressed in AT-I (70% of final score) as well as the transversal theory-based competencies of Primary Care and Clinical Examination (both 40% of final score).

The purpose of this study was to assess whether a common competence evaluation test, performed as an OSCE test, would result in different scores for second-year students after completing a pre-med internship at a primary care centre or in a hospital.

Methods

A descriptive study was conducted during the 2010–2014 academic year on examination results obtained by all second-year students from the Faculty of Medicine of the University of Lleida (Catalonia) in the three subjects of the second semester (Assisted Training I, Primary Care, and Clinical Examination), and their OSCE test score.

The seminars were held in groups of 30 students; groups were allocated alphabetically and distributed considering the teaching capacities of the hospital and primary care services.

The OSCE test method (Harden, 1975) is considered the gold standard for evaluating the acquisition of clinical competencies,^17,18 and has been implemented as an evaluation method by a growing number of health care institutions, faculties of medicine, medical boards and scientific societies. Most English-speaking countries have adopted the OSCE, and it has been used for years as a standardised way to evaluate students’ professional competencies.^19,20

The OSCE test is divided into three parts based on the targeted competencies taught during the training period of the aforementioned subjects: Assisted Training-I, Primary Care, and Clinical Examination. The test had a standardised distribution along three school days. Students were divided into six groups of 20 people performing at different stations for three teaching days, with each test lasting two and a half hours. There was a rotation of consecutive learner pairs, lasting fifteen minutes per station to complete the circuit on which the OSCE test was structured.

The performed OSCE test meets the recommended quality requirements for this kind of multistage test evaluations. The test was implemented according to the format recommendations of the National Conference of Deans, which will be shared by all the Spanish Faculties of Medicine at the end of the degree. The OSCE duration is 2.5 h, with 10 min stages and 20 students participating in each test cycle. The structure of the test was designed according to a set of specifications elaborated by the OSCE Test Committee in order to distribute the competences shown in Table 1 and to assign the score percentage of the evaluation methodology to each part (anamnesis, physical examination, skills and communication). The test cycle is divided in seven stages with simulated patients previously trained by the teaching staff that has elaborated the specifications sheet of the clinical case. The players are resident physicians instead of professional actors, with previous experience in this evaluation methodology. Physicians specifically trained for this purpose act as observers in all stages, and evaluate each case with filling a 20 item predefined checklist (anamnesis, physical examination and communication, and two physical examination procedures). The stages with simulated patient include the presentation of a clinical case adapted to the competences and the academic level of the students (thoracic pain and abdominal pathology), with a correct clinical history taking as the main objective. Clinical assessment competences are evaluated principally according to the physical examination performed (cardiorespiratory, basic neurological and abdominal). The test is completed with four static stages involving visualisation of recorded clinical interviews (basic communication and delivering bad news competences) and four written tests (anamnesis writing, clinical and therapeutic management, and preventive actions).

All enrolled students (n = 423) completed the OSCE test. Evaluation panels comprised teaching staff and voluntary healthcare professionals featuring simulated patients who followed a previously drafted and rehearsed script.

Variables for analysis included the location at which practices took place (Primary and Hospital Care), the marks obtained in the individual subjects (Assisted Training-I, Clinical Examination and Primary Care) and the marks obtained by the students in practice. Data also included each student's final OSCE score as well as notes for each section of the test. Group comparison was conducted based on area of experimental practices: Primary Care or Hospital. The student data were collected at the end of the July 2013 examination using a template to create a posteriori calculation sheet for each student. The qualifications were valuated between 0 and 10 points.

Descriptive analyses included the age of the global average and standard deviation of test scores and standard deviation, the average grade in each section of the test and the distribution for the different variables. Percentiles were calculated on the distribution of final scores, and the normal distribution of scores was obtained.

The results are presented in tables and figures. Presented data include the obtained values, descriptive statistics as well as results from between-group comparison using the Student's t-test. Analyses were conducted using SPSS Version 12 statistical software.

Ethics

The paper has been supervised and authorised by the CEIC of the University Hospital Arnau de Vilanova in Lleida.

Results

Of the 423 students registered in second-year courses for the degree of medicine in the 2010–2014 academic years, 122 were trained in the Primary Health Centre and 301 in the Hospital.

The average global score and standard deviation (SD) global on the OSCE was 7.21 (0.83), ranging from 4.08 to 9.16. The 25th, 50th, and 75th percentiles were 6.70, 7.25 and 7.77, respectively.

Analyses showed that the average scores obtained by the students from the practical rotation of the Primary Health Centre did not significantly differ from those obtained by students from the Hospital (p = 0.07) (Table 2).

Table 2.

Descriptive analysis of OSCE transversal competencies results by practice location (n = 423).

	Primary care (n = 122)	Hospital (n = 301)	p
	Mean (DS; IC)	Mean (DS; IC)
OSCE average rating	7.32 (0.82; 7.18–7.47)	7.17 (0.83; 6.07–7.26)	0.91
Assisted Training I	8.17 (0.77; 8–8.3)	8.19 (0.75; 8.1–8.3)	0.81
Primary Care	6.78 (1; 6.6–7)	6.37 (1.2; 6.2–6.5)	<0.001
Clinical Examination	7.03 (1.75; 6.7–7.3)	6.9 (1.71; 6.7–7.1)	0.5

Statistically significant differences were not observed in the average competency ratings obtained in the subject of Assisted Training. Students trained in the Hospital received an average qualification (SD) of 8.19 (0.75) while students trained in the Primary Health Centre received an average (SD) of 8.17 (0.77) points (p = 0.81).

The results of the ECOE test regarding the competency related to Primary care showed that the average (SD) for the students who conducted their rotation in the hospital was 6.37 (1.2) while students who conducted their rotation in the Primary Attention received an average (SD) of 6.78 (1) points, which was significantly different (p < 0.001).

The average (SD) score on the ECOE in the subject of Clinical Exam was 6.9 points (1.71) for the students who carried out their rotation in the Hospital and 7.03 (1.75) points for those who trained in the Primary Health Centre, which was not significantly different (p = 0.5).

Fig. 1 illustrates the comparison between both student training possibilities and the associated results for all the students evaluated.

Figure 1.

Box plot comparing the two teaching scenarios and the respective test results for all students tested.

Discussion

The results show that second-year medical students acquired similar levels of competencies when comparing the two classic learning scenarios, PHC and Hospital. This result was expected because that is precisely what the competence-based curriculum fosters, that is, the student's acquisition of knowledge and skills in the setting in which it is needed, debunking the old tradition that required students to learn certain subjects, almost exclusively, within certain knowledge areas. While the rotating models of care practices are different for the PHC and Hospital regarding student training, the clinical interview methods, accessibility and student's role along with the targeted skills are equivalent. These training characteristics will be of particular relevance to future student's choices of medical specialties.^21–23

One of the strengths of this work is that analyses included all the students in four courses. The results are also interesting because they represent a prospective study of the revisions to the new educational programmes in which many of the acquired competencies have a transversal component and where the involved teaching staff is large and comprises different medical practice profiles. It will be necessary in each of these subject areas to examine the student's level of learning, but even so, our study confirms the success of involving multiple clinical teachers because in this way results can generalise to the potential organisational needs of other faculties with similar improvement possibilities.

Our work also suggests that an individualised practice model ensures the internalisation of medical competencies and is also a way of giving curriculum importance to community-oriented health care processes, health promotion and communication-related aspects.²⁴ The physician–patient relationship model is central to medical practice. Its trained development and evaluation in real-life scenarios provides medical students with more realistic experience and vision as well as more patient contact, allowing them to acquire the communication skills and clinical reasoning that will be essential to their future professional practice.^25–27 Moreover, our study contributes to the improvement of curricular design in showing that students of the Faculty of Medicine of Lleida who have received a medical degree with basic competencies in primary medical care have a favourable opinion and also appreciate the opportunity to have acquired those specific competences during the degree programme.²⁸ This approach allows a closer approximation of medical training to the perceived needs in the community and a progressive convergence of educational programmes with the legal guidelines that are common to the surrounding countries.

Learning in a variety of settings has generally meant that, in addition to hospital-based experience, students now learn in general practices, community health facilities and patients’ homes.²⁹ These individual locales naturally introduce some of the complexities of context and relationships that are so often lost, when patients are seen out of context, in a hospital bed. In this way, EURACT (European Academy of Teachers in General Practice and Family Medicine) states that each university medical school educating future doctors must offer all students an adequate level of knowledge, practice and exposure to family practice care to gain the requisite knowledge, skills, attitudes and scope of primary care approach.³⁰ Family medicine teaching as well as training is a fundamental resource for medicine today and for future generations as urged by all the International Organisations and Institutions of Medicine (EURACT 2014).

With regard to limitations of our work, it would be interesting to analyse each competence separately to check whether there are individual differences beyond the mean deviation. However, it is clear that such a disaggregated level of analysis does not permit a comprehensive work. It is also possible that the analysis of a single cadre limited the generalizability of our results, influenced by the level of motivation of the involved teaching staff (remarkably high among second-year training supervisors) or by differences between groups of students. In our assessment, we did not find any factor that may have changed; however, we aim to analyse students from other academic years to see if the outcomes in learning transversal competencies are retained. Furthermore, we have not examined whether the two alphabetically assigned groups were homogeneous for age and sex, or for the number of attained credits before the OSCE test was conducted.

Another potential limitation is that we do not yet know what would happen if there was a third group of students (control group) who had no practical medical training. Can we be sure that the skills were acquired during the training? In the process of learning and evaluation, it should be considered whether the knowledge was acquired in the classroom and if the competencies were acquired during the Assisted Training. By extension, one might ponder whether the OSCE test accurately reflects this training model and the mastery of clinical skills. This would be a field to explore in future works.

Moreover, we cannot guarantee that the learning methodology was the same in the two teaching scenarios. Therefore, we know that the evaluation has measured the result but not necessarily the method of learning. We have evaluated the same learning outcome (competencies) without knowing exactly whether the aforementioned two aspects were addressed in a similar manner. The methods of teaching in a medical office can vary widely and are largely dependent on the teacher and the circumstances surrounding the teaching environment because training and competence evaluation is a complex field.^31–33 The OSCE test could be considered an additive assessment, and therefore, an assessment of the final outcome that measures attainment of competencies relevant to the EHEA. As a supplemental evaluation, the supervisor's report, if properly designed, would give us a better view of formative assessment, and thus, a successful continuous assessment.³⁴ In an article published in 2002, Blay³⁵ explained that the foregoing is particularly true when the choice arises between an assessment intended to certify the competence (summative) and one that seeks to guide the training of individuals (formative).

In conclusion, we found no learning differences in the acquisition of transversal clinical competencies when conducted in two different educational areas (Hospital and Primary Health Centre) and assessed by the OSCE test. In any field of education, the programming, teaching objectives (which should be clear and common) and supervisor involvement should be distributed equally. The present study confirms the efficacy of including primary care practice in the curricular design of the educational programme because it complements the hospital setting in the acquisition of the transverse clinical competencies assigned to the medical degree.

Practice points

• •

  Medical students attained a similar level of competency acquisition when comparing the two classical practical medical training scenarios: primary health centre and hospital.

• •

  This work expands a prospective method of learning with the revisions to the educational programmes in which most competencies have a transversal approach.

• •

  Results demonstrated the efficacy of involving many healthcare professionals with different medical practice profiles in the training of medical students.

• •

  The inclusion of practical training in primary care in the curricular design of the educational programme was validated because it contributes equally when compared to hospital training in the acquisition of the transversal clinical competencies in the degree of medicine.

Financing

This project has been carried out without external financing.

Conflict of interest

The authors declare that there is no conflict of interest.