The curricular educational and academic models need to focus on the integral education of the physician by competences. One of the challenges for teachers in the health area is competency assessment, as they tend to evaluate skills, knowledge, and attitudes separately; likewise, it is done with instruments that lack relevance, validity, and reliability. Currently, there are more medical schools that are developing their curricula by competencies, from a sustainable perspective, which refers to update, transform, mobilize, and transfer the knowledge and skills: cognitive, practical, social, affective, and creative to the solution of complex problems. The main characteristic of their curricula is that they are renewed continuously to face the current challenges of health and medical training in the knowledge society through different approaches.1–3

The socioformation emerged in 2002 from research in Latin America through collaborative work of professionals, researchers, and teachers from various countries seeking a comprehensive training and, in turn, offered a new educational dimension of society using new information technologies and with this, giving meaning to a knowledge society, more reflective, with greater entrepreneurship, addressing problems of the context, and solving the immediate and raised needs.4

One of the essential aspects of the socioformation are the competences, because they contribute a new perspective of the education from the challenges of the society of knowledge.5

In socioformation, the socioformative evaluation is proposed, which is oriented in the integral formation, articulating the diagnostic assessment, continuous and summative, starting from the self-evaluation, co-evaluation, and hetero evaluation in a collaborative work environment, and for this, it is necessary to use instruments that enable continuous improvement based on the socioformative taxonomy of 5 levels (preformal, receptive, resolutive, autonomous, and strategic). One of the most relevant instruments is the rubric.6

The rubrics evaluate the approach of problems of the context, in which knowledge, abilities, attitudes, strategies, and tendencies are put in play that are related to the constructive elements to face and solve the problematics raised, besides it accommodates the metacognition for the improvement from the feedback. Cebrián defines the rubric as a valid tool for the teaching-learning process, which helps to define and explain to students what is expected to be learned.7

The Faculty of Medicine innovated its curriculum by competencies, to contribute with the training of physicians committed to solving the current and future problems in the area of health through basic, professional, and specific competencies with sustenance in the educational model by competences focused on the learning. Being focused on learning, the curriculum places the student in a leading position to create and recreate their activities through the use and appropriation of the necessary theoretical-practical instruments. This structure of appropriate procedures for self-directing learning allows students to comprehensively address the health-disease process.8

One way to evaluate the clinical competence of physicians is through the Objective Structured Clinical Examination (OSCE), which is designed to assess the performance of students in specific clinical situations, where the students are tested their theoretical knowledge, clinical reasoning, skills, and abilities, as well as their attitude and interpersonal communication skills. It is a timed test, structured as a circuit of stations that resemble clinical scenarios, with specific tasks that allow evaluating the clinical competence of students.9 The stations include the presence of an evaluator or a standardized patient, who carries at the moment a performance rating scale, either in the form of a checklist, a perception list, or a rubric.

The objective of this study was to assess the level of achievement of the competencies proposed in the graduates' profile by applying the OSCE with socioformative rubrics and assessing their relationship with the written final exam and the general average of the degree.

A cross-sectional study was carried out with graduates of the career of medicine of the Faculty of Medicine and Biomedical Sciences of the Autonomous University of Chihuahua, in the city of Chihuahua, Chihuahua, Mexico in 2018; in which an OSCE was applied as part of its degree exam.

The professional examination of the career of medicine integrates two evaluations: (1) Written exam of general knowledge of medicine and (2) OSCE where the domains belonging to the competences (basic, professional, and specific) that make up the profile of the physician.

Regard to the level of achievement of the competences comprising the profile of the physician, of the 7 competences that make up the physician profile, 6 obtained an autonomous level of achievement: Diagnosis and management of pathologies; Communication; Sociocultural; Culture in health; Basic conceptual elements; Public health and health systems. Only the Problem-Solving competence reached a level of resolving achievement (Table 2). These results integrate the analysis of the 4 dates which OSCE develop, where applied the same socioformative rubric. It is important to mention that the sample was divide in different groups because of the size of population.

Concerning to domains of the competencies with higher achievement levels, of the 22 domains evaluated, the one that obtained the highest average was: Relates the health and disease elements with the person and their environment, with an autonomous level of achievement. Instead, the domain with the lowest average was: Identifies solutions based on different reliable sources of information, including the review of the international literature (in other languages), which reached a level of resolutive achievement (Table 3).

Regarding the dates, in the month of May, 12 domains obtained the highest averages, 4 in the month of April, 3 in January, and 3 in March.

A significant positive correlation was observed between the average of the OSCE and the average of the written exam of the students through the Pearson correlation coefficient (P = <0.01). As well as between the average of the OSCE and the general average of the career. Another important finding is that a correlation was observed between the general average of the career and the average of the written exam (Table 4).

Through the comparative analysis between the levels of achievement in each domain and the date they performed the OSCE, the results reveal significant differences, mentioning on which date they reached autonomous and / or resolutive levels of achievement. The level of autonomous achievement was the highest in the development of the OSCE and the lowest level was the resolutive one.

In the months of March, April, and May, the students reached an autonomous level of achievement in: Promoting the improvement of the quality of human life attending to established welfare indicators; compared to the month of January, where the students only reached a level of resolutive achievement.

In the months of January, April, and May, the domain: Analyses, integrates, and contrasts the anatomy and physiology, both general and local or regional, with their knowledge about the patient's disease, reached an autonomous level of achievement, instead in March, they achieved a resolutive level.

In April, the highest average corresponds to the domain: Examines the patient in a complete and systematic way, respecting the dignity and integrity of the patient, reaching an autonomous level of achievement as in the months of March and May; however, in January, they only reached a level of resolutive achievement.

During the month of May, the domains: (1) Apply structured interrogation (organized and coherent) on the patient's state of health; (2) identify the clinical-pathological alterations of the examined patient, reached an autonomous level of achievement, and (3) propose the treatment or reclassify and refer to another level of competence; got the highest average in May. However, the third domain reached levels of resolutive achievement in the months of January, March, and April.

The statistical differences observed in the results of the OSCEs between the sessions of January, March, and April in relation to May, were also observed in the averages of the written exam carried out on those same dates (Table 5), which suggests that the students who carried out the OSCE in the month of May developed in better way their skills during their training process. General average obtained by groups according to the dates on which they took the written exam and the OSCE.

Survey analysis: Regarding the survey responses, the results of the means analysis are shown; students believe that clinical cases are consistent with their level of preparation (X¯  = 3.73 ± 3.14), an average that exceeds the upper limit (X¯ + 1σ = 3.656). On the other hand, compliance with the established schedules (X¯ = 2.82 ± 1.21), feedback by the evaluators (X¯ = 2.73 ± 1.15), the promotion of reflection (X¯ = 2.71 ± 1.11) presented values below the lower limit (X¯-1σ = 3.097).

In the opinions and suggestions section, the students felt that the OSCE was an appropriate strategy to evaluate the competences which the physician should graduate; Likewise, they commented that the clinical cases used for the evaluation were concrete and adequate according to the general medicine consultation. They also said that the OSCE allowed them to determine their strengths and the areas in which they should improve. On the other hand, they felt that the time in each station was not enough to adequately develop the skills that were evaluated in each of the stations and suggested that they be offered in advance instructions regarding the organization of the OSCE to avoid confusion when entering each of the stations; They also recommended greater feedback from the evaluator. Another important opinion on the part of the students, is the fact of having simulated patients and mannequins; since it made them nervous and considered that since they are not real patients it is more complicated to determine a diagnosis since there is no real clinical case.

This study highlights that the OSCE, being an objective and reliable exam, is ideal as part of the professional examination for the career of Medical due to its flexibility in terms of the number of students that can be evaluated, the type of patients represented, and the format of the exam itself, including the length of the exam, the number and duration of stations. In addition, by using the rubric as an evaluation instrument, it is possible to determine the level of achievement of the competencies with which the students graduate, which allows feedback on their training process.

The socioformative rubrics is an instrument that explains to students what is expected of their professional performance and it has criteria on how it will be assessed, thus promoting reflection and metacognition. Likewise, it provides feedback on achievements and aspects to continue improving. It is considered as a strategy that unfolds the potential of students by providing information on each competence and contributes to improving teaching strategies.5

The averages of the students who presented the OSCE in the month of May, were the highest. This correlates with the averages of the written exam carried out on that same date (Table 5). These results indicate that students developed in a better way their skills during their training process. However, according to Callavier et al. (1993), it is also possible that it is due to a “copy effect” in which the followers exchange information with the rest of the participants, a situation in which 3 effects can occur:

1. Positive: It makes the last candidates to take the exam obtain better averages.

2. Negative: The possession of the information causes a negative effect among the applicants, presenting themselves to the exam too confident or with prejudices, which has a downward impact on their averages.

3. No effect: Does not affect the averages; the information that circulates among supporters is usually not relevant to improve their average.10

Therefore, divide application in different dates can be a limitation for this study.

The organization of the OSCEs, where the clinical cases used in each application were different, allows us to infer that had a greater impact the previous training of the students on the results of the evaluation than the information that circulated among them.

These results are comparable to those of a previous investigation which aimed to present the results of the students of the Faculty of Medicine who applied the practical phase of the professional exam with the OSCE. It was observed that the correlation of the average of the 5 years of the career with the result of the OSCE varied since they have different degrees of development of clinical skills, so that the grades influence other variables, in addition to that they are often overvalued.11

The correlational analysis highlights a significant positive correlation between the average of the OSCE and the average of the written exam of the students, which makes it clear that an adequate performance in the written exam that assesses the theoretical knowledge is related to an adequate performance in the clinical skills demonstrated in the OSCE. It was also found a correlation between the average of the OSCE and the general average of the career, which is of importance since it allows to know if the competencies established in the career profile are related at the time of demonstrating them in the OSCE.

Regarding the surveys carried out on the students, it was observed that they consider to a greater extent that the clinical cases are consistent with their level of preparation. Instead, they consider to a lesser extent that there was compliance with the established schedules, feedback from the evaluators, and promotion of the reflection by the evaluators, which indicates that there are areas for improvement in future examinations; however, this evaluation is carried out by completing the OSCE and the feedback from the evaluator is given to the student in a written format later.

After the end of the circuit of stations, in most of the OSCE, a feedback session needs to be done, where the evaluators review the actions carried out with the student, in order to bring reflection and self-criticism. Getting involved in feedback is mandatory. This is a valuable moment for meaningful learning, since it allows to recognize the student's development during the evaluation, as well as the analysis of their performance, highlighting those actions that were performed correctly and helping the student to discover and understand what to improve and how to do it.12

One of the main recommendations of Harden and other authors to improve the validity, reliability, and objectivity of the exam is to be very careful in its design, planning, and administration.13

It is also important to use the rubrics, which provide information on the effectiveness of the teaching techniques used during educational activities. They also allow each course and the evaluation process to be analyzed by external peers, which is reflected in a continuous improvement process.14

This study leads to the conclusion that the OSCE, being a reliable evaluation process, is suitable as part of the professional examination of the medical career, which as previously mentioned, it correlates with the results of other evaluations. Thus, with the use of the rubric, it is possible to determine the level of achievement of the competencies which the student graduates. It also allows for feedback to improve the academic performance of the graduate and the institution.

Therefore, the OSCE is one of the best methodological strategies to assess competencies for their objectivity and compliance with the requirements of validity and reliability. In addition, it is more reliable when applied through rubrics, because they report greater accuracy of the performance levels achieved in each competition, considering the specificity of each clinical situation.

It was determined that the OSCE pertinently assesses the competences established in the graduate profile of the career of medicine which is based on the health problems in the different population groups and the challenges that faces to give them effective solution. In this way, the graduate physician has been able to reach mostly the level of autonomous and resolutive achievement of the domains that integrate the proposed competencies in the physician profile.