We are living in an era of change in the paradigm of training future doctors that conditions the competencies required by doctors. These changes should be reflected in the goals of medicine which must be moderate cautious, affordable, and economically sustainable, fair, and equitable and focus on the person behind the illness (person-centered medicine). There are also changes in demography with aging, socioeconomic factors, and epidemiology with new diseases (COVID-19) and disappearance of other diseases, emergence of new therapeutic approaches, and the prevalence of chronic diseases. But also, organizational changes, as the use of information and communication technologies, artificial intelligence, the reduction of time and cost in information, changes in the doctor–patient relationship, and better-informed patients. And finally, changes in resource management. The economic resources are not unlimited, and it is necessary to prioritize. All these aspects must be part of the physicians learning process and translated into competencies.1

The process of defining competences and curricular goals is dynamic. Neither the goals that medical training seeks to cover nor the competencies that must be demonstrated at the end of each stage are immutable. Instead, they must be adapted to meet the societal needs, which represent an improvement in the quality of the healthcare received by patients.1

Regarding specific competencies and learning outcomes, advances in the field of molecular biomedicine and technology have provided understanding of human physiology, pathophysiology, and pathology, allowing better prevention and improved diagnosis and treatment of diseases, and sometimes individualized therapies. Currently, these advances are part of clinical practise, and the incipient technologies derived from them show a horizon of future applications that, in the short term, will be implemented in the field of healthcare. All these changes will have an impact on society. They will require that future medical professionals adapt and, that their training throughout the training continuum contemplates these changes in order to acquire the competencies that will enable them to face their medical practise in the best interest of the health and safety of patients and of society.

Moreover, the recent health crisis due to the COVID-19 pandemic has shown the need to revise our current curricula to include new competencies and expand competences with currently little space as Public Health, Epidemiology in a globalized world, Clinical management, Bioethics, Telemedicine/Digital medicine, Individual and collective protection measures of the health of professionals, etc.2

Lately, we have also seen an increase of the interest in generic or independent domain skills among which can be considered Ethics and Professionalism, Communication Skills, Teaching and Research, Leadership, Teamwork, Management of Health Resources, etc. The acquisition of such competences is important to develop adequate clinical practise and a global doctor. The independent domain skills or generic competencies are complex behaviors. For its correct acquisition, a longitudinal development is required along the training program3 as well as introduction of more adequate assessment tools.

The goal of medical education is to foster future professionals, with an ability to practise medicine according to the highest standards and to contribute to the development of medicine and a healthy society. The expectations on future doctors are similar regardless of where in the world they are formulated.4–8

Medical education has 3 phases: undergraduate, postgraduate, and continuing professional development throughout professional life (Fig. 1). After graduation, the doctor must be able to practise safely during postgraduate training to become a specialist. Both patients and employers must have trust in the ability of the graduated doctor to fulfill entrustable professional activities,9 to gradually become more and more independent, and at the same time, a professional member of various teams, contributing to their performance.

Fig. 1.

The 3 phases of medical education. The 2 first have specified learning objectives while the long period of professional life-long learning is mostly unregulated.

(0.08MB).

Assessment plays a fundamental role in the training of doctors and students to ensure that each graduate has demonstrated all the competencies expected. During undergraduate and postgraduate education, assessment has 2 major roles; to support the development of each student to reach his or her full potential, mainly through continuing formative assessment and feedback, and to guarantee the quality of each issued individual diploma through summative assessments covering all aspects of professional competence. To meet these expectations, assessment should focus on the expected competence outcomes of the whole program and support the gradual development of these competences rather than assessing each course separately. An additional requirement is global trust in diplomas issued by individual universities, which requires international recognition of national or regional accrediting agencies.10

To assure a real acquisition of the expected competencies along the continuum of medical education, a new paradigm has emerged; programmatic assessment.

A new paradigm in evaluation; programmatic assessment

In the last 2 decades, a new philosophy of understanding the evaluation of medical students and doctors has emerged around the idea of “programmatic assessment.”11–15 First of all, it should be said that it is not a new instrument but rather a way of understanding evaluation. That is why we refer to it as a new paradigm. Beyond the classic parameters of validity and reliability, the objective is to assume also others like fairness, trustworthiness, and dependability.

The evaluation practise that we carry out generally starts from the idea that competencies are stable and generic. That is to say, the way of measuring them is good for all circumstances (place, time, type of student…), and that there is no relationship between these competencies. Therefore, each competence has its own measuring instrument. If, for example, we want to measure the ability to resolve a myocardial infarction in a patient, we measure communication on a Likert scale of generic concepts (relationship, empathy…), and we measure decision-making based on questions generated in a clinical case. It does not occur to us to think that both competencies influence each other and that we could use both instruments to measure both competencies.

To achieve this, the evaluation must start from a program that holistically conceives the evaluative objective and frames it within the study plan or academic curriculum itself. Each competence must be measured with different instruments, so that we can perceive the relational aspects with other competences. And of course, the same instrument can be used in different competences. So, instead of a one-to-one relationship between instrument and competence, we will have a matrix with several lines and columns.

We assume that when evaluating we carry out a measurement that aims to assess the learning of students. Instead, we should use measurements that are useful for learning. This is consistent with the idea that since the student is the center of the learning system, the evaluation must also serve to help the students realize where they are and which their level of learning is, but also which their strengths are and which aspects they need to improve. The final objective should be to contribute diagnostic (what is the level of competence in each student) and prognostic (what will the future competence be) information, and treatment (proposals to improve the current situation). These different measures regarding each competence allows us to recognize patterns of behavior (diagnosis) in each individual student and propose specific activities of remediation (treatment).

But the most important aspect of the new paradigm lies in decision-making for each student. It involves the team of responsible teachers openly deliberating on the results measured in a system such as the one mentioned above and making a consensual decision about each student's grade. Thus, the challenge of assuming the variability of each student and the relationship between the different measurements is invoked: in short, it is about making a judgment that allows each student to be treated as an individual.

The process of decision is crucial in this paradigm. Students generate measures coming from standardized instruments based on quantitative inputs, and others based on more qualitative approaches. The team in charge of this evaluation must decide in each case the value of the assessment through a process of discussion and deliberation. In order to avoid possible bias, it is important to separate teachers responsible for assessment and those responsible for training and for giving feedback and mentoring.

With this, it is easy to awake the myth of subjectivity in decision-making. We must remember that what we must avoid is arbitrariness, but that subjectivity without bias, well-reasoned and founded, is a fairer way of measuring competencies since it observes the individual in his or her entire performance, taking into account all the relationships and interactions that may exist.

This programmatic approach to competence development requires an overarching leadership of the whole program and is less likely to be compatible with independent leadership of separate courses lacking a holistic perspective. In addition, systems that allow characterization, visualization, and support of the gradual development of individual students are needed. To achieve this, a multitude of assessment methods should be combined throughout the program and used in a way that allows evaluation of all aspects of professional competence and stimulates students to develop these competencies. Many different methods have been documented, all with varying strengths and weaknesses. No method alone meets all expectations. Medical schools should select a set of methods that are compatible with their needs and resources, develop expertise in using these methods and evaluate their performance. Emphasis should be on methods that request that students show how they master a specific task, according to the competence pyramid of Miller16 (Fig. 2), and at the same time, illustrate the level of support and supervision needed. Knowledge tests alone are not sufficient although knowledge is the fundamental basis for all aspects of professional performance. There is always a temptation to rely heavily on knowledge tests since these are comparably easy to construct and manage and comfortably allow grading of students based on the number of points achieved. The scores may even contribute to an impression of “objectivity.” To avoid this, knowledge tests should be the basis of the assessment system only. One model is to use “progress testing” with a multitude of single best-answer questions covering all domains of knowledge from the whole program with multiple sampling from each domain.17 This enables visualization of the progression of knowledge throughout the program while at the same time, knowledge from previous parts of the program is kept alive. If the individual student follows his or her “development curve,” there is no need for re-assessments due to a temporary dip, while further actions must be taken if the development curve flattens. Thus, the main emphasis in the assessment system can be on demonstrated skills and professional performance and attitudes, with a basis in a portfolio system where the students themselves are primarily responsible for providing evidence of competence.18

Fig. 2.

Miller´s competence pyramid. Assessment during undergraduate education should move upwards, while the later phases must not forget to include the theoretical and scientific basis of medical practise.

(0.08MB).

In the last years, Spanish medical schools have partially incorporated into their curriculum the requirements established by the European Higher Education Area or Bologna Process,25 among others, the introduction of outcome-based medical education.26 Moreover, specialization training programs in Spain have been oriented towards the outcome-based paradigm. This paradigm means going from assessing the theoretical–practical content to the assessment of acquired competencies.27 Since the acquisition of competencies is a continuous process, the assessment of competencies must be continuous and progressive. Integrated and progressive competency assessment programs should be established, based on the programmatic evaluation paradigm.12 The adoption of such model runs better in a horizontally and vertically integrated medical curriculum.

Unfortunately, in general, Spanish medical schools continue to work with non-integrated curricula, structured on independent and non-integrated subjects or topics. Each subject is assessed independently, without connection among them. Furthermore, the concentration of assessments of different subjects at a few time points during the year, leads students not to prepare continuously throughout the training program, but to do so intensively and even compulsively when the date of the exam is approaching, causing significant distortions in the normal follow-up of the course.8 Spanish medical schools have introduced different assessment tools, but multiple-choice questions tests (MCQ) continue to be the most used, in spite of the fact that this tool is insufficient by itself to adequately assess the competencies of a medical student.8

To overcome these problems, Spanish medical schools have developed and implemented a structured objective clinical evaluation (OSCE), which is performed generally at the end of the supervised internship period in the last year of the learning process (6th year). This OSCE serves as an assessment of this internship period. However, it is important to point out that the end-of-degree OSCE which involves a major implementation effort with complex logistics and high economic and time costs, does not ensure the overall acquisition of competencies, nor can it replace the lack of a true continuous assessment of competencies with sufficient feedback throughout the degree. There is no doubt about the educational impact of the OSCEs, but it cannot be the only tool on which almost exclusively, the final verification of the competency status of our students is based. Furthermore, without considering the difficulties in the design, implementation, and evaluation of these tests in our context, we cannot from a psychometric point of view, ensure the total validity and reliability of the test. Other weak points are the lack of remediation mechanisms for students who do not pass (very low percentage) or the difficulty to give feedback on a test that is performed at the end of the degree.8,28,29

According to the considerations above, medical schools and their faculty should not be satisfied with the progress that the generalized implementation of the end-of-degree OSCE means. It is necessary to assume the need to advance in the development of a continuous and truly integrated assessment system and apply it globally and not to individual subjects. It should be emphasized that achieving this objective is unfeasible without the necessary means in terms of organizational, time, material, and human resources, and the participation of experts in assessment, working within the framework of a stable academic structure of medical education (medical education department), whose core function is to centralize and evaluate the assessment processes throughout the degree.

A real continuous assessment of the performance of students with continuous feedback that help them in their learning, leads us to the concept of continuous and integrated assessment. For this, powerful coordination mechanisms between different subjects in the curriculum are essential, which is not the case in our medical schools. Continuous assessment linked to feedback is useful and necessary to monitor learning progression and to take summative decisions (pass/fail). This statement is valid both for the teacher who teaches a subject and for the faculty that certifies a degree.28,29 This continuous assessment must be embedded in a program of institutional assessment or programmatic assessment referred to previously. The more important the decision to be made according to the results of the assessment is (e.g., to certify the acquisition of a degree), the greater the number and quality of the reports must be. Data and assessment activities should therefore be available throughout the evaluation period. Only enough assessment points, structured and organized within the institution's program, will make it possible to make the coherent and fair decisions that society demands.

In this assessment model, Spanish medical schools can use all available instruments, diversifying them and using those that are most valid, reliable, and feasible in their context. Among these tools, medical schools should consider some of those referred to previously for different competencies. In this sense, we propose the following model:

• 1.

  Regarding the assessment of knowledge and its application, clinical reasoning, and decision-making, one option to consider is the “progress test.” Progress test is a long-term, feedback-oriented educational assessment tool to evaluate the development and sustainability of cognitive knowledge during a learning process, used in different countries.17

• 2.

  For the assessment of clinical procedures to be acquired by all students at different stages of the degree, the use of the OSATS (Objective Structured Assessment of Technical Skills) tests in a simulated setting should be considered by means of a tunnel of activities that would assess the basic skills and procedures that every student should know how to perform.30 These skills tunnels could be carried out 3 times throughout the curriculum, one of them in the final year. To this end, all medical schools should agree on the common skills and procedures that students are expected to achieve at each stage of the degree. When students have obtained enough experience, or in residents, DOPS (Directly Observed Procedural Skills) is another valid tool for assessment.21

• 3.

  To assess clinical performance, a minimum number of Mini-Cex could be used during the clinical internship periods to assess the acquisition of basic clinical competencies in a real professional context.22

• 4.

  Regarding the OSCEs, as previously mentioned, it does not seem correct to establish only a final OSCE at the end of the degree, given its complexity and cost. Several and simpler OSCEs should be carried out throughout the degree, basically with a formative objective.20

• 5.

  Finally, the use of portfolio should be also considered for the assessment of generic competencies.18

Medical schools should make an in-depth analysis of the structure of their current curricula, adopting a real curricular integration and the most convenient assessment procedures to progress towards a continuous assessment model, in accordance with the programmatic assessment paradigm including those tools that are more easily applied in each school that will allow to support and stimulate the learning of students to help them reach their full potential.

This research did not receive any specific grants from funding agencies in the public, commercial or non-for-profit sections.

Ethical statement is incorporated with files attached. Authors agree with ethical code of the editors.