Portfolio is a new tool that gathers evidence of knowledge, abilities, attitudes, and values. This form of evaluation has arisen in the context of a competency based learning and evaluation. Conventional portfolios can be mainly divided in summative and formative. Summative portfolio can evaluate the final learning product of the student and formative portfolio can guide the student through the learning process with various evaluations. Also, electronic portfolios have increased the range of evidences that can be added to the portfolio, is highly accessible and is more agreeable to students. Recommendations are made on how to start and maintained a portfolio in medical residencies.
El portafolio es una nueva herramienta que recopila evidencia de conocimiento, habilidades, actitudes y valores. Esta forma de evaluación ha surgido de la necesidad de un aprendizaje y evaluación basados en competencias. Los portafolios convencionales se dividen principalmente en sumativos y formativos. El portafolio sumativo evalúa el producto final del estudiante y el portafolio formativo guía al estudiante a través del proceso de aprendizaje mediante numerosas evaluaciones. Además, el portafolio electrónico incrementa el rango de evidencias que pueden ser añadidas, es altamente accesible y cuenta con una gran aceptación por parte de los estudiantes. Se realizan recomendaciones de cómo empezar y mantener un portafolio en las residencias médicas.
By definition, competency-based learning involves acquiring a combination of cognitive, sensory, and motor competencies and abilities. This type of education developed based on the needs resulting from modernization and industrialization. Previous educational models focused exclusively on knowledge acquisition and not on performance.1,2
The curricular advances that are being made around the world are impossible to ignore. In addition to modifying to competencies, programs, and objectives of courses, evaluation methods should also be updated. We will not reach the educational forefront if courses continue to be evaluated as they have been traditionally. We are approaching an educational revolution that represents a fundamental change in the way that we teach and evaluate.
In our path toward comprehensive competency-based learning in which the teacher participates not only as a trainer of independent students but also as a facilitator of self-teaching, new tools of measurement and evaluation are fundamental.1–3 The portfolio is a new tool that can encourage reflection and self-evaluation, in addition to being a collection of evidence of learning and experiences.4,5 As portfolios can be either physical or electronic, both modalities are addressed in this work.
It is therefore important to discuss evaluation, which is defined as an act that gathers data of a person, deed, situation, or phenomenon to deliver an evaluative judgment.3 Various authors have expressed that the solitary performance of the traditional exam is insufficient as the final evaluation of a student.1,2 Nevertheless, despite this shift in the literature, exams remain the only form of evaluation in the majority of graduate programs; thus, it is crucial to make progress in creating new evaluation processes, particularly at the graduate level.
A portfolio can be valuable when it is used to evaluate a graduate student. However, a portfolio is a tool that remains under-utilized in graduate medical studies and in graduate programs in general. A portfolio can be used when students are expected to attain more than theoretical understanding of a subject; in other words, portfolios are particularly useful when the educational goal is to shape abilities, competencies, analysis, deep understanding, and a reflection of the learning process, among other things.4,5 Notably, comprehensive evaluation of student performance is often critical, particularly in a competencies-based curriculum.2
In addition to being heterogeneous and dynamic, a portfolio can be an instrument for teachers’ reflection and students’ self-reflection.2,5 It accumulates evidence of performance and allows for this evidence to be properly considered,4,5 particularly evidence regarding whether a student has acquired the competencies required by a specific course. Significant learning arises from critical analysis and reflection.1,2
The basis for the use of portfolios emerged from pedagogical constructivism.1 This foundation is based on the acquisition of a new understanding through analysis and by reflecting on previous knowledge, with its subsequent application to new and diverse situations. Along with pedagogical constructivism, a comprehensive evaluation of learning allows us to evaluate, analyze, reflect, and act on learning objectives. In order words, to perform a formative evaluation.2 Thus, a portfolio may be the ideal tool to engage in this evaluation and improve student learning.
PortfolioPortfolios began to be employed in pedagogy in 1980 and were initially used by architects, photographers, designers, etc. In 1991, Paulson wrote that the portfolio is a collection of student works that reflects the progress, effort, and achievements of an individual. Portfolio and electronic portfolio is the recollection of evidence of individual development, evaluation, student motivation and self-assessment.4
Thistlethwaite5 described portfolio as a collection of documents that provides proof of learning and a reflective analysis of the documented events. The portfolio should show that the collector has acquired the necessary competency level in his/her training.
A portfolio can apply both to teachers and students. Universities can use portfolios to document teaching and learning in graduate programs.
There are various types of portfolios,2 including:
- •
Work portfolios: general and deliberate collections of evidence that show that the student is meeting the objectives of a specific project;
- •
Exhibition portfolios: selections of best works; and
- •
Diagnostic evaluation portfolios: documentations of learning that facilitate feedback on a student's performance.
Portfolios aggregate the evaluation of knowledge, abilities, attitudes, and values. In addition, they create a dynamic learning environment because they also allow for self-evaluation. Both professors and students can self-evaluate based on the results presented in portfolios.
Portfolio learning can be understood as a type of experience-based learning. According to Rodríguez-Weber,6 the following 4 abilities should be present in any experience-based learning:
- •
Drawing from experience;
- •
Performing reflective observation;
- •
Conceptualizing or deducing abstract generalizations; and
- •
Applying knowledge to new situations.
According to other author,7 portfolio learning consists on the following:
- •
Identifying experiences that the student defines as significant;
- •
Identifying the fact that learning is derived through experience;
- •
Specifying how this learning is demonstrated in practice; and
- •
Identifying additional learning needs and methods with which to accomplish these actions.
The graduate student, in reflection on what was learned, achieves a combination of previous learning with new learning and thus acquires significant understanding that can be applied to practice and in various situations.1 Rodríguez-Weber6 emphasizes performance-based evaluation and is thus able to focus on the competencies acquired by the student.
As discussed above, a portfolio represents a solution for evaluating a competencies-based program. Furthermore, a portfolio requires the student's active participation in converting the portfolio into a student-centered tool.8 The student can complete and be responsible for most of the content, which thus promotes reflection.
Formative and summative evaluations in portfoliosThere are 3 phases in the evaluation process: initial evaluations, formative evaluations, and summative evaluations,3 which are defined as follows:
- •
“Formative evaluation: Is a systematic and continuous activity, with the objective of providing necessary information about the educational process, to readjust its objectives, critically review plans, programs, methods, and resources, orienting students, and providing feedback on the process itself.
- •
Summative evaluation: the activity of synthesis of a topic, course, or educational level which allows recognition of whether students have achieved the desired results in function of the proposed situations of teaching and learning”.
We conceive of formative evaluation as a method of following the student in their successes and failures and as an invitation for guidance from the tutor to help the student improve and acquire the competencies that are the objective of a course. The student controls the learning goals and creates plans of action. Formative evaluation is thus a personalized academic guide.
Summative evaluation is also valid and necessary in some graduate courses and is generally performed at the end of a course or school year. The quality and total number of items are evaluated and whether the student has achieved the desired competencies is then determined. There are various rubrics used to evaluate the student's final product. This evaluation invites less student reflection but offers a measure of total competency that is more rigid and complete. The product of a summative evaluation is a final numerical score.
Fig. 1 explains the various ways in which a portfolio can be implemented. Different authors6–14 recommend various uses for portfolios. For example, some authors recommend that a portfolio only be used to document evidence, and they use the portfolio as a summative evaluation. In the final evaluation, the portfolio is created by the student who should have met certain requirements and who finally receives a quantitative evaluation based on the use of a checklist or rubric. By contrast, a formative evaluation includes reflection on the content and subsequent improvement of competencies and abilities based on a plan that is outlined by the student and the teacher.
A portfolio can also be used as a tool for self-evaluation, the hetero- and co-evaluation of student and teacher, which assesses the learning acquired based on evidence. A portfolio is also an invaluable tool in a competencies-based graduate program because it documents cognitive, sensory, and motor competencies and abilities.2,5
In graduate programs – and particularly in health sciences – portfolios help to assess student progress over the years, facilitating reflection on what was learned and presenting a current picture of acquired competencies. In addition, a portfolio facilitates an organization of what has been learned, in order to generate realistic goals. For students, a portfolio makes it easier to demonstrate their process of learning when they apply for work or to another graduate program, and this benefit also applies to teachers’ portfolios.5
Some studies suggest the simultaneous use of summative and formative evaluations in portfolio applications. Other authors, however, prefer formative evaluations.10 In our opinion, a formative evaluation is more likely to result in significant student learning. In this manner, the portfolio's scope is multidimensional because it achieves the metacognition necessary in the student.
It is recommended that the teachers in charge of the formative evaluation are not the same as those in charge of the summative evaluation because performing both roles interferes with the objectives of reflection. However, this is a decision for the head of the graduate program.
Dannefer,12 a professor at the Cleveland Clinic (Department of Medicine), developed an evaluation model in which the summative and formative portfolio are the top evaluations in a medical residency based on competencies.
Practical adviceWhen beginning to implement portfolio evaluations in a graduate program, it is essential to train both students and teachers,13 as these groups must know the purposes of portfolios and what is expected to comprise their contents and how they are to be created. In the case of electronic portfolios, it is recommended that training be directed toward the use of portfolio technology. Before implementing portfolios, an introductory class should be provided.
If this is the first experience with portfolios at the institution, it is advisable to begin with the most indispensable of contents13 (the material that is already on hand), to allow for the gradual acceptance of portfolios in this context. It is undeniable that portfolios require more work on the part of students and teachers, which can result in some reticence and obstacles to their application.
An exhibition portfolio is also a practical and motivational way to begin this process. All personnel can be invited to participate with their best work examples.
On occasion, it is important to take specific actions to enrich our portfolios, whether through projects, self-evaluations, or reflection. We do not have to consider only those activities that already form a part of our teaching routine.
The following are some recommended self-reflection questions.
What do I know about this topic? How did I learn it? What else should I learn? How can I learn it? These questions represent cases or situations with a certain degree of complexity, unfamiliarity, or unexpected results that we should evaluate regularly, and we should propose goals that help us clarify issues and learn.5
Self-evaluation can be assessed using the scale devised by Dreyfus,15 as described in Table 1.
Evaluations can be performed monthly, each semester, or annually; the more frequent the evaluation, the more effective they are at estimating a student's progress. Furthermore, to ensure that students comply with the purposes of portfolios, it is recommended that teachers request frequent feedback from students and that filling portfolios does not consume an inordinate amount of time.
To prepare an electronic and/or conventional portfolio in graduate work, the following should be considered.6–14
- •
Objective
- •
Students
- •
Teachers
- •
Audience
- •
Content
- •
Design
- •
Management
- •
Communication
- •
Evaluation method
- •
Technology used
Once these points are defined, the person responsible for creating the portfolio will be able to formulate it more effectively.
For purposes of evaluation, the portfolio should have the following elements.6–14
- •
A structure for learning;
- •
A record of learning;
- •
Evidence from reflective practices; and
- •
Evidence from the learning cycle.
When creating a portfolio for medical graduate work, the following points are recommended for students.
- •
Title page
- •
Index or table of contents
- •
Curriculum vitae
- •
Plan of study
- •
Education
- •
Summary of articles
- •
Tests
- •
Classes, etc.
- •
Experience and skills
- •
Practical evaluations
- •
Skills according to each branch
- •
Reflection
- •
Analysis of exams
- •
Analysis of cases
- •
Self-evaluation
- •
Critique of articles
- •
Publications
- •
Conference presentations
- •
Prizes and awards
- •
Learning plan
- •
Research projects (thesis)
Various studies,6–14 particularly in graduate programs in medicine, demonstrate the success of portfolios in numerous courses. Some uses are shown in Table 2. Unfortunately, the majority of studies are recommendations and satisfaction surveys that do not describe the results derived from this educational technique.
Studies documenting the use of portfolios in graduate medicine.
| Author | Course | Implementation method and results |
|---|---|---|
| Duque16 (2006) | Geriatrics | Authors used an introductory class; achieving 30 evaluations monthly. |
| Clay17 (2007) | Critical medicine | Authors enumerated the list of competencies for development; performed exercises directed toward self-reflection; performed exclusive didactic exercises for the portfolio; multidisciplinary; created a web page as a showcase portfolio. |
| Ruiz-Moral26 (2009) | Family medicine | They undertook a formative portfolio. The authors concluded portfolio may be useful in rotations and in strengthening the relationship between teacher and resident. |
| Cabalier9 (2009) | Dermatology | Authors reported the following results: 54.78% (n=229) created a complete portfolio, 38.99% (n=163) completed 80% of extra work necessary to complete the portfolio; 6.22% (n=26) did not create a portfolio. Students completed reflection on their learning process in general. |
| Casanova10 (2011) | Dermatology | Suggests a list of competencies to be acquired and activities to acquire each. |
| Donato13 (2012) | Internal medicine | Suggests training, gradual implementation, and convincing of teachers; reports a rubric based on the reflective capacity of students, in ascending order: minimal effort to report, interpret, manage, and/or teach. |
Electronic portfolios have emerged as a new modality of portfolio16–18 that has arisen with the advent of information technology and a society that is continually more technologically sophisticated.
One of the advantages of electronic portfolios, or e-portfolios, is that they can be viewed from any computer or electronic device. Because of their accessibility, the student and teacher are more likely to be able to use its content and to thus achieve the reflective learning that is sought.
An e-portfolio can have the same content guidelines as a conventional portfolio, with the advantage that videos, images, and voice recordings can be attached.
Although there are specialized and advanced programs for e-portfolios, these programs require special training and, in most cases, come at a price. These portfolios have been adopted by numerous university programs in the United States and the United Kingdom. Nonetheless, there are other free programs for e-portfolios available to any teacher and student that will be described below.
One of the disadvantages of free programs is the low amount of storage. Thus, some of the information should be uploaded as links (from DropBox, YouTube, SlideShare, Google Drive, etc.). Moreover, when choosing a portfolio service, it must be determined whether information will be managed privately or publicly. In some portfolios, privacy is recommended (accessible by the advisor and student only), particularly with regard to evaluations, self-evaluations, and personal reflections.
We recommend the following sites,19–25 among others:
Blogging in blogger.com,19 is the easiest to use, great privacy control, and comes with widgets. Wordpress20 is easy to use, popular, and versatile. Mahara21 is a pre-established portfolio design, requires training. Edu-portfolio22 is from the From the University of Montreal, requires training. Education Blogging in edublogs.org,23 is a free basic service, other services require a fee. Gloogesites24 offers web pages with more control and organization of information, privacy management, easy to use. Wikispaces25 is another useful program for various learning activities, including the electronic portfolio.
An electronic portfolio, particularly one that is available to the entire public, has an important mission as an educational tool in and of itself. This mission becomes even more important when using tools such as YouTube and SlideShare, which provide information to anyone. At Hospital Civil de Culiacán, we have published various videos on YouTube and classes on SlideShare that have been visited by the general public, which grants (international) public access to the program and students.
Education should be open and accessible to all students. This is the great change that Information and Communication Technologies (ICTs) have brought in terms of the democratization of education.
Disadvantages of portfolio useThe time consumed in the completion and organization of a portfolio is a disadvantage that has repeatedly been shown by articles assessing portfolios as an educational tool.5,26 It is important to us as evaluators that we concentrate more on content and reflection than over design and form. Although it is important to have a balance of both content and creativity, in graduate studies, we consider content more important than creativity, which is evaluated, for example, in lower school grades.
Other student criticisms that have been documented26 include portfolios that require daily evaluations or exhausting self-evaluation content. There should be a certain freedom for the student to express his or her successes, reflections, and self-evaluations. A portfolio should not require a certain number of pages but should instead provide a structure that awaits construction based on the free expression of each student. We recommend monthly evaluations – including evaluations with the teacher each trimester or semester – and entries in the portfolio each month, which should not exhaust teachers and students.
A portfolio should be a user-friendly tool that is useful in the development and upkeep of information.
The experience of implementing a portfolio in a teaching hospitalFormative and summative evaluations in portfolios are currently being implemented at Hospital Civil de Culiacán. The graduate department has portfolios that document the progress of all students, and these emphasize educational progress not only in research but also in administrative procedures, such as rotations, vacations, and licenses. These portfolios allow teachers to have clear and up-to-date evidence regarding students’ progress, and they serve in the external supervision from the head office that oversee the quality of educational institutions.
In addition, portfolios have been utilized in every service that is a basis for the formative evaluation of students. These portfolios include more details, the educational progress of students, exams, rubrics, checklists, research, etc. This evaluation allows teachers to more objectively observe the abilities and competencies that are acquired by students during the year and to compare them easily with their peers. Even more importantly, different learning techniques or goals can be planned from the correct evaluation of each portfolio, as the result of a more personalized learning. For this reason, student portfolios represent an important reflection of teacher activity.
Additionally, media such as YouTube and Vimeo could be used to upload conferences and classes, and SlideShare has been utilized to uploaded PowerPoint presentations and photos from classes. Since current students prefer technology, e-portfolios could be more appealing and in this way increase the consciousness of their own learning and could come closer to meta-cognition. Meta-cognition is the ability of the student to understand and be aware of his/her own process of thinking and learning.1,2 For example, students can see and analyze themselves during conference or poster presentations and view their classes or articles; which is impossible to achieve with paper portfolios. There is also the issue of increase availability of information–for residents and mentors- with e-portfolios.5 Currently, we are conducting research at our organization regarding reflection in formative evaluations, as enabled by portfolios.
Resistance to change in modes of evaluation from both teachers and students is natural and is caused by a long history of objective-based education. However, this resistance can be alleviated with training and with the assignment of personnel who are exclusively in charge of supervising portfolios. A professor, teacher, service or administrative personnel can assist in verifying the completion of portfolios.
Portfolios have been implemented in various residency programs as a formative tool with good results. The use of portfolio is recommended in a graduate competency based program.
ConclusionsThe portfolio is a valuable tool for classifying evidence and promoting reflective learning in a competencies-based graduate learning environment.
Conflict of interestThe authors of this article declare no conflict of interest.
We acknowledge the support of CIDOCS and the master's program: teaching in health sciences at the Autonomous University of Sinaloa (UAS).
This article is based on the presentation, “The portfolio as a tool for learning and evaluation in medical residencies” that was given in October 2013 at the 1st International Congress: Common Space for Teacher Training, Mazatlán, México.
