Concept maps and simulations in a computer system for learning Psychology

Abstract

PSICO-A is a computer system for learning Psychology. It is specially designed for secondary school children. It is the first system in Psychology designed for learning didactic units of the subject. PSICO-A is based on many pedagogical influences, such as concept maps, free retrieval practice, effective feedback, simulations, digital games, and metacognition. A significant improvement has been shown in the conceptual performance in those children that constructed computer-generated maps using the system compared to those that have drawn them by hand. An evaluation was also made of the interactions between concept mapping and simulations, demonstrating that the first group of pupils performed better in simulations than the second group. Further studies are needed to study the influence of these two conditions of concept mapping on the performance of digital games.

Resumen

PSICO-A es un sistema informático para el aprendizaje de la Psicología. Está especialmente destinado a estudiantes de Educación Secundaria. Es el primer sistema integrado computacional concebido para la enseñanza de unidades didácticas de Psicología. PSICO-A combina diversas herramientas e influencias didácticas: introduce mapas conceptuales, recuperación libre del recuerdo, un mecanismo efectivo de “feedback”, simulaciones, juegos digitales y explora la capacidad metacognitiva de los alumnos. Hemos confirmado una mejora significativa en el rendimiento conceptual en aquellos alumnos que construyeron mapas conceptuales a través del sistema frente a aquellos que tuvieron que trazarlos a mano. Además, hemos analizado la interrelación entre mapas conceptuales y simulaciones, comprobando que el primer grupo de alumnos rindió más en una simulación que el segundo grupo. Quedaría para un trabajo futuro demostrar qué sucedería si, después de realizar el mapa conceptual (en sus dos condiciones), los alumnos fueran expuestos a un juego relacionado con una unidad didáctica de la asignatura.

Introduction

PSICO-A is a computer system designed for teaching Psychology students of Secondary Education and first courses of University. Taking as reference agent-based systems like MetaTutor (Azevedo, Witherspoon, Chauncey, Burkett, & Fike, 2009), Betty's Brain (Davis et al., 2003) and REAL (Bai, Black & Vitale, 2007), it is a modular design system that introduces games and simulations inspired by Black's “representational theory” (Black, 1992), according to which, knowledge is best represented through images (Finke, 1989) and mental models (Gentner & Stevens, 1983) that configure virtual learning environments (Jonassen & Land, 2012). PSICO-A introduces teaching tools like concept maps (Novak, 1977), the free memory retrieval (Karpicke & Blunt, 2011), a mechanism of “feedback” self-generated (Slamecka & Graf, 1978) and the emphasis on metacognition (Dunlosky & Metcalfe, 2008) through the ideas of the global metacognitive model of Mayor, Suengas, and González Marqués (1993).

PSICO-A was developed as a system for the web in language PHP5 (Lerdorf, Tatroe, & MacIntyre, 2006) programming, but is also available as an accessible from the desktop of a computer program (the desktop version has been initially tested in Microsoft Windows XP and Microsoft Windows 7) operating systems. Our system has own hosting domain “psico-a.org” and the URL is http://psico-a.org/secure/login/.

PSICO-A consists of a “front-end” or user interface, which is the area in which the student interacts with the system and, in turn, of a “back-end” or teacher interface. For a detailed description of the system and its computational architecture can be seen (González Marqués & Pelta, 2013).

As is well known, one of the most interesting legacies of the educational constructivist theory of Ausubel (1963) is the concept of “advance organizers”. It is an introductory-type of material located at a higher level of generality and inclusiveness than the material to be studied, and provides support for the incorporation of the material that students should learn. According to Ausubel (2002, p. 117), the role of the organizers is to “provide an ideational anchor for the stable incorporation and retention of more detailed and differentiated learning materials.” The organizers must explicitly relate to the specific fragment that follows and must also be linkable with the established ideas in cognitive structure. This process that connects new information with existing, relevant and higher-order informational segments, Ausubel (2002, p. 155) is called “subsuming” learning. Thus, the emergence of new propositional meanings reflects a subordinate relationship of the new material with higher-order thoughts, creating a hierarchical organization of the cognitive structure. It is precisely like this how the concept maps-like advance organizers – “organize the new facts related to a common theme, integrating the elements of the new and the existing knowledge with each other” (Ausubel, 2002, p. 156). Novak (1977) showed that concept maps were a very suitable technique to serve as advance organizers. Concept maps are graphical schemes consisting of concepts, word-links used to connect the concepts and propositions that form a semantic unit configured by the interrelated concepts. Concept maps are characterized by their hierarchical nature (the most inclusive concepts are at the top and the examples in the lower part), they are selective (they summarize the most important or significant) and promote visual memory.

The process of creating a concept map can be tedious and even boring for students. If pen and paper are used to build it, you often need some time when the complexity of the material to be learned is not minimal (Schau, Mattern, Zeilik, Teague, & Weber, 2001). In addition, revisions tend to be complicated and even frustrating (Chang, Sung, & Chen, 2001). The advent of information technology has enabled the introduction of computers into small schools. Computerized concept maps are easy to build and revise (Anderson-Inman & Ditson, 1999; Plotnick, 1997; Zeitz & Anderson-Inman, 1992). Errors in describing concepts can be easily modified and students can customize their maps more effectively than using pencil and paper. Indeed, the first authors to demonstrate the superiority of performing electronic or digital concept maps versus traditional technique were Anderson-Inman and Zeitz (1993) and since then, their results have been replicated by other authors. And so, Royer and Royer (2004) used the software “Inspiration” to confirm that students who used them, designed more complex and precise than that created using pencil and paper maps. In turn, they were much more motivated on the task. The first aim of this study is to see whether PSICO-A improves the production of concept maps against the old method of pen and paper but we also take care to analyze the effect of its construction on performing computer simulations.

Hagemans, van der Meij, and de Jong (2013) showed that the prior execution of concept maps (conventional or marking the deployed concepts with colors) improved the performance of the subjects in a kinematic learning activity, based on a simulation. His experiment investigated whether a concept map could improve the performance of a group of high school students in a simulation that raised a classic problem of a mobile speed. The management concepts on the simulation screen followed the management concepts in the concept map. In the simulation the students could manipulate the variable speed and put in action the movement of the car. When students thought they knew the right answer, they could stop the simulation and they received a feedback about the correctness of that. Students in the experimental group were significantly benefited from the presence of the concept map on the screen before the simulation, compared with the control group students who did not have this support tool. Moreover, when in another experimental condition they were presented with a concept map with the concept boxes colored, displaying the progression of the map, it was found that the student's performance was still above those of the first experimental condition. It is as if the colored concept map generated a learning route in the minds of the students that specified a sequence of appropriate codes to work in a certain domain of knowledge; that is, the concept maps as “advance organizers” would help students acquire a better knowledge of the domain structure furthermore reinforced by dynamic clues given by the colors. In short, exposing students to an optimal learning route clearly marked through concept maps would support the planning and monitoring of it and it would them to control settings such as simulations. This only serves to emphasize the importance of presenting information in the form of an organized structure for learning and not present it at random (Glynn, Yeany, & Britton, 1991).

Increasingly frequent studies show that individuals benefit from learning in environments such as simulations only if they receive some support (Alfieri, Brooks, Aldrich, & Tenenbaum, 2011; Mayer, 2004). Concept maps are privileged instruments in this sense because, as good advance organizers, they inform the learning material to the students and facilitate them to link such material to the already learned. If they also transmit the information in a visual format, its effectiveness will be understood as an aid to learning environments based on simulations (Hagemans et al., 2013). In this article, we consider whether the design of a concept map using PSICO-A facilitates the performance of students in the corresponding simulation compared with classic pencil and paper. In PSICO-A, the concept map consists of a series of boxes occupied by concepts connectable to each other through various types of arrows that express three types of relations: causal, hierarchical and descriptive. Students must access the Mode menu in a way of “Game” and “Simulation”. The simulation consists of a simple animation and game is based on a digital game. In both our virtual agent intervenes (Del Bimbo & Vicario, 1995), a white rat called MOUSI that their behavior follows the parameters defined by Crespi (1942) in his experiment. This experiment influenced the reformulation of the drive theory reduction by Hull (1952). This theory is the content of the teaching unit section as a basis for our experimental design. The isomorphism between the information, as it is organized in concept maps, and the simplified representation thereof as may be organized in a simulation or a game, could explain the generation of very suitable for learning mental models. In PSICO-A, the task required by the simulation and the game and the distribution of parameters in both fits very well to the data studied by the students and the development of their concept maps. In addition, and in line with the improvement produced in small learning tests (Roediger, Agarwal, McDaniel, & McDermott, 2011), we added small activities that allowed the next level both in simulation and in the game. Following Forbus and Gentner (1997) or Thomas and Neilson (1995), we have designed an environment as a simplified representation of a real experiment that allows students to develop mental models very suitable for learning. Obviously, it also has a principle of parsimony pedagogical: not unnecessarily complicate the approach of the students to these learning tools.

A theoretically interesting point of our study is the influence of concept mapping on the performance of the subjects in simulations. Does the use of generation system concept maps could influence the performance of students in the corresponding simulation? To try to answer this question is addressed also our experimental study.

Method

Participants

The participants were 68 students with an average age of 17.4 years (SD = .76) of two High schools in Madrid. Both groups have been integrated by students of Psychology as an optional choice in the Second Year of Bachillerato (A-level). The sample was divided into two groups, an experimental group (EG) and a control group (CG), each corresponding to students enrolled in the course of each center. In the distribution by gender, 29 (43.7%) were male and 39 (57.3%) were women.

Materials

It was applied the computer system PSICO-A (González Marqués & Pelta, 2013). It consists of a “front-end” or user interface and, in turn, of a “back-end” or teacher interface. In the “front-end”, we used only the didactic unit, concept map and simulation zones. The didactic unit was based on the textbook of Psychology (Alonso, 2012).

Design and procedure

To determine the effectiveness of the process of generation of concept maps in PSICO-A, the study included, as independent variable, using the system to make concept maps and the dependent variable, the students’ performance in the following two parameters: number of concepts in the concept map and conceptual relations percent correct in that map.

This was a design of two groups: the control group (CG) or construction of concept map using pencil and paper, had 33 students and the experimental group (EG) or construction of concept map using the automatic generation method of the system, was composed of 35 students. Both groups used only PSICO-A to access the study of the didactic unit (the two groups), the concept map design mode (in the case of group EG) and simulation mode (in the case of both groups). The two groups performed a pre-test before starting the experiment (phase 1). This pre-test was used as a covariate control analysis of individual differences and previously demanded to explain the methodology for creating concept maps with three types of relations: causal, hierarchical and descriptive. In the pretest, students had 15 min to study Section 5.3 of the textbook of Psychology (Alonso, 2012). This section introduces the basics of psychological experimentation such as the dependent variable and independent variable. From there they had to draw using pencil and paper for 20 min, a concept map of the fragment studied. Subsequently, the two groups performed the post-test (phase 2). This time, the students had to account for 15 min, the corresponding study material to paragraph 2.1 about drive reduction theory. While the CG group had to make the map in hand (for 20 min), after studying the content of the paragraph, the EG group built the map in the system itself. Finally, both groups run the simulation for 10 min.

All the sessions take place on a day when there was no interference with examinations or tests of other subjects. The meetings were performed in parallel, i.e., the same day, the same time, attending two professors the sessions in the computer room.

Data analysis

We have used the SPSS 14.0 statistical program and have accepted a degree of significance of .05. A Levene's test and an One-Way ANOVA were used to determine if there was a significant difference in the pre-test received by the groups. After evaluating the results, there were no significant differences between groups in any of the categories, in other words they were homogeneous groups regarding their performance on the test.

In the post-test we have evaluated the same parameters as in the pre-test. In each case an One-Way ANOVA for comparison between groups was applied. The effect size was measured by Cohen's d. Then we analyzed the results by different parameters or categories.

Results

In the generation of concepts in the concept map, the post-test made a significant difference in the number of concepts in favor of the EG group, as revealed ANOVA, F(1, 66) = 237.92, p < .001. The effect size was high (see Table 1).

Table 1.

One-Way and repeated measures ANOVAs for the number of concepts generated in the concept map.

	Pretest				Postest				ANOVA pre-test			ANOVA post-test
	Experimental		Control		Experimental		Control		F(1, 66)	p	d	F(1, 66)	p	d
	M	SD	M	SD	M	SD	M	SD
Number	7.52	1.12	7.46	.95	13.69	1.40	8.24	1.50	.05	.818	.02	237.92	.000^*	.88

Note: d = Cohen's d. Experimental n = 35, control n = 33.

*

p < .05.

Regarding the percentage of correct relations in the concept map, ANOVA, F(1, 66) = 191.65, p < .001 found significant differences in the post-test between the groups, also surpassing EG to CG. A team of three teachers judged the correction of relations (see Table 2).

Table 2.

One-Way and repeated measures ANOVAs for the correct relations in the concept map.

	Pretest				Postest				ANOVA pre-test			ANOVA post-test
	Experimental		Control		Experimental		Control		F(1, 66)	p	d	F(1, 66)	p	d
	M	SD	M	SD	M	SD	M	SD
Relations	51.29	7.79	48.64	6.76	92.43	9.42	53.18	13.68	2.22	.140	.17	191.65	.000^*	.85

Note. d = Cohen's d. Experimental n = 35, control n = 33.

*

p < .05.

It was suspected that in the post-test there could have been some correlation between the ease to generate concepts in the concept map and the student's performance in the simulation. This is a theoretically interesting question that will be addressed in the discussion of the experimental results, since it seems that the influence of pedagogical tools are well tested as the advance organizers (concept maps, etc.) in simulations solving. It is known that a correlation does not necessarily imply a causal relationship but to locate it can be a very good sign to explain a possible influence of authentic theoretical importance. Therefore, we investigated the following correlations:

• (A)

  Correlation between the number of concepts in the concept map and performance and simulation performance: here there was a significant correlation between the average number of concepts generated by the EG group and the high scores obtained in the simulation, putting perhaps evidence the influence of an “advance organizer” (Ausubel, 1963) as concept maps (Novak, 1998), on the development of mental models that enable a better performance in a simulation which, in turn, increases general learning on one subject by the students (Hagemans et al., 2013). It could also be an indirect connection between making concept maps using a computer program and increased student performance in simulations compared to traditional pen and paper technique.

• (B)

  Correlation between the number of correct relations in the concept map and performance in the simulation: there was a significant bivariate correlation.

We show these results in Tables 3 and 4.

Table 3.

Pearson's correlation between number of concepts in the map and performance in the simulation.

		Map	Simulation
Map	r	1	.324^*
	Sig.		.033
	N	68	68
Simulation	r	.324^*	1
	Sig.	.033
	N	68	68

*

Correlation is significant at the .05 level (2-tailed).

Table 4.

Pearson correlation between percentage of correct relations between concepts in the map and performance in the simulation.

		Map	Simulation
Map	r	1	.950^*
	Sig.		.000
	N	68	68
Simulation	r	.950^*	1
	Sig.	.000
	N	68	68

*

Correlation is significant at the .05 level (2-tailed).

There were indications that the profit achieved by the EG group in generating concept maps using PSICO-A, was not limited to mere recovery memorization of concepts, but impacted on a greater understanding of what had been studied, and therefore a better performance in the simulation proposal.

An One-Way ANOVA of the difference in percentage points regarding the maximum score achievable in the simulation (620 points), between CG and EG, allowed to go beyond the above-established correlations, emphasizing the significance of the found differences (see Table 5).

Table 5.

One-Way ANOVA of percentage points of the simulation.

	Experimental		Control		ANOVA
	M	SD	M	SD	F(1, 66)	p	d
Percentage points	89.34	7.94	45.15	9.64	427.57	.000^*	.92

Note: d = Cohen's d. Experimental n = 35, control n = 33.

*

p < .05.

Discussion

Using PSICO-A appears to confirm that the design of concept maps as “advance organizers” exceeds the use of pen and paper in terms of richness of maps, in line with other results previously achieved (Anderson-Inman & Zeitz, 1993; Royer & Royer, 2004). On the other hand, we could have an indirect evidence that tools used for the generation of maps help to improve student achievement in computer simulations. As regards the number of concepts in the concept map, Karpicke and Blunt (2011) stated that the concept mapping without having in front the text to study, is also a practical demonstration of information retrieval. The improvement brought about by the system could be explained by the superiority of an automatic method of constructing concept maps (e.g. CmapTools, Cañas, 2004) compared to using traditional pen and paper. If we analyze the percentage of correct conceptual relations (Ruiz-Primo & Shavelson, 1996) in the concept map, in the post-test, F(1, 66) = 191.65, p < .001, there was a EG group clear superiority over the other group. A bivariate correlation analysis revealed a correlation between the number of concepts and the percentage of correct relations and the yield obtained in the simulation by the EG group. Although it is known that correlations do not necessarily imply causal explanations, we found a significant correlation at the .05 level between the number of correct relations in the concept map and performance in the simulation of PSICO-A, r(68) = .32, p = .03, and the number of concepts in the concept map automatically generated and performance in this simulation, r(68) = .95, p < .001. These results could credit the beneficial influence of concept maps on the simulations and could indirectly support the findings of Hagemans et al. (2013). In fact, an One-Way ANOVA showed that the EG group scored comparatively more in the simulation students CG group, F(1, 66) = 427.57, p < .001.

It is commonly accepted that the games are useful for learning because they involve understand and manipulate many variables (Greenfield, 2010). However, Charsky and Ressler (2011) have shown that the concept maps do not seem to be an effective tool for improving motivation of individuals involved in computer games. Previously, Charsky and Mims (2008) had suggested that in order for computer games to be useful for instruction, they need to be complemented by activities that help students relate the games with the theoretical content and, among these activities could be the design and the visualization of concept maps. In their experiment they examined the effect of using concept maps in learning History concepts through the game “Civilization III”. They divided a sample of American ninth graders into three groups: one group that, prior to use of the game, had manipulated an automatically generated concept map, a second group that built their own concept maps and a control group that did not use any concept map. The individuals who used concept maps showed a lower motivation (assessed in terms of trust, satisfaction and care) in the game that the students in the control group. According to the authors (art. cit., p. 604), concept maps can reduce such motivation because they focus the students’ attention too much on the inherent difficulty of the conceptual learning and convert the game into a less autonomous, creative and active task. Indeed, we have performed an experiment that we could have again confirmed the superiority of concept mapping by computer and perhaps more importantly, an indirect evidence to support the findings of researchers who insist on the importance of concept maps to help improve performance in the simulations. In the immediate future it would be interesting to check if the concept mapping in PSICO-A has a positive influence or no on the resolution of digital games by students.

Conflict of interest

The authors of this article declare no conflict of interest.