The Effect of Adjunct Questions Emphasizing the Particulate Nature of Matter on Students' Understanding of Chemical Concepts in Multimedia Lessons

Abstract

This study investigated the use of three types of adjunct questions to enhance and assess student learning of molecular-level chemistry concepts. Subjects were 98 volunteers in a college–level general chemistry course. The subjects were divided into three groups, each of which completed the same multimedia computer lessons, but different types of adjunct questions: (a) text-based macroscopic/symbolic questions, (b) text-based questions emphasizing the particulate nature of matter, and (c) pictorial questions emphasizing the particulate nature of matter. A conceptual test was administered, consisting of three parts: (1) macroscopic/symbolic text-based items, (2) molecular-level text-based items, and (3) molecular-level pictorial items. No significant differences were found among the three groups on any part of the test using scores on the TOLT (an assessment of formal reasoning ability), FIT (an assessment of mental capacity), and a pre-test as covariates. There was a significant positive relationship between TOLT scores and each of the three parts of the test. There was also a significant positive relationship between FIT scores and scores on the macroscopic and pictorial parts of the test. Formal reasoning ability accounted for more variance among test scores for all three parts of the test than did mental capacity.

Resumen

Este estudio investigó el uso de tres tipos de preguntas adjuntas para mejorar y evaluar el aprendizaje estudiantil de conceptos de química a nivel molecular. Los sujetos fueron 98 voluntarios en un curso de química general a nivel universitario. Fueron divididos en tres grupos, cada uno de los cuales completó las mismas lecciones multimedia computacionales, pero con diferentes tipos de preguntas adjuntas: (a) preguntas macroscópico/simbólicas basadas en textos, (b) preguntas basadas en textos con énfasis en la naturaleza corpuscular de la materia, (c) preguntas pictóricas con énfasis en la naturaleza corpuscular de la materia. Se les puso una prueba conceptual consistente de tres partes: (1) preguntas macroscópico/simbólicas basadas en textos, (b) preguntas basadas en textos sobre el nivel molecular, (c) preguntas pictóricas a nivel molecular. No se encontraron diferencias significativas entre los tres grupos en ninguna parte de la prueba utilizando las notas de TOLT (que evalúa la capacidad de razonamiento formal), FIT (que evalúa la capacidad mental) y una prueba previa como covariantes. Existió una relación positiva significativa entre las calificaciones de TOLT y cada una de las tres partes de la prueba. También hubo una relación positiva significativa entre las calificaciones de FIT y los resultados de las partes macroscópica y pictórica de la prueba. La capacidad de razonamiento formal registró más varianza que la capacidad mental en las tres partes de la prueba.

Introduction

Chemistry is regarded by chemists as a well-ordered realm of atoms, molecules, and ions, governed by scientific laws. However, to beginners, chemistry may be perceived as a collection of sometimes smelly reactions described by arcane equations, with little relationship to the molecular level. Chemistry's nature as a science is characterized by three interrelated levels (macroscopic or observable, symbolic, and particulate or molecular), which poses difficulties for learners, particularly in building their understanding of particulate-level concepts (Gabel, Samuel, & Hunn, 1987; James & Nelson, 1981; Johnstone, 1993; Lee, 1999; Nakhleh, 1992).

Other factors that contribute to the difficulty of learning chemistry are the intellectual level of students (Herron, 1996; Karplus, 1977) and a lack of understanding of fundamental models and concepts of chemistry (Lythcott, 1990; Treagust and Chittleborough, 2008). Consequently, many students regard chemistry as a difficult subject (Rowe, 1983; Ward & Herron, 1980).

The Molecular Visualization in Science Education Workshop funded by the United States National Science Foundation emphasized the importance of helping students to understand particulate-level concepts in their learning of chemistry, physics, and biology. The report of this workshop described a variety of methods instructors could employ to introduce students to molecular-level topics (Jones, Jordan, & Stillings, 2005). These recommendations are supported by the findings of several researchers. For example, Griffiths and Preston (1992) found that an understanding of the concepts of atom and molecule is fundamental to the learning of other chemistry concepts. Gabel, Samuel and Hunn (1987) reported that

... students do not understand the meaning of the symbols chemists use to represent the macroscopic and microscopic levels.... Students are able to use formulas in equations and even balance equations correctly without understanding the meaning of the formula in terms of particles that the symbols represent. (p. 695)

Previous research has found a positive relationship between viewing animations or visualizations of the particulate level of matter and achievement. Ardac and Akaygün (2005) and Williamson and Abraham (1995) found that viewing animations of molecular processes supported student learning of concepts better than did still images. Kelly, Phelps, and Sanger (2004) used an animation in conjunction with a chemistry demonstration to promote connections between the macroscopic and molecular levels of chemistry.

Haidar and Abraham (1991) found that development of students' conceptions and their use of particulate theory require formal reasoning ability. They suggested that students need instruction to help them link macroscopic observations in the laboratory with the submicroscopic models that chemists use to explain them. Some investigators have developed animations of molecular behavior to accompany related laboratory activities (Abraham, Gelder, & Haines, 2001; Suits & Diack, 2002). These enhancements have been shown to have a positive impact on student understanding of the molecular nature of the phenomena (Supasorn, Suits, Jones, & Vibuljun, 2008).

Several researchers have used pictures and student drawings to probe how beginning learners understand and visualize the particulate level of matter. Sanger (2000) used student drawings to show that successful students use better strategies than unsuccessful students when distinguishing elements, compounds, and mixtures in pictorial images. Suits and Hypolite (2004) reported that students can actually develop misconceptions from drawing static representations of dynamic processes, such as electron-photon interactions in atoms. Kelly and Jones (2007) found that students could draw correct representations of the particulate nature of matter after viewing animations of molecular behavior, without actually understanding what their pictures meant. They also found that although students who viewed animations of the particulate level of matter showed learning gains, they had difficulty transferring that learning to new situations (Kelly & Jones, 2008). For example, students could not draw images of a solution of sodium chloride only a few weeks after viewing an animation of the dissolving process.

Studies have found that even when chemistry students perform well on problem-solving tasks, they may poorly understand the chemistry concepts involved. In such cases, students may have memorized problem-solving algorithms that they apply without understanding (Nakhleh & Mitchell, 1993; Nurrenbern & Pickering, 1987; Pushkin, 1998). Because traditional assessments often cannot distinguish deep student understanding from skill in applying problem-solving algorithms, conceptual examination questions have been developed (Bowen & Bunce, 1997). Some of the questions employ pictorial representations of atoms and molecules to assess student understanding of particulate-level concepts and to identify student misunderstandings.

To date, little research has been reported on using written materials to supplement multimedia computer chemistry lessons, especially materials linking recorded demonstrations and experiments to molecular-level representations of observable phenomena (Ardac & Akaygün, 2004; Kozma, 2000). In addition, little information is available on the value of pictorial representations in examinations to assess students' understanding of chemical concepts at the microscopic level and some researchers have questioned whether test items with novel formats (for example, questions that include pictures of the particulate level of matter) are appropriate measures for students not previously exposed to this format (Bodner, 1997; Noh & Scharmann, 1997).

One strategy that may focus student attention on molecular-level interpretations of observable phenomena is the use of adjunct questions; that is, questions that students encounter as they proceed through a lesson. Holliday and McGuire (1992) found that adjunct questions helped to focus eighthgrade students' attention and enhance concept learning during a computer-animated science lesson.

Additional information concerning instructional characteristics that promote learning of molecular-level concepts and their interaction with student characteristics is needed. This study investigated the impact of different types of adjunct questions used in conjunction with multimedia chemistry lessons on students' understanding of chemical concepts. It also investigated the role of pictorial representations in assessments of student conceptual understanding of chemistry concepts.

Purpose

In this study, three kinds of adjunct questions were used during multimedia computer lessons to focus students' attention and to stimulate thought about content presented in the multimedia computer lessons at the macroscopic or molecular level:

• •

  TM: Questions emphasizing the macroscopic/symbolic level (visible chemical changes and change represented by chemical equations)

• •

  TP: Questions employing text to emphasize connections between the particulate nature of matter and macroscopic demonstrations and equations

• •

  PP: Questions employing simple pictures of molecular behavior along with text to emphasize connections between the particulate nature of matter and macroscopic demonstrations and equations

The study was designed to investigate these research questions:

• 1.

  How do the three types of adjunct questions (TM, TP, and PP) used during multimedia computer lessons affect students' understanding of molecular level concepts?

• 2.

  How is formal reasoning ability related to the understanding of chemical concepts for students receiving each of the three types of adjunct questions?

• 3.

  How is mental capacity related to the understanding of chemical concepts for students receiving each of the three types of adjunct questions?

• 4.

  How do attitudes of students toward the different types of adjunct questions compare?

Method

Subjects

This study was conducted with 98 undergraduate volunteers (41 male, 57 female) enrolled in first-semester general chemistry for science majors at a university in the western United States. Sixty percent of the subjects were less than 20 years old, 35% were 20-25 years old, and 5% were 26-35 years old. Most students were majoring in scientific fields, with biology the most popular major (40%); only 8% of the students were majoring in chemistry. Most subjects (89%) had completed at least one year of chemistry in secondary school. Nine instructional laboratory sections were randomly divided into three groups; volunteers were solicited from each group and assigned to use one of three types of adjunct questions: text-only macroscopic/symbolic (Group TM), text-only molecular (emphasizing the particulate nature of matter) (Group TP) and pictorial molecular (emphasizing the particulate nature of matter) (Group PP).

Instruments

1

Adjunct Questions

Three types of adjunct questions were prepared for use with the multimedia computer lessons: text-only macroscopic/ symbolic questions, text-only questions emphasizing the particulate nature of matter and pictorial questions emphasizing the particulate nature of matter (Figure 1). The content of these three adjunct question types was similar. Some pictorial adjunct questions were developed from ideas suggested by James and Nelson (1981). All adjunct questions were validated by a panel of chemistry instructors.

Figure 1.

Examples of the three types of adjunct questions used in this study.

2

Multimedia Computer Lessons

The multimedia computer lessons used in this study (Figure 2) were drawn from the Exploring Chemistry multimedia courseware (Smith, Jones, & Gammon, 1994). Students using these lessons are challenged to make predictions about chemical reactions and phenomena such as gas behavior and reaction rates. They select reagents and then observe videos depicting what happens when the reactants are mixed. Feedback guiding students through explanations of the phenomena is provided. Some explanations refer to the molecular level, but neither pictures nor animations of the molecular level of the phenomena are included. Rather, lessons are focused on the macroscopic and symbolic levels of chemistry.

Figure 2.

A screen image from the Exploring Chemistry multimedia lessons used in the study. In this lesson, students select reactants, view the reaction that occurs, and balance the equation representing the reaction.

3

Pre-test

The Pre-test assessed students' prior chemistry knowledge. Test items consisted of text-only two-tiered questions on the macroscopic and particulate levels of chemistry that were similar to those in Parts 1 and 2 in the Test of Conceptual Understanding in Chemistry.

4

Test of Conceptual Understanding in Chemistry (TCC)

The TCC consisted of three parts (Figure 3):

Figure 3.

Examples of items from the three parts of the Test of Conceptual Understanding in Chemistry (TCC)

Part 1. Text-only questions about the macroscopic level of matter.

Part 2. Text- only questions about the particulate level of matter.

Part 3. Pictorial questions about the particulate level of matter.

Parts 1 and 2 of the TCC contained multiple-choice and shortanswer items. Part 3 contained multiple-choice and drawing completion items (items that required students to answer questions by completing simple drawings of atoms and molecules). Eleven of the multiple-choice items in Parts 1 and 3 and two of the drawing items were two-tiered questions that required explanations or reasons to be given (Treagust, 1988). Most students finished the test in about 30 minutes.

5

Test of Logical Thinking (TOLT)

The TOLT assessed student formal reasoning ability. The test consists of ten items focused on proportional reasoning, control of variables, probabilistic reasoning, correlational reasoning, and combinatorial reasoning (Tobin & Capie, 1981). A split-half reliability coefficient of 0.70 was found for the present sample.

6

Figural Intersections Test (FIT)

The FIT is a 35-item paper and pencil instrument developed by Burtis and Pascual-Leone (1974) to assess mental capacity (the number of different items or steps that can be coordinated at the same time) (Niaz, 1987; Niaz & Lawson, 1985). A split-half reliability coefficient of 0.80 was obtained for the present sample.

7

Attitude Questionnaire

The 15-item attitude survey instrument used in this study is a slightly modified version of a survey assessing attitude toward adjunct questions developed by Liao (1995). It employed a five-point Likert-scale to assess student attitudes toward the three types of adjunct questions and had a Cronbach alpha reliability of 0.93. Attitude Questionnaire items were validated by three professors of education.

Procedure

Three sections of a 15-week introductory chemistry course included four one-hour lectures taught by three experienced professors and one three-hour laboratory weekly. Each laboratory section, which was taught by graduate teaching assistants, included students from all three lecture sections.

Multimedia lessons were scheduled during the laboratory periods in weeks 4, 9, and 11 of the course. The chemistry content addressed by multimedia lessons during these three weeks included states of matter, elements, compounds, mixtures, chemical reactions, balancing chemical equations, solubility, net ionic equations, acids and bases, and predicting reaction products. Students were paired and instructed to work with their partners while completing these lessons. The three different types of adjunct questions were provided to students on worksheets at the start of each multimedia session. No feedback was given to students on their answers to worksheet questions to reduce any “training” effect.

The consent forms and demographic forms were administered during the laboratory period in week 3 along with the pre-test. Students completed the TOLT and FIT during the week 9 and 11 laboratory periods, respectively. The TCC was administered in week 13 during the three lecture sessions. A counterbalanced design was used to control for possible sequencing effects within the TCC. Three versions of the TCC were prepared, in which the order of the parts varied as follows: 1-2-3, 2-3-1, 3-1-2. The three versions were mixed and distributed randomly to the students.

Interviews were conducted following the TCC. The primary purpose of the interviews was to discover what strategies students had used to answer the questions and whether they were correctly interpreting the questions. A secondary purpose was to identify students' misconceptions. Nine student volunteers were interviewed. Three students were randomly selected from each of three TCC score ranges: the top third of the scores, the middle third, and the lowest third. At the beginning of the interviews, students received their original written TCC answer sheets without any scoring marks and were asked to explain their reasons for each answer. Tape recordings of the interviews were transcribed.

In both the pretest and the TCC, open-ended responses for items requiring a reason or explanation were categorized according to a rubric that was based on a rubric employed by Abraham, Williamson and Westbrook (1994). The reliability of the categorizations of students' TCC responses was assessed by comparing independent scoring assigned by two graduate teaching assistants with scores assigned by the primary researcher to three papers selected at random. A Pearson product moment correlation coefficient was used to assess the correspondence of the scores. The correlation coefficients among the three raters ranged from r = .94 (p < .0001) to r =.99 (p < .0001), indicating that the ratings corresponded very closely.

Results

No significant differences were detected among the three treatment groups on the three TCC parts, using multivariate analysis of covariance, F(6,180) = 2.07, p > .05. Pre-test scores, TOLT scores, and FIT scores served as covariates. A significant positive relationship was noted among scores on all three parts of the test. Correlation coefficients for scores of all subjects were 0.48 (p < .0001) for TCC Parts 1 and 3, 0.54 (p < .0001) for TCC Parts 2 and 3, and 0.46 (p < .0001) for TCC Parts 1 and 2. However, a trend can be noted among the scores outlined in rectangles: the apparent high score on each TCC part was achieved by the group that had used the corresponding type of adjunct question. In addition, the mean score of PP-group students (who had used pictoral adjunct questions) on pictorial questions was marginally higher than were mean scores of TP- and TM-group students (at the 94% confidence level, p = 0.059). Each treatment group achieved significantly higher scores on Part 3 than on Part 1. In addition, groups TP and PP achieved significantly higher scores on Part 2 than on Part 1. Table 1 summarizes the percent means and standard deviations of TCC scores across Parts 1-3.

Table 1.

Means and Standard Deviations on TCC Part 1-3 and the Pre-test for the three groups.

Group n		TCC Part 1		TCC Part 2		TCC Part 3		Pre-test
		M	SD	M	SD	M	SD	M	SD
TM	33	53.73^a	18.17	62.46	20.41	65.91^a	21.31	44.24	13.67
TP	37	52.86^b,c	18.21	67.15^b	20.31	68.92^c	18.95	45.27	15.48
PP	28	45.32^d,e	14.76	63.73^d	19.61	72.62^e	15.94	43.68	14.32

Part 1: Macroscopic/symbolic text-based test.

Part 2: Molecular-level text-based test.

Part 3: Molecular-level pictorial test.

TM = text-only macroscopic/symbolic group, TP = text-only particulate nature of matter group, PP = pictorial particulate nature of matter group. Superscript letters indicate significant differences.

A MANCOVA was completed to determine whether the three different lecturers affected the TCC scores. The TOLT, FIT, and pre-test scores were used as covariates. No significant differences were noted among lecture sections, nor were there any significant differences attributable to student gender.

A significant positive relationship among TOLT and each of the three TCC parts was found. The correlation coefficients were .20 (p < .05) for TOLT and TCC Part 1, .47 (p < .0001) for TOLT and TCC Part 2, and .33 (p < .001) for TOLT and TCC Part 3. There was also discernable interplay of students' mental capacities with their TCC performances. A significant positive relationship was found between FIT and TCC Part 1 (r = .24, p < .05), and FIT and TCC Part 3 (r = .26, p < .05), but no significant relationship was found between FIT and TCC Part 2 (r = .20, p > .05). Table 2 summarizes correlations among scores on TOLT, FIT, the pre-test, and TCC Parts 1-3.

Table 2.

Intercorrelations among TOLT, FIT, Pre-test, and TCC Part 1-3 Scores.

	TOLT	FIT	Pretest	TCC Part 1	TCC Part 2	TCC Part 3
TOLT		.41^****	.24^*	.20^*	.47^****	33^***
FIT			16	24^*	.20	.26^*
Pre-test				.40^****	.27^**	.27^**
TCC Part 1					.47^****	.48^****
TCC Part 2						.54^****

*

p < .05

**

p < .01

***

p < .001

****

p <.0001

The Attitude Questionnaire (AQ) results were subjected to a factor analysis. A principal components analysis followed by a Varimax rotation yielded two factors. Factor 1 (13 items) represented students' perception of the usefulness of their adjunct questions. Factor 2 (2 items) represented students' perception of the difficulty of their adjunct questions. The factors were analyzed with Scheffe's Test (Zar, 1999). Significant mean differences (p < .05) were found among the three treatment groups on Factor 1 (usefulness), with group TM (macroscopic-level questions) indicating the most positive attitude and group PP (pictorial particulate-level questions) the least positive. No significant differences were found among the three groups concerning Factor 2 (difficulty).

Discussion

This study did not reveal significant differences in student performance on the test of conceptual understanding in chemistry (TCC) among the three groups receiving different types of adjunct questions. One explanation might be that the time over which adjunct questions were used (three three-hour sessions) was too short and was not continuous. Even though there were no significant differences across groups on any of the three parts of the TCC, the trend in mean scores from each part of the TCC suggests that further testing may reveal an effect of prior exposure to pictorial questions. Students receiving the pictorial adjunct questions achieved their highest mean score on Part 3 (molecular-level pictorial items) and their lowest mean score of the three groups on Part 1 (macroscopic/symbolic items). Although scores on Part 3 were higher for all three groups than scores on Part 1, the difference was greatest for the pictorial group. This pattern is congruent with the finding of Dwyer (1972), who reported that pictorial representations did not affect scores on a verbal posttest, but facilitated student performance on a pictorial test.

Table 3.

Attitude Survey Means and Standard Deviations of Two Factors for Treatment Groups.

	TM (n = 38)		TP (n = 48)		PP (n = 40)		Total (n = 126)
	M	SD	M	SD	M	SD	M	SD
1. Usefulness (Max = 65)	44.74	9.05	39.31	11.74	35.60	11.25	39.88	10.68
2. Difficulty (Max = 10)	6.18	1.52	6.13	1.66	5.90	2.15	6.07	1.78

TM = Text-only macroscopic/symbolic group.

TP = Text-only particulate nature of matter group. PP = Pictorial particulate nature of matter group.

TOLT mean scores in this study ranged from 5.27 to 6.32 for these undergraduate general chemistry students, higher means than that of 4.4 reported by Tobin and Copie (1981), for 247 college science students. This difference may be due to the fact that the students in this study were volunteers.

In this study, TCC Part 1 scores were moderately correlated with pre-test scores (r = .40, p < .0001), while the correlation between scores on TCC Part 1 and formal reasoning ability as assessed by TOLT was lower (r = .20, p < 0.05). These two correlation coefficients were compared (Glass & Hopkins, 1984, pp. 310-311); a significant difference between correlation coefficients was found (t(95) = 1.74, p < .05). This result indicated that prior knowledge played a greater role in the scores on TCC Part 1 than did reasoning ability.

The correlation between scores on TCC Part 2 and TOLT (r = .47, p < .0001) was much stronger than that between TCC Part 2 and pre-test scores (r = .27, p < .01). The difference between those correlation coefficients was investigated (Glass & Hopkins, 1984, pp. 310-311); a significant difference was found (t(95) = 1.82, p < .05). This finding suggests that reasoning ability played a greater role regarding scores on TCC Part 2 than did prior knowledge.

The correlation between scores on TCC Part 3 and the pre-test (r = .27, p < .01) appeared a little lower than the correlation between scores on the TCC Part 3 and TOLT (r = .33, p < .001). However, no significant difference was found between these correlation coefficients. These results suggest that both prior knowledge and reasoning ability played comparable roles in TCC Part 3 performance.

TOLT scores appeared to have been more associated with TCC scores than did FIT scores. There were low correlations between FIT scores and scores on all three parts of the TCC. This finding suggests that mental capacity was not highly associated with student performance on any of the three parts of the TCC.

The finding that there were no significant differences among the three parts of the TCC suggests that using text or pictorial representations of molecular level chemistry concepts, whether familiar or unfamiliar, can be used to assess student understanding of these concepts. However, the trend in scores noted earlier (that the highest score in each part of the TCC appears to have been achieved by the group that had received adjunct questions of that type) suggests that further study may be required to provide a definitive answer to this question.

The macroscopic section of the TCC (Part 1) was the most difficult for all students in this study. One reason for the higher difficulty of TCC Part 1 may be that four of the nine items were concerned with writing chemical formulas or chemical equations, and predicting the products of chemical reactions. For example, when asked to write a chemical equation from a sentence (“When calcium reacts with water, calcium hydroxide and diatomic hydrogen gas are produced. Write the balanced chemical equation for this reaction”), only 35.71% of the students in all treatments were able correctly to write the balanced chemical equation. When asked to choose the correct observation for the reaction that occurs when sodium metal is added to an excess of water, 70.41% of the students chose the correct answer of gas formation. However, only 28.57% of the students were able to write the correct balanced chemical equation for this reaction. In contrast, when asked to choose the correct observation for the reaction that occurs when adding a solution of NaOH to a solution of HCl, only 27.55% of the students chose the correct answer of no visible change. Many students chose solid formation as the result that would be observed for this reaction (perhaps thinking of the formation of NaCl). However, 84.69% of the students wrote the correct balanced chemical equation for this reaction.

The pictorial section of the TCC (Part 3) appeared to be the easiest test for all three groups of students in this study. These results suggest that the pictorial representations may have removed obstacles for students who have difficulty recalling elemental symbols, or who are not familiar with a particular chemical reaction. The types of student errors noted in responses to the pictorial questions were similar to those reported by others (Nakhleh & Mitchell, 1994; Sanger, 2000). For example, 75.5% of students in all groups could balance a chemical equation provided in symbolic form, but only 31.6% could correctly balance a similar chemical equation represented by pictures of atoms and molecules. They knew that there should be the same number of each type of atom on each side of the equation, but attempted to balance the pictorial equation by adding individual atoms to the side that needed atoms or by changing the formulas of products. Part of the reason for this behavior may have been due to uncertainty about how to proceed, rather than to conceptual misunderstanding. For example, some of the same students who changed formulas or added individual atoms in balancing an equation could correctly categorize pictures depicting chemical or physical changes, offering reasons such as, “same combination of atoms as before” for depictions of physical changes and “The 2 different elements react together in a combination reaction” for a depiction of chemical change.

Data from the nine student interviews provide some interesting insights into students' misconceptions and the way that students explain their knowledge. Findings from the interviews can be summarized as follows:

• •

  Students who provided correct answers in the TCC were able to explain their answers clearly, but students who provided wrong answers could not explain how they got their answers.

• •

  Students revealed misconceptions about distinctions between compound and mixture; even among high-scoring students. This misconception involved the belief that any substance containing a bond should be classified as a compound, no matter whether bonding is between the same or different kinds of atoms.

• •

  No student in the low-scoring group could balance a pictorial chemical equation using circles and squares to represent atoms. They knew that equal numbers of each type of atom should appear on both sides of the equation, but they did not seem to understand that a correctly written chemical equation must represent a particular chemical reaction. Students devised new compounds instead; to equalize mass, they simply added additional atoms to the side of the equation that needed more mass. This behavior was not found, however, among middle- or high-scoring students.

• •

  No student in the low-scoring group gave a correct answer to the question in which students represent with drawings a chemical reaction involving a limiting reactant. For example, when asked to draw molecular pictures for the equation 2 S(s) + 3 O₂(g) → 2 SO₃(g) they did not understand the meaning of coefficients in the chemical equation, such as the 2 in 2 SO₃ and tried to devise a product that contained all atoms in one molecule, S₂O₆. That behavior was not observed among any middle- or high-scoring students.

The attitude survey results imply that students feel more comfortable with adjunct questions that are more similar to the lesson they are studying. Students in the TM group, who completed the macroscopic adjunct questions, were able to find answers to the questions by observing the multimedia lessons, while students in the TP and PP groups had to draw on their knowledge and understanding of chemical concepts to answer the questions. The latter process required more thought and more time. Therefore, these questions may have seemed less enjoyable to students, even though the questions themselves were not viewed as difficult. They were also question types not encountered in other parts of their chemistry course and thus were not familiar and may not have seemed relevant to course learning goals.

Conclusions

Based on the results of this study, these conclusions can be drawn:

• 1.

  The fact that no differences were found among mean scores for the three treatment groups on TCC Parts 1-3 implies that testing students with unfamiliar visual representations such as those employed in this study may not disadvantage students who had not previously encountered these types of representations.

• 2.

  The significant relationship between scores on TCC Part 2 and TCC Part 3 implies that text-based test questions may be as effective as pictorial test questions for assessing students' understanding at the molecular level.

• 3.

  Reasoning ability was found to play a role in students' molecular-level understanding. The use of pictures and requiring students to respond with drawings (TCC Part 3) was less strongly correlated with their reasoning abilities than were textual questions (TCC Part 2). This finding suggests that pictorial questions may have been easier for students to interpret than were written descriptions of the particulate level of matter.

• 4.

  The small positive relationship between mental capacity and overall score on the TCC implied that mental capacity did not play a large role in the ability of students to answer conceptual questions, as measured by the TCC.

• 5.

  The students in this study did not readily perceive the connections between observable macroscopic events and underlying processes occurring at the particulate level. This finding suggests that it may be important for chemistry instructors to emphasize these fundamental interrelationships frequently.

Noh and Scharmann (1997) reported that secondary school students receiving extensive instruction with molecular-level drawings exhibited better understanding of chemistry concepts on text-based questions than students who had not received such instruction, but not when tested with questions using pictures of atoms and molecules. In that study, text-based questions were as effective as pictorial questions for assessing understanding of particulate-level chemical concepts. In the current research, all three treatment groups scored higher on pictorial questions about the particulate level of matter than they did on text questions about the macroscopic level of matter. However, the processes required to solve a pictorial problem may need clarification for some students. For example, instructors can provide examples of how to balance pictorial representations of chemical reactions on tests.

Further research may provide a deeper understanding of how instructional materials containing text and/or pictures of the particulate level of matter affect student learning. In this study, students used adjunct questions only for three weeks and only in their laboratory sessions. In addition, student feedback regarding their responses to adjunct questions was not provided in an attempt to replicate situations where students study alone and also to ensure that the TCC assessed understanding rather than extent of training. A more comprehensive implementation of instructional materials emphasizing the particulate level of matter, combined with supporting classroom instruction, feedback on student work, and consistent use of particulate-level questions on examinations, may result in greater discernable impact on student learning.

Acknowledgements

The authors thank Henry Heikkinen, Clark Fields, and John Cooney for the time they generously gave to this study. This material was based upon work funded in part by the National Science Foundation under Grants No. REC 0095023 and REC 0440103.