A protocol analysis of the influence of technology on students' actions,verbal commentary,and thought processes during the performance of acid-base titrations

We investigated students' initial and final understanding of acid-base concepts and their concurrent thought processes and actions during the process of acid-base titrations. Here we report students' actions and thought processes while titrating. Different levels of information were presented by three technologies: chemical indicators, pH meters, and microcomputer-based laboratories. We speculated that the level of information would influence students' actions and thought processes, as expressed in verbal commentary. Data were collected from 14 secondary chemistry students. Each student used one technology to titrate a strong acid, a weak acid, and a polyprotic acid with a strong base. They verbalized their thoughts while titrating. Students then graphed pH versus volume of base and discussed the titration with the investigator. Verbal commentaries were coded and analyzed for patterns in actions and for frequency of statement categories. Drawings were analyzed for shape, scale, and direction; discussions were analyzed for understanding of acid-base neutralization. We found that the technology's level of information affected the focus of students' observations. The microcomputer group focused primarily on the graph; other groups exhibited multiple foci. We speculate the screen display functions as an auxiliary short-term memory. The discussion data also reveal that students held three main ideas about how acids and bases behave when mixed. Implications for instruction are discussed.     

Concept map structure,gender and teaching methods: an investigation of students' science learning

Background: This study deals with the application of concept mapping to the teaching and learning of a science topic with secondary school students in Germany.

Purpose: The main research questions were: (1) Do different teaching approaches affect concept map structure or students' learning success? (2) Is the structure of concept maps influenced by gender? (3) Is the concept map structure a reliable indicator of students' learning success?

Sample: One hundred and forty-nine high-achieving 5th-grade students from four German secondary schools participated in the study. The average age of participants was 10½ years. Gender distribution was balanced. Students produced concept maps working in small, single-sex groups.

Design and methods: There were two teaching approaches used: one based upon teacher-centred instruction and one consisting of student-centred learning. Both were followed by a concept-mapping phase. Student groups experienced either one or the other teaching approach. Concept map structures were analysed using of the method of Kinchin, Hay and Adams. We defined three different possible types of concept map structure: spokes, chains and nets. Furthermore, for assessing a student's short- and longer-term learning success, we constructed a multiple-choice knowledge test applied in a pre-, post-, retention-test design. Parametric tests, such as MANOVA, one-way ANOVA and t-tests were used to identify any differences in gender, teaching approach, number of nets per concept map and their interactions.

Results: Type of teaching approach had an effect on concept map structure but not on students' longer-term learning success. Students of the teacher-centred approach produced more net structures than those students who participated in the hands-on instruction. Subsequent analyses showed in total more net structures for female groups. The interaction of gender and number of nets per concept map showed a significant effect on students' longer-term learning success.

Conclusion: The study suggests that Kinchin's classification scheme for assessing concept map quality may be a good indicator of students' learning success when applied in combination with a knowledge test.     

Identifying critical junctures in learning in a college course on evolution

The purpose of this research study was to (a) describe how concept mapping can be used as an integral instructional strategy for teaching a college course on evolution, (b) evaluate the utility of incorporating concept mapping in a college course on evolution, (c) determine whether students' concept maps reveal “critical junctures” in learning as the course unfolds, and (d) assess the impact of concept mapping on students' study practices and on students' understanding of course content. Key findings include: (a) Critical junctures in learning evolution can be identified by monitoring the degree of concordance of superordinate concepts appearing on the class set of concept maps submitted after each of the course lectures; (b) students who made concept maps reported spending an average of 37% more study time on this college biology course than on their previous biology courses; and (c) the use of “seed concepts,” “micromapping,” a standard concept map format, and a standard concept map checklist made the strategy feasible for the instructor to implement and for the student to adopt. A concept map performance index formulas was also developed for this research study in order to assess students' overall mapping performance.     

A three-tier system for assessing concept map links: a methodological study

A novel method is described for analysing concept maps for research and analysis purposes. The coding system rates the use, the stability, and the complexity of each link, which is a unique way of representing students' knowledge. The analysis scheme affords a look at how students may integrate new knowledge into their existing structures and may be used for assessment purposes or research on how students learn. This coding system was successfully applied to a sample of 56 complex concept maps that had been generated from student interviews on the topic of chemical bonding. The coding system is of particular use when analysing complex concept maps with a large number of concept nodes and links. The system described here was also particularly useful for assessing complex, non-hierarchical concept maps.     

A time sequence-oriented concept map approach to developing educational computer games for history courses

Concept maps have been recognized as an effective tool for students to organize their knowledge; however, in history courses, it is important for students to learn and organize historical events according to the time of their occurrence. Therefore, in this study, a time sequence-oriented concept map approach is proposed for developing a game-based environment to facilitate students' learning of historical events and their organization during the gaming process. With this approach, students can easily learn the precedence relationships among the historical events that occurred in different time periods with the time sequence-oriented concept map. To evaluate the effectiveness of the proposed approach, a historical role-playing game has been developed for an elementary school history course to examine the students' performance in terms of learning motivation, self-efficacy and learning achievements. A subject unit, the “Siege of Fort Zeelandia by Zheng Cheng-Gong,” was chosen as the history topic. The results show that the proposed approach can significantly enhance the students' learning achievement, but did not affect their learning motivation or self-efficacy for the history course. As a consequence, it is concluded that students can benefit from concept maps in terms of enhancing their learning achievement, but they do not necessarily enjoy using concept maps in game-based learning activities.     

An innovative concept map approach for improving students' learning performance with an instant feedback mechanism

Concept maps have been widely employed for helping students organise their knowledge as well as evaluating their knowledge structures in a wide range of subject matters. Although researchers have recognised concept maps as being an important educational tool, past experiences have also revealed the difficulty of evaluating the correctness of a concept map. It usually takes days or weeks for teachers to manually evaluate the concept maps developed by students; consequently, the students cannot receive timely feedback from the teachers, which not only affects their learning schedules, but also significantly influences the students' learning achievements. In this paper, a computer‐based concept map‐oriented learning strategy with real‐time assessment and feedback is proposed in order to cope with the problems mentioned above. Our approach provides immediate evaluation of concept maps and gives also real‐time feedback to the students. An experiment has been conducted to evaluate the effectiveness of this new strategy in comparison with the conventional computer‐based concept map approach. It is found that our innovative approach can be significantly beneficial to promote learning achievements as well as the learning attitudes of students.     

Research report

We examined ninth-grade students' explanations of chemical reactions using two forms of an open-ended essay question during a learning cycle. One form provided students with key terms to be used as 'anchors' upon which to base their essay, whereas the second form did not. The essays were administered at three points: pre-learning cycle, post-concept application, and after additional concept application activities. Students' explanations were qualitatively examined and grouped according to common patterns representing their understandings or misunderstandings. Findings indicated that more misunderstandings were elicited by the use of key terms as compared to the non-use of key terms in the pre-test. Misunderstandings in the key term essay responses generally involved the misuse of these terms and their association with the concept. Findings also indicated significant positive shifts in students' understanding over the learning cycle. No perceptible increase in understanding occurred after additional application activities. Differences in gender were observed, with females showing equal or greater understanding compared to males, contradicting reports that males typically outperform females in the physical sciences and supporting the need to reconstruct assessment techniques to better reveal the conceptual understandings of all students.     

A window into thinking: Using student writing to understand conceptual change in science learning

This study, conducted in an inner-city middle school, followed the conceptual changes shown in 25 students' writing over a 12-week science unit. Conceptual changes for 6 target students are reported. Student understanding was assessed regarding the nature of matter and physical change by paper-and-pencil pretest and posttest. The 6 target students were interviewed about the goal concepts before and after instruction. Students' writing during lesson activities provided qualitative data about their understandings of the goal concepts across the science unit. The researcher constructed concept maps from students' written statements and compared the maps across time to assess changes in the schema of core concepts, complexity, and organization as a result of instruction. Target students' changes were studied in detail to determine patterns of conceptual change. After patterns were located in target students' maps, the remaining 19 students' maps were analyzed for similar patterns. The ideas that students identified in their writing showed changes in central concepts, complexity, and organization as the lessons progressed. When instructional events were analyzed in relation to students' demonstrated ideas, understanding of the goal conceptions appeared in students' writing more often when students had opportunities to explain their new ideas orally and in writing.     

What is this Substance? What Makes it Different? Mapping Progression in Students’ Assumptions about Chemical Identity

Given the diversity of materials in our surroundings, one should expect scientifically literate citizens to have a basic understanding of the core ideas and practices used to analyze chemical substances. In this article, we use the term ‘chemical identity' to encapsulate the assumptions, knowledge, and practices upon which chemical analysis relies. We conceive chemical identity as a core crosscutting disciplinary concept which can bring coherence and relevance to chemistry curricula at all educational levels, primary through tertiary. Although chemical identity is not a concept explicitly addressed by traditional chemistry curricula, its understanding can be expected to evolve as students are asked to recognize different types of substances and explore their properties. The goal of this contribution is to characterize students' assumptions about factors that determine chemical identity and to map how core assumptions change with training in the discipline. Our work is based on the review and critical analysis of existing research findings on students' alternative conceptions in chemistry education, and historical and philosophical analyses of chemistry. From this perspective, our analysis contributes to the growing body of research in the area of learning progressions. In particular, it reveals areas in which our understanding of students' ideas about chemical identity is quite robust, but also highlights the existence of major knowledge gaps that should be filled in to better foster student understanding. We provide suggestions in this area and discuss implications for the teaching of chemistry.     

Developing causal understanding with causal maps: the impact of total links, temporal flow, and lateral position of outcome nodes

This study examined some of the methodological approaches used by students to construct causal maps in order to determine which approaches help students understand the underlying causes and causal mechanisms in a complex system. This study tested the relationship between causal understanding (ratio of root causes correctly/incorrectly identified, number of correctly identified root-cause links explaining how root causes directly/indirectly impact final outcomes) and three attributes observed in students’ causal maps (total links, temporal flow, lateral position of final outcome) that students produced before and after online discussions on noted similarities and differences between students’ causal maps. The findings suggest that: (a) causal understanding can be adversely affected if students are instructed before group discussion to temporally sequence nodes to flow from left to right and to position the outcome node farther away from the left edge of the map relative to other nodes in the map; (b) causal understanding following group discussion can be increased by instructing students to minimize the number of causal links and create a map with temporally flow; (c) promoting temporal flow following discussion may be the most effective means of helping students to identify root causes; and (d) instructing students to minimize the number of links following discussion may be the most effective means to helping students explain root causes directly/indirectly impact outcomes. These findings provide insights on what processes and constraints can be formalized and integrated into causal mapping software when used as an instructional and assessment tool.     

Science Teachers’ Use of a Concept Map Marking Guide as a Formative Assessment Tool for the Concept of Energy

In this study, we investigated the value of a concept map marking guide as an alternative formative assessment tool for science teachers to adopt for the topic of energy. Eight high school science teachers marked students’ concept maps using an itemized holistic marking guide. Their marking was compared with the researchers’ marking and the scores of a multiple-choice diagnostic test. Statistical analysis revealed that teachers evaluated students’ concept maps consistently, but the concept map scores were only mildly correlated with the multiple-choice test scores. We explored what each assessment method revealed in terms of students’ understanding with one student’s performance as a showcase. Teachers’ views on using the marking guide of concept maps as an assessment tool were also included. We discuss the value of the concept map marking guide as a formative assessment tool for science teachers to create and modify in order to encourage students’ conceptual learning.     

Revealing conceptual understanding of international business

This study aims to identify an adequate approach for revealing conceptual understanding in higher professional education. Revealing students’ conceptual understanding is an important step towards developing effective curricula, assessment and aligned teaching strategies to enhance conceptual understanding in higher education. Essays and concept maps were used to determine how students’ conceptual understanding of international business can be revealed adequately. To this end, 132 international business students in higher professional education were randomly assigned to four conditions to write essays and to construct concept maps about an international business research topic. The conditions were: essay alone, essay after concept map, concept map alone, and concept map after essay. An assessment rubric was used to assess the breadth and depth of students’ conceptual understanding. Results show essays are the most adequate approach for revealing conceptual understanding of international business. In particular, concept maps revealed fewer facts and less reasoning than essays. Essays written after concept maps were less effective than essays, possibly since students perceived these essays as redundant. Further research is suggested on how educators can foster conceptual understanding.     

Comparison of the reliability and validity of scores from two concept‐mapping techniques*

This paper reports the results of a study that compared two concept‐mapping techniques, one high‐directed, “fill‐in‐the‐map,” and one low‐directed, “construct‐a‐map‐from‐scratch.” We examined whether: (1) skeleton map scores were sensitive to the sample of nodes or linking lines to be filled in; (2) the two types of skeleton maps were equivalent; and (3) the two mapping techniques provided similar information about students' connected understanding. Results indicated that fill‐in‐the‐map scores were not sensitive to the sample of concepts or linking lines to be filled in. Nevertheless, the fill‐in‐the‐nodes and fill‐in‐the‐lines techniques were not equivalent forms of fill‐in‐the‐map. Finally, high‐directed and low‐directed maps led to different interpretations about students' knowledge structure. Whereas scores obtained under the high‐directed technique indicated that students' performance was close to the maximum possible, the scores obtained with the low‐directed technique revealed that students' knowledge was incomplete compared to a criterion map. We concluded that the construct‐a‐map technique better reflected differences among students' knowledge structure. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 260–278, 2001     

An investigation into high-achiever and low-achiever knowledge organisation and knowledge processing in concept mapping: A case study

This study used concept map content analysis and interviews to gain insights into the knowledge organisation and knowledge processing of pre-service teachers. Forty-eight preservice teachers of elementary science from a teachers’ training college in Sarawak, Malaysia, participated in this study. Correlations between achievement and five concept map characteristics showed that there were significant positive correlations (p<.01) between achievement and: the number of appropriate links; the average number of appropriate concepts per cluster; and, the hierarchy score of subjects’ concept maps, and significant negative correlations (p<.01) between achievement and: the number of inappropriate links; and, the average number of inappropriate concepts per cluster. Interviews with high-achievers and lowachievers revealed that there were differences in the way they processed knowledge during concept mapping. The high-achievers were more thorough than were the low-achievers in cognitive processing of knowledge, taking time to make sense of concepts, sort and group concepts, form relevant links between concepts, and organise concepts hierarchically. Active cognitive processing of knowledge seems to be related to more complex, well-integrated cognitive structures for the material learned.     

Reducing Task-related Variance in Performance Assessment Using Concept Maps

Task-related variance causes scores from performance assessments not to be generalizable and thus inappropriate for high stakes use. It is possible that task-related variance is due, in part, to students’ inability to transfer their knowledge consistently from one assessment task to another. Therefore, concept-mapping, a cognitive tool, might be useful to aid this transfer. This study examines the effects of concept maps on the task-related variance components of Political Science performance assessments. On three quizzes, some students used concept maps while writing two essays, while other students did not. The task variance components remained unchanged across groups, but the person main effect components increased and the task-by-person interaction components decreased for those using concept maps. Also, the scores from the concept mapping groups had higher generalizability coefficients than for those who did not use a concept map.     

Student affect and conceptual understanding in learning chemistry

This study explores the relationship between affective and cognitive variables in grade 9 chemistry students (n = 73). In particular, it explores how students' situational interest, their attitudes toward chemistry, and their chemistry‐specific self‐concept influence their understanding of chemistry concepts over the course of a school year. All affective variables were assessed at two time points: at the middle of the first semester of grade 9, and at the end of the second semester of grade 9, and then related to students' postinstructional understanding of chemical concepts. Results reveal that none of the affective variables measured at the earliest time point have a significant direct effect on postinstructional conceptual understanding. Looking at the different affective variables as intermediary constructs, however, reveals a pattern in which self‐concept and situational interest measured at the middle of grade 9 contribute to self‐concept measured at the end of grade 9, which in turn, has a positive, significant effect on students' postinstructional conceptual understanding. These results reveal the importance of a strong and positive self‐concept, the feeling of doing well in the chemistry class, for developing a meaningful understanding of scientific concepts. © 2006 Wiley Periodicals, Inc. J Res Sci Teach 44: 908–937, 2007     

Measuring the quality of e‐learning

This paper shows how concept mapping can be used to measure the quality of e‐learning. Six volunteers (all of them 3rd‐year medical students) took part in a programme of e‐learning designed to teach the principles of magnetic resonance imaging (MRI). Their understanding of MRI was measured before and after the course by the use of concept mapping. The quality of change in individuals' maps was assessed using criteria developed to distinguish between meaningful and rote‐learning outcomes. Student maps were also scored for evidence of conceptual richness and understanding. Finally, each map was compared directly with the content of the electronic teaching material. The results show that many of the student misconceptions were put right in the course of their learning but that many of the key concepts introduced in the teaching were ignored (or sometimes learnt by rote) by the students. This was because the teaching material locked these new ideas in structures and terminology that precluded meaning‐making among non‐experts. Our data suggest that students' prior knowledge is a key determinant of meaningful learning. We suggest that this must be acknowledged if the design and use of electronic teaching material is also to be meaningful. Ultimately, measures of student learning are the only authentic indicators of the quality of teaching through technology.     

Problems and issues in the use of concept maps in science assessment

The search for new, authentic science assessments of what students know and can do is well under way. This has unearthed measures of students' hands-on performance in carrying out science investigations, and has been expanded to discover more or less direct measures of students' knowledge structures. One potential finding is concept mapping, the focus of this review. A concept map is a graph consisting of nodes representing concepts and labeled lines denoting the relation between a pair of nodes. A student's concept map is interpreted as representing important aspects of the organization of concepts in his or her memory (cognitive structure). In this article we characterize a concept map used as an assessment tool as: (a) a task that elicits evidence bearing on a student's knowledge structure in a domain, (b) a format for the student's response, and (c) a scoring system by which the student's concept map can be evaluated accurately and consistently. Based on this definition, multiple concept-mapping techniques were found from the myriad of task, response format, and scoring system variations identified in the literature. Moreover, little attention has been paid to the reliability and validity of these variations. The review led us to arrive at the following conclusions: (a) an integrative working cognitive theory is needed to begin to limit this variation in concept-mapping techniques for assessment purposes; (b) before concept maps are used for assessment and before map scores are reported to teachers, students, the public, and policy makers, research needs to provide reliability and validity information on the effect of different mapping techniques; and (c) research on students' facility in using concept maps, on training techniques, and on the effect on teaching is needed if concept map assessments are to be used in classrooms and in large-scale accountability systems. © 1996 John Wiley & Sons, Inc.     

Role of the microcomputer‐based laboratory display in supporting the construction of new understandings in thermal physics

Teachers' failure to use the microcomputer‐based laboratory (MBL) more widely may be a result of not recognizing its capacity to transform laboratory activities. This research aimed to increase understanding of how MBL activities designed to be consistent with a constructivist theory of learning support or constrain student construction of understanding. The first author conducted the research with his Year 11 physics class of 15 students. Dyads addressed 10 tasks in thermal physics using a predict–observe–explain format. Data sources included video and audio recordings of students and teacher during four 70‐minute sessions, students' computer data and written notes, semistructured student interviews, and the teacher's journal. Analysis of students' discourse identified many instances in which students' initial understandings of thermal physics were mediated in multiple ways by the screen display. The findings are presented as eight assertions. Recommendations are made for developing pedagogical strategies incorporating MBL activities that will likely catalyze student construction of understanding. © 2004 Wiley Periodicals, Inc. J Res Sci Teach 41: 165–185, 2004