System thinking skills at the elementary school level

This study deals with the development of system thinking skills at the elementary school level. It addresses the question of whether elementary school students can deal with complex systems. The sample included 40 4th grade students from one school in a small town in Israel. The students studied an inquiry‐based earth systems curriculum that focuses on the hydro‐cycle. The program involved lab simulations and experiments, direct interaction with components and processes of the water cycle in the outdoor learning environment and knowledge integration activities. Despite the students' minimal initial system thinking abilities, most of them made significant progress with their ability to analyze the hydrological earth system to its components and processes. As a result, they recognized interconnections between components of a system. Some of the students reached higher system thinking abilities, such as identifying interrelationships among several earth systems and identifying hidden parts of the hydrological system. The direct contact with real phenomena and processes in small scale scenarios enabled these students to create a concrete local water cycle, which could later be expanded into large scale abstract global cycles. The incorporation of outdoor inquiry‐based learning with lab inquiry‐based activities and knowledge integration assignments contributed to the 4th grade students' capacity to develop basic system thinking abilities at their young age. This suggests that although system thinking is regarded as a high order thinking skill, it can be developed to a certain extent in elementary school. With a proper long‐term curriculum, these abilities can serve as the basis for the development of higher stages of system thinking at the junior–high/middle school level. © 2009 Wiley Periodicals, Inc. J Res Sci Teach 47: 540–563, 2010     

Four case studies,six years later: Developing system thinking skills in junior high school and sustaining them over time

This study examines the process by which system thinking perceptions develop within the context of a water cycle curriculum. Four junior high school students undergoing an especially designed inquiry‐based intervention were closely observed before, during, immediately after, and 6 years after completing a year long systems‐based learning program. The employed research tools included observations, semi‐structured interviews, and a number of “concept viewing” tools (drawings, concept maps, and repertory grids). Out of the data, four distinct “stories,” each presenting a different way of constructing hydro system mental models, are described. The paper's main conclusion is that students develop their systems mental models and remember the learned material based on learning patterns that tend to remain unchanged over time. Consequently, in order to facilitate efficient and lasting construction of students' system models, learning experiences should harness these, and especially the meta‐cognitive learning pattern, which holds special significance for constructing systems. © 2010 Wiley Periodicals, Inc. J Res Sci Teach 47: 1253–1280, 2010     

Enhancing systems‐thinking skills with modelling

Systems thinking is an essential cognitive skill that enables individuals to develop an integrative understanding of a given subject at the conceptual and systemic level. Yet, systems thinking is not usually an innate skill. Helping students develop systems‐thinking skills warrants attention from educators. This paper describes a study examining the effects of utilising systems modelling as a cognitive tool in enhancing a group of graduate students' systems‐thinking skills. A significant improvement was observed in the systems‐thinking practises of the students. A theoretical rationale for enhancing systems‐thinking skills with modelling and the results of the study will be discussed.     

Revitalizing Algebra: the effect of the use of a cognitive tutor in a remedial course

This study investigated the effect of instruction with a cognitive tutoring software system in a remedial algebra course. The performance on algebra tasks of students who attended the experimental (cognitive tutor) and a control class was compared. The results indicated that the two groups of students were equally proficient with respect to algebraic manipulation skills. However, students who attended the experimental algebra section performed significantly better in problem solving than students in the control section. This finding suggested that the use of the cognitive tutor (a) improved students' problem‐solving abilities; (b) fostered student development of richer concepts of variable and function; and (c) improved students' procedural abilities in approaching and carrying through mathematical analyses of relatively complex situations.     

Effect of knowledge integration activities on students' perception of the earth's crust as a cyclic system

Systems thinking is regarded as a high‐order thinking skill required in scientific, technological, and everyday domains. However, little is known about systems thinking in the context of science education. In the current research, students' understanding of the rock cycle system after a learning program was characterized, and the effect of a concluding knowledge integration activity on their systems thinking was studied. Answers to an open‐ended test were interpreted using a systems thinking continuum, ranging from a completely static view of the system to an understanding of the system's cyclic nature. A meaningful improvement in students' views of the rock cycle toward the higher side of the systems thinking continuum was found after the knowledge integration activity. Students became more aware of the dynamic and cyclic nature of the rock cycle, and their ability to construct sequences of processes representing material transformation in relatively large chunks significantly improved. Success of the knowledge integration activity stresses the importance of postknowledge acquisition activities, which engage students in a dual process of differentiation of their knowledge and reintegration in a systems context. We suggest including such activities in curricula involving systems‐based contents, particularly in earth science, in which systems thinking can bring about environmental literacy. © 2003 Wiley Periodicals, Inc. J Res Sci Teach 40: 545–565, 2003     

Facilitation prompts and rubrics on higher‐order thinking skill performance found in undergraduate asynchronous discussion boards

Over the past 20 years, researchers have begun to examine data from asynchronous computer‐mediated student discussions in courses. Some results have shown students to demonstrate lower or mid‐level thinking skills, while others suggest students routinely demonstrate higher‐order thinking skills. The authors investigated the relationship between scaffold types and the level of students' thinking skill performance, learning achievement and attitudes, in a two‐by‐two factorial, quasi‐experimental study. Participants included 216 undergraduate preservice K‐12 teachers who were presented with one of four versions of an asynchronous discussion board assignment. Resulting discussion interactions were evaluated for demonstration of low‐, mid‐ and higher‐order thinking skills. Findings revealed students who were given a scaffold demonstrated higher‐level thinking skills more frequently than did students who received no scaffold. No significant differences in learning achievement associated with test performance were found in test results. The treatment variables did significantly affect effect survey ratings associated with students' attitudes.     

From nationwide standardized testing to school‐based alternative embedded assessment in Israel: Students' performance in the matriculation 2000 project

Matriculation 2000 was a 5‐year project aimed at moving from the nationwide traditional examination system in Israel to a school‐based alternative embedded assessment. Encompassing 22 high schools from various communities in the country, the Project aimed at fostering deep understanding, higher‐order thinking skills, and students' engagement in learning through alternative teaching and embedded assessment methods. This article describes research conducted during the fifth year of the Project at 2 experimental and 2 control schools. The research objective was to investigate students' learning outcomes in chemistry and biology in the Matriculation 2000 Project. The assumption was that alternative embedded assessment has some effect on students' performance. The experimental students scored significantly higher than their control group peers on low‐level assignments and more so on assignments that required higher‐order thinking skills. The findings indicate that given adequate support and teachers' consent and collaboration, schools can transfer from nationwide or statewide standardized testing to school‐based alter‐native embedded assessment. © 2003 Wiley Periodicals, Inc. J Res Sci Teach 40: 34–52, 2003     

Assessment of mathematics and reading performance: An examination of the correspondence between direct assessment of student performance and teacher report

Teacher judgments have been identified as a primary source of information regarding student academic achievement. Research examining the accuracy of teachers' judgments in assessing students' academic abilities has shown relatively high accuracy. However, previous studies have relied primarily on norm‐referenced measures to obtain estimates of students' achievement in reading and mathematics. Recent developments in the assessment of students' academic skills, such as Curriculum‐Based Measurement (CBM), provide a direct estimate of students' skill levels in basic areas such as reading and mathematics. The purpose of the present study was to examine the extent to which teachers' perceptions of students' reading and mathematics skills corresponded to direct estimates of students' reading and mathematics skills. Two second‐grade teachers estimated the reading and mathematics skills of 33 second‐grade students. Results of this study indicated that teachers were not accurate in assessing their students' mathematics functioning. Teachers were more accurate in assessing the occurrence of Mastery mathematics levels in basic addition, but were very inaccurate in assessing the occurrence of Mastery, Instructional, or Frustrational mathematics levels in all other skills assessed. In reading, teachers' judgment accuracy varied as a function of grade‐level material and instructional level. Specifically, teachers experienced considerable difficulty accurately identifying students who were reading at a Mastery level in grade‐level or above‐grade‐level material. © 2006 Wiley Periodicals, Inc. Psychol Schs 43: 247–265, 2006.     

First-generation college students: Characteristics,experiences, and cognitive development

This study sought answers to three questions: (1) Do the precollege characteristics of first-generation students differ from those of traditional students? (2) Do first-generation students' college experiences differ from those of other students? (3) What are the educational consequences of any differences on first-year gains in students' reading, math, and critical thinking abilities? Answers come from 2,685 students (825 first-generation and 1,860 traditional students) who entered 23 diverse institutions nationwide in Fall 1992 and who completed one year of study. First-generation students differ from their traditional peers in both entering characteristics and college experiences. Although traditional students make greater net gains in reading during their first year, the two groups gain to about the same degree in math and critical thinking skills. Those gains, however, appear to result from somewhat different experiences.     

Speaking Personally—With Robert G. Holmberg

This study investigated the effects of using Socratic questioning to enhance students' critical thinking (CT) skills in asynchronous discussion forums (ADF) in university-level distance learning courses. The research effort empirically examined two coherent subjects: (a) the efficacy of teaching and modeling Socratic questioning for developing students' CT skills in ADF and (b) the persistence of students' CT skills following the teaching and modeling of Socratic questioning in the ADF. The results indicate (a) teaching and modeling of Socratic questioning helped students demonstrate a higher level of CT skills and (b) students maintained their CT skills after exposure to and modeling of Socratic questioning in the ADF.     

The effects of instructional methods on students' learning outcomes requiring different cognitive abilities: context-aware ubiquitous learning versus traditional instruction

The purpose of this study was to explore the effects of the context-aware ubiquitous learning (u-learning) approach versus traditional instruction on students' ability to answer questions that required different cognitive skills, using the framework of Bloom's taxonomy of educational objectives, including knowledge, comprehension, application, analysis, and synthesis. In this study, 230 third- and fourth-grade students in 8 classes were counterbalanced and assigned to the u-learning approach and traditional instruction for learning different topics in two separate plant-observing activities. The results showed that the students who learned with traditional instruction performed better than those who used the u-learning approach in terms of answering questions that required their cognitive abilities of knowledge, comprehension, application, analysis, and synthesis. Moreover, an in-depth analysis of the students' learning behaviors in the u-learning context revealed that most of their learning behaviors recorded in the u-learning system were not significantly related to their cognitive abilities.     

Predictors of well‐structured and ill‐structured problem solving in an astronomy simulation

This study compared the problem‐solving skills required for solving well‐structured problems and ill‐structured problems in the context of an open‐ended, multimedia problem‐solving environment in astronomy. Two sets of open‐ended questions assessed students' abilities for solving well‐structured and ill‐structured problems. Generalized, rubric scoring systems were developed for assessing problem‐solving skills. Instruments were also developed and administered to assess cognitive and affective predictors of problem‐solving performance. By regressing the scores on the cognitive and affective predictors onto students' scores on the well‐structured and ill‐structured problems, we concluded that solving well‐structured and ill‐structured problems require different component skills. Domain knowledge and justification skills were significant predictors of well‐structured problem‐solving scores, whereas ill‐structured problem‐solving scores were significantly predicted by domain knowledge, justification skills, science attitudes, and regulation of cognition. Implications for problem solving in science education are presented. © 2003 Wiley Periodicals, Inc. J Res Sci Teach 40: 6–33, 2003     

Science Process Skills and Cognitive Preference Styles

The study examined the differences in cognitive styles between two comparable groups of students at the Grade 9 (Secondary 3) level, namely the LSS (Lower Secondary Science) group who had been exposed to the practical-based, inquiry-oriented type of science and the non-LSS group of students who had studied the more traditional type of science in the “old” science curriculum. Their differences in science achievement are measured by the common IEA Science Paper-Pencil, Multiple Choice Criterion Test and also, by the Science Process or Practical Test (which measured three levels of process skills, such as the observation/manipulation, reasoning and investigation skills). Variance in science achievement thus measured is examined against the 4 cognitive preference styles of the students, (measured by the Combined Cognitive Preference Inventory) namely the “recall”, “principles”, “applications” and “questioning” modes of thinking. The findings indicated that (a) the attainment of the science process or practical skills was characterised by the type of science curriculum (LSS or non-LSS) and it was significantly associated with the achievement level of students, (b) the cognitive preference pattern covaried according to the students' level of science achievement and the type of curriculum and (c) the weak but significant relationship between performance in the science practical skills and the students' modes of cognitive style have important implications for teachers who are concerned about the intended effects of changes in the science curriculum on the consequent learning behaviour or cognitive outcome of their students.     

Explaining systems: Investigating middle school students' understanding of emergent phenomena

Science, as with all cognitive activities, is fundamentally a matter of interpretation, sense‐making, and explanation. This study focused on a small group of middle school students as they developed understanding of a particular type of phenomena: emergent systems. Such systems are notable in that macro‐level properties emerge as the result of micro‐level interactions between system components. I describe students' initial understanding of emergent systems, as well as the ways in which their thinking came to reflect the following heuristics: (a) recognition that there may not be a singular causal force underlying the system; (b) distinguishing between micro‐ and macro‐levels of analysis; and (c) comprehending that even small changes at the micro‐level can have significant effects at the macro‐level (Resnick, 1994 ). I conclude by considering some implications for science education. © 2000 John Wiley & Sons, Inc. J Res Sci Teach 37: 784–806, 2000     

The effects of environment‐based education on students' critical thinking skills and disposition toward critical thinking

This study examined the relationship between environment‐based education and high school students' critical thinking skills and disposition toward critical thinking. Four hundred four 9^th and 12^th grade students from 11 Florida high schools participated in the study. A Pretest‐Posttest Nonequivalent Comparison Group Design (9^th grade) and a Posttest Only Nonequivalent Comparison Group Design (12^th grade) were used. Interviews of students and teachers were used in the classic sense of triangulation. Data collection took place over the 2001–2002 school year. When controlling for pretest score, grade point average (GPA), gender, and ethnicity, environment‐based programs had a positive effect on 9^th grade students' critical thinking skills (p=.002). When controlling for GPA, gender, and ethnicity, environment‐based programs had a positive effect on 12^th grade students' critical thinking skills (p < .001) and disposition toward critical thinking (p < .001). The results of this study support the use of environment‐based education for improving critical thinking and can be used to guide future implementation.     

Engaging students in environmental research projects: Perceptions of fluency with innovative technologies and levels of scientific inquiry abilities

The purpose of this mixed‐method study was to investigate the changes in high school students' perceptions of fluency with innovative technologies (IT) and the levels of students' scientific inquiry abilities as a result of engaging students in long‐term scientific research projects focusing on community‐based environmental issues. Over a span of 3 years, a total of 125 ninth‐ through twelfth‐grade students participated in this study. A project‐specific Likert‐scale survey consisting of three parts (fluency with All Technologies, GPS/GIS, and CBL2/EasyData) was administered to all students as a pre‐ and post‐test. At the end of the study, 45 students were randomly interviewed and asked to elaborate on the changes in their perceptions of fluency with IT. The results indicated statistically significant increases (p < 0.001) in students' perceptions of their fluency with IT. Qualitative analysis of students' interview results corroborated the statistical findings of students' changes in perceptions of their fluency with IT. Students' research papers based on the environmental studies conducted at the interface of classroom and community were analyzed using the Scientific Inquiry Rubrics, which consist of 11 criteria developed by the researchers. Results indicated the students' abilities to conduct scientific inquiry for 7 out of 11 criteria were at the proficient level. This study clearly points to the correlation between the development of IT fluency and ability levels to engage in scientific inquiry based on respective competencies. Ultimately, this research study recommends that students' IT fluency ought to be developed and assessed concurrently with an emphasis on contemporary higher order scientific inquiry abilities. © 2010 Wiley Periodicals, Inc. J Res Sci Teach 48: 94–116, 2011     

Learning about energy: The influence of alternative frameworks,cognitive levels,and closed-mindedness

This article reports on a two-part study designed (a) to find the relation, if any, between junior high school students' alternative frameworks on energy held prior to instruction and their cognitive level of operations, either preformal or formal, and (b) to determine the relation, if any, between junior and senior high school students' success or failure in learning about energy and their alternative frameworks, their levels of cognitive operations, and their tendencies toward open- or closed-mindedness. In Part 1, we found no significant relation between junior high school students' prior alternative frameworks on energy and their cognitive levels of operations. In Part 2, we found that significantly better learning outcomes were achieved by students who had higher cognitive level scores. We also found that the extent to which students succeeded in learning the energy concept was a function of their prior knowledge. That is, certain alternative frameworks held prior to instruction may have facilitated the learning process. Finally, we found that the two groups (i.e., those who succeeded in learning about energy and those who did not) could not be distinguished according to scores in open- or closed-mindedness. These finding reinforce the assumption that learning is more domain specific than earlier theorists believed.     

Classification and framing in the case method: discussion leaders' questions

Basil Bernstein's classification and framing was adopted as a theoretical model to analyse the instruction of two university professors who incorporated case studies into their graduate business and education courses. Classification and framing allows for a meaningful analysis of the discussion leader's questions that facilitate students' understanding of a case. The investigation identified values related to the questioning process that purportedly fosters students' communication skills, critical‐thinking abilities, and problem‐solving skills in a case discussion. This study contributes to a growing body of literature that seeks a deeper understanding of the facilitator's role in case‐based discussions.     

Cognitive foundations for science assessment design: Knowing what students know about evolution

To improve assessments of academic achievement, test developers have been urged to use an “assessment triangle” that starts with research‐based models of cognition and learning [NRC (2001) Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press]. This approach has been successful in designing high‐quality reading and math assessments, but less progress has been made for assessments in content‐rich sciences such as biology. To rectify this situation, we applied the “assessment triangle” to design and evaluate new items for an instrument (ACORNS, Assessing Contextual Reasoning about Natural Selection) that had been proposed to assess students' use of natural selection to explain evolutionary change. Design and scoring of items was explicitly guided by a cognitive model that reflected four psychological principles: with development of expertise, (1) core concepts facilitate long‐term recall, (2) causally‐central features become weighted more strongly in explaining phenomena, (3) normative ideas co‐exist but increasingly outcompete naive ideas in reasoning, and (4) knowledge becomes more abstract and less specific to the learning situation. We conducted an evaluation study with 320 students to examine whether scores from our new ACORNS items could detect gradations of expertise, provide insight into thinking about evolutionary change, and predict teachers' assessments of student achievement. Findings were consistent with our cognitive model, and ACORNS was revealing about undergraduates' thinking about evolutionary change. Results indicated that (1) causally‐central concepts of evolution by natural selection typically co‐existed and competed with the presence of naïve ideas in all students' explanations, with naïve ideas being especially prevalent in low‐performers' explanations; (2) causally‐central concepts were elicited most frequently when students were asked to explain evolution of animals and familiar plants, with influence of superficial features being strongest for low‐performers; and (3) ACORNS scores accurately predicted students' later achievement in a college‐level evolution course. Together, findings illustrate usefulness of cognitive models in designing instruments intended to capture students' developing expertise. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 744–777, 2012     

Perceived Role of Multimedia Instructional Materials on Multicriteria Technology and Engineering Decisions*

Studies have shown that it is difficult for people to deal with multicriteria decision‐making situations. Information technology tools such as decision‐support systems and expert systems have been developed in order to help them in such situations. Another tool that has been identified as helping managers understand complex engineering decision‐making situations is multimedia instructional materials. This research investigates the perceptions of business versus engineering students on improvement of their higher‐level cognitive skills when they participated in a multimedia‐based case study that used an expert system to model a complex engineering and technical problem. Two groups of students, business and engineering, participated in an experiment, where they analyzed the case study and made their recommendations. Two questionnaires measured their perceptions on the improvements achieved on different constructs. A structural equations model was developed in order to test the research hypotheses with business students being the experimental group and the engineering students as a control group. The major findings are no significant relationship between student major and higher‐order cognitive skills improvement, business students perceived more higher‐order skills improvement compared to engineering students, both groups perceived an improvement in learning‐driven factor, and business students valued learning‐driven factors more compared to engineering students. These results show that multimedia instructional materials were perceived to be very useful in making multicriteria engineering and technical decisions.