More on time series designs: A reanalysis of mayer and Kozlow's data

A recent article by Mayer and Kozlow introduces a time series experiment to science education researchers. This article examines in greater detail design considerations and reanalyzes their data using time series analysis. The first applications of time series experimental methodology to science education research were reported by Mayer and Lewis (1979) and by Mayer and Kozlow (1980). This article discusses Mayer and Kozlow's work and presents a reanalysis of their data which is intended to be heuristic, not critical. (All data were supplied by Dr. Mayer and permission was granted to use the data.) Both design and analysis will be examined. It is important to distinguish time series experimental design from time series analysis. The former, as presented by Campbell and Stanley (1966) and detailed by Glass, Willson, and Gottman (1975), is concerned with structuring treatment conditions and observations over time to make causal inferences. Time series analysis is a set of statistical techniques from which an appropriate technique can be selected to analyze collected data in the framework of a time series design. Just as data from in a randomized experiment might be analyzed using a one-way analysis of variance or a Kruskal- Wallis H-test, data from a time series design may be subjected to several alternative analysis techniques.     

A comparison of concrete and formal science instruction upon science achievement and reasoning ability of sixth grade students

Several recent studies suggest concrete learners make greater gains in student achievement and in cognitive development when receiving concrete instruction than when receiving formal instruction. This study examined the effect of concrete and formal instruction upon reasoning and science achievement of sixth grade students. Four intact classes of sixth grade students were randomly selected into two treatment groups; concrete and formal. The treatments were patterned after the operational definitions published by Schneider and Renner (1980). Pretest and posttest measures were taken on the two dependent variables; reasoning, measured with Lawson's Classroom Test of Formal Reasoning, and science achievement, measured with seven teacher made tests covering the following units in a sixth grade general science curriculum: Chemistry, Physics, Earth Science, Cells, Plants, Animals, and Ecology. Analysis of covariance indicated significantly higher levels (better than 0.05 and in some cases 0.01) of performance in science achievement and cognitive development favoring the concrete instruction group and a significant gender effect favoring males.     

Identification and description of the momentum effect in studies of learning: An abstract science concept

Several studies of the validity of the intensive time series design have revealed a post-intervention increase in the level of achievement data. This so called “momentum effect” has not been demonstrated through the application of an appropriate analysis technique. The purpose of this study was to identify and apply a technique that would adequately represent and describe such an effect if indeed it does occur, and to use that technique to study the momentum effect as it is observed in several data sets on the learning of the concept of plate tectonics. Subsequent to trials of several different analyses, a segmented straight line regression analysis was chosen and used on three different data sets. Each set revealed similar patterns of inflection points between lines with similar time intervals between inflections for those data from students with formal cognitive tendencies. These results seem to indicate that this method will indeed be useful in representing and identifying the presence and duration of the momentum effect in time series data on achievement. Since the momentum effect could be described in each of the data sets and since its presence seems a function of similar circumstances, support is given for its presence in the learning of abstract scientific concepts for formal cognitive tendency students. The results indicate that the duration of the momentum effect is related to the level of student understanding tested and the cognitive level of the learners.     

The nature of advanced reasoning and science instruction

Although the development of reasoning is recognized as an important goal of science instruction, its nature remains somewhat of a mystery. This article discusses two key questions: Does formal thought constitute a structured whole? And what role does propositional logic play in advanced reasoning? Aspects of a model of advanced reasoning are presented in which hypothesis generation and testing are viewed as central processes in intellectual development. It is argued that a number of important advanced reasoning schemata are linked by these processes and should be made a part of science instruction designed to improve students' reasoning abilities. Concerning students' development and use of formal reasoning, Linn (1982) calls for research into practical issues such as the roles of task-specific knowledge and individual differences in performance, roles not emphasized by Piaget in his theory and research. From a science teacher's point of view, this is good advice. Accordingly, this article will expand upon some of the issues raised by Linn in a discussion of the nature of advanced reasoning which attempts to reconcile the apparent contradiction between students' differential use of advanced reasoning schemata in varying contexts with the notion of a general stage of formal thought. Two key questions will be discussed: Does formal thought constitute a structured whole? And what role does propositional logic play in advanced reasoning? The underlying assumption of the present discussion is that, among other things, science instruction should concern itself with the improvement of students' reasoning abilities (cf. Arons, 1976; Arons & Karplus, 1976; Bady, 1979; Bauman, 1976; Educational Policies Commission, 1966; Herron, 1978; Karplus, 1979; Kohlberg & Mayer, 1972; Moshman & Thompson, 1981; Lawson, 1979; Levine & linn, 1977; Pallrand, 1977; Renner & Lawson, 1973; Sayre & Ball, 1975; Schneider & Renner, 1980; Wollman, 1978). The questions are of interest because to date they lack clear answers, yet clear answers are necessary if we hope to design effective instruction in reasoning.     

Cognitive Development and Comprehension of Physics Concepts

Summaries

English

The purpose of this study was twofold: (1) to investigate the cognitive development of llth‐grade science students in Jordan; and (2) to investigate the relationship between cognitive development of science students and their comprehension of concrete and formal physics concepts.

The sample of this study consisted of 389 secondary science students (209 males and 180 females) of the school year 1979‐1980. Cognitive development of the students was measured by an Arabic version of the Longeot Test. Comprehension of physics concepts was assessed by an achievement test constructed by the researchers and validated by teachers of physics and the science supervisor in the Irbid district of Jordan.

The study did not reveal any significant difference between males and females in cognitive development. Moreover, it revealed that only 17% of the students were formal thinkers, whereas 52% were concrete thinkers.

The 2×3 ANOVA revealed that cognitive level was a highly significant factor in the comprehension of both concrete and formal concepts. It also revealed that while sex was not a significant factor in the comprehension of concrete concepts, it was a significant factor in the comprehension of formal concepts. The interaction between sex and cognitive level was not significant in the comprehension of both types of concepts.     

Effects of traditional and discovery instructional approaches on learning outcomes for learners of different intellectual development: A study of chemistry students in Zambia

This study examined the differential effectiveness of traditional and discovery methods of instruction for the teaching of science concepts, understandings about science, and scientific attitudes, to learners at the concrete and formal level of cognitive development. The dependent variables were achievement, understanding science, and scientific attitude; assessed through the use of the ACS Achievement Test (high school chemistry, Form 1979), the Test on Understanding Science (Form W), and the Test on Scientific Attitude, respectively. Mode of instruction and cognitive development were the independent variables. Subjects were 120 Form IV (11th grade) males enrolled in chemistry classes in Lusaka, Zambia. Sixty of these were concrete reasoners (mean age = 18.23) randomly selected from one of the two schools. The remaining 60 subjects were formal reasoners (mean age 18.06) randomly selected from a second boys' school. Each of these two groups was randomly split into two subgroups with 30 subjects. Traditional and discovery approaches were randomly assigned to the two subgroups of concrete reasoners and to the two subgroups of formal reasoners. Prior to instruction, the subjects were pretested using the ACS Achievement Test, the Test on Understanding Science, and the Test on Scientific Attitude. Subjects received instruction covering eight chemistry topics during approximately 10 weeks. Posttests followed using the same standard tests. Two-way analysis of covariance, with pretest scores serving as covariates was used and 0.05 level of significant was accepted. Tukey WSD technique was used as a follow-up test where applicable. It was found that (1) for the formal reasoners, the discovery group earned significantly higher understanding science scores than the traditional group. For the concrete reasoners mode of instruction did not make a difference; (2) overall, formal reasoners earned significantly higher achievement scores than concrete reasoners; (3) in general, subjects taught by the discovery approach earned significantly higher scientific attitude scores than those taught by the traditional approach. The traditional group outperformed the discovery group in achievement scores. It was concluded that the traditional approach might be an efficient instructional mode for the teaching of scientific facts and principles to high school students, while the discovery approach seemed to be more suitable for teaching scientific attitudes and for promoting understanding about science and scientists among formal operational learners.     

Cognitive development in a secondary science setting

Observations were made of the progressive change in the cognitive development of 141 students over the course of their secondary education in an Australian private school. Cognitive development was measured in years 8, 10 and 12 usingBond's Logical Orerations Test. Rasch analysis of each of the data sets provided ability estimates for students in the year groups of 1993 (year 8), 1995 (year 10) and 1997 (year 12). Twenty-nine students from the year group of 1993 were tested on all three occasions. We analysed data from these 29 students in order to investigate the children's cognitive development across years 8, 10 and 12. We also examined the influence of the Cognitive Acceleration through Science Education (CASE)Thinking Science program on the cognitive development and scholastic achievement of these students. We found increased mental growth between years 8 and 10 for most students in theThinking Science cohort, which could not be predicted from their starting levels. There was a significant correlation between cognitive development and the scholastic achievement of these students. Although boys as a group were more advanced in cognitive development than girls in years 8 and 10, no difference was found in the rate of cognitive change based on sex up to year 10. However girls showed cognitive gains across years 10–12 which were not found in boys. The students who were new to the school also showed increased cognitive development in years 11 and 12. Students who had experienced theThinking Science course were more cognitively developed than students who joined the school after the intervention had taken place. This study supports the claim of Adey and Shayer that there is a relationship between cognitive development and scholastic achievement, even though we used different measures of cognitive development and scholastic achievement.     

Accelerating the development of formal thinking in middle and high school students II: Postproject effects on science achievement

A one-year lag was found between the effect of an intervention intended to promote formal operational thinking in students initially 11 or 12 years of age and the appearance of substantial science achievement in the experimental groups. A one-year lag was also reported on cognitive development: Whereas at the end of the two-year intervention the experimental groups were up to 0.9σ ahead of the control groups, one year later the differential on Piagetian measures had disappeared, but the experimentals now showed better science achievement of even greater magnitude. Although the control groups showed normal distribution both on science achievement and cognitive development, the experimental groups showed bi- or trimodal distribution. Between one-half and one-quarter of the students involved in the experiment in different groups showed effects of the order of 2σ both on cognitive development and science achievement; some students appeared unaffected (compared with the controls), and others demonstrated modest effects on science achievement. An age/gender interaction is reported: the most substantial effects were found in boys initially aged 12+ and girls initially 11+. The only group to show no effects was boys initially aged 11+. It is suggested that the intervention methods may have favored the abstract analytical learning style as described by Cohen 1986.     

The four-card problem resolved? Formal operational reasoning and reasoning to a contradiction

To test the hypothesis that adolescents classified as formal operational, based upon use of proportional reasoning on the “Pouring Water Task” (Lawson, Karplus, & Adi, 1978) have acquired the mental structures necessary to comprehend hypothetico-deductive arguments of a pattern referred to as “reasoning to a contradiction,” while adolescents classified as concrete operational, based upon use of additive reasoning on the same task have not, a sample of 100 high school students were administered the task and three versions of a problem requiring use of reasoning to a contradiction before, immediately after, and one month after brief verbal instruction in use of that reasoning pattern. Results were generally supportive of the hypothesis as most of the concrete students failed the immediate and delayed posttest problems (62 and 80%, respectively) while most of the formal students succeeded (80 and 71%, respectively). Group differences were significant (p < .001) in both cases. These results suggest that, contrary to those who have argued that content plays a substantial role in logical performance, a general hypothetico-deductive reasoning competence exists in some adolescents and is applicable across a wide variety of task domains. Science instruction which aims to teach this competence is recommended.     

Systematic modeling versus the learning cycle: Comparative effects on integrated science process skill achievement

This study assessed the effectiveness of the systematic modeling teaching strategy on integrated science process skills and formal reasoning ability. Urban middle school students received a three-month process skill intervention treatment from teachers trained in either the use of systematic modeling or the learning-cycle model. A third, control group received traditional science instruction. The analysis of data revealed that (a) students receiving modeled instruction demonstrated a significant difference in their achievement of process skills when compared to either of the control groups. (b) Students taught by teachers who had received special process skill and strategy training demonstrated a significant difference in their process skill achievement when compared with the control group. (c) Students at different cognitive reasoning levels demonstrated significantly different process skill ability.     

Climate change education: quantitatively assessing the impact of a botanical garden as an informal learning environment

Although informal learning environments have been studied extensively, ours is one of the first studies to quantitatively assess the impact of learning in botanical gardens on students' cognitive achievement. We observed a group of 10th graders participating in a one-day educational intervention on climate change implemented in a botanical garden. The students completed multiple-choice questionnaires in a pre-post-retention test design. Comparing the test scores revealed a significant short-term knowledge gain as well as a long-term knowledge gain. Consequently, our results show the potentials of botanical gardens as effective learning environments, and for complementing formal school-based learning settings regarding climate change education.     

Association between schoolwide positive behavioural interventions and supports and academic achievement: a 9-year evaluation

This study evaluated the long-term impact of schoolwide positive behavioural interventions and supports (PBIS) on student academic achievement. In this quasi-experimental study, academic achievement data were collected over 9 years. The 21 elementary, middle, and high schools that achieved moderate to high fidelity to the Save & Civil Schools’ PBIS model were matched with 28 control schools to assess academic gains. There were 5 years of baseline data (no intervention in treatment schools) and 4 years of intervention data, including 1 year of maintenance. Results indicate that implementation of the PBIS programme was significantly associated with increased student academic achievement (p = .001) and that the rate of change for students’ academic achievement in treatment schools was greater than for students in control schools. This study suggests that PBIS programmes, such as Safe & Civil Schools’ Foundations, may complement other efforts to improve academic outcomes.     

The effect of frequency of testing upon the measurement of achievement in an intensive time-series design

Progress in the development of an intensive time-series design for use in classroom situations has raised several questions regarding the validity of the data obtained. One such question is related to the effect of frequency of testing upon the data. This study was designed to determine the presence of such an effect. Use of the design with eighth grade students studying a unit on crustal evolution indicated that the frequency of data collection did not adversely affect student attitudes toward the science class nor did it have any measurable effect upon class achievement data. The results of a previous study were also replicated, giving increased confidence in the validity of the data yielded by the intensive time-series design as used in these studies.     

Generalized achievement goals and situational coping in inquiry learning

This study examined situation-specific coping strategies of students with different achievement goals as they take part in inquiry learning. A case study was conducted in a Finnish elementary school. 21 ten-year-old students participated in the study. Two types of data were collected: (1) Students self-reported questionnaires on achievement goals, (2) Video data on the students' learning processes and social interaction. The results revealed the importance of a situative perspective in instructional design, since the students not only differed in their coping attempts regarding their initial goals but also according to their individual situational interpretations that mediated their active coping attempts. Some of the non-learning-focused students had difficulties engaging in the working procedures. A lack of teacher's concrete and precise guidance in both cognitive and motivational sense appeared to explain this phenomenon.     

Facilitating successful prediction problem solving in biology through application of skill theory

The purpose of this study was to identify cognitive factors associated with differences in prediction problem-solving success among high school biology students, and to determine whether guided practice facilitated successful prediction. The Group Assessment of Logical Thinking was used to evaluate subjects' cognitive operational level, written prediction worksheets and think-aloud interviews were used to measure predictive success and identify problem-solving tendencies. Treatment group subjects received 8 hr of directed prediction practice using interactive computer simulations, whereas the control group practiced prediction without focusing on specific skills, after which all subjects were retested. Predictive reasoning success showed a significant correlation (p < .01) to both formal operational development and five specific cognitive skills: (a) identifying relevant knowledge in long-term memory, (b) using a systematic problem-solving strategy, (c) applying cause-effect reasoning, (d) reviewing solutions for logical inconsistency, and (e) evaluating alternative solutions. Analysis of covariance indicated significantly increased prediction success for treatment group subjects following practice in the five identified skills (p < .01). © 1996 John Wiley & Sons, Inc.     

Immediate and delayed effects of meta-cognitive instruction on regulation of cognition and mathematics achievement

The present study addressed two research questions: (a) the extent to which students who were exposed to meta-cognitive instruction are able to implement meta-cognitive processes in a delayed, stressful situation, in our case—being examined on the matriculation exam; and (b) whether students preparing themselves for the matriculation exam in mathematics, attain a higher level of mathematics achievement and meta-cognitive awareness (knowledge about cognition and regulation of cognition) as a result of being exposed to meta-cognitive instruction. Participants were 61 Israeli high school students who studied mathematics for four-point credit on the matriculation exam (middle level). About half of the students (N = 31) were assigned to meta-cognitive instruction, called IMPROVE, and the others (N = 30) studied with no explicit meta-cognitive guidance (control group). Analyses included both quantitative and qualitative methods. The later was based on students’ interviews, conducted about a couple of months after the end of the intervention, immediately after students completed the matriculation exam in mathematics. Results indicated that IMPROVE students outperformed their counterparts on mathematics achievement and regulation of cognition, but not on knowledge about cognition. Furthermore, during the matriculation exam, IMPROVE students executed different kinds of cognitive regulation processes than the control students. The theoretical and practical implications of the study are discussed.     

Examining the effectiveness of guided inquiry with problem-solving process and cognitive function training in a high school chemistry course

The purpose of this study was to examine the effectiveness of traditional versus guided inquiry (with problem-solving process and cognitive function training) on high school chemistry knowledge, science process skills, scientific attitudes, and problem-solving competency. Two classes of students were recruited from three classes of Grade 11 students at one school in North-eastern Thailand. Using a split-plot design, students were assigned to an experimental (N = 34) and a control group (N = 31), and were administered (a) learning achievement tests (chemistry knowledge, science process skills, and scientific attitude), (b) a problem-solving competency test, and c) tests of cognitive functioning. The findings showed that students’ learning achievement and problem-solving competency in the guided inquiry group were significantly higher than in the traditional group. The effect of the new teaching method does not seem to stem solely from improvement in cognitive functioning. We attributed the improvement to greater flexibility in the amount of information provided by the teachers, more effortful processing by the students, and greater collaboration amongst the students.     

Effects of integrated video media on student achievement and attitudes in high school chemistry

This study explored the effects of an integrated video media curriculum enhancement on students' achievement and attitudes in a first-year general high school chemistry course within a multiculturally diverse metropolitan school district. Through the use of a treatment-control experimental design, approximately 450 students in Grades 9–12 were sampled on measures of chemistry achievement and attitude over the period of 1 academic year. The results revealed significantly higher achievement scores on standardized measures of achievement as well as on microunit researcher-designed, criterion-referenced quizzes for the treatment students who experienced a general chemistry course enhanced with an integrated use of a structured chemistry video series. Correlation of student achievement with logical thinking ability revealed that students with high levels of logical thinking ability benefited most from the video-enhanced curriculum. Treatment students also scored significantly higher than control students on the chemistry attitude instrument. These results along with qualitative supportive evidence suggest that this integrated video media curriculum intervention can positively affect student chemistry achievement and attitude across ability levels and across a diverse multicultural population. Furthermore, the data suggest that educational science video media in general, and the World of Chemistry video series in particular, are instructional tools that can be used effectively to bring the often abstract, distant worlds of science into close focus and within the personal meaningful realm of each individual student. © 1997 John Wiley & Sons, Inc. J Res Sci Teach 34: 617–631, 1997.     

Computer simulations in the high school: Students' cognitive stages,science process skills and academic achievement in microbiology

Computer-assisted learning, including simulated experiments, has great potential to address the problem solving process which is a complex activity. It requires a highly structured approach in order to understand the use of simulations as an instructional device. This study is based on a computer simulation program, 'The Growth Curve of Microorganisms', which required tenth grade biology students to use problem solving skills whilst simultaneously manipulating three independent variables in one simulated experiment. The aims were to investigate the computer simulation's impact on students' academic achievement and on their mastery of science process skills in relation to their cognitive stages. The results indicate that the concrete and transition operational students in the experimental group achieved significantly higher academic achievement than their counterparts in the control group. The higher the cognitive operational stage, the higher students' achievement was, except in the control group where students in the concrete and transition operational stages did not differ. Girls achieved equally with the boys in the experimental group. Students' academic achievement may indicate the potential impact a computer simulation program can have, enabling students with low reasoning abilities to cope successfully with learning concepts and principles in science which require high cognitive skills.