Design,validation, and use of an evaluation instrument for monitoring systemic reform

Over the past decade, state and national policymakers have promoted systemic reform as a way to achieve high‐quality science education for all students. However, few instruments are available to measure changes in key dimensions relevant to systemic reform such as teaching practices, student attitudes, or home and peer support. Furthermore, Rasch methods of analysis are needed to permit valid comparison of different cohorts of students during different years of a reform effort. This article describes the design, development, validation, and use of an instrument that measures student attitudes and several environment dimensions (standards‐based teaching, home support, and peer support) using a three‐step process that incorporated expert opinion, factor analysis, and item response theory. The instrument was validated with over 8,000 science and mathematics students, taught by more than 1,000 teachers in over 200 schools as part of a comprehensive assessment of the effectiveness of Ohio's systemic reform initiative. When the new four‐factor, 20‐item questionnaire was used to explore the relative influence of the class, home, and peer environment on student achievement and attitudes, findings were remarkably consistent across 3 years and different units and methods of analysis. All three environments accounted for unique variance in student attitudes, but only the environment of the class accounted for unique variance in student achievement. However, the class environment (standards‐based teaching practices) was the strongest independent predictor of both achievement and attitude, and appreciable amounts of the total variance in attitudes were common to the three environments. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 646–662, 2001     

Classroom Environment Instruments: Development,Validity and Applications

Few fields of educational research have such a rich diversity of valid, economical and widely-applicable assessment instruments as does the field of learning environments. This article describes nine major questionnaires for assessing student perceptions of classroom psychosocial environment (the Learning Environment Inventory, Classroom Environment Scale, Individualised Classroom Environment Questionnaire, My Class Inventory, College and University Classroom Environment Inventory, Questionnaire on Teacher Interaction, Science Laboratory Environment Inventory, Constructivist Learning Environment Survey and What Is Happening In This Class) and reviews the application of these instruments in 12 lines of past research (focusing on associations between outcomes and environment, evaluating educational innovation, differences between student and teacher perceptions, whether students achieve better in their preferred environment, teachers' use of learning environment perceptions in guiding improvements in classrooms, combining quantitative and qualitative methods, links between different educational environments, cross-national studies, the transition from primary to high school, and incorporating educational environment ideas into school psychology, teacher education and teacher assessment).     

When Does 'Good News' Become 'Bad News'? Relationships between Government and the Integrated Schools in Northern Ireland

The development of a set of religiously integrated schools in Northern Ireland since 1981 is often portrayed in wholly positive terms. However, the continued growth of the movement has generated serious tensions at a number of levels which have wider implications for any analysis of the implications of parental choice.     

What do we know about early sport specialization? Not much!

In November 2005, the International Olympic Committee's Medical Commission issued a statement on Training the elite child athlete recommending that “more scientific research be done to better identify the parameters of training the elite child athlete”. This paper focuses on a specific issue related to training the child athlete: early specialization. While associations between early specialization and expertise development are highlighted, much research also suggests early specialization is associated with a range of negative consequences affecting physical, psychological, and social development. Alternately, some researchers have proposed that an early diversification approach does not disadvantage athletes in acquiring expertise, and is important for the development of intrinsic motivation and skill transferability. However, this review suggests that inadequate evidence exists to resolve the issue in favor of either approach.     

The advance of research and evaluation skills by EPs: implications for training and professional development

Research and evaluation has become increasingly important in many professions, including educational psychology, particularly as the profession moves to becoming more evidence‐based in its practice. The recent move to a three year doctoral programme in England, Wales and Northern Ireland may be viewed as a positive one in developing research skills, although it is also crucial to consider what could be done to extend the research skills of those already qualified. Lessons may be learned by looking to clinical psychology and some of the health professions. Finally, some further implications for the training and professional development of Educational Psychologists (EPs) in the UK are considered.     

Applied psychology and the case for individual casework: some reflections on the role of the educational psychologist

The authors consider the changing role of educational psychology in the current era and emphasise the importance of casework‐based interventions not only for influencing systemic work but also for maintaining credibility within the educational marketplace. With in‐school counselling and intervention becoming more popular, the natural providers of this service should be educational psychologists but this may not be the case in many local authorities. The authors consider the possibility of the profession becoming obsolete if there is a continuation of the move away from individual interventions to a more consultative based approach.     

The ongoing education of engineering practitioners: how do they perceive the usefulness of academic research?

This study explores the usefulness of academic research to engineering practitioners. Issues such as knowledge transfer, the research-practice gap and the changing social/political environment are discussed. An empirical survey of practitioners is analysed to better understand the engineering professions use of academic material, such as research journals, with the medical profession acting as the benchmark. The study also examines the future role for academia from the perspective of practitioners, with areas such as engagement, relevance, dissemination and education highlighted. While the study found there was a reasonable level of utilisation of academic material, there are still some significant gaps. The study brings to the fore the issue of relevance in applied research fields and highlights the need to evaluate the usefulness of our research to practitioners and the betterment of society.     

A cross-national mixed-method study of reality pedagogy

This mixed-methods cross-national study investigated the effectiveness of reality pedagogy (an approach in which teachers become part of students’ activities, practices and rituals) in terms of changes in student perceptions of their learning environment and attitudes towards science. A questionnaire was administered to 142 students in grades 8–10 in the Bronx, New York City and Dresden, Germany. The questionnaire combines learning environment scales from the Constructivist Learning Environment Survey and the What Is Happening In this Class? Questionnaire with attitude scales from the Test of Science-Related Attitudes. Student interviews were used to support questionnaire findings. Quantitative data analyses revealed that reality pedagogy had a greater impact on students in the Bronx than in Dresden, with qualitative data clarifying differences in how reality pedagogy was enacted in each geographic area. Overall, our findings add to the body of evidence concerning the effectiveness of reality pedagogy as an approach to teaching and learning science across a variety of contexts.     

Differential Effectiveness of Computer-assisted Instruction for Boys and Girls

A sample of 671 secondary school students in 24 classes of nominally slow learners was involved in investigating not only the relative efficacy of computer-assisted instruction (CAI) and traditional expository teaching, but particularly the differential effectiveness of the two teaching methods for boys and girls. Both achievement and attitude outcomes were investigated. The difference in mean scores between CAI and traditional classes overall was 3.5 standard deviations for achievement and 1.4 standard deviations for attitude. However, although there was no overall effect of gender on either outcome, a significant treatment-gender interaction occurred for the achievement outcome (but not for the attitude outcome). Boys achieved better than girls in the CAI groups, but boys and girls achieved similarly in the control group.     

Relationship between perceived levels of classroom individualization and science-related attitudes

Relationships between perceived levels of classroom individualization and science-related attitudes were explored for a sample of 712 junior high school science students. Five dimensions of perceived individualization (personalization, participation, independence, investigation, and differentiation) were measured with the Individualized Classroom Environment Questionnaire (ICEQ), while seven distinct attitudes were measured with the Test of Science-Related Attitudes (TOSRA). Hierarchical regression analyses revealed that the set of individualization variables accounted for a significant increment in end-of-year attitude scores, beyond that attributable to corresponding beginning-of-year attitude scores, for four of the seven attitudes considered. Significant associations between an individual individualization variable and an attitudinal dimension were positive in all cases. The study also provided support for the reliability and validity of the ICEQ and TOSRA and for their general usefulness in science education research.