Design,validation, and use of an evaluation instrument for monitoring systemic reform

Over the past decade, state and national policymakers have promoted systemic reform as a way to achieve high‐quality science education for all students. However, few instruments are available to measure changes in key dimensions relevant to systemic reform such as teaching practices, student attitudes, or home and peer support. Furthermore, Rasch methods of analysis are needed to permit valid comparison of different cohorts of students during different years of a reform effort. This article describes the design, development, validation, and use of an instrument that measures student attitudes and several environment dimensions (standards‐based teaching, home support, and peer support) using a three‐step process that incorporated expert opinion, factor analysis, and item response theory. The instrument was validated with over 8,000 science and mathematics students, taught by more than 1,000 teachers in over 200 schools as part of a comprehensive assessment of the effectiveness of Ohio's systemic reform initiative. When the new four‐factor, 20‐item questionnaire was used to explore the relative influence of the class, home, and peer environment on student achievement and attitudes, findings were remarkably consistent across 3 years and different units and methods of analysis. All three environments accounted for unique variance in student attitudes, but only the environment of the class accounted for unique variance in student achievement. However, the class environment (standards‐based teaching practices) was the strongest independent predictor of both achievement and attitude, and appreciable amounts of the total variance in attitudes were common to the three environments. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 646–662, 2001     

Big girls don't cry: the effect of child witness demeanor on juror decisions in a child sexual abuse trial

OBJECTIVE: This study investigated the effect of child witness demeanor (defined as crying) on mock jurors' decisions in a simulated First-Degree rape trial. METHOD: One hundred and thirty-three undergraduates serving in the role of mock jurors read a trial summary in which the primary independent variable was the demeanor of the alleged child victim (i.e., calm, teary, hysterical crying). In addition to reading the summary, participants viewed pencil drawings of the witnesses that were presented as "courtroom drawings." RESULTS: The results showed that the teary condition led to more guilty verdicts and a greater belief in the alleged victim than the other demeanor conditions. CONCLUSIONS: Findings from this study indicate that demeanor can impact the perception of a child who is an alleged sexual assault victim in court. However, it is not simply the case that any display of demeanor will lead to a positive outcome for the alleged victim. Instead, it appears that too little or too much emotion from the alleged child victim negatively affected credibility in the eyes of the mock jurors.     

Measuring Effectiveness of Health Education in a Web‐based Learning Environment: a preliminary report

Driven by claims of efficacy, flexibility and resource effectiveness, higher education is increasingly utilising the Web as an instructional tool. The claims for pedagogical effectiveness are often just that – claims — and appear not to have been proven in the reality of subject presentation and evaluation. Thus, it is necessary to examine assumptions regarding the benefits of Web‐based instruction in terms of effectiveness. This article discusses aspects of an investigation which examined and compared the effectiveness of HIV/AIDS related collaborative tutorial activities carried out in both a Web‐based learning environment and a face‐to‐face class situation within an undergraduate health education subject. Effectiveness of the pedagogical strategy and the different learning environments were measured in terms of observed learning outcomes and reported perceptions of the learners regarding their learning experience. Preliminary results based on measured learning outcomes related to the subject matter, HIV/AIDS, demonstrated that collaborative learning activities were significantly more effective in the Web‐based than in the class environment. Additionally, the vast majority of learners perceived the Web‐based environment to be as effective or more effective than the face‐to‐face, class environment in terms of facilitating their understanding of the issues explored in the subject.     

The Macro- and Micropolitics of Personnel Evaluation: A Framework

In this article, the authors explicate a conceptual framework for analyzing the politics of personnel evaluation in an educational context. Using several elements of their framework, they discuss the politics of teacher evaluation in California in relation to the types of personnel evaluation decisions, the actors, their access to these decisions, their sources and level of power, and the outcomes related to their decisions. The authors further portray how politics enters into personnel evaluation at the district level by describing an actual case.     

The Efficacy of Preschool Developmental Indicators as a Screen for Early Primary School-Based Literacy Interventions

Literacy success lays the foundation for children's later educational, health, and well-being outcomes. Thus, early identification of literacy need is vital. Using data from New Zealand's national preschool health screening program for fiscal years 2010/2011–2014/2015, demographic and health variables from 255,090 children aged 4 years were related to whether they received a literacy intervention in early primary school. Overall, 20,652 (8.1%) children received an intervention. Time-to-event analysis revealed that all considered variables were significantly related to literacy intervention (all p < .01), but the full model lacked reasonable predictive power for population screening purposes (Harrell's c-statistic = .624; 95% CI [.618, .629]). Including more direct literacy measures in the national screening program is likely needed for improvement.     

Classroom Environment Instruments: Development,Validity and Applications

Few fields of educational research have such a rich diversity of valid, economical and widely-applicable assessment instruments as does the field of learning environments. This article describes nine major questionnaires for assessing student perceptions of classroom psychosocial environment (the Learning Environment Inventory, Classroom Environment Scale, Individualised Classroom Environment Questionnaire, My Class Inventory, College and University Classroom Environment Inventory, Questionnaire on Teacher Interaction, Science Laboratory Environment Inventory, Constructivist Learning Environment Survey and What Is Happening In This Class) and reviews the application of these instruments in 12 lines of past research (focusing on associations between outcomes and environment, evaluating educational innovation, differences between student and teacher perceptions, whether students achieve better in their preferred environment, teachers' use of learning environment perceptions in guiding improvements in classrooms, combining quantitative and qualitative methods, links between different educational environments, cross-national studies, the transition from primary to high school, and incorporating educational environment ideas into school psychology, teacher education and teacher assessment).     

Research on Teacher Effectiveness: Implications for Remedial Teachers

Over the past decade considerable progress has been made in defining and describing teacher behaviours associated with high levels of pupil achievement. Collectively described as teacher effectiveness research, this body of knowledge is now influencing conceptions of effective teaching in special education settings. What is unclear, however, is the extent to which special education teachers are aware of this research, perceive it to be important, and actually use the findings in their day‐to‐day teaching. Nor is it clear what, if any, modifications are needed when findings are applied to instruction with low achieving and mildly handicapped children. The present study addresses the questions of the perceived importance of and the need for adaptations to teacher effectiveness findings as they relate to a sample of Queensland remedial teachers.     

A Cross-cultural Study of Classroom Learning Environments in Australia and Taiwan

This research is distinctive in that it not only provides an example of one of the few cross-cultural studies in science education, but also it used multiple research methods from different paradigms in exploring classroom learning environments in Taiwan and Australia. This article describes the validation and use of an English and Mandarin version of the What is Happening in this Class? (WIHIC) questionnaire in junior high school science classes in Australia and Taiwan. When the WIHIC was administered to 1,081 students in 50 classes in Australia and to 1,879 students in 50 classes in Taiwan, data analysis supported the reliability and factorial validity of the questionnaire, and revealed differences between Taiwanese and Australian classrooms. Although the study commenced from a more positivistic framework, favouring a more objectivist view, as the study progressed, it employed an interpretative framework and drew on elements of constructivist and critical theory paradigms. This article outlines the researchers' use of multiple research methods including classroom observations, in-depth interviews and narratives. The themes which emerged from the data gathered using these methods helped to make sense of classroom environments that were created in each country.

     

What does it mean to be an exemplary science teacher?

In order to provide a refreshing alternative to the majority of research reports, which malign science education and highlight its major problems and shortcomings, a series of case studies of exemplary practice was initiated to provide a focus on the successful and positive facets of schooling. The major data-collection approach was qualitative and involved 13 researchers in hundreds of hours of intensive classroom observation involving 20 exemplary teachers and a comparison group of nonexemplary teachers. A distinctive feature of the methodology was that the qualitative information was complemented by quantitative information obtained from the administration of questionnaires assessing student perceptions of classroom psychosocial environment. The major trends were that exemplary science teachers (1) used management strategies that facilitated sustained student engagement, (2) used strategies designed to increase student understanding of science, (3) utilized strategies that encouraged students to participate in learning activities, and (4) maintained a favorable classroom learning environment.