Design,validation, and use of an evaluation instrument for monitoring systemic reform

Over the past decade, state and national policymakers have promoted systemic reform as a way to achieve high‐quality science education for all students. However, few instruments are available to measure changes in key dimensions relevant to systemic reform such as teaching practices, student attitudes, or home and peer support. Furthermore, Rasch methods of analysis are needed to permit valid comparison of different cohorts of students during different years of a reform effort. This article describes the design, development, validation, and use of an instrument that measures student attitudes and several environment dimensions (standards‐based teaching, home support, and peer support) using a three‐step process that incorporated expert opinion, factor analysis, and item response theory. The instrument was validated with over 8,000 science and mathematics students, taught by more than 1,000 teachers in over 200 schools as part of a comprehensive assessment of the effectiveness of Ohio's systemic reform initiative. When the new four‐factor, 20‐item questionnaire was used to explore the relative influence of the class, home, and peer environment on student achievement and attitudes, findings were remarkably consistent across 3 years and different units and methods of analysis. All three environments accounted for unique variance in student attitudes, but only the environment of the class accounted for unique variance in student achievement. However, the class environment (standards‐based teaching practices) was the strongest independent predictor of both achievement and attitude, and appreciable amounts of the total variance in attitudes were common to the three environments. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 646–662, 2001     

Classroom Environment Instruments: Development,Validity and Applications

Few fields of educational research have such a rich diversity of valid, economical and widely-applicable assessment instruments as does the field of learning environments. This article describes nine major questionnaires for assessing student perceptions of classroom psychosocial environment (the Learning Environment Inventory, Classroom Environment Scale, Individualised Classroom Environment Questionnaire, My Class Inventory, College and University Classroom Environment Inventory, Questionnaire on Teacher Interaction, Science Laboratory Environment Inventory, Constructivist Learning Environment Survey and What Is Happening In This Class) and reviews the application of these instruments in 12 lines of past research (focusing on associations between outcomes and environment, evaluating educational innovation, differences between student and teacher perceptions, whether students achieve better in their preferred environment, teachers' use of learning environment perceptions in guiding improvements in classrooms, combining quantitative and qualitative methods, links between different educational environments, cross-national studies, the transition from primary to high school, and incorporating educational environment ideas into school psychology, teacher education and teacher assessment).     

Measuring Effectiveness of Health Education in a Web‐based Learning Environment: a preliminary report

Driven by claims of efficacy, flexibility and resource effectiveness, higher education is increasingly utilising the Web as an instructional tool. The claims for pedagogical effectiveness are often just that – claims — and appear not to have been proven in the reality of subject presentation and evaluation. Thus, it is necessary to examine assumptions regarding the benefits of Web‐based instruction in terms of effectiveness. This article discusses aspects of an investigation which examined and compared the effectiveness of HIV/AIDS related collaborative tutorial activities carried out in both a Web‐based learning environment and a face‐to‐face class situation within an undergraduate health education subject. Effectiveness of the pedagogical strategy and the different learning environments were measured in terms of observed learning outcomes and reported perceptions of the learners regarding their learning experience. Preliminary results based on measured learning outcomes related to the subject matter, HIV/AIDS, demonstrated that collaborative learning activities were significantly more effective in the Web‐based than in the class environment. Additionally, the vast majority of learners perceived the Web‐based environment to be as effective or more effective than the face‐to‐face, class environment in terms of facilitating their understanding of the issues explored in the subject.     

The Macro- and Micropolitics of Personnel Evaluation: A Framework

In this article, the authors explicate a conceptual framework for analyzing the politics of personnel evaluation in an educational context. Using several elements of their framework, they discuss the politics of teacher evaluation in California in relation to the types of personnel evaluation decisions, the actors, their access to these decisions, their sources and level of power, and the outcomes related to their decisions. The authors further portray how politics enters into personnel evaluation at the district level by describing an actual case.     

The Efficacy of Preschool Developmental Indicators as a Screen for Early Primary School-Based Literacy Interventions

Literacy success lays the foundation for children's later educational, health, and well-being outcomes. Thus, early identification of literacy need is vital. Using data from New Zealand's national preschool health screening program for fiscal years 2010/2011–2014/2015, demographic and health variables from 255,090 children aged 4 years were related to whether they received a literacy intervention in early primary school. Overall, 20,652 (8.1%) children received an intervention. Time-to-event analysis revealed that all considered variables were significantly related to literacy intervention (all p < .01), but the full model lacked reasonable predictive power for population screening purposes (Harrell's c-statistic = .624; 95% CI [.618, .629]). Including more direct literacy measures in the national screening program is likely needed for improvement.     

Negotiating textual talk: Conversation analysis,pedagogy and the organisation of online asynchronous discourse

This paper uses Conversation Analysis to investigate the ways in which participants in an online asynchronous postgraduate reading group managed and negotiated their contributions within the discussion. Using the conversation analytic concerns with sequential organisation, adjacency pairs and topicality, this article shows the analytic insights that this perspective can bring to the examination of written asynchronous discourse. The paper shows that in the section of the discussion analysed here, the discourse displayed remarkable similarities to the ways in which face-to-face conversation has been seen to operate in terms of the organisation of conversational turns, the application of specific interactional rights, the lineal development of topics of conversation and the structural use of question-answer turn pairs. The paper concludes by showing how this form of analysis can relate to the formation of reflexive pedagogy in which course design can be created to take account of such findings. It shows how a detailed understanding of how pedagogy is played out in interaction is fundamental for reflecting on the relationship between pedagogic aims and educational practice.     

The effects of mood,cognitive style,and cognitive ability on implicit learning

In an experiment with 109 undergraduates, we examined the effect of mood, cognitive style, and cognitive ability on implicit learning in the Artificial Grammar (AG) and Serial Reaction Time (SRT) tasks. Negative mood facilitated AG learning, but had no significant effect on SRT learning. Rational cognitive style predicted greater learning on both tasks, but this effect on SRT was mediated by cognitive ability. SRT, but not AG learning was significantly correlated with Math and English scores on the ACT. These findings confirm and contradict previous research. The association of negative mood and rational cognitive style with AG confirms that AG learning is facilitated by systematic, bottom-up processing. However, the lack of converging evidence for the SRT task suggests that the tasks involve different aspects of implicit processing. Theoretical explanations and suggestions for future research are discussed.