On the evaluation of systemic science education reform: Searching for instructional sensitivity

We propose a multilevel‐multifaceted approach to evaluating the impact of education reform on student achievement that would be sensitive to context and small treatment effects. The approach uses different assessments based on their proximity to the enacted curriculum. Immediate assessments are artifacts (students' products) from the enactment of the curriculum; close assessments parallel the content and activities of the unit/curriculum; proximal assessments tap knowledge and skills relevant to the curriculum, but topics can be different; and distal assessments reflect state/national standards in a particular knowledge domain. To provide evidence about the sensitivity of the multilevel approach in ascertaining outcomes of hands‐on science programs we administered close, proximal, and distal performance assessments to evaluate the impact of instruction based on two Full Option Science System units—Variables, and Mixtures and Solutions—in a Bay Area school district. Results indicated that close assessments were more sensitive to the changes in students' pre‐ to post‐test performance than proximal assessments. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 369–393, 2002     

Revolution is in the Everyday: Métissage as place of education

Positing that place has a pedagogy that can be harnessed for educative means and ends or left to chance and that partnering place and its pedagogy with teaching magnifies the influence of place and teaching, I conceptualize métissage as place of education. Because few scholars in education have written about métissage and even fewer have conceptualized it as place of education, I begin by defining métissage illustrating the multi-layered definition with literary, historical, and autobiographical examples. I then examine métissage as place of education with its own pedagogy whose influence increases when partnered with teaching, portray this phenomenon at play in one urban school, demonstrate ways for teachers to subvert negative school places by rejecting, challenging, and countering them and by using the everyday as a revolutionary tool, and pose questions about such a pedagogy's potential dangers.     

The surprising viability of a simple alternate estimation procedure for construction of large‐scale structural equation measurement models

As part of developing a comprehensive strategy for structural equation model building and assessment, a large‐scale Monte Carlo study of 7,200 covariance matrices sampled from 36 population models was conducted. This study compared maximum likelihood with the much simpler centroid method for the confirmatory factor analysis of multiple‐indicator measurement models. Surprisingly, the contribution of maximum likelihood to model analysis is limited to formal evaluation of the model. No statistically discernible differences were obtained for the bias, standard errors, or mean squared error (MSE) of the estimated factor correlations, and empirically obtained maximum likelihood standard errors for the pattern coefficients were only slightly smaller than their centroid counterparts. Further supporting the recommendations of Anderson and Gerbing (1982), the considerably faster centroid method may have a useful role in the analysis of these models, particularly for the analysis of large models with 50 or more input variables. These results encourage the further development of a comprehensive research paradigm that exploits the relative strengths of both centroid and maximum likelihood as complementary estimation procedures along an integrated exploratory‐confirmatory continuum of model specification, revision, and formal evaluation.     

Predicting response to early reading intervention from verbal IQ, reading-related language abilities, attention ratings, and verbal IQ-word reading discrepancy: failure to validate discrepancy method

Additional analyses of a previously published study addressed three questions about growth in word reading during early reading intervention: (1) How well do Verbal IQ, reading-related language abilities (phonological, rapid naming, and orthographic), and attention ratings predict reading growth? (2) How well do language deficits predict reading growth? and (3) How well does Verbal IQ-word reading discrepancy predict reading growth? Univariate analyses showed that Verbal IQ, phonological skills, orthographic skills, rapid automatized naming (RAN), and attention ratings predicted the response to early intervention, but multivariate analyses based on a combination of predictors for real-word reading and pseudoword reading showed that Verbal IQ was not the best unique predictor. Students with double or triple deficits in language skills (RAN, phonological, and orthographic processing) responded more slowly to early intervention than students without language deficits. Verbal IQ-word reading discrepancy did not predict the response to early intervention in reading. Overall results supported the use of reading-related language and attention measures rather than IQ-achievement discrepancy in identifying candidates for early reading intervention.     

ENGAGING WITH FACULTY TO DEVELOP,IMPLEMENT, AND PILOT ELECTRONIC PERFORMANCE ASSESSMENTS OF STUDENT TEACHERS USING MOBILE DEVICES

This article discusses the development and implementation of a technology-supported student teacher performance assessment that supports integration with a larger electronic assessment system. The authors spearheaded a multidisciplinary team to develop a comprehensive performance assessment based on the Pathwise framework. The team collaborated with a third party vendor to implement and field test these assessments supported with mobile technologies in the field. Twenty supervisors completed the field test using 10 iPAQ PDAs and 10 laptops. Eighteen participated in a posttest survey and eight in a focus group debriefing session. Supervisors were overwhelmingly positive about the experience, the value of the updated assessments, and the ongoing plans to support field observations with mobile technologies. However, issues related to training, support, and trust loom largely for successful adoption, diffusion, and successful use of technology-supported assessments.     

Issues in secondary teacher preparation

As teacher educators who have had extensive experience working in the field‐based secondary programs of two universities, the authors recognize the benefits, as well as the challenges, of forming partnerships with secondary educators to design and deliver collaboratively the teacher preparation program. In this article, experiences are shared and some of the major issues related to secondary teacher preparation that have been identified and contemplated are discussed.     

Work placement experience: should I stay or should I go?

The opportunity to experience work placements that complement taught and practical courses in higher education has become a central strand of many undergraduate degree programmes. While there is tacit agreement that such placements are a good thing, in recent years the numbers of students opting for work placements has been declining. This paper uses a mixed method research design to probe the learning outcomes, attitudes and perceptions of undergraduate students who choose not to go on a work placement. Findings highlight some areas of concern that could be considered by institutions of higher education working to enhance good practice in students’ work placement experiences.