Managing What We Can Measure: Quantifying the Susceptibility of Automated Scoring Systems to Gaming Behavior

As methods for automated scoring of constructed‐response items become more widely adopted in state assessments, and are used in more consequential operational configurations, it is critical that their susceptibility to gaming behavior be investigated and managed. This article provides a review of research relevant to how construct‐irrelevant response behavior may affect automated constructed‐response scoring, and aims to address a gap in that literature: the need to assess the degree of risk before operational launch. A general framework is proposed for evaluating susceptibility to gaming, and an initial empirical demonstration is presented using the open‐source short‐answer scoring engines from the Automated Student Assessment Prize (ASAP) Challenge.     

Nonscience majors learning science: A theoretical model of motivation

A theoretical model of nonscience majors' motivation to learn science was tested by surveying 369 students in a large‐enrollment college science course that satisfies a core curriculum requirement. Based on a social‐cognitive framework, motivation to learn science was conceptualized as having both cognitive and affective influences that foster science achievement. Structural equation modeling was used to examine the hypothesized relationships among the variables. The students' motivation, as measured by the Science Motivation Questionnaire (SMQ), had a strong direct influence on their achievement, as measured by their science grade point average. The students' motivation was influenced by their belief in the relevance of science to their careers. This belief was slightly stronger in women than men. Essays by the students and interviews with them provided insight into their motivation. The model suggests that instructors should strategically connect science concepts to the careers of nonscience majors through such means as case studies to increase motivation and achievement. © 2006 Wiley Periodicals, Inc. J Res Sci Teach 44: 1088–1107, 2007     

Opening classroom interaction: the importance of feedback

Efforts to encourage ‘interactive practice’ in the National Literacy (NLS) and Numeracy (NNS) Strategies in the UK, have led to an emphasis on teacher questions. Recent research into classroom interactions, however, indicate that the pattern of interaction remains largely unchanged since the introduction of these strategies in terms of the type and amount of questions teachers are asking, and the opportunities for extended pupil participation. This article uses evidence gathered from a large‐scale research project examining classroom interactions during literacy and numeracy lessons, and the researchers' critical reflections upon this process, to examine conceptions of interactive pedagogy. It is argued that in order to ‘open’ classroom interaction, emphasis should be less on the questions teachers ask, and more on the manner with which teachers react to pupils' responses to questions. Episodes of classroom interaction from video recorded literacy and numeracy lessons taken as part of the study are used to support this argument. They present evidence of teacher behaviours in reaction to pupils' responses which succeed in facilitating a more interactive learning environment. The implication that such behaviour will contribute towards a model of effective interactive practice is also discussed.     

IMPROVING OUTCOMES IN MATHEMATICS IN NEW ZEALAND: A DYNAMIC APPROACH TO THE POLICY PROCESS

Intervention at scale with the aim of improving student participation, engagement and outcomes in mathematics education is a challenge for educational policy makers and reformers. This article argues that an iterative annual cycle of policy formulation, implementation and evaluation enabled ongoing adjustments to the strategic focus, the professional development model and the system infrastructure as the New Zealand Numeracy Development Project was taken to scale. The analysis draws on the project’s evaluation data over a 6-year period to demonstrate how adjustments were made over time to the pedagogical tools and to the professional development processes. The ongoing development of knowledge supported the management of strategic risks in taking the project to scale: the ongoing appropriation of adequate levels of resourcing to support the school-based professional development model and the availability of system-wide expertise for effective implementation. The analysis suggests that conceptualising implementation as an interdependent and interrelated component of an iterative policy process and as an opportunity for knowledge building ensured a continuing focus on student outcomes. The dynamic approach to the policy process appeared central to building this intervention’s effectiveness and feasibility at scale.     

Anti-bullying procedures for schools in Ireland: principals’ responses and perceptions

The role of the principal is imperative in creating a school climate that nurtures differences and equality at the same time. Policy and legislation in Ireland dictates the responsibility of school principals in preventing and intervening in issues relating to bullying. The current research investigates the extent to which principals consider the impact of bullying on their students and the strategies they take to combat it in their schools. We consider how principals respond to bullying and what supports they feel are missing for them to adhere to relevant policy. A sample of 918 principals from a range of schools across Ireland completed an online survey. The results demonstrated some positive actions by principals such as the provision of anti-bullying policies. However, they also document areas that need more action such as increased access to counsellors and appointing a specific staff member to deal with and tackle bullying in each school. Guidance from the Department of Education and Skills in Ireland on the development and provision of an anti-bullying programme is called for. Results are discussed in light of the current Anti-Bullying Procedures established by the Department of Education and Skills and the Action Plan for Bullying 2013.     

Teaching Instructional Design: An Apprenticeship Model

Several cognitive psychologists have written about the importance of placing instruction within “authentic” contexts that mirror real-life situations. They argue that knowledge learned in academic settings does not necessarily transfer to non-academic settings. Whether preparing performance technologists or instructional designers, educators must strive to create meaningful problem-solving contexts that enable students to define, and subsequently solve, real-world problems. In an attempt to address this issue we have modified the way we teach instructional design. This paper discusses a cognitive apprenticeship approach to teaching design, which incorporates elements of modeling, coaching, reflection, articulation, and exploration. We describe how these features are embedded within three phases (orientation, situated training/learning, and exploration) of an introductory instructional design course designed to move our novice designers along a continuum of expertise as they develop and refine their own professional design skills. Although the apprenticeship model described here specifically addresses concerns within the context of preparing instructional designers, we believe that this model can be adapted to address similar issues in the education of performance technologists.     

Predicting reading achievement for Kindergarten boys and girls

This study assessed the validity of the Kindergarten Teacher Rating Scale (KTRS) in predicting reading achievement for male and female students. The KTRS was a significant predictor of reading achievement for both boys and girls; differential predictive validity for boys and girls was not found. The KTRS explained about 30% of the variance in reading achievement both at the end of the 1st grade and the beginning of 2nd grade. The proportion of variance in reading achievement explained by variance in KTRS scores was significantly greater than the proportion of variance in reading achievement explained by variance in reading readiness scores. There were no significant differences in the mean KTRS scores for male and female students.