The Impact of Performance Level Misclassification on the Accuracy and Precision of Percent at Performance Level Measures

No Child Left Behind (NCLB) performance mandates, embedded within state accountability systems, focus school AYP (adequate yearly progress) compliance squarely on the percentage of students at or above proficient. The singular importance of this quantity for decision-making purposes has initiated extensive research into percent proficient as a measure of school quality. In particular, technical discussions have scrutinized the impact of sampling, measurement, and other sources of error on percent proficient statistics. In this article, we challenge the received orthodoxy that measurement error associated with individual students' scores is inconsequential for aggregate percent proficient statistics. Synthesizing current classification accuracy research with techniques from randomized response designs, we establish results which specify the extent to which measurement error—manifest as performance level misclassifications—produces bias and increases error variability for percent at performance level statistics. The results have direct relevance for the design of coherent and fair accountability systems based upon assessment outcomes.     

Introducing school-based assessment as part of junior cycle reform in Ireland: a bridge too far?

Assessments and examinations play a critical role in certifying student achievement in secondary education. Prompted by concerns about the negative effects of examinations on curriculum, teaching and learning, elements of School-Based Assessment (SBA) have been introduced into certification systems, sometimes modelled on practice in other jurisdictions. This case study investigates factors influencing efforts to introduce SBA in lower secondary education in the Republic of Ireland and reactions from the main stakeholders. Policymakers’ perspectives were informed by national consultations, results of international assessments, trends towards skills-based curricula and practices in relation to SBA as part of high-stakes assessment internationally. Despite broad enthusiasm for the reforms from most stakeholders, teachers remained opposed. A series of compromise proposals shifted the reforms far from their intended nature, leaving in place a dual system of assessment that incorporates continued centralised examining by the state along with some non-certified SBA by teachers. The efficacy of this solution in relation to the original aims of the reform remains to be seen. The analysis explores relevant substantive and methodological issues. The complex interplay between international, national and very local influences on policy implementation is highlighted, suggesting the need for due diligence in anticipating and managing stakeholder responses to reform initiatives. Readers’ attention is also drawn to the intricacy of undertaking qualitative case study inquiry and the need for awareness in relation to possible alternative interpretations of data.     

Understanding the nature of school partnerships with business in delivery of vocational programmes in schools in Australia

This article seeks to provide a school perspective on the nature and quality of the partnerships which schools form with businesses in order to deliver work placements and workplace learning in Australia. It found that the ability of schools to engage with external partners depended on the ability of school leaders to define and communicate the role of VET within the school and its broader community. This dependence on individuals and leadership is vulnerable to changes in key personnel and the informality of some of the processes and relationships can lead to problems in monitoring, evaluating and replicating programmes. Our study shows that a balance is required between carefully documented processes and the flexibility required to operate programmes successfully. The study also noted the tension between the perceived needs of the school and those of industry. A successful partnership necessarily requires school flexibility – in the decisions as to what programmes should be offered and how work placements and timetabling should be organised.     

The effects of caffeine ingestion on performance time, speed and power during a laboratory-based 1 km cycling time-trial

There is little published data in relation to the effects of caffeine upon cycling performance, speed and power in trained cyclists, especially during cycling of approximately 60 s duration. To address this, eight trained cyclists performed a 1 km time-trial on an electronically braked cycle ergometer under three conditions: after ingestion of 5 mg x kg-1 caffeine, after ingestion of a placebo, or a control condition. The three time-trials were performed in a randomized order and performance time, mean speed, mean power and peak power were determined. Caffeine ingestion resulted in improved performance time (caffeine vs. placebo vs. control: 71.1 +/- 2.0 vs. 73.4 +/- 2.3 vs. 73.3 +/- 2.7 s; P = 0.02; mean +/- s). This change represented a 3.1% (95% confidence interval: 0.7-5.6) improvement compared with the placebo condition. Mean speed was also higher in the caffeine than placebo and control conditions (caffeine vs. placebo vs. control: 50.7 +/- 1.4 vs. 49.1 +/- 1.5 vs. 49.2 +/- 1.7 km x h-1; P = 0.0005). Mean power increased after caffeine ingestion (caffeine vs. placebo vs. control: 523 +/- 43 vs. 505 +/- 46 vs. 504 +/- 38 W; P = 0.007). Peak power also increased from 864 +/- 107 W (placebo) and 830 +/- 87 W (control) to 940 +/- 83 W after caffeine ingestion (P = 0.027). These results provide support for previous research that found improved performance after caffeine ingestion during short-duration high-intensity exercise. The magnitude of the improvements observed in our study could be due to our use of sport-specific ergometry, a tablet form and trained participants.     

Bring your own device – a snapshot of two Australian primary schools

Background: The use of 1:1 and Bring Your Own Device strategies in schools is in its infancy and little is known about how mobile devices such as tablets are being used to support educational practice.

Purpose: In this article, two suburban primary schools in Sydney, Australia were focused on with an aim to understand how mobile device strategies were developed and implemented and how the devices were being used in the schools.

Design and method: This qualitative study uses a case study method. It draws upon questionnaires, interviews and classroom observations, and builds upon previous research in English and Australian schools.

Findings: Results of the research indicate that the devices have only recently been incorporated into the school and suggest that their usage has been generally embraced by both school staff and parents. Key issues highlighted by these two schools included the importance of the school’s vision and uncertainty about the differences between models of provision. Participant responses also referenced some positive impacts on classroom practice, which amplify constructivist pedagogy: there were examples of device use extending student learning by supporting peer assessment, collaboration, research skills and projects.     

Wechsler scale performance patterns of American Indian children

Using WISC, WISC-R, and WPPSI subtest scores recategorized according to the Bannatyne scheme, the study uncovered evidence for the existence of an Indian Wechsler Scale performance pattern that is different from that found in normal and learning disabled groups. The Indian pattern finds spatial abilities more well developed than sequencing skills, which are superior to conceptual and acquired knowledge performances. The sample was composed of 142 Indian children, primarily Ojibwa (Chippewa). Supporting hypotheses addressed differences between groups of traditional Indian children and those who are more Anglo acculturated. The traditional children evidenced the Indian pattern of recategorized Wechsler subtest performance, while the more acculturated groups did not. Discussion of results centers on factors that may be related to the Indian pattern, such as cultural heritage, otitis media, and the school curriculum.