Examining the Precision of Cut Scores Within a Generalizability Theory Framework: A Closer Look at the Item Effect

An Angoff standard setting study generally yields judgments on a number of items by a number of judges (who may or may not be nested in panels). Variability associated with judges (and possibly panels) contributes error to the resulting cut score. The variability associated with items plays a more complicated role. To the extent that the mean item judgments directly reflect empirical item difficulties, the variability in Angoff judgments over items would not add error to the cut score, but to the extent that the mean item judgments do not correspond to the empirical item difficulties, variability in mean judgments over items would add error to the cut score. In this article, we present two generalizability-theory–based analyses of the proportion of the item variance that contributes to error in the cut score. For one approach, variance components are estimated on the probability (or proportion-correct) scale of the Angoff judgments, and for the other, the judgments are transferred to the theta scale of an item response theory model before estimating the variance components. The two analyses yield somewhat different results but both indicate that it is not appropriate to simply ignore the item variance component in estimating the error variance.     

Interdisciplinary Research: A Promising Approach to Investigate Elite Performance in Sports

While the call for interdisciplinary studies has been loud and clear over the last decade or so, the number of interdisciplinary publications in sport sciences is rather limited. One of the reasons for this scarcity is the complexity of the research process itself. Still, the combination and integration of information from different scientific disciplines seems to be important to better explain how elite performance comes about. In this article, we focus on this issue and discuss the advantages of interdisciplinary research for understanding how elite athletes are able to achieve such high-proficiency levels. The first section provides an overview of the studies that investigated elite performance in sports. In the second section, we focus on the issue of interdisciplinarity and illustrate how the ecological–dynamical approach and the concept of dimensional reduction can be used to explain how elite performers cope with the complex nature of sport skills. In the last section, we present a research outline and some practical considerations that can be helpful for researchers who aim to study sport skills from an interdisciplinary perspective.     

The Rating and Matching Item-Objective Alignment Methods

Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two methods and on the item and objective characteristics that influence judges' decisions. We randomly assigned judges to either rate item-objective links or match items to objectives while reviewing the 2004 Arizona high school mathematics standards and assessment. Across items we found moderate convergence between methods, and we detected apparent reasons for divergently scored items. We also found that judges relied on item and objective content and intellectual skill features to render decisions. Based on our evidence, we contend that a thorough alignment analysis would involve judges using both rating and matching, while focusing on both content and intellectual skill. The findings have important implications for states when examining the alignment between their standards and assessments.     

The role of teachers in facilitating situational interest in an active-learning classroom

The study sought to explore whether interactional teacher characteristics such as social congruence, subject-matter expertise, and cognitive congruence increase situational interest in students. Correlational and path analyses were conducted on a sample of 498 polytechnic students to assess potential differences in situational interest based on these three teacher characteristics and how well these characteristics predicted situational interest. Results revealed that students supported by teachers who scored high on these characteristics reported significantly higher levels of situational interest. Cognitive congruence was the most significant predictor of situational interest. Overall, the findings suggest that teachers play an influential role in increasing students’ situational interest in the active-learning classroom.