IRT Model Misspecification and Measurement of Growth in Vertical Scaling

Functional form misfit is frequently a concern in item response theory (IRT), although the practical implications of misfit are often difficult to evaluate. In this article, we illustrate how seemingly negligible amounts of functional form misfit, when systematic, can be associated with significant distortions of the score metric in vertical scaling contexts. Our analysis uses two‐ and three‐parameter versions of Samejima's logistic positive exponent model (LPE) as a data generating model. Consistent with prior work, we find LPEs generally provide a better comparative fit to real item response data than traditional IRT models (2PL, 3PL). Further, our simulation results illustrate how 2PL‐ or 3PL‐based vertical scaling in the presence of LPE‐induced misspecification leads to an artificial growth deceleration across grades, consistent with that commonly seen in vertical scaling studies. The results raise further concerns about the use of standard IRT models in measuring growth, even apart from the frequently cited concerns of construct shift/multidimensionality across grades.     

Straw Godzilla: Engaging the academy and research ethics in artistic research projects

Abstract

In Australia, the university ethics approval process is guided by the National Statement on Ethical Conduct in Human Research. The National Statement does not provide a hurdle to be overcome or avoided, nor is it a Godzilla-like monster that must be slain for truth to survive. Rather the National Statement provides an affirmation of an abiding respect for all life and a mechanism for beginning the intelligent questioning, theorization and contextualization for a work of art. Focusing specifically on the emergent discipline of artistic research, this article addresses the question of how the supervisory process may take on the ‘spirit’ of the National Statement to engender genuine ethical debate, rather than merely focus on the instrumental obstacles that seem to get in the way of the research. It specifically addresses the emergent field of artistic research and the concomitant resistance to ethical regulation of artistic research, in which bureaucratic instrumentalism and compliance or censorship are considered to potentially emasculate the vitality of an art work and the ability of art to serve as a truthsayer or agent provocateur. In this, the attitude of the supervisor toward research ethics in artistic research is considered a key factor in determining their students attitudes towards the ethics process. This article will present several initiatives undertaken with the intent of more fully engaging supervisors and students with research ethics.     

Investigation of a Nonparametric Procedure for Assessing Goodness-of-Fit in Item Response Theory

There have been many studies of the comparability of computer-administered and paper-administered tests. Not surprisingly (given the variety of measurement and statistical sampling issues that can affect any one study) the results of such studies have not always been consistent. Moreover, the quality of computer-based test administration systems has changed considerably over recent years, as has the computer-experience of students. This study synthesizes the results of 81 studies performed between 1997 and 2007. The estimated effect size across all studies was very small (–.01 weighted, .00 unweighted). Meta-analytic methods were used to ascertain whether grade (elementary, middle, or high school) or subject (English Language Arts, Mathematics, Reading, Science, or Social Studies) had an impact on comparability. Grade appeared to have no affect on comparability. Subject did appear to affect comparability, with computer administration appearing to provide a small advantage for English Language Arts and Social Studies test (effect sizes of .11 and .15, respectively), and paper administration appearing to provide a small advantage for Mathematics tests (effect size of??.06).     

Modified Multiple-Choice Items for Alternate Assessments: Reliability,Difficulty, and Differential Boost

Federal policy on alternate assessment based on modified academic achievement standards (AA-MAS) inspired this research. Specifically, an experimental study was conducted to determine whether tests composed of modified items would have the same level of reliability as tests composed of original items, and whether these modified items helped reduce the performance gap between AA-MAS eligible and ineligible students. Three groups of eighth-grade students (N?=?755) defined by eligibility and disability status took original and modified versions of reading and mathematics tests. In a third condition, the students were provided limited reading support along with the modified items. Changes in reliability across groups and conditions for both the reading and mathematics tests were determined to be minimal. Mean item difficulties within the Rasch model were shown to decrease more for students who would be eligible for the AA-MAS than for non-eligible groups, revealing evidence of differential boost. Exploratory analyses indicated that shortening the question stem may be a highly effective modification, and that adding graphics to reading items may be a poor modification.     

Forthcoming events

The aims of this study were to quantify the effects of factors such as mode of exercise, body composition and training on the relationship between heart rate and physical activity energy expenditure (measured in kJ?·?min^?1) and to develop prediction equations for energy expenditure from heart rate. Regularly exercising individuals (n = 115; age 18?–?45 years, body mass 47?–?120?kg) underwent a test for maximal oxygen uptake ([Vdot]O_2max test), using incremental protocols on either a cycle ergometer or treadmill; [Vdot]O_2max ranged from 27 to 81?ml?·?kg^?1?·?min^?1. The participants then completed three steady-state exercise stages on either the treadmill (10?min) or the cycle ergometer (15?min) at 35%, 62% and 80% of [Vdot]O_2max, corresponding to 57%, 77% and 90% of maximal heart rate. Heart rate and respiratory exchange ratio data were collected during each stage. A mixed-model analysis identified gender, heart rate, weight, [Vdot]_2max and age as factors that best predicted the relationship between heart rate and energy expenditure. The model (with the highest likelihood ratio) was used to estimate energy expenditure. The correlation coefficient (r) between the measured and estimated energy expenditure was 0.913. The model therefore accounted for 83.3% (R ²) of the variance in energy expenditure in this sample. Because a measure of fitness, such as [Vdot]O_2max, is not always available, a model without [Vdot]O_2max included was also fitted. The correlation coefficient between the measured energy expenditure and estimates from the mixed model without [Vdot]O_2max was 0.857. It follows that the model without a fitness measure accounted for 73.4% of the variance in energy expenditure in this sample. Based on these results, we conclude that it is possible to estimate physical activity energy expenditure from heart rate in a group of individuals with a great deal of accuracy, after adjusting for age, gender, body mass and fitness.     

Evaluation of a lower-body compression garment

The aims of this study were to determine how custom-fit compression shorts affect athletic performance and to examine the mechanical properties of the shorts. Ten male and 10 female track athletes on a university's nationally competitive track team, specializing in sprint or jump events, participated in the study. Testing utilized the compression shorts with loose-fitting gym shorts as the control garment. Several significant effects were revealed for the custom-fit compressive garment. Although 60 m sprint time was not affected, hip flexion angle was reduced. Skin temperature increased more and at a faster rate during a warm-up protocol. Muscle oscillation was decreased during vertical jump landing. Countermovement vertical jump height increased when the participants were wearing the custom-fit compression garment. In materials testing, the elasticity of the compressive garment provides increased flexion and extension torque at the end range of extension and flexion, respectively, and may assist the hamstrings in controlling the leg at the end of the swing phase in sprinting. The compressive garment significantly reduced impact force by 27% compared with American football pants alone. Through various mechanisms, these findings may translate into an effect on athletic performance and a reduction in injuries.     

A Nested Logit Approach for Investigating Distractors as Causes of Differential Item Functioning

In multiple‐choice items, differential item functioning (DIF) in the correct response may or may not be caused by differentially functioning distractors. Identifying distractors as causes of DIF can provide valuable information for potential item revision or the design of new test items. In this paper, we examine a two‐step approach based on application of a nested logit model for this purpose. The approach separates testing of differential distractor functioning (DDF) from DIF, thus allowing for clearer evaluations of where distractors may be responsible for DIF. The approach is contrasted against competing methods and evaluated in simulation and real data analyses.