UNDERSTANDING TEST‐TYPE ASSIGNMENT: WHY DO SPECIAL EDUCATORS MAKE UNEXPECTED TEST‐TYPE ASSIGNMENTS?

We interviewed special educators (a) whose students with disabilities (SWDs) were proficient on the 2008 general education assessment but were assigned to the 2009 alternate assessment based on modified achievement standards (AA‐MAS), and (b) whose students with mild disabilities took the 2008 alternate assessment based on alternate achievement standards (AA‐AAS) and then the 2009 AA‐MAS. We explored teachers’ rationales for test‐type assignment, student characteristics, and quality of instruction to determine the test‐type decisions’ appropriateness. All teachers based their decisions on combinations of factors in the guidelines plus subjective and noninstructional factors. Findings raised concerns about the subjectivity of the assessment assignment system and the inappropriate grade‐level instruction for SWDs. Future research, implications of these findings, and limitations are discussed.     

Considering the Use of General and Modified Assessment Items in Computerized Adaptive Testing

This article used several data sets from a large-scale state testing program to examine the feasibility of combining general and modified assessment items in computerized adaptive testing (CAT) for different groups of students. Results suggested that several of the assumptions made when employing this type of mixed-item CAT may not be met for students with disabilities that have typically taken alternate assessments based on modified achievement standards (AA-MAS). A simulation study indicated that the abilities of AA-MAS students can be underestimated or overestimated by the mixed-item CAT, depending on students’ location on the underlying ability scale. These findings held across grade levels and test lengths. The mixed-item CAT appeared to function well for non-AA-MAS students.     

Sensitivity of Cross-State Assessment Item Difficulty to Differences in State Curricular Content Standards

It has long been argued that U.S. states’ differential performance on nationwide assessments may reflect differences in students’ opportunity to learn the tested content that is primarily due to variation in curricular content standards, rather than in instructional quality or educational investment. To quantify the effect of differences in states’ intended curricular goals on test item performance in the mid-to-late 2000s, we use fractional logit regression of state-specific mathematics item difficulty values on a measure of content emphasis in state elementary school mathematics curricular standards documents. Finding weak but positive associations between content emphasis in state standards and proportion-correct item difficulty, we conclude that variations in states’ intended curriculum content, alone, appear to have had limited influence on cross-state mathematics test item performance during the time frame examined. Implications for cross-state assessment are discussed.     

Opportunities and Options for Facilitating and Evaluating Access to the General Curriculum for Students With Disabilities

Federal regulations concerning the development and implementation of alternate assessments based on modified achievement standards include a set of safeguards intended to ensure that eligible students have access to grade-level general curriculum. These regulations concerning curricular access and opportunity to learn for students with disabilities may prove difficult for educational researchers and policymakers to operationalize and evaluate. This article provides a historical and policy context for efforts to ensure curricular access. In addition, this article reviews research on potential indicators of (or ways of measuring) access to the general curriculum and opportunity to learn. Finally, best practices for facilitating and evaluating access to the general curriculum are discussed.     

Using Test Scores From Students With Disabilities in Teacher Evaluation

Much of the recent focus of educational policymakers has been on improving the measurement of teacher effectiveness. Linking student growth to teacher effects has been a large part of reform efforts. To date, neither researchers nor practitioners have arrived at a consensus on how to treat test scores from students with disabilities in growth‐based teacher effectiveness indicators, despite the fact that these students make up approximately 13% of the K‐12 student population. In this study, we leverage longitudinal data from the population of teachers in one state to explore practical questions related to including general assessment scores from students with disabilities in teacher evaluation. Findings suggest that including test scores from students with disabilities allows more teachers to be evaluated and does not substantially affect teachers’ scores. Moreover, including disability‐related covariates can allow for fairer evaluations for teachers with many students with disabilities in their class.     

Building Validity Evidence for Scores on a State-Wide Alternate Assessment: A Contrasting Groups, Multimethod Approach

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54 students who had disabilities, but were not officially eligible for the alternate assessment also was assessed. Evidence to support the validity of the inferences about IAA scores was mixed, yet promising. Specifically, the relationship among the reading, language arts, and mathematics achievement level ratings on the IAA and the concurrent scores on the ACES-Academic Skills scales for the eligible students varied across grade clusters, but in general were moderate. These findings provided evidence that IAA scales measure skills indicative of the state's content standards. This point was further reinforced by moderate to high correlations between the IAA and Idaho State Achievement Test (ISAT) for the not eligible students. Additional evidence concerning the valid use of the IAA was provided by logistic regression results that the scores do an excellent job of differentiating students who were eligible from those not eligible to participate in an alternate assessment. The collective evidence for the validity of the IAA scores suggests it is a promising assessment for NCLB accountability of students with significant disabilities. The methods of establishing this evidence have the potential to advance validation efforts of other states' alternate assessments.     

Modified Multiple-Choice Items for Alternate Assessments: Reliability,Difficulty, and Differential Boost

Federal policy on alternate assessment based on modified academic achievement standards (AA-MAS) inspired this research. Specifically, an experimental study was conducted to determine whether tests composed of modified items would have the same level of reliability as tests composed of original items, and whether these modified items helped reduce the performance gap between AA-MAS eligible and ineligible students. Three groups of eighth-grade students (N?=?755) defined by eligibility and disability status took original and modified versions of reading and mathematics tests. In a third condition, the students were provided limited reading support along with the modified items. Changes in reliability across groups and conditions for both the reading and mathematics tests were determined to be minimal. Mean item difficulties within the Rasch model were shown to decrease more for students who would be eligible for the AA-MAS than for non-eligible groups, revealing evidence of differential boost. Exploratory analyses indicated that shortening the question stem may be a highly effective modification, and that adding graphics to reading items may be a poor modification.     

Trends in Performance and Growth by Students With and Without Disabilities on Five State Summative Assessments

Indicators of student academic growth are desired in state accountability systems in order to approximate student learning over time and attribute observed growth to schooling inputs. Through an extant analysis of five states’ assessment data, this study offers evidence about whether longitudinal match rates and measures of growth differ at the state level for students with disabilities, relative to students without disabilities. There were three main findings: 1) In states in which a modified assessment was offered, students with disabilities were more likely to have missing prior year scores, and consequently missing growth scores; 2) Low scoring students, many of whom had a disability, were more likely to have missing prior scores on the state general assessment, and consequently missing growth scores; 3) Students with and without disabilities showed similar growth using transition and gain score definitions of growth, but students with disabilities had lower growth when estimated via a regression-based model. Measurement and policy considerations are discussed.     

Relationship between IEP Characteristics and Test Scores on an Alternate Assessment for Students with Significant Cognitive Disabilities

Although many studies have examined the alignment of state standards with large-scale assessment and instruction, fewer have attended to alignment concerning alternate assessments for students with significant disabilities. This study was designed to (1) compare expectations in one state's alternate assessment (AA) with curricular priorities reflected in students' Individualized Education Programs (IEPs), and (2) consider the effect of this relationship on AA scores. The study was conducted in a state whose AA consisted of standardized performance tasks measuring reading comprehension (RC) and number systems (NUM). Archival data, including AA scores and IEPs for 292 students, were analyzed. The average IEP emphasized speaking, writing, and measurement, and objectives primarily required simple recall skills. Half of IEPs contained no objectives aligned with RC. More than one third of IEPs did not align with NUM. Assessment–IEP alignment had a moderate effect on Reading test score, but not Math test score. Recommendations are made for future investigations of the taught curriculum for this population, and professional development to improve alignment of instruction with assessments.     

An Examination of the Relationship between Computation,Problem Solving,and Reading

The purpose of this study is to evaluate the relationship of mathematics calculation rate (curriculum-based measurement of mathematics; CBM-M), reading rate (curriculum-based measurement of reading; CBM-R), and mathematics application and problem solving skills (mathematics screener) among students at four levels of proficiency on a statewide test. It was hypothesized that CBM-M provides insufficient information to make good screening decisions and that other measures with content more similar to that of large-scale tests of mathematics would function to improve screening. One hundred and seventy students in third grade from a rural elementary school in the Midwestern United States participated. Structural equation modeling was used to evaluate direct, mediator, and latent growth models. In general, CBM-R mediated the relationship between the mathematics ability screener and passing the state assessment, while CBM-M did not have any significant paths within these models. Results are discussed in terms of the utility of CBM-M and CBM-R procedures in screening for success on state test performance in mathematics.     

Dynamic Assessment for Identification of Twice-Exceptional Learners Exhibiting Mathematical Giftedness and Specific Learning Disabilities

The purpose of this research was to investigate the utility of dynamic assessment (DA) in identifying a group of twice-exceptional students who exhibited mathematical giftedness and specific learning disabilities (MG/LDs). A quantitative method was applied by a multidisciplinary team to identify 30 students (16 girls and 14 boys) in the fifth and sixth grades, aged 10 years to 11 years 11 months, at three public elementary schools in Amman, Jordan. The findings of the dynamic mathematics assessment showed (a) a high variance in performance between the pre- and posttests and (b) a lack of relation with conventional static assessment (e.g., mathematics achievement tests and the arithmetic subtest in the Wechsler Intelligence Scale for Children, Jordanian version). DA was found to be beneficial for elementary school boys and girls and to affect their mathematics achievement to the same extent. No group differences were observed between low and high achievers in their learning potential as revealed by their performance on DA.     

Effects of Item Modifications on Test Accessibility for Persistently Low-Performing Students with Disabilities

ABSTRACT

Construct-irrelevant cognitive complexity of some items in the statewide grade-level assessments may impose performance barriers for students with disabilities who are ineligible for alternate assessments based on alternate achievement standards. This has spurred research into whether items can be modified to reduce complexity without affecting item construct. This study uses a generalized linear mixed modeling analysis to investigate the effects of item modifications on improving test accessibility by reducing construct-irrelevant cognitive barriers for persistently low-performing fifth-grade students with cognitive disabilities. The results showed item scaffolding was an effective modification for both mathematics and reading. Other modifications, such as bolding/underlining of key words, hindered test performance for low-performing students. We discuss the findings’ potential impact on test development with universal design.     

Score Comparability of a State Reading Assessment Across Selected Groups of Students With Disabilities

This study investigated the factorial invariance of scores from a 7th-grade state reading assessment across general education students and selected groups of students with disabilities. Confirmatory factor analysis was used to assess the fit of a 2-factor model to each of the 4 groups. In addition to overall fit of this model, 5 levels of constraint, including equal factor loadings, intercepts, error variances, factor variances, and factor covariances, were investigated. Invariance across the factor loadings and intercepts was supported across the groups of students with disabilities and general education students. Invariance for these groups was not supported for the error variances. For the students with mental retardation, the lack of fit of the 2-factor model and the observed score results suggested a mismatch between the difficulty level of this test and the ability level of these students. Although the results generally supported the score comparability of the reading assessment across these groups, further research is needed into the nature of the larger error variances for the student with disabilities groups and into accommodations and modifications for the students with mental retardation.     

Examining calculator use among students with and without disabilities educated with different mathematical curricula

This study assessed if students with and without disabilities used calculators (fourfunction, scientific, or graphing) to solve mathematics assessment problems and whether using calculators improved their performance. Participants were sixth and seventh-grade students educated with either National Science Foundation (NSF)-funded or traditional mathematics curriculum materials. Students solved multiple choice and open-ended problems based on items from the State’s released previous assessments. A linear mixed model was conducted for each grade to analyze the factors impacting students’ self-reported calculator use. Chi Square tests were also performed on both grade’s data to determine the relationship between using a calculator and correctly solving problems. Results suggested only time as a main factor impacting calculator use and students who self-reported using a calculator were more likely to answer questions correctly. The results have implications for practice given the controversy over calculator use by students both with and without disabilities.     

Culturally Relevant Assessment and Support of Grade 1 Students with Mild Disabilities in Tanzania: An Exploratory Study

ABSTRACT

In Tanzania, many people are not aware of the concept of learning disabilities. Parents and teachers often do not realise that children who do poorly in school and do not have visible disabilities, could still have a disability that is causing their poor school performance. This paper presents the results of a research and service project to identify and support students with mild disabilities in an inclusive school in Tanzania during the three-year pilot process. An international NGO implemented a culturally-relevant, curriculum-based assessment for screening grade 1 students with potential delays or disabilities and provided Tier 2 interventions through small group lessons. In this three-year study, 413 students were screened, 108 students received Tier 2 services and six students received Tier 3 services. Students in Tier 2 received small group instruction in mathematics and literacy as a pull-out programme. Students in Tier 3 attended a special school for children with disabilities. Results showed that students who received Tier 2 intervention made statistically significant gains in their assessment scores after six and twelve months of services. This study demonstrated one successful assessment and intervention model in a typical Tanzanian school to support young children with disabilities to improve their academic outcomes.     

Shifting Engagements in Figured Worlds: Middle School Mathematics Students' Participation in an Architectural Design Project

Project-based curricula have the potential to engage students' interests. But how do students become interested in the goals of a project? This article documents how a group of 8th-grade students participated in an architectural design project called the Antarctica Project. The project is based on the imaginary premise that students need to design a research station in Antarctica. This premise is meant to provide a meaningful context for learning mathematics. Using ethnography and discourse analysis, the article investigates students' engagement with the imaginary premise and curricular tasks during the 7-week project. A case study consisting of scenes from main phases of the project shows how the students took on concerns and responsibilities associated with the figured world proposed by the Antarctica Project and how this shaped their approaches to mathematical tasks (Holland, Lachicotte, Skinner, & Cain, 1998). Participating in the figured world of Antarctica and evaluating situations within this world was important for how students used mathematics meaningfully to solve problems. Curricular tasks and classroom activities that facilitated students in assuming and shifting between roles relevant to multiple figured worlds (i.e., of the classroom, Antarctica, and mathematics) helped them engage in the diverse intentions of curricular activities.     

Differential Improvement in Student Understanding of Mathematical Principles Following Formative Assessment Intervention

ABSTRACT

The authors describe results from a study of a middle school mathematics formative assessment strategy. They employed a randomized, controlled design to address the following question: Does using our strategy improve student performance on assessments of key mathematical ideas relative to a comparison group? Eighty-five teachers and 4,091 students were included. Students took a pretest and a transfer measure at the end of the year. Treatment students completed formative assessments. Treatment teachers had exposure to professional development and instructional resources. Results indicated students with higher pretest scores benefited more from the treatment compared to students with lower pretest scores. In addition treatment students significantly outperformed control students on distributive property items. This effect was larger as pretest scores increased. Results, limitations, and future directions are discussed.     

The Role of Extended Time and Item Content on a High‐Stakes Mathematics Test

The premise of a great deal of current research guiding policy development has been that accommodations are the catalyst for student performance differences. Rather than accepting this premise, two studies were conducted to investigate the influence of extended time and content knowledge on the performance of ninth‐grade students who took a statewide mathematics test with and without accommodations. Each study involved 1,250 accommodated students (extended time only) with learning disabilities and 1,250 nonaccommodated students demonstrating no disabilities. In Study One, a standard differential item functioning (DIF) analysis illustrated that the usual approach to studying the effects of accommodations contributes little to our understanding of the reason for performance differences across students. Next, a mixture item response theory DIF model was used to explore the most likely cause(s) for performance differences across the population. The results from both studies suggest that students for whom items were functioning differently were not accurately characterized by their accommodation status but rather by their content knowledge. That is, knowing students' accommodation status (i.e., accommodated or nonaccommodated) contributed little to understanding why accommodated and nonaccommodated students differed in their test performance. Rather, the data would suggest that a more likely explanation is that mathematics competency differentiated the groups of student learners regardless of their accommodation and/or reading levels.     

Teaching biology through statistics: application of statistical methods in genetics and zoology courses

Incorporation of mathematics into biology curricula is critical to underscore for undergraduate students the relevance of mathematics to most fields of biology and the usefulness of developing quantitative process skills demanded in modern biology. At our institution, we have made significant changes to better integrate mathematics into the undergraduate biology curriculum. The curricular revision included changes in the suggested course sequence, addition of statistics and precalculus as prerequisites to core science courses, and incorporating interdisciplinary (math-biology) learning activities in genetics and zoology courses. In this article, we describe the activities developed for these two courses and the assessment tools used to measure the learning that took place with respect to biology and statistics. We distinguished the effectiveness of these learning opportunities in helping students improve their understanding of the math and statistical concepts addressed and, more importantly, their ability to apply them to solve a biological problem. We also identified areas that need emphasis in both biology and mathematics courses. In light of our observations, we recommend best practices that biology and mathematics academic departments can implement to train undergraduates for the demands of modern biology.     

A Meta‐Analysis of Research on the Read Aloud Accommodation

Read aloud is a testing accommodation that has been studied by many researchers, and its use on K‐12 assessments continues to be debated because of its potential to change the measured construct or unfairly increase test scores. This study is a summary of quantitative research on the read aloud accommodation. Previous studies contributed information to compute average effect sizes for students with disabilities, students without disabilities, and the difference between groups for reading and mathematics using a random effects meta‐analytic approach. Results suggest that (1) effect sizes are larger for reading than for math for both student groups, (2) the read aloud accommodation increases reading test scores for both groups, but more so for students with disabilities, and (3) mathematics scores gains due to the read aloud accommodation are small for both students with and without disabilities, on average. There was some evidence to suggest larger effects in elementary school relative to middle and high school and possible mode effects, but more studies are needed within levels of the moderator variables to conduct statistical tests.