Examining Effectiveness and Validity of Accommodations for English Language Learners in Mathematics: An Evidence-Based Computer Accommodation Decision System

Research indicates that the performance-gap between English Language Learners (ELLs) and their non-ELL peers is partly due to ELLs' difficulty in understanding assessment language. Accommodations have been shown to narrow this performance-gap, but many accommodations studies have not used a randomized design and are based on relatively small sample sizes. Addressing such issues, we administered a standard-based mathematics assessment to approximately 3,000 Grade 9 ELL and non-ELL students under five different language-based accommodations. Results indicate that many of these accommodations did not produce significant gains for the recipients. Some even had a negative impact. We believe several factors may explain these findings. First, newer assessments, including those developed for this study, may have been linguistically modified to the point that further modification has only a limited effect. Second, the language of instruction may have not adequately prepared students for the assessment. If the language of instruction (textbook, etc.) contains unnecessary linguistic complexity, then students may not have had the opportunity to learn the assessed content. A third factor is students’ unfamiliarity with these accommodations because they are seldom used in classroom instruction and teacher assessments. We discuss our findings and implications for policymakers, assessment developers, practitioners, and researchers.     

Do Proper Accommodation Assignments Make a Difference? Examining the Impact of Improved Decision Making on Scores for English Language Learners

Does it matter if students are appropriately assigned to test accommodations? Using a randomized method, this study found that individual students assigned accommodations keyed to their particular needs were significantly more efficacious for English language learners (ELLs) and that little difference was reported between students receiving incomplete or not recommended accommodations and no accommodations whatsoever. A sample of third and fourth grade ELLs in South Carolina (N = 272) were randomly assigned to various types of test accommodations on a mathematics assessment. Results indicated that those students who received the appropriate test accommodations, as recommended by a version of a computerized accommodation taxonomy for ELLs (the selection taxonomy for English language learners accommodations; STELLA), had significantly higher test scores than ELLs who received no accommodations or those who received incomplete or not recommended accommodation packages. Additionally, students who were given no test accommodations scored no differently than those students that received accommodation packages that were incomplete or not recommended, given the students' particular needs and challenges. These findings are important in light of research and anecdotal reports that suggest a general lack of systematicity in the current system of assigning accommodations and a tendency to give all available accommodations regardless of individual child characteristics. The results also have important implications for how future accommodation research should be structured to determine the benefits of particular accommodations and accommodation packages. This study would suggest that control and treatment groups should be assembled based on specific student needs in order for direct comparisons to be made.     

Mean Effects of Test Accommodations for ELLs and Non‐ELLs: A Meta‐Analysis of Experimental Studies

The objective was to examine the impact of different types of accommodations on performance in content tests such as mathematics. The meta‐analysis included 14 U.S. studies that randomly assigned school‐aged English language learners (ELLs) to test accommodation versus control conditions or used repeated measures in counter‐balanced order. Individual effect sizes (Glass's d) were calculated for 50 groups of ELLs and 32 groups of non‐ELLs. Individual effect sizes for English language and native language accommodations were classified into groups according to type of accommodation and timing conditions. Means and standard errors were calculated for each category. The findings suggest that accommodations that require extra printed materials need generous time limits for both the accommodated and unaccommodated groups to ensure that they are effective, equivalent in scale to the original test, and therefore more valid owing to reduced construct‐irrelevant variance. Computer‐administered glossaries were effective even when time limits were restricted. Although the Plain English accommodation had very small average effect sizes, inspection of individual effect sizes suggests that it may be much more effective for ELLs at intermediate levels of English language proficiency. For Spanish‐speaking students with low proficiency in English, the Spanish test version had the highest individual effect size (+1.45).     

Framing Appropriate Accommodations in Terms of Individual Need: Examining the Fit of Four Approaches to Selecting Test Accommodations of English Language Learners

Providing appropriate test accommodations to most English language learners (ELLs) is important to facilitate meaningful inferences about learning. This study compared teacher large-scale test accommodation recommendations to those from a literature- and practitioner-grounded accommodation selection taxonomy. The taxonomy links student-specific needs, strengths, and schooling experiences to large-scale test accommodation recommendations that differentially minimize barriers of access for students with different profiles. A blind panel of experts rated four sets of recommendations for each of 114 ELLs. Results found the taxonomy was a significantly better fit for distinguishing accommodations by student need than teacher recommendations. Further, the fit of teacher recommendations showed no difference when the teacher used a structured data collection procedure to gather profile information about each of their ELLs and when they did not, and teachers’ recommendations were not found to differ significantly from a random set of accommodations. Findings are consistent with previous literature that suggests the task of matching specific accommodations to individual needs, rather than the task of identifying individual needs, is where teachers struggle in recommending appropriate test accommodations.     

The Effects of Test Accommodations for English Language Learners: A Meta-Analysis

A meta-analysis using Hierarchical Linear Modeling (HLM) was conducted to examine the effects of test accommodations on the test performance of English language learners (ELLs). The results indicated that test accommodations improve ELLs' test performance by about 0.157 standard deviations—a relatively small but statistically significant increase. Once the potential predictors that may have contributed to the variance of the effect sizes across studies had been accounted for, only English proficiency was found to be significant. Further, the results indicated that ELLs with a low level of English proficiency benefited much more from test accommodations than did those with a high level of English proficiency. Little difference was observed in regard to other factors such as students' ethnicity, students' grade level, or the subject for which they were being examined. Although previous studies have suggested that linguistic simplification may be more effective than other methods, results from this meta-analysis offered no support for that suggestion.     

Bilingual school psychologists' assessment practices with English language learners

This study explored bilingual school psychologists' assessment practices with students identified as English language learners (ELL). One thousand bilingual National Association of School Psychologist members were recruited nationwide, and 276 participated. Among those conducting language proficiency assessments of ELLs, many (58%) use comprehensive methods across four domains of language use. Participants generally use multifaceted assessment approaches and most (84.1%) assess acculturation as part of the assessment battery. Although few use interpreters, those who did reported a mix of best practices and discouraged practices in their use. Multiple regression results showed that engaging in continuing education activities about assessing ELLs predicted “best practices” when assessing language proficiency, selecting and using measures with ELLs with whom they share a second language, and using interpreters. A relationship was also found between engaging in applied preservice training with bilingual supervision and best practice behaviors assessing student acculturation. © 2010 Wiley Periodicals, Inc.     

The Effect of Linguistic Simplification of Science Test Items on Score Comparability

The use of accommodations has been widely proposed as a means of including English language learners (ELLs) or limited English proficient (LEP) students in state and districtwide assessments. However, very little experimental research has been done on specific accommodations to determine whether these pose a threat to score comparability. This study examined the effects of linguistic simplification of 4th- and 6th-grade science test items on a state assessment. At each grade level, 4 experimental 10-item testlets were included on operational forms of a statewide science assessment. Two testlets contained regular field-test items, but in a linguistically simplified condition. The testlets were randomly assigned to LEP and non-LEP students through the spiraling of test booklets. For non-LEP students, in 4 t-test analyses of the differences in means for each corresponding testlet, 3 of the mean score comparisons were not significantly different, and the 4th showed the regular version to be slightly easier than the simplified version. Analysis of variance (ANOVA), followed by pairwise comparisons of the testlets, showed no significant differences in the scores of non-LEP students across the 2 item types. Among the 40 items administered in both regular and simplified format, item difficulty did not vary consistently in favor of either format. Qualitative analyses of items that displayed significant differences in p values were not informative, because the differences were typically very small. For LEP students, there was 1 significant difference in student means, and it favored the regular version. However, because the study was conducted in a state with a small number of LEP students, the analyses of LEP student responses lacked statistical power. The results of this study show that linguistic simplification is not helpful to monolingual English-speaking students who receive the accommodation. Therefore, the results provide evidence that linguistic simplification is not a threat to the comparability of scores of LEP and monolingual English-speaking students when offered as an accommodation to LEP students. The study findings may also have implications for the use of linguistic simplification accommodations in science assessments in other states and in content areas other than science.     

Differential Performance by English Language Learners on an Inquiry-Based Science Assessment

The performance of English language learners (ELLs) has been a concern given the rapidly changing demographics in US K-12 education. This study aimed to examine whether students' English language status has an impact on their inquiry science performance. Differential item functioning (DIF) analysis was conducted with regard to ELL status on an inquiry-based science assessment, using a multifaceted Rasch DIF model. A total of 1,396 seventh- and eighth-grade students took the science test, including 313 ELL students. The results showed that, overall, non-ELLs significantly outperformed ELLs. Of the four items that showed DIF, three favored non-ELLs while one favored ELLs. The item that favored ELLs provided a graphic representation of a science concept within a family context. There is some evidence that constructed-response items may help ELLs articulate scientific reasoning using their own words. Assessment developers and teachers should pay attention to the possible interaction between linguistic challenges and science content when designing assessment for and providing instruction to ELLs.     

The Critical Role of Vocabulary Development for English Language Learners

English language learners (ELLs) who experience slow vocabulary development are less able to comprehend text at grade level than their English‐only peers. Such students are likely to perform poorly on assessments in these areas and are at risk of being diagnosed as learning disabled. In this article, we review the research on methods to develop the vocabulary knowledge of ELLs and present lessons learned from the research concerning effective instructional practices for ELLs. The review suggests that several strategies are especially valuable for ELLs, including taking advantage of students' first language if the language shares cognates with English; ensuring that ELLs know the meaning of basic words, and providing sufficient review and reinforcement. Finally, we discuss challenges in designing effective vocabulary instruction for ELLs. Important issues are determining which words to teach, taking into account the large deficits in second‐language vocabulary of ELLs, and working with the limited time that is typically available for direct instruction in vocabulary.     

Teaching English Language Learners At Risk for Reading Disabilities to Read: Putting Research into Practice

The purpose of this article is to describe features of interventions that are empirically validated for use with first‐grade students at risk for reading disabilities who are English language learners (ELLs) and whose home language is Spanish. The empirical evidence supporting these interventions is summarized. Interventions for improving oral language and reading abilities with struggling readers who are ELLs taught in either Spanish or English are described as a means to assist school districts and teachers in defining and implementing effective interventions for ELLs at risk for reading difficulties. The interventions described may be useful to educators seeking information about Response to Intervention as a means of identifying ELLs who require services for learning disabilities.     

A Randomized Experiment of a Cognitive Strategies Approach to Text-Based Analytical Writing for Mainstreamed Latino English Language Learners in Grades 6 to 12

Abstract

This study reports Year 1 findings from a multisite cluster randomized controlled trial of a cognitive strategies approach to teaching text-based analytical writing for mainstreamed Latino English language learners (ELLs) in 9 middle schools and 6 high schools. There were 103 English teachers stratified by school and grade and then randomly assigned to the Pathway Project professional development intervention or control group. The Pathway Project trains teachers to use a pretest on-demand writing assessment to improve text-based analytical writing instruction for mainstreamed Latino ELLs who are able to participate in regular English classes. The intervention draws on well-documented instructional frameworks for teaching mainstreamed ELLs. Such frameworks emphasize the merits of a cognitive strategies approach that supports these learners’ English language development. Pathway teachers participated in 46 hrs of training and learned how to apply cognitive strategies by using an on-demand writing assessment to help students understand, interpret, and write analytical essays about literature. Multilevel models revealed significant effects on an on-demand writing assessment (d = .35) and the California Standards Test in English language arts (d = .07).     

English Language Learners and Learning Disabilities: Research Agenda and Implications for Practice

Although little is known about learning disabilities (LDs) in English language learners (ELLs), there is a substantial knowledge base about the identification, assessment, and intervention of and for LDs in monolingual native English‐speaking students. Building on this knowledge, participants at an October 2003 National Symposium on Learning Disabilities in English Language Learners were asked to suggest research questions, priorities, and suggestions on how to build the necessary infrastructure to address critical research needs. In the discussions that took place, important themes emerged: (1) identification and assessment of LD and/or reading disabilities (RD) in ELLs (ELL/Ds), (2) understanding of the language and literacy developmental trajectories of ELLs, (3) understanding of the individual and contextual factors affecting outcomes, (4) the intersection of all of these areas with neurobiology, and (5) developing and testing the effectiveness of interventions for learning disabilities in ELL/Ds. These themes, and the research agenda that was forged around them, are presented. In addition, the practice implications of this agenda are presented, along with some suggestions for current practice while we await future research findings.     

Evaluating Computer-Based Test Accommodations for English Learners

Test accommodations for English learners (ELs) are intended to reduce the language barrier and level the playing field, allowing ELs to better demonstrate their true proficiencies. Computer-based accommodations for ELs show promising results for leveling that field while also providing us with additional data to more closely investigate the validity and effectiveness of those accommodations. In this study, we evaluate differences across non-ELs and two EL groups in their decision to use either of two computer-based accommodations on high school history and math assessments. We also evaluate differences in response times across these groups. Results showed that ELs used accommodations more than non-ELs; however, many students did not use any accommodations, and use decreased as the assessment progressed. In addition, students had longer response time for items with accommodations in history but not mathematics. Recommendations for future research in accommodations for ELs are discussed.     

Threats to the validity of science performance assessments for English language learners

In recent years, at the same time that performance assessments in science have become more popular, the number of English language learners (ELLs) (i.e., students whose native language is other than English) served by the U.S. educational system has also increased rapidly. While the research base is growing in each of these areas independently, little attention has been paid to their intersection. This case study of the use of a science performance assessment with 96 ELLs in five high school science classes investigated the face, construct, and consequential validity of this intersection. Qualitative and quantitative data analyses showed that both teachers and students had an overall favorable response to the assessment, although students' English comprehension and expression skills were determining factors for certain items. While most responses were reliably scored, ELL spelling and syntax on certain responses were significant sources of error. The degree of specificity of teachers' guidance also significantly affected students' scores. Recommendations from this study include increasing the clarity of an assessment's design, allowing ELLs more time to complete assessments, and scoring by raters who are knowledgeable about typical patterns in written English for this student population. Furthermore, it is recommended that the use of performance assessments with ELLs be exploratory until such time as their validity and reliability with this population can be more adequately established. J Res Sci Teach 34: 721–743, 1997.     

Integrating Science And English Proficiency For English Language Learners

Despite the expectation that all students should achieve high academic standards, content area instruction and English for speakers of other languages instruction for English language learners (ELLs) have traditionally been conceptualized as separate domains, resulting in educational inequities for ELLs. This is because effective instruction to promote academic achievement for ELLs requires integration of content and language. Such inequities are more pronounced in urban schools where ELLs are disproportionately represented. In science education, research on instructional interventions to simultaneously promote science and English proficiency of ELLs has begun to emerge in recent years. Grounded in this emerging research literature, we offer specific instructional strategies to integrate science and English proficiency for ELLs in five domains: (a) literacy strategies with all students, (b) language support strategies with ELLs, (c) discourse strategies with ELLs, (d) home language support, and (e) home culture connections.     

Examining the Validity of Standards-Based Assessments for Initially Fluent Students and Former English Language Learners

To date, assessment validity research on non-native English speaking students in the United States has focused exclusively on those who are presently English language learners (ELLs). However, little, if any, research has been conducted on two other sizable groups of language minority students: (a) bilingual or multilingual students who were already English proficient when they entered the school system (IFEPs), and (b) former English language learners, those students who were once classified as ELLs but are now reclassified as being English proficient (RFEPs). This study investigated the validity of several standards-based assessments in mathematics and science for these two student groups and found a very high degree of score comparability, when compared with native English speakers, for the IFEPs, whereas a moderate to high degree of score comparability was observed for the RFEPs. Thus, test scores for these two groups on the assessments we studied appear to be valid indicators of their content knowledge, to a degree similar to that of native English speakers.     

Linguistic Complexity,Schematic Representations,and Differential Item Functioning for English Language Learners in Math Tests

This article examines nonmathematical linguistic complexity as a source of differential item functioning (DIF) in math word problems for English language learners (ELLs). Specifically, this study investigates the relationship between item measures of linguistic complexity, nonlinguistic forms of representation and DIF measures based on item response theory difficulty parameters in a state fourth-grade math test. This study revealed that the greater the item nonmathematical lexical and syntactic complexity, the greater are the differences in difficulty parameter estimates favoring non-ELLs over ELLs. However, the impact of linguistic complexity on DIF is attenuated when items provide nonlinguistic schematic representations that help ELLs make meaning of the text, suggesting that their inclusion could help mitigate the negative effect of increased linguistic complexity in math word problems.     

The Impact of a Study Abroad Experience on Preservice Teachers' Dispositions and Plans for Teaching English Language Learners

This study explored how a study abroad experience affected preservice teachers' (PSTs') dispositions towards and plans for teaching English Language Learners (ELLs). This qualitative study involved 16 elementary education initial licensure PSTs who participated in a 2-month study abroad trip to Germany. Data in the form of surveys, interviews, and reflections were collected. The PSTs' experiences left them feeling like outsiders, fostering greater empathy for ELLs. As the PSTs faced the challenges of studying abroad, they developed ways of coping, which influenced their dispositions toward ELLs and the accommodations they wanted to make when teaching ELLs in their future classrooms. A model of the PSTs' learning progression within the study abroad experience is offered as a lens to consider their transformations. Findings suggest it would be beneficial for educators traveling abroad with PSTs to find ways to make explicit the connections between personal encounters and future teaching experiences.     

Imagining and Moving Beyond the ESL Bubble: Facilitating Communities of Practice Through the ELL Ambassadors Program

When educators do not facilitate English language learners’ (ELLs) social integration in schools, this can perpetuate ELLs’ marginalized status and the plateauing of ELLs’ English language development. This study highlights a program for secondary ELLs called the ELL Ambassadors program, which partnered ELLs with non-ELLs based on shared extracurricular interests. Comparing the stories, perspectives, and test scores of five newcomer ELLs from varied countries of origin, this article shows how program participants strengthened their English language skills and achieved academic success, demonstrating tremendous agency as they gained access to, and were socialized within, interest-based communities of practice. Further, this article documents how youth imagined and claimed new identities, moving beyond the insulation and isolation of the ESL bubble to gaining confidence through interest-based learning with other peers. Findings suggest that interest-based peer programs at schools may create important opportunities for ELLs’ academic, language, and identity formation.