Striving for equitable classroom assessments for linguistic minorities: Strategies for and effects of revising life science items

Striving for equitable assessments that can contribute to classroom learning for linguistic minorities is a goal of increasing importance as the national population of English language learners continues to rise. This study investigated classroom assessments for English learners in middle school life science courses in two California schools. A framework for equitable classroom assessments, “McCes—Sounds like Success,” was used to refine and evaluate assessments in the study. Ways to improve two written assessments for advanced English learners were developed through teacher research and tested with a pretest/posttest design. Eleven changes to the items were developed, such as adding visual supports and dividing prompts into smaller units. Regression analyses of raw and Rasch modeled data from the pretest/posttest showed that both English only students and advanced English learners scored significantly better on the modified classroom assessments. A new perspective on validating equitable classroom assessments as opposed to standardized assessments for English learners is discussed. © 2006 Wiley Periodicals, Inc. J Res Sci Teach 44: 864–881, 2007     

Equitable Written Assessments for English Language Learners: How Scaffolding Helps

This study investigated the effects of the use of scaffolds in written classroom assessments through the voices of both native English speakers and English language learners from two middle schools. Students responded to assessment tasks in writing, by speaking aloud using think aloud protocols, and by reflecting in a post-assessment interview. The classroom assessment tasks were designed to engage students in scientific sense making and multifaceted language use, as recommended by the Next Generation Science Standards. Data analyses showed that both groups benefitted from the use of scaffolds. The findings revealed specific ways that modifications were supportive in helping students to comprehend, visualize and organize thinking, and elicit responses. This study offers a model for both sensitizing teachers and strengthening their strategies for scaffolding assessments equitably.     

Young learners’ interactional development in task-based paired-assessment in their first and foreign languages: a case of English learners in China

In response to the growing interest in evaluating young learners’ foreign language (FL) performance, this study aims to deepen our understanding of young learners’ developmental differences in interaction during task-based paired-language assessments. To examine age effects separately from the effect of general language proficiency, we analysed learners’ interaction in both their first language (L1) and FL. We observed the students as they engaged in an assessment consisting of a set of decision-making tasks with their peers. After each task, the students self-assessed their performance. We found differences in interactions across grade levels in both L1 and FL with respect to holistic interactional patterns, topic development, and turn-taking patterns. The students’ understanding of communicative interactive tasks also appeared to differ between grade levels, perhaps reflecting their experiential differences with communicative tasks in class. The paper concludes with implications for implementing task-based paired-language assessments for young learners.     

Threats to the validity of science performance assessments for English language learners

In recent years, at the same time that performance assessments in science have become more popular, the number of English language learners (ELLs) (i.e., students whose native language is other than English) served by the U.S. educational system has also increased rapidly. While the research base is growing in each of these areas independently, little attention has been paid to their intersection. This case study of the use of a science performance assessment with 96 ELLs in five high school science classes investigated the face, construct, and consequential validity of this intersection. Qualitative and quantitative data analyses showed that both teachers and students had an overall favorable response to the assessment, although students' English comprehension and expression skills were determining factors for certain items. While most responses were reliably scored, ELL spelling and syntax on certain responses were significant sources of error. The degree of specificity of teachers' guidance also significantly affected students' scores. Recommendations from this study include increasing the clarity of an assessment's design, allowing ELLs more time to complete assessments, and scoring by raters who are knowledgeable about typical patterns in written English for this student population. Furthermore, it is recommended that the use of performance assessments with ELLs be exploratory until such time as their validity and reliability with this population can be more adequately established. J Res Sci Teach 34: 721–743, 1997.     

英语书面语中模糊限制语的使用差异研究

基于BROWN,LOB,CLEC三个书面语语料库,通过对比英语本族语者与中国英语学习者英语书面语中模糊限制语的使用情况,探讨影响中国英语学习者英语写作中模糊限制语使用的因素,指出学生在英语写作中应尽可能避免文化思维差异及母语迁移对写作的影响,而教师在教学中应尽量加强对模糊限制语的研究,使学生在表达中增强正确使用模糊限制语的意识,达到成功交流的目的。     

Toward dynamic assessment of reading: applying metacognitive awareness guidance to reading assessment tasks

This paper focuses on research that aimed to provide a theoretical–practical framework to link literacy assessment practice with learning theory. An experimental study was designed with reference to three theoretical axes: ‘metacognitive awareness’ theory, ‘schema’ theory and the Vygotskian ‘zone of proximal fevelopment’. The study tested the effect of using written metacognitive awareness guidance (MCAG) as a tool for activating and engaging learners’Habits of Mind while processing authentic reading assessment tasks taken from Israeli kits of assessment tasks (Guterman, 2000). The study on 300 Grade–4 pupils used three modalities: a control group, which received no treatment; a placebo group, which received content instructions (CI); and a treatment group, which was given written MCAG. The findings confirmed that applying metacognitive awareness guidance to reading assessment tasks makes a difference in the learners’ level of performance and achievement on those tasks, and also increases learners’ chances of internalising the guidance components.     

The self‐assessed portfolio: a case study

This paper reports on a successful attempt to use the portfolio as a sole assessment tool for an upper level language arts course at an English‐medium university in Lebanon. Over four consecutive years in the spring semester, the teacher/researcher devised a special syllabus based on the teaching/learning of text discourses and other language tasks emphasizing skills to improve the English language of the learners. Only students’ portfolios were used to evaluate students’ work. The learners, majoring in Education or English, worked on various language tasks. They presented and assessed their work according to rubrics. Learners had to self‐reflect on each task, have a one‐to‐one conference with the class teacher, and assign a letter grade to their work. Results indicated that though using only portfolios for assessment purposes was a rather difficult task, it was still more effective than traditional assessment. Working on self‐assessed portfolios actively engaged learners in the learning process.     

远程英语写作教学中的书面反馈调查与研究

在远程语言教学中,由于交流机会有限,书面反馈就成为了开启和维持指导教师与学习者对话的重要手段。基于网络平台开放英语写作课程,以书面反馈为研究对象,采用问卷和访谈等方式,探索在远程语言教学环境下,指导教师和学习者对反馈的理念与实践。研究发现不同指导教师所提供的反馈类型和形式有明显差异,而不同学习者之间的期望和理解也差异较大。建议指导教师提供反馈前应了解学习者的学习需求与特点,同时,需要对指导教师和学习者进行有关有效利用书面反馈系统的培训,以提高书面反馈的对语言教学的促进效果。     

Validating English Language Proficiency Assessment Uses for English Learners: Academic Language Proficiency and Content Assessment Performance

States use standards‐based English language proficiency (ELP) assessments to inform relatively high‐stakes decisions for English learner (EL) students. Results from these assessments are one of the primary criteria used to determine EL students’ level of ELP and readiness for reclassification. The results are also used to evaluate the effectiveness of and funding allocation to district or school programs that serve EL students. In an effort to provide empirical validity evidence for such important uses of ELP assessments, this study focused on examining the constructs of ELP assessments as a fundamental validity issue. Particularly, the study examined the types of language proficiency measured in three sample states’ ELP assessments and the relationship between each type of language proficiency and content assessment performance. The results revealed notable variation in the presence of academic and social language in the three ELP assessments. A series of hierarchical linear modeling (HLM) analyses also revealed varied relationships among social language proficiency, academic language proficiency, and content assessment performance. The findings highlight the importance of examining the constructs of ELP assessments for making appropriate interpretations and decisions based on the assessment scores for EL students. Implications for policy and practice are discussed.     

English language learners and automated scoring of essays: Critical considerations

This article presents considerations for using automated scoring systems to evaluate second language writing. A distinction is made between English language learners in English-medium educational systems and those studying English in their own countries for a variety of purposes, and between learning-to-write and writing-to-learn in a second language (Manchón, 2011a), extending Manchón's framework from instruction to assessment and drawing implications for construct definition. Next, an approach to validity based on articulating an interpretive argument is presented and discussed with reference to a recent study of the use of e-rater on the TOEFL. Challenges and opportunities for the use of automated scoring system are presented.     

Examining the Validity of Standards-Based Assessments for Initially Fluent Students and Former English Language Learners

To date, assessment validity research on non-native English speaking students in the United States has focused exclusively on those who are presently English language learners (ELLs). However, little, if any, research has been conducted on two other sizable groups of language minority students: (a) bilingual or multilingual students who were already English proficient when they entered the school system (IFEPs), and (b) former English language learners, those students who were once classified as ELLs but are now reclassified as being English proficient (RFEPs). This study investigated the validity of several standards-based assessments in mathematics and science for these two student groups and found a very high degree of score comparability, when compared with native English speakers, for the IFEPs, whereas a moderate to high degree of score comparability was observed for the RFEPs. Thus, test scores for these two groups on the assessments we studied appear to be valid indicators of their content knowledge, to a degree similar to that of native English speakers.     

Issues in the assessment of literacy‐related difficulties across language backgrounds: a cross‐linguistic comparison

This paper focused on the assessment of phonological skills amongst children with developmental dyslexia. Findings from assessments of English and Hungarian monolingual children with and without literacy deficits and bilingual Filipino children with and without literacy deficits in English indicated that performance on phonological‐based tasks often used in dyslexia assessment batteries was influenced by the language background. Monolingual English children with poor literacy skills showed characteristic deficits in most areas of phonological ability, whereas Hungarian counterparts showed little evidence of such difficulties. Bilingual Filipino children with poor English literacy skills showed equivalent profiles to their monolingual counterparts only when assessments in both English and Filipino were considered. The paper discusses difficulties of generalising assessment procedures from one language context to another.     

研究生公共英语课程体系改革构想

时代的挑战对研究生英语水平提出了更高的要求,研究生英语教学也需要进一步改革和完善。在调查分析的基础上,找出了当前研究生英语课程体系中的问题和困惑主要体现在课程设置、教学形式和评估体系三方面,并以此为切入点,在研究生英语教学大纲的理论指导和外语教学理论的指导下,提出了从课程设置、教学形式和评估体系三个方面进行课程体系改革的构想,以期更有效地激发研究生英语学习潜能,培养和提高学生的英语应用能力、学术交流能力和文化创新能力。     

Gender and practical skill performance in science

The performance of 18 boys and 18 girls on four problem-solving tasks set in science contexts was compared. The tasks were administered in a one-to-one testing situation and assessments were made by direct observation, questioning, and by using written records. The tasks were valid and reliable, and the samples of boys and girls were matched for ability and curriculum background. Past studies have identified gender differences in performance on science tasks; however, this study found little evidence to support these findings. Few significant differences in performance were found. No gender differences were detected in observation, reporting, or planning skills, and there was no differential performance on the use of scientific language. Girls performed less well in relation to self-reliance, and performance differences on the interpretation skill approached significance with boys' performance superior.     

Thinking beyond the score: Multidimensional analysis of student performance to inform the next generation of science assessments

Conventional assessment analysis of student results, referred to as rubric-based assessments (RBA), has emphasized numeric scores as the primary way of communicating information to teachers about their students’ learning. In this light, rethinking and reflecting on not only how scores are generated but also what analyses are done with them to inform classroom practices is of utmost importance. Informed by Systemic Functional Linguistics and Latent Dirichlet Allocation analyses, this study utilizes an innovative bilingual (Spanish–English) constructed response assessment of science and language practices for middle and high school students to perform a multilayered analysis of student responses. We explore multiple ways of looking at students’ performance through their written assessments and discuss features of student responses that are made visible through these analyses. Findings from this study suggest that science educators would benefit from a multidimensional model which deploys complementary ways in which we can interpret student performance. This understanding leads us to think that researchers and developers in the field of assessment need to promote approaches that analyze student science performance as a multilayered phenomenon.     

Bilingual school psychologists' assessment practices with English language learners

This study explored bilingual school psychologists' assessment practices with students identified as English language learners (ELL). One thousand bilingual National Association of School Psychologist members were recruited nationwide, and 276 participated. Among those conducting language proficiency assessments of ELLs, many (58%) use comprehensive methods across four domains of language use. Participants generally use multifaceted assessment approaches and most (84.1%) assess acculturation as part of the assessment battery. Although few use interpreters, those who did reported a mix of best practices and discouraged practices in their use. Multiple regression results showed that engaging in continuing education activities about assessing ELLs predicted “best practices” when assessing language proficiency, selecting and using measures with ELLs with whom they share a second language, and using interpreters. A relationship was also found between engaging in applied preservice training with bilingual supervision and best practice behaviors assessing student acculturation. © 2010 Wiley Periodicals, Inc.     

Differential Performance by English Language Learners on an Inquiry-Based Science Assessment

The performance of English language learners (ELLs) has been a concern given the rapidly changing demographics in US K-12 education. This study aimed to examine whether students' English language status has an impact on their inquiry science performance. Differential item functioning (DIF) analysis was conducted with regard to ELL status on an inquiry-based science assessment, using a multifaceted Rasch DIF model. A total of 1,396 seventh- and eighth-grade students took the science test, including 313 ELL students. The results showed that, overall, non-ELLs significantly outperformed ELLs. Of the four items that showed DIF, three favored non-ELLs while one favored ELLs. The item that favored ELLs provided a graphic representation of a science concept within a family context. There is some evidence that constructed-response items may help ELLs articulate scientific reasoning using their own words. Assessment developers and teachers should pay attention to the possible interaction between linguistic challenges and science content when designing assessment for and providing instruction to ELLs.     

Integrating Scaffolding Strategies Into Technology-Enhanced Assessments of English Learners: Task Types and Measurement Models

This article investigates ways to improve the assessment of English learner students' English language proficiency given the current movement of creating next-generation English language proficiency assessments in the Common Core era. In particular, this article discusses the integration of scaffolding strategies, which are prevalently utilized as an instructional strategy for English learner students, into the design of technology-enhanced assessment tasks. The article includes sample tasks and student responses to illustrate the design of scaffolding assessment tasks and their potential to increase the accuracy of measuring students' English language proficiency. We also explore possible scoring and psychometric models for the scaffolding tasks in large-scale standardized assessments.     

Using Educative Assessments to Support Science Teaching for Middle School English-language Learners

Grounded in Hallidayan perspectives on academic language, we report on our development of an educative science assessment as one component of the language-rich inquiry science for English-language learners teacher professional learning project for middle school science teachers. The project emphasizes the role of content-area writing to support teachers in diagnosing their students’ emergent understandings of science inquiry practices, science content knowledge, and the academic language of science, with a particular focus on the needs of English-language learners. In our current school policy context, writing for meaningful purposes has received decreased attention as teachers struggle to cover large numbers of discrete content standards. Additionally, high-stakes assessments presented in multiple-choice format have become the definitive measure of student science learning, further de-emphasizing the value of academic writing for developing and expressing understanding. To counter these trends, we examine the implementation of educative assessment materials—writing-rich assessments designed to support teachers’ instructional decision making. We report on the qualities of our educative assessment that supported teachers in diagnosing their students’ emergent understandings, and how teacher–researcher collaborative scoring sessions and interpretation of assessment results led to changes in teachers’ instructional decision making to better support students in expressing their scientific understandings. We conclude with implications of this work for theory, research, and practice.     

Roles of definitional and assessment models in the identification of new or second language learners of English for special education

This article examines the efficacy of current definitional perspectives on learning disabilities (LD) and related assessment models to support appropriate instructional and support services for learners of English with learning-related difficulties. A revised framework for defining LD and an associated assessment model, curriculum-based dynamic assessment (CDA), are proposed. The results of a teacher assessment study are reported to exemplify how this revised framework may be studied. The study examined the following questions: (a) Can curriculum-based dynamic assessments of authentic learning tasks help educators to differentiate between the work of students with limited English proficiency and their peers identified as having LD? (b) What are the characteristics of curriculum-based work samples of limited English proficient students with LD that may predictably differentiate them from their peers without LD?