A psychometric measurement model for adult English language learners: Pearson Test of English Academic

The aim of this study was to apply Rasch modeling to an examination of the psychometric properties of the Pearson Test of English Academic (PTE Academic). Analyzed were 140 test-takers' scores derived from the PTE Academic database. The mean age of the participants was 26.45 (SD = 5.82), ranging from 17 to 46. Conformity of the participants' performance on the 86 items of PTE Academic Form 1 of the field test was evaluated using the partial credit model. The person reliability coefficient was .96, and item reliability was .99. The results showed that no significant differential item functioning was found across subgroups of gender and spoken-language context, indicating that the item data approximated the Rasch model. The findings of this study validated the test stability of PTE Academic as a useful measurement tool for English language learners' academic English assessment.     

A tale of two tests: The role of topic and general academic knowledge in traditional versus contemporary scenario-based reading

We compared high school students’ performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Performance on the two assessments were moderately correlated (r = 0.57), but the SBA was more difficult (Study 1; n = 342). The two assessments similarly depended on basic reading skills but diverged in the relation to academic knowledge and (non-academic) topic knowledge (Study 2; n = 1107). Academic knowledge was highly predictive of traditional comprehension, but less so for SBA. Topic knowledge was more predictive of SBA than traditional comprehension. Thus, the two assessments tap into similar constructs related to comprehension; however, the level of topic knowledge is more important for performance on scenario-based, multiple-source reading tasks, whereas academic knowledge is more important for traditional reading comprehension tasks.     

Assessing construct of DIBELS: Measurement invariance of DIBELS subscales,from kindergarten to the first grade

A challenge using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in studying reading growth is that reading skills children exhibit change by age. In order to study growth using changing subscales, it is necessary to examine measurement invariance and measurement structure underlying the different subscales. The purpose of this paper is to examine the measurement structure of the DIBELS subscales, particular measurement invariance. The results indicate that the DIBELS subscales do not seem to have metric invariance but they do share a common factor over time, suggesting that the same construct of reading skills were measured but they manifested in the different fashion over time.     

Grade-Level Invariance of a Theoretical Causal Structure Predicting Reading Comprehension With Vocabulary and Oral Reading Fluency

This research investigates the relative importance of vocabulary and oral reading fluency as measurement dimensions of reading comprehension as the student passes from elementary to high school. Invariance of this model over grades 4 through 8 is tested using two independent student samples reading grade-level appropriate passages. Results from structural equation modeling indicate that the model is not invariant across grade levels. Vocabulary knowledge is a significant and constant predictor of overall reading comprehension irrespective of grade level. While significant, fluency effects diminish over grades, especially in the later grades. Lack of grade level invariance was obtained with both samples. Results are discussed in light of vertically linked reading assessments, adequate yearly progress, and instruction.     

The developmental trajectory of intrinsic reading motivation: Measurement invariance,group variations,and implications for reading proficiency

The goal of the present study is to extend previous research on the developmental trajectory of intrinsic reading motivation during early adolescence. Using large-scale panel data on secondary school students in Germany, we examined: (1) the longitudinal measurement invariance of intrinsic reading motivation, (2) the generalizability of the developmental trajectory of intrinsic reading motivation across students’ gender, parental socioeconomic status (SES), and school tracks (academic vs. vocational), and (3) the associations between the developmental trajectory of intrinsic reading motivation and the developmental trajectory of reading proficiency. The scale we used to measure intrinsic reading motivation showed the (strict) measurement invariance across six occasions of measurement from Grades 5 to 10, indicating the high structural similarity (e.g., factor loadings, intercepts) of intrinsic reading motivation during early adolescence. Our analyses of latent growth curve models also confirm previous findings that students tend to experience a steady and significant linear decline in intrinsic reading motivation from Grades 5 to 10. This developmental decline also seems to be more pronounced in size (Δ =  − 0.772, p < .001) than previously reported. The developmental decline in intrinsic reading motivation was observed irrespective of students’ gender, parental SES, and school tracks. Male students expressed lower mean-levels of intrinsic reading motivation across the waves and exhibited a steeper motivational decline compared to female students. Despite mean-level differences across the waves, students showed similar degrees of a motivational decline across parental SES and school tracks. Finally, the larger decline in students’ intrinsic reading motivation was associated with the smaller growth of their reading proficiency from Grades 5 to 10. Our study provides further support for the high prevalence of the developmental decline in intrinsic reading motivation during early adolescence, its generalizability across students’ demographic characteristics, and its implications for the development of reading proficiency.     

Are Accommodations for English Learners on State Accountability Assessments Evidence-Based? A Multistudy Systematic Review and Meta-Analysis

The objectives of this two-part study were to: (a) investigate English learner (EL) accommodation practices on state accountability assessments of reading/English language arts and mathematics in grades 3–8, and (b) conduct a meta-analysis of EL accommodation effectiveness on improving test performance. Across all distinct testing programs, we found that at least one EL test accommodation was provided for both test content areas. The most popular accommodations provided were supplying students with word-to-word dual language dictionaries, reading aloud test directions and items in English, and allowing flexible time/scheduling. However, we found minimal evidence that testing programs provide practitioners with recommendations on how to assign relevant accommodations to EL test takers’ English proficiency level. To evaluate whether accommodations used in practice are supported with evidence of their effectiveness, a meta-analysis was conducted. On average, across 26 studies and 95 effect sizes (N = 11,069), accommodations improved test performance by .16 standard deviations. Both test content and sampling design were found to moderate accommodation effectiveness; however, none of the accommodations investigated were found to have intervention effects that were statistically different from zero. Overall, these results suggest that currently employed EL test accommodations lack evidence of their effectiveness.     

How invariant and accurate are domain ratings in writing assessment?

The use of evidence to guide policy and practice in education (Cooper, Levin, & Campbell, 2009) has included an increased emphasis on constructed-response items, such as essays and portfolios. Because assessments that go beyond selected-response items and incorporate constructed-response items are rater-mediated (Engelhard, 2002, Engelhard, 2013), it is necessary to develop evidence-based indices of quality for the rating processes used to evaluate student performances. This study proposes a set of criteria for evaluating the quality of ratings based on the concepts of measurement invariance and accuracy within the context of a large-scale writing assessment. Two measurement models are used to explore indices of quality for raters and ratings: the first model provides evidence for the invariance of ratings, and the second model provides evidence for rater accuracy. Rating quality is examined within four writing domains from an analytic rubric. Further, this study explores the alignment between indices of rating quality based on these invariance and accuracy models within each of the four domains of writing. Major findings suggest that rating quality varies across analytic rubric domains, and that there is some correspondence between indices of rating quality based on the invariance and accuracy models. Implications for research and practice are discussed.     

Phonological Awareness and Rapid Automatized Naming Predicting Early Development in Reading and Spelling: Results from a Cross-Linguistic Longitudinal Study

In this study, the relationship between latent constructs of phonological awareness (PA) and rapid automatized naming (RAN) were investigated and related to later measures of reading and spelling in children learning to read in different alphabetic writing systems (i.e., Norwegian/Swedish vs. English). 750 U.S./Australian children and 230 Scandinavian children were followed longitudinally between kindergarten and 2nd grade. PA and RAN were measured in kindergarten and Grade 1, while word recognition, phonological decoding, and spelling were measured in kindergarten, Grade 1, and Grade 2. In general, high stability was observed for the various reading and spelling measures, such that little additional variance was left open for PA and RAN. However, results demonstrated that RAN was more related to reading than spelling across orthographies, with the opposite pattern shown for PA. In addition, tests of measurement invariance show that the factor loadings of each observed indicator on the latent PA factor was the same across U.S./Australia and Scandinavia. Similar findings were obtained for RAN. In general, tests of structural invariance show that models of early literacy development are highly transferable across languages.     

Gendered pathways to educational aspirations: The role of academic self-concept,school burnout,achievement and interest in mathematics and reading

The aim of this study was to examine lower secondary school students' (N = 1152) pathways to educational aspirations. The study used multi-group structural equation modelling to investigate the predictions of academic self-concept, school burnout, achievement, and interest in mathematics and reading, in relation to educational aspirations for boys and girls. While certain factors were influential for students' academic aspirations irrespective of gender, some interesting differences also emerged. Academic self-concept and interest in reading predicted educational aspirations for both groups. However, gendered pathways emerged in how achievement and interest in mathematics predicted educational aspirations. Interest in mathematics predicted girls’ educational aspirations, whereas mathematics achievement was a significant predictor for boys. School burnout had negative indirect effects through interest in reading and mathematics in both groups, but for girls, there was also a direct positive effect on educational aspirations.     

Accuracy of teacher assessments of second-language students at risk for reading disability

This study examined the accuracy of teacher assessments in screening for reading disabilities among students of English as a second language (ESL) and as a first language (L1). Academic and oral language tests were administered to 369 children (249 ESL, 120 L1) at the beginning of Grade 1 and at the end of Grade 2. Concurrently, 51 teachers nominated children at risk for reading failure and completed rating scales assessing academic and oral language skills. Scholastic records were reviewed for notation of concern or referral. The criterion measure was a standardized reading score based on phonological awareness, rapid naming, and word recognition. Results indicated that teacher rating scales and nominations had low sensitivity in identifying ESL and L1 students at risk for reading disability at the 1-year mark. Relative to other forms of screening, teacher-expressed concern had lower sensitivity. Finally, oral language proficiency contributed to misclassifications in the ESL group.     

Equivalence of Paper-and-Pencil and Online Administration Modes of the Statewide English Test for Students With and Without Disabilities

This study investigated whether scores obtained from the online and paper-and-pencil administrations of the statewide end-of-course English test were equivalent for students with and without disabilities. Score comparability was evaluated by examining equivalence of factor structure (measurement invariance) and differential item and bundle functioning analyses for the online and paper groups. Results supported measurement invariance between the online and paper groups, suggesting that it is meaningful to compare scores across administration modes. When the data were analyzed at both the item and item bundle (content area) levels, similar performance appeared between the online and paper groups.     

英语阅读理解题型分析及应对策略

目前各类英语水平考试中,阅读理解都占有很重要的比例,特别是英语教学和英语考试改革以后,对学习者听说和读写能力有了更高的要求。但从考生的分数来看,阅读理解经常是失分最多的部分,因此在阅读过程中对其题型进行分析并及时总结应对策略是非常有必要的。     

Predicting math outcomes: reading predictors and comorbidity

This commentary addresses issues concerning (a) the measurement of numbers, letters, and words versus cognitive processes in early screening batteries, and (b) comorbid associations of reading, math, and attention disorders. Based on reading prediction studies, assessments that include numbers should be most predictive of math outcomes. However, given the comorbid association of reading, math, and attention disorders, measures sensitive to reading and attention difficulties may be necessary in early screening batteries for math disabilities.     

Improving content area reading comprehension of Spanish speaking English learners in Grades 4 and 5 using web-based text structure instruction

Reading and comprehending content area texts is important for academic and professional success as well as life skills necessary to maintain good health and quality lifestyle. Spanish speaking English language learners have shown poor performance on high-stakes assessments in reading comprehension. The number of Spanish speaking English learners (ELs) in our schools continues to increase at a fast pace, and therefore it is imperative that we address their reading comprehension needs swiftly and effectively. The text structure strategy has shown positive results on comprehension outcomes in many research studies with students at Grades 2, 4, 5, and 7. This study is the first implementation of instruction about the text structure strategy expressly designed to accommodate the linguistic and comprehension needs of Spanish speaking ELs in Grades 4 and 5. Strategy instruction on the web for English learners (SWELL) was designed to deliver instruction about the text structure strategy to Spanish speaker English learners. A randomized controlled study with pre and post-tests was conducted with 14 classrooms at fourth-grade and 17 classrooms at fifth-grade in high poverty schools where over 85% of students were Spanish speaking bilinguals or ELs. Analysis of data using multi-level models show moderate to large-effects favoring the students in the SWELL classrooms over the business as usual control classrooms on important measures such as a standardized reading comprehension test and main idea and cloze tasks. This research has practical implications for the use of web-based tools to provide high-quality and supportive instruction to improve Spanish speaking ELs reading comprehension skills.     

Assessment in an era of accessibility: Evaluating rules for scripting audio representation of test items

Standardized, large-scale assessment of educational outcomes has become a global phenomenon over the past three decades (Smith, 2016 ). A key challenge facing assessment designers is that standard formats may be inaccessible or may create barriers to student performance. Schwanke, Smith, and Edyburn's ( 2001 ) A3 model describes how advocates have reacted to structural barriers by providing accommodations and, ultimately, accessibility. This paper synthesizes and evaluates three studies that attempted to improve accessibility in assessments for students who struggle with print reading through audio presentation of assessment items. Cross-study implications for policy and practice are considered.     

Evaluating item response theory linking and model fit for data from PISA 2000–2012

ABSTRACT

Based on concerns about the item response theory (IRT) linking approach used in the Programme for International Student Assessment (PISA) until 2012 as well as the desire to include new, more complex, interactive items with the introduction of computer-based assessments, alternative IRT linking methods were implemented in the 2015 PISA round. The new linking method represents a concurrent calibration using all available data, enabling us to find item parameters that maximize fit across all groups and allowing us to investigate measurement invariance across groups. Apart from the Rasch model that historically has been used in PISA operational analyses, we compared our method against more general IRT models that can incorporate item-by-country interactions. The results suggest that our proposed method holds promise not only to provide a strong linkage across countries and cycles but also to serve as a tool for investigating measurement invariance.     

大学英语课外阅读的情感处理策略

国内外语言学家的研究表明,大量课外阅读有助于外语语言水平的提高,而情感因素是影响外语学习的重要因素之一。通过对某院校三个文科班级非英语专业本科生的实验研究,采用情感量表和阅读测试等研究工具,探求从情感维度对英语课外阅读进行指导性和任务性阅读,对学生英语学习情感的影响以及与阅读成绩之间的关系。结果表明。从情感维度对英语课外阅读材料进行处理有助于学生对英语阅读的积极情感体验;对学生进行指导性和任务性课外阅读,对阅读成绩有一定积极作用;在不同的阅读任务下,学生的英语阅读情感认识以及阅读成绩存在显著性差异。     

The relationship between strategic reading instruction,student learning of L2‐based reading strategies and L2 reading achievement

This study investigates the relationship between strategic reading instruction, the process of learning second language‐based reading strategies and English reading achievement for Thai university students of science and technology. In a course in reading general English texts for 16 weeks, 82 students were taught using a strategies‐based approach (experimental cohort), whereas another 82 students were taught using a traditional, teacher‐centred approach (control cohort). A pre‐test/post‐test research design was employed, and a portfolio approach was used to investigate the experimental cohort students' process of learning reading strategies. The results showed that the experimental cohort outperformed the control cohort in the post‐course standardised English test. The higher‐level reading proficiency learners in the experimental cohort were also found to be better than their low‐level peers at learning to use second language‐based reading strategies effectively. Implications are discussed for alternative instructional practice of reading in English as a foreign language (EFL).     

基于AWL的大学学术英语词汇与阅读教学

以Coxhead学术英语词汇表(AWL)为基础,探讨基于语料库方法的词汇分类方法,以及学术英语词汇表与学术阅读能力之间的关系。结合目前大学英语课程设置和教学的实际情况,提出开设大学学术英语系列课程,指导大学生通过英语学习促进其专业能力的发展。作为学术英语课程的先行课程,大学学术英语阅读课程将学术英语词汇作为积极词汇列入教学重点,并结合学术词汇、句法、语篇多个层面进行教学内容设计。     

Structural validity and invariance of the Feedback Perceptions Questionnaire

Despite a growing interest in instructional feedback, students’ feedback perceptions received limited attention. We examined the structural validity and measurement invariance of the Feedback Perceptions Questionnaire (FPQ). The FPQ measures feedback perceptions in terms of perceived fairness, usefulness, acceptance, willingness to improve, and affect. Secondary school students (N = 1486) received a fictional scenario containing Concise General Feedback or Elaborated Specific Feedback by a fictional peer. Students rated their perceptions as if they had received the feedback themselves. Confirmatory Factor Analysis (CFA) supports the structural validity of the FPQ and its invariance for the two types of peer feedback, gender, four grade-levels and two tracks. Perceived fairness of peer feedback was a strong positive predictor of willingness to improve and affect, whereas perceived usefulness and acceptance of peer feedback showed a more complex pattern in predicting willingness to improve and affect.