Investigating the validity of two widely used quantitative text tools

In recent years, readability formulas have gained new prominence as a basis for selecting texts for learning and assessment. Variables that quantitative tools count (e.g., word frequency, sentence length) provide valid measures of text complexity insofar as they accurately predict representative and high-quality criteria. The longstanding consensus of text researchers has been that such criteria will measure readers’ comprehension of sample texts. This study used Bormuth’s (1969) rigorously developed criterion measure to investigate two of today’s most widely used quantitative text tools—the Lexile Framework and the Flesch–Kincaid Grade-Level formula. Correlations between the two tools’ complexity scores and Bormuth’s measured difficulties of criterion passages were only moderately high in light of the literature and new high stakes uses for such tools. These correlations declined a small amount when passages from the University grade band of use were removed. The ability of these tools to predict measured text difficulties within any single grade band below University was low. Analyses showed that word complexity made a larger contribution relative to sentence complexity when each tool’s predictors were regressed on the Bormuth criterion rather than their original criteria. When the criterion was texts’ grade band of use instead of mean cloze scores, neither tool classified texts well and errors disproportionally placed texts from higher grade bands into lower ones. Results suggest these two text tools may lack adequate validity for their current uses in educational settings.     

The relations among L1 (Spanish) literacy skills,L2 (English) language,L2 text reading fluency,and L2 reading comprehension for Spanish-speaking ELL first grade students

We investigated the relations of L2 (i.e., English) oral reading fluency, silent reading fluency, word reading automaticity, oral language skills, and L1 literacy skills (i.e., Spanish) to L2 reading comprehension for Spanish-speaking English language learners in the first grade (N = 150). An analysis was conducted for the entire sample as well as for skilled and less skilled word readers. Results showed that word reading automaticity was strongly related to oral and silent reading fluency, but oral language skill was not. This was the case not only for the entire sample but also for subsamples of skilled and less skilled word readers, which is a discrepant finding from a study with English-only children (Kim et al., 2011). With regard to the relations among L2 oral language, text reading fluency, word reading automaticity, reading comprehension, and L1 literacy skills, patterns of relations were similar for skilled versus less skilled word readers with oral reading fluency, but different with silent reading fluency. When oral and silent reading fluency were in the model simultaneously, oral reading fluency, but not silent reading fluency, was uniquely related to reading comprehension. Children's L1 literacy skill was not uniquely related to reading comprehension after accounting for other L2 language and literacy skills. These results are discussed in light of a developmental theory of text reading fluency.     

The effect of language specific factors on early written composition: the role of spelling,oral language and text generation skills in a shallow orthography

Spelling skills have been identified as one of the major barriers to written text production in young English writers. By contrast oral language skills and text generation have been found to be less influential in the texts produced by beginning writers. To date, our understanding of the role of spelling skills in transparent orthographies is limited. The current study addressed this gap by examining the contribution of spelling, oral language and text generation skills in written text production in Italian beginner writers. Eighty-three children aged 7–8 years participated in the study. Spelling, lexical retrieval, receptive grammar, and written sentence generation and reformulation skills were assessed and children were asked to write a text on a set topic. A factor analysis revealed that the children’s written text production was captured by three factors: productivity, complexity and accuracy. In contrast to results from children learning to write in opaque orthographies, such as English, this study showed that receptive grammar and written sentence generation skills accounted for significant variance in measures of productivity, complexity and accuracy in Italian children’s written text production. Spelling skills contributed to text accuracy and quality and explained more variance than receptive grammar in microstructural accuracy. By contrast, oral grammatical skills explained more variance in text quality than spelling. The current study shows the differential impact of language systems, such as Italian, on written text production. Implications for assessment and instruction are outlined.     

Prior knowledge,reading skill,and text cohesion in the comprehension of science texts

This study examined how text features (i.e., cohesion) and individual differences (i.e., reading skill and prior knowledge) contribute to biology text comprehension. College students with low and high levels of biology knowledge read two biology texts, one of which was high in cohesion and the other low in cohesion. The two groups were similar in reading skill. Participants' text comprehension was assessed with open-ended comprehension questions that measure different levels of comprehension (i.e., text-based, local-bridging, global-bridging). Results indicated: (a) reading a high-cohesion text improved text-based comprehension; (b) overall comprehension was positively correlated with participants' prior knowledge, and (c) the degree to which participants benefited from reading a high-cohesion text depended on participants' reading skill, such that skilled participants gained more from high-cohesion text.     

Validating Automated Measures of Text Complexity

Automated text complexity measurement tools (also called readability metrics) have been proposed as a way to help teachers, textbook publishers, and assessment developers select texts that are closely aligned with the new, more demanding text complexity expectations specified in the Common Core State Standards. This article examines a critical element of the validity arguments presented in support of proposed metrics: the claim that criterion text complexity scores developed from students’ responses to reading comprehension test items are reflective of the difficulties actually experienced by students while reading. Evidence that fails to support this assertion is examined, and implications relative to the goal of obtaining valid, unbiased evidence about the measurement properties of proposed readability metrics are discussed.     

Simplifying informational text structure for struggling readers

Direct instruction of reading strategies, such as the ‘structure strategy’, is demonstrated to be effective for the development of more mature and skilled reading processes in struggling readers. This instructional intervention approach, aimed at directly improving reading ability, can be used in combination with text simplification. Text simplification is the modification of the text in order to make it more understandable or readable for target groups of readers. In this article, we discuss a theoretically-driven text simplification approach, inspired by cognitive models of reading comprehension. Differently from classical approaches to linguistic text simplification, the aim of cognitive text simplification is not simply to reduce the linguistic complexity of the text, but to improve text coherence and the structure of information in the text. This can be achieved by using rhetorical devices, like signaling or discourse markers, which specify relationships among ideas at a global level (macrostructural) and work as processing instructions for the reader, scaffolding reading comprehension. The goal of this paper is to discuss, in light of the literature, the effectiveness of these adaptations for improving struggling readers’ understanding and learning from informational texts.     

Rethinking literacy: communication,representation and text

In this article I want to consider shifts in the use of the word ‘literacy’ and the implications for classroom work with texts, particularly the implications of the rapid and radical emergence of new relationships between different modes of representation and communication ( Kress, 2003; Raney, 1996; Unsworth, 2001 ). My concern is to argue that any approach to classroom literacy needs not only to recognise the new forms of text which children meet every day but to give multimodal texts a firm place in the curriculum. Further, if the text experience of young learners about these new combinations of modes of representation are to be realised in the classroom, then we need a framework for describing those texts. An approach which takes account of the rhetoric of design may be a way forward.     

The effects of inter-letter spacing in visual-word recognition: Evidence with young normal readers and developmental dyslexics

Recent research has demonstrated that slight increases of inter-letter spacing have a positive impact on skilled readers' recognition of visually presented words. In the present study, we examined whether this effect generalises to young normal readers and readers with developmental dyslexia, and whether increased inter-letter spacing affects the reading times and comprehension of a short text. To that end, we conducted a series of lexical decision and continuous reading experiments in which words were presented with the default settings or with a small increase in inter-letter spacing. Increased spacing produced shorter word identification times not only with adult skilled readers (Experiment 1), but also with young normal readers (Grade 2 and Grade 4 children; Experiment 2) and, even to a larger degree, with readers with dyslexia (Experiments 3 and 4). These experiments suggest that slight increases in inter-letter spacing would improve the readability of texts aimed at children, especially those with dyslexia.     

Career readiness: An analysis of text complexity for occupational reading materials

The Common Core State Standards (CCSS) and the broader college and career readiness agenda encourage educators, researchers, and other stakeholders to focus on preparing students for life after high school. A key emphasis is literacy, as the ability to read and comprehend written language is critical to success in college and careers. Understanding the level of reading comprehension needed for college and careers has important instructional implications. This study examined text complexity levels of various career texts using the Reading Maturity Metric and compared them to expectations in the CCSS. Text samples were selected for jobs from the five job zones in the Occupational Information Network database. Text complexity demands for all careers were generally in the CCSS range of college and career readiness and increased as job zone and required preparation increased. Results could provide specific career-related targets to make the CCSS reading requirements more relevant for students.     

Key skills for science learning: the importance of text cohesion and reading ability

To explore the importance of text cohesion, we conducted two experiments. We measured online (reading times) and offline (comprehension accuracy) processes for texts that were high and low cohesion. In study one (n?=?60), we manipulated referential cohesion using noun repetition (high cohesion) and synonymy (low cohesion). Students showed enhanced comprehension accuracy and faster comprehension responses for text that were high in referential cohesion. In study two (n?=?52), we examined connective text designs (‘because’, ‘and’ and ‘no connective’). Students demonstrated enhanced reading times for text using a ‘because’ connective. Additionally, we examined the individual differences (reading ability, science self-concept and self-esteem) as predictors of achievement with science comprehension tasks. Across both experiments reading ability predicted comprehension with both high (noun-repetition text and ‘and’ text) and low cohesion text (synonym text and ‘no connective’ text). These findings highlight the importance of good reading abilities and text cohesion for promoting science comprehension and learning.     

A study of readability of texts in Bangla through machine learning approaches

In this work, we have investigated text readability in Bangla language. Text readability is an indicator of the suitability of a given document with respect to a target reader group. Therefore, text readability has huge impact on educational content preparation. The advances in the field of natural language processing have enabled the automatic identification of reading difficulty of texts and contributed in the design and development of suitable educational materials. In spite of the fact that, Bangla is one of the major languages in India and the official language of Bangladesh, the research of text readability in Bangla is still in its nascent stage. In this paper, we have presented computational models to determine the readability of Bangla text documents based on syntactic properties. Since Bangla is a digital resource poor language, therefore, we were required to develop a novel dataset suitable for automatic identification of text properties. Our initial experiments have shown that existing English readability metrics are inapplicable for Bangla. Accordingly, we have proceeded towards new models for analyzing text readability in Bangla. We have considered language specific syntactic features of Bangla text in this work. We have identified major structural contributors responsible for text comprehensibility and subsequently developed readability models for Bangla texts. We have used different machine-learning methods such as regression, support vector machines (SVM) and support vector regression (SVR) to achieve our aim. The performance of the individual models has been compared against one another. We have conducted detailed user survey for data preparation, identification of important structural parameters of texts and validation of our proposed models. The work posses further implications in the field of educational research and in matching text to readers.     

The paradox of text in the culture of literacy

This paper invites a reconsideration of the divide between oral and literate cultural practices in the research on literacy. It begins by challenging the distinction which David Olson has made between text andutterance as one which overstates the separateness of literate and oral practices. To supplement Olson's position, the author describes the continuing importance of oral practices for the promotion of intellectual activities which are centred on texts. He does so, in part, by drawing on the literacy lessons of the Passover Haggadah. The Haggadah proves to be a text which celebrates the intellectual necessity of interruption and the ongoing need for interpretation, two critical practices firmly rooted in an oral regard for the word. This picture of an alternative literacy tradition is put forward as a possibility for a renewed regard for reading and for the text in the classroom.     

Students’ genre expectations and the effects of text cohesion on reading comprehension

This study raises the question what makes school texts comprehensible by analyzing whether students’ genre expectations about literary or expository texts moderate the impact of different forms of text cohesion on reading comprehension, even when the texts are similar regarding their genre. 754 students (Grade 9) from comprehensive schools read one of four text versions with similar content, but different degrees of local and global text cohesion. The four more or less cohesive texts were introduced as literary texts (part of a story) or as expository texts (newspaper article), although the different genres were only purported and the texts contained both literary and expository passages. Reading comprehension was assessed with multiple-choice-items, semi-open, and open-ended questions. Results demonstrate that global cohesion was profitable for reading comprehension with expository expectations, but not with literary ones. Local text cohesion and both forms of cohesion in combination did not interact with students’ genre expectations and had no main effect on comprehension. When students reading skills and prior knowledge were considered, the interaction was still apparent. Moreover, students with lower levels of reading skills tended to profit especially from texts with global cohesion, whereas the readers with higher reading skills achieved equal means in reading comprehension irrespective of the degree of global text cohesion. The findings are discussed with respect to theoretical aspects of text–reader-interactions, cognitive and emotional components of genre expectations, and the composition and instruction of comprehensible school texts.     

An Examination of Text Complexity as Characterized by Readability and Cohesion

To better understand dimensions of text complexity and their effect on the comprehension of adolescents, 103 high school seniors were randomly assigned to 4 groups. Each group read versions of the same 2 informational passages and answered comprehension test items targeting factual recall and inferences of causal content. Group A passages had a challenging readability level and high cohesion; Group B passages had an easier readability and low cohesion; Group C passages had a challenging readability level and low cohesion; and Group D passages had an easier readability and high cohesion. Students in Group D significantly outperformed students in Group C (g = 0.78). Although the effect sizes of comparisons among all groups ranged from g = 0.13 to 0.73, no other comparisons were statistically significant. Results indicate that adolescents’ reading comprehension is dually influenced by a text's readability and cohesion. Implications for matching readers to instructional text are discussed.     

What oral text reading fluency can reveal about reading comprehension

Text reading fluency – the ability to read quickly, accurately and with a natural intonation – has been proposed as a predictor of reading comprehension. In the current study, we examined the role of oral text reading fluency, defined as text reading rate and text reading prosody, as a contributor to reading comprehension outcomes in addition to decoding efficiency and language comprehension. One hundred and six Dutch primary school children from fourth grade participated in this study and were assessed on decoding efficiency, vocabulary, syntactic ability, reading fluency performance and reading comprehension skills. Regression analysis showed that text reading prosody, not text reading rate, explained additional variance in reading comprehension performance when decoding efficiency and language comprehension were controlled for. This result suggests that the inclusion of text reading prosody as an aspect of text reading fluency is justified and that a natural intonation is associated with better comprehension of what is read.     

Individual variation in children's reading comprehension across digital text types

The present study examined children's digital text comprehension of digital text types linear digital text vs hypertext, with or without graphical navigable overviews. We investigated to what extent individual variation in children's comprehension could be explained by lexical quality (word reading efficiency and vocabulary knowledge), cognitive load factors (prior knowledge and working memory), text type and graphical overview. Participants were 93 sixth graders in a within‐subject design. Word reading efficiency, vocabulary knowledge and prior knowledge predicted children's digital comprehension scores, while working memory did not. Reading comprehension was equal for linear text or hypertext. However, the presence of an overview facilitated reading comprehension for readers with lower prior knowledge. It can be concluded that hypertexts with basic digital text features and accompanying comprehension questions are not more difficult for children than linear digital texts, that similar individual factors predict reading comprehension of linear text and hypertext, and that a graphical overview helps when prior knowledge is low.     

Role of text and student characteristics in real‐time reading processes across the primary grades

Although much is known about beginning readers using behavioural measures, real‐time processes are still less clear. The present study examined eye movements (skipping rate, gaze, look back and second‐pass duration) as a function of text‐related (difficulty and word class) and student‐related characteristics (word decoding, reading comprehension, short term and working memory). Twenty‐four third and 20 fifth graders read a relatively easy (below grade level) and more difficult text (at grade level). The results showed that skipping rate mainly relied on text characteristics and a three‐way interaction of grade, text difficulty and word class. Gaze durations depended mostly on student characteristics. Results on look backs showed more and longer look backs in difficult texts. Finally, second‐pass duration mostly relied on grade level. To conclude, this study shows that both student and text characteristics should be taken into account when studying online text reading development.     

Exploring the effect of background knowledge and text cohesion on learning from texts in computer science

In this study, we examine the effect of background knowledge and local cohesion on learning from texts. The study is based on construction–integration model. Participants were 176 undergraduate students who read a Computer Science text. Half of the participants read a text of maximum local cohesion and the other a text of minimum local cohesion. Afterwards, they answered open-ended and multiple-choice versions of text-based, bridging-inference and elaborative-inference questions. The results showed that students with high background knowledge, reading the low-cohesion text, performed better in bridging-inference and in elaborative-inference questions, than those who read the high-cohesion text. Students with low background knowledge, reading the high-cohesion text, performed better in all types of questions than students reading the low-cohesion text only in elaborative-inference questions. The performance with open-ended and multiple-choice questions was similar, indicating that this type of question is more difficult to answer, regardless of the question format.     

Reading development in European Portuguese: relationships between oral reading fluency,vocabulary and reading comprehension

This study investigated direct and indirect effects between oral reading fluency, vocabulary and reading comprehension across reading development in European Portuguese. Participants were 329 children attending basic education, from grade 1 to grade 6. The results of path analyses showed that text reading fluency is much more dependent on the foundational skills of word recognition than reading comprehension, and the later, in turn, depends crucially on the specific constituent skill of text reading fluency. Text reading fluency has a significant influence on vocabulary from the beginning, but vocabulary contributed to reading comprehension only in more advanced grades. These results, obtained with an orthography of intermediate depth, are in line with the Simple View of Reading (SVR). However, they also highlight the importance of textual cues—besides the pivotal role of decoding—from the beginning of learning to read, which must be taken into account in the SVR.