Assessing comprehension during reading with the Reading Strategy Assessment Tool (RSAT)

Comprehension emerges as the results of inference and strategic processes that support the construction of a coherent mental model for a text. However, the vast majority of comprehension skills tests adopt a format that does not afford an assessment of these processes as they operate during reading. This study assessed the viability of the Reading Strategy Assessment Tool (RSAT), which is an automated computer-based reading assessment designed to measure readers’ comprehension and spontaneous use of reading strategies while reading texts. In the tool, readers comprehend passages one sentence at a time, and are asked either an indirect (“What are your thoughts regarding your understanding of the sentence in the context of the passage?”) or direct (e.g., why X?) question after reading each pre-selected target sentence. The answers to the indirect questions are analyzed on the extent that they contain words associated with comprehension processes. The answers to direct questions are coded for the number of content words in common with an ideal answer, which is intended to be an assessment of emerging comprehension. In the study, the RSAT approach was shown to predict measures of comprehension comparable to standardized tests. The RSAT variables were also shown to correlate with human ratings. The results of this study constitute a “proof of concept” and demonstrate that it is possible to develop a comprehension skills assessment tool that assesses both comprehension and comprehension strategies.     

Answering reading Comprehension Items Without Passages on the SAT-I,the ACT,and the GRE

Performance on the reading comprehension (RC) tasks of the Scholastic Assessment Test-I (SAT-I or the "new" SAT), the Enhanced American College Testing Assessment (ACT), and the Graduate Record Examination (GRE) when passages were missing was examined. For the SAT-I and ACT, scores were well above chance and correlated substantially with verbal score on the earlier version of the SAT, indicating that examinees perform similarly with or without passages. Comparable but weaker results were found for the GRE. The findings raise doubts about the construct validity of the RC task. We argue that performance is influenced by the plausibility of item choices with or without the passages and that this, in turn, is the result of the construction of test items with little knowledge of the underlying reading process.     

Scholastic Aptitude Test preparation for the adolescent dyslexic

Residual signs characteristic of a specific language disability, such as vocabulary deficits, trouble differentiating between literal and figurative words, difficulty paraphrasing a reading selection, and problems with abstract reasoning make it arduous for dyslexic adolescents to demonstrate their maximum potential on the verbal section of the Scholastic Aptitude Test. This paper illustrates ways in which SAT skills can be enhanced through a structured program of vocabulary development, an understanding of test taking strategies for specific SAT questions, and the learning of a pattern of analysis to improve reading comprehension. It includes case studies, as well as a discussion of the types of special test arrangements available for dyslexic students.     

A Description of What Happens When an Examinee Takes a Multiple-Choice Reading Comprehension Test

This study investigated the strategies a group of college students used to complete a portion of a standardized reading comprehension test. Twenty-six students were randomly assigned to either an introspective interview, in which the subjects explained to a researcher what they were doing and thinking as they read the test passages and answered the multiple-choice questions, or a retrospective interview, in which the students completed the test without interruption and then recounted for the researcher how they had gone about the task. Data analysis resulted in the identification of three broad categories of processing behavior: an overall approach to the test task, reading strategies, and test-taking strategies. In addition, difficulties encountered by the subjects were identified. Results indicate that the common element in each subject's approach to the test was a focus on getting to the questions as quickly as possible and then using the questions to direct a search of the passage to locate the best possible information to answer the questions. The implications of these results for better understanding the relationship between test-taking behavior and reading are discussed.     

An evaluation of two approaches for teaching reading comprehension strategies in the primary years using science information texts

There are few research studies on the effects of teaching comprehension strategies to young children in the primary grades. Using a Dominant–Less Dominant Mixed Model design employing both qualitative and quantitative data collection, we evaluated two approaches for teaching comprehension strategies to 7- and 8-year-old children in four second-grade classrooms using science information texts. The first approach focused upon explicitly teaching a series of single comprehension strategies, one-at-a-time (SSI). The second approach focused on teaching a “set” or “family” of transacted comprehension strategies within a collaborative, interactive and engaging routine (TSI). Results showed no difference between teaching young children a “set” of comprehension strategies and teaching comprehension strategies explicitly, one-at-a-time on their reading comprehension performance as measured by a standardized test of reading comprehension, recall of main ideas from reading two 200 word passages from information texts, a reading motivation survey and a strategy use survey. Results showed significant differences between students taught a set of comprehension strategies on measures of elaborated knowledge acquisition from reading science books (detail idea units recalled), retention of science content knowledge, and significantly improved criterion or curriculum-based reading comprehension test scores. These benefits favoring TSI over SSI are important because the learning curve is relatively steep for teachers to develop the ability to teach and for young children to develop the ability to coordinate a “set” of transacted comprehension strategies.     

Assessing Reading Comprehension: The Effects of Text-Based Interest,Gender, and Ability

Previous research has established that interest in a text is related to better comprehension and recall, but the impact of interest on comprehension in testing situations has not been widely investigated. This study used Hierarchical Linear Models (HLM) to examine the relationship between students' interest in reading passages and their performance on reading comprehension test items over those passages. The study involved 19,735 students in Grades 4 through 8. Stimuli consisted of 98 different reading passages. A small but significant relationship between interest and test performance was found, which was stronger for girls and for students of higher ability levels. A second HLM analysis was employed to explore whether certain passage characteristics were associated with higher or lower interest. The implications for test construction are discussed.     

Self-generated questions and reading recall: Does training help?

Several authors have recommended that reading teachers encourage students to ask questions while reading to improve comprehension and recall. This improvement seems to be influenced by the quality of the questions generated, and therefore it has been suggested that poor comprehenders be trained how to construct good questions before using this technique. Trained junior high school students were compared with untrained students on comprehension and recall after both were told to construct questions while reading text passages. Results indicate that training improved the question quality only for those students who had above average pretest free recall scores, although improvements in the question quality were directly related to improvements in reading recall. The implications of these results for teachers and for theories of the development of comprehension strategies are discussed.     

Examining the prediction of reading comprehension on different multiple‐choice tests

In this study, 180 Norwegian fifth‐grade students with a mean age of 10.5 years were administered measures of word recognition skills, strategic text processing, reading motivation and working memory. Six months later, the same students were given three different multiple‐choice reading comprehension measures. Based on three forced‐order hierarchical multiple regression analyses, results indicated that the unique contribution of measured skills and processes to performance varied across comprehension tests. In particular, when the test consisted of a longer passage, contained a larger proportion of inferential questions and was answered without access to relevant text passages, the relative importance of word recognition skills seemed to be reduced while working memory emerged as a relatively strong, unique positive predictor of comprehension performance. These findings have important practical implications for the assessment of reading comprehension.     

不确定判断与阅读多选题的猜测策略

本研究应用不确定判断理论,采用有声思维的方法,探讨受试在不确定的情况下如何猜测多项选择阅读题。研究发现受试采用了代表性、可得性、锚定与调整等搜索性策略。而锚定与调整策略最为有效,常被用作"锚"去调整猜测的有背景知识、常识、逻辑推理、应试型知识等。这一发现对命题有启示,有助于改善试题质量,降低猜测的命中率,提高考试效度。     

Does formative feedback on search behaviour help students in answering comprehension questions from an available text? / ¿Ayuda la retroalimentación formativa sobre el comportamiento de búsqueda a contestar preguntas de comprensión en lecturas con texto disponible?

This study analyses the efficacy of formative feedback to boost students’ search behaviour when answering comprehension questions in a with-text reading situation, which is a common reading situation in instructional and assessment settings. In these reading situations search strategies play an important role to predict students’ performance. Sixty-five high school students read two texts and answered eight multiple-choice comprehension questions per text using the software Read&Answer, which recorded all the students’ actions. After answering each question, students received either global-search-feedback or specific-search-feedback, which differed in the specificity of their information, or no-feedback. Participants who received any feedback had a second chance to correct their wrong answers. Specific-search-feedback increased students’ search decisions and improved their use of relevant information to repair wrong answers over global-search-feedback. Consequently, specific-search-feedback improved students’ performance when they corrected wrong answers over global-search-feedback. These results have implications for the design and implementation of formative feedback in computer-based systems aimed at improving students’ performance and teaching reading literacy skills.     

How Individual Differences Interact With Task Demands in Text Processing

Reading is affected by both situational requirements and one’s cognitive skills. The current study investigated how individual differences interacted with task requirements to determine reading behavior and outcome. We recorded the eye movements of college students, who differed in reading efficiency, while they completed a multiple-choice (MC) comprehension test in two within-subject conditions: one in which they read passages and answered MC questions as in a typical reading test and one in which they wrote a summary before answering the MC questions. We found students spent longer time reading the text in the summary-writing condition, resulting in a benefit in the time they spent when answering MC questions. This time benefit was larger for students who had relatively low reading efficiency. These results demonstrated that both task requirements and individual differences can interact to affect reading behavior and performance. Implications for reading practice and assessment are discussed.     

Schema effects in EFL reading comprehension

Schemata are known to play an important a role in reading comprehension. Comprehension is seen as the interaction between top-down processing from activated schemata and bottom-up processing from concepts expressed by the text. If readers activate an inappropriate schema, they may miss the meaning of the text. The present study examines the effects of faulty schemata on reading comprehension. At the end of an advanced English reading course at Haifa University, a test of reading comprehension was administered to 125 students. One section of the test contained an advanced level text about love and marriage, a text close to the personal experience of the examinees. Using dictionaries, students translated expressions and sentences and answered short-answer comprehension questions in English. Results indicated that 23% of the wrong answers to the comprehension questions were driven by schemata which differed substantially from the actual content of the text.     

Examining listening previewing as a classwide strategy to promote reading comprehension and vocabulary

Classwide instructional strategies to improve not only reading fluency but also comprehension and vocabulary knowledge are essential for student reading success. The current study examined the immediate effects of two classwide listening previewing strategies on reading comprehension and vocabulary knowledge. Twenty‐one, fourth‐grade general education students were exposed to three experimental conditions including a silent reading control condition, a listening previewing condition, and a listening previewing with vocabulary previewing condition. For all conditions, students read grade‐level passages, answered 10 comprehension questions, and completed a vocabulary‐matching task. Results showed that both listening previewing conditions led to improvements in comprehension as compared to silent reading. Adding a vocabulary previewing component to listening previewing procedures resulted in the highest levels of comprehension and vocabulary. Applied implications and directions for future research are discussed. © 2010 Wiley Periodicals, Inc.     

Listening Comprehension in Primary Level Grades Two,Four and Six

Aspects of listening comprehension skills in second-, fourth- and sixth-grade students (n=107) were investigated. The design employed the Sentence Verification Technique (SVT). Participants listened to three narrative and three expository passages, which had been recorded onto a CD, together with test sentences. Compared to the narrative passages, the expository material was more difficult to understand. Girls systematically scored higher than boys on all measures of listening comprehension although this sex difference was small and mostly nonsignificant. These results are comparable with previous findings in reading comprehension research. Age-related improvement in listening comprehension was observed with the narrative, but not with the expository passages. Cluster analysis produced four different comprehension strategies: Good Listening, Poor Listening, and two poor strategies previously hypothesised by Royer et al . (1987), viz. Top-down and Verbatim Strategies. The results are discussed from the point of view of their possible practical significance.     

The role of question type and reading ability in reading comprehension

Performance on a standardized reading comprehension test reflects the number of correct answers readers select from a list of alternate choices, but fails to provide information about how readers cope with the various cognitive demands of the task. The aim of this study was to determine whether three groups of readers: normally achieving (NA), poor comprehenders (CD), with no decoding disability, and reading disabled (RD), poor comprehenders with poor decoding skills, differed in their ability to cope with reading comprehension task demands. Three task variables reflected in the question-answer relations that appear on standardized reading comprehension tests were identified.Passage Independent (PI) question can be answered with reasonable accuracy based on the reader's prior knowledge of the passage content.Inference (INFER) questions required the reader to generate an inference at the local or global test level.Locating (LOCAT) questions require the reader to match the correct answer choice to a detail explicitly stated in the text either verbatim or in paraphrase form. The relations among reader characteristics, cognitive task factors and reading comprehension test scores were analyzed using a structural relations equation with LISREL. It was found that the three reading groups differed with respect to the underlying relationship between their performance on specific question-answer types and their standardized reading comprehension score. For the NA group, a high score on PI was likely to be accompanied by a low score on INFER, whereas in the CD and RD groups, PI and INFER are positively related. The finding of a negative relationship between background knowledge and inference task factors for normally achieving readers suggests that even normal readers may have comprehension difficulties that go undetected on the basis of a standardized scores. This study indicates that current comprehension assessments may not be adequate for assessing specific reading difficulties and that more precise diagnostic tools are needed.     

Exploring dynamic assessment as a means of identifying children at risk of developing comprehension difficulties

In this study, the authors explore a newly constructed dynamic assessment (DA) intended to tap inference-making skills that they hypothesize will be predictive of future comprehension performance. The authors administered the test to 100 second-grade children using a dynamic format to consider the concurrent validity of the measure. The dynamic portion of the assessment comprised teaching children to be "reading detectives" by using textual clues to solve what was happening in the story. During the DA children listened to short passages and answered three inferential questions (i.e., one setting, two causal). If children were unable to answer a question, they were reminded what a reading detective would do and given a set of increasingly concrete prompts and clues to orient them to the relevant portion of text until they could answer the question correctly. Results showed that the DA correlated significantly with a standardized measure of reading comprehension and explained a small but significant amount of unique variance in reading comprehension above and beyond vocabulary and word identification skills. In addition, results suggest that DA may be better than the standardized measure of reading comprehension at identifying intraindividual differences in young children's reading abilities.     

The Interaction of Reader and Task Factors in the Assessment of Reading Comprehension

This study examined the effects of selected reader and task variables on reading comprehension performance. Fifty deaf students and 61 hearing students of comparable reading skill level were blocked by cognitive style scores into field dependent or field independent groups. Subjects read 12 passages and completed selected-response and constructed-response question tasks under both lookback and no-lookback conditions. The passage questions tapped both text-explicit and text-implicit information. Several reader and task interaction effects were found to be significant, particularly for lookback conditions and constructed-response tasks. Moreover, cognitive style interacted with hearing state on tasks involving lookback options. Implications are drawn for further consideration of differential test administration and training in test-taking skills for certain types of readers.     

Silent versus oral reading comprehension and efficiency

Seventy‐four students read passages from an individually administered test of reading comprehension (a subtest from the Test of Dyslexia, a test of reading and related abilities currently in development; McCallum & Bell, 2001), and then answered literal and inferential questions. Students were randomly assigned to one of two conditions; 39 students read the passages silently and 35 read orally, with time recorded for each passage read. Comprehension and time were dependent measures for a Multivariate Analysis of Covariance (MANCOVA) and two follow‐up Analyses of Covariance (ANCOVA). After controlling for reading ability, results from the MANCOVA showed a significant combined effect ( p < .05); however, a comparison of mean reading comprehension scores showed no significant difference between silent readers and oral readers ( p > .05). On the other hand, with reading ability controlled, silent readers took significantly less time to complete passages compared to those who read orally ( p < .02). In fact, students took 30% longer to read orally than silently, on average. When test directions do not specify either oral or silent reading and error analysis is not a goal, testing will be more efficient via silent responding with no loss of comprehension. © 2004 Wiley Periodicals, Inc. Psychol Schs 41: 241–246, 2004.     

上下义关系理论与TEM-8阅读理解相关性研究

上下义关系作为一项重要的语义学理论对英语阅读理解能力的提高有着较强的理论指导作用。TEM-8阅读体裁决定了其篇章结构同上下义关系理论有着天然的契合之处。文章运用上下义关系理论中词汇的外延和内涵关系,通过收集分析2005年至2007年的TEM-8阅读真题,从词、句、段、篇章的层面上分析了上下义关系理论与TEM-8阅读理解的相关性,指出上下义关系作为一种科学解题的方法论,可以正确地指导考生猜测词义、概括段落主题句以及理解篇章,从而有效提高TEM-8阅读成绩。     

Interest,inferences, and learning from texts

Topic interest and learning from texts have been found to be positively associated with each other. However, the reason for this positive association is not well understood. The purpose of this study is to examine a cognitive process, inference generation, that could explain the positive association between interest and learning from texts. In Study 1, sixty undergraduate students participated by reading two science texts, which differed in coherence levels, silently. The results replicated previous findings that topic interest is positively associated with recall and accurate answers to comprehension questions for both texts. In Study 2, sixty-nine undergraduate students participated by reading the same two science texts while thinking aloud. The results indicated that topic interest was positively associated with inference generation while reading for the more coherently-written text. Subsequent analyses indicated inference generation partly explained the positive association between topic interest and accurate answers to comprehension questions for the more coherently-written text. The findings from Study 2 were independent of the effects of reading comprehension skill. Theoretical implications of the findings, in regard to standards of coherence and depth of processing while reading, are discussed.