The influence of question type,text availability,answer confidence and language background on student comprehension of an expository text

Reading is an essential activity for learning at university, but lecturers are not always experienced in setting appropriate questions to test understanding of texts. In other words, their assessments may not be ‘constructively aligned’ with the learning outcomes they hope their students to exhibit. In examination conditions, questions may be set with insufficient time for re-reading available texts, thus drawing more on students' powers of recall than on deeper learning and comprehension. Previous research has been undertaken on reading comprehension generally, but no research has yet explored the interaction of factors such as text availability (re-reading of texts), text layout, question type and respondents' language background. This study explores the correctness of 50 participants' responses to a set reading task based on an expository text, and participants' confidence in giving those answers, in relation to four factors: the effects of question type; text availability; text layout; and language background. The main findings are that non-native speakers of English have more difficulty and less confidence in answering implicit questions and that reviewing the text has a significant effect on response correctness for implicit questions. The form of text layout did not show a significant effect, however. Our results have implications for lecturers who set readings and questions for comprehension and others who use reading comprehension as part of their ‘hidden curriculum’. Further research in this area is required to determine more precisely the effects of language background.     

Not Read,but Nevertheless Solved? Three Experiments on PIRLS Multiple Choice Reading Comprehension Test Items

Multiple-choice (MC) reading comprehension test items comprise three components: text passage, questions about the text, and MC answers. The construct validity of this format has been repeatedly criticized. In three between-subjects experiments, fourth graders (N ₁ = 230, N ₂ = 340, N ₃ = 194) worked on three versions of MC items from the Progress in International Reading Literacy Study 2001 reading comprehension test with relevant components successively deleted: “original version” (text, questions, MC-answers), “version without text” (questions, MC-answers), “version without text and without questions” (only MC-answers). Answering correctly the MC items became more difficult as the relevant information was eliminated. In the two narrative fictional texts presented, the students' performance of the version without text was not better than chance. Conversely in the informational (fictional) text, the students' performance of the version without text was better than chance. In the third condition, students' performance was never better than chance.     

The implications of content versus item validity on science tests

The use of content validity as the primary assurance of the measurement accuracy for science assessment examinations is questioned. An alternative accuracy measure, item validity, is proposed. Item validity is based on research using qualitative comparisons between (a) student answers to objective items on the examination, (b) clinical interviews with examinees designed to ascertain their knowledge and understanding of the objective examination items, and (c) student answers to essay examination items prepared as an equivalent to the objective examination items. Calculations of item validity are used to show that selected objective items from the science assessment examination overestimated the actual student understanding of science content. Overestimation occurs when a student correctly answers an examination item, but for a reason other than that needed for an understanding of the content in question. There was little evidence that students incorrectly answered the items studied for the wrong reason, resulting in underestimation of the students' knowledge. The equivalent essay items were found to limit the amount of mismeasurement of the students' knowledge. Specific examples are cited and general suggestions are made on how to improve the measurement accuracy of objective examinations.     

Staying with the text: the contribution of gender,achievement orientations,and interest to students’ performance on a literacy task

This study addresses concerns about boys’ underperformance on literacy tasks compared to girls, by investigating male and females students’ responses to narrative texts. Participants were 142 Grade 9 and 10 students. Achievement orientations, including goals, self‐efficacy, and self‐handicapping, were measured and approach and avoidance factors identified. Boys scored higher than girls on the avoidance factor. The task presented to participants involved reading sections from two narrative texts representing typically male or typically female reading interests. Interest ratings for the narrative text topics and interest levels while reading the texts were monitored, and students’ answers to multiple‐choice questions on text content recorded. Regression analyses confirmed that the influence of gender, achievement orientation, and on‐task text interest on reading performance varied with different patterns of task interest. The findings suggest that students’ achievement orientations and task interests are both important for understanding gender differences in students’ response to narrative texts.     

Effects of text-belief consistency and reading task on the strategic validation of multiple texts

In the comprehension of multiple controversial scientific texts, readers with strong prior beliefs tend to construct a one-sided mental representation that is biased towards belief-consistent information. In the present study, we examined whether an argument in contrast to a summary task instruction can increase the resource allocation to and strategic validation of belief-inconsistent information which should be positively related to comprehension. Undergraduate students read one belief-consistent and one belief-inconsistent text about a controversial scientific issue either with an argument or a summary task instruction. The use of strategic validation and memorization strategies was assessed with think-aloud protocols, and a verification task was used to investigate comprehension outcomes. As predicted, readers following a summary task read belief-consistent information longer and used more memorization strategies for such information. Readers following an argument task spent similar time reading both text types and used more validation strategies when reading the belief-inconsistent text. In addition, the use of strategic validation during reading the belief-inconsistent text improved comprehension for this text type but hindered the comprehension of the belief-consistent text.     

Optimising web site designs for people with learning disabilities

Much relevant internet‐mediated information is inaccessible to people with learning disabilities because of difficulties in navigating the web. This paper reports on the methods undertaken to determine how information can be optimally presented for this cohort. Qualitative work is outlined where attributes relating to site layout affecting usability were elicited. A study comparing web sites of different design layouts exhibiting these attributes is discussed, with the emphasis on methodology. Eight interfaces were compared using various combinations of menu position (vertical or horizontal), text size and the absence or presence of images to determine which attributes of a site have the greatest performance impact. Study participants were also asked for their preferences, via a ‘smiley‐face’ rating scale and simple interviews. ‘Acquiescence bias’ was minimised by avoiding polar (‘yes/no’) interrogatives, achieved by asking participants to compare layouts (such as horizontal versus vertical menu), with reasons coaxed from those able to articulate them. Preferred designs were for large text and images. This was the reverse of those facilitating fastest retrieval times, a discrepancy due to preferences being judged on aesthetic considerations. Design recommendations that reconcile preference and performance findings are offered. These include using a horizontal menu, juxtaposing images and text, and reducing text from sentences to phrases, thus facilitating preferred large text without increasing task times.     

Using a Multidimensional Differential Item Functioning Framework to Determine if Reading Ability Affects Student Performance in Mathematics

Many teachers and curriculum specialists claim that the reading demand of many mathematics items is so great that students do not perform well on mathematics tests, even though they have a good understanding of mathematics. The purpose of this research was to test this claim empirically. This analysis was accomplished by considering examinees that differed in reading ability within the context of a multidimensional DIF framework. Results indicated that student performance on some mathematics items was influenced by their level of reading ability so that examinees with lower proficiency classifications in reading were less likely to obtain correct answers to these items. This finding suggests that incorrect proficiency classifications may have occurred for some examinees. However, it is argued that rather than eliminating these mathematics items from the test, which would seem to decrease the construct validity of the test, attempts should be made to control the confounding effect of reading that is measured by some of the mathematics items.     

The inattentive on-screen reading: Reading medium affects attention and reading comprehension under time pressure

This study explored the influence of reading media and reading time-frame on readers' on-task attention, metacognitive calibration, and reading comprehension. One hundred and forty undergraduates were allocated to one of four experimental conditions varying on the reading medium (in print vs. on screen) and on the reading time-frame (free vs. pressured time). Readers' mindwandering while reading, prediction of performance on a comprehension test, and their text comprehension were measured. In-print readers, but not on-screen readers, mindwandered less on the pressured than in the free time condition, indicating higher task adaptation in print. Accordingly, on-screen readers in the pressured condition comprehended less than the other three groups. Mindwandering and text comprehension were similar under free reading time regardless of medium. Lastly, there were no differences in readers’ metacognitive calibration. The results support the hypothesis of shallow information processing when reading on screen under time constraints.     

Understanding metacognitive inferiority on screen by exposing cues for depth of processing

Paper-and-pencil learning and testing are gradually shifting to computerized environments. Cognitive and metacognitive researchers find screen inferiority compared to paper in effort regulation, test performance, and extent of overconfidence, in some cases, with unknown differentiating factors. Notably, these studies used reading comprehension tasks involving lengthy texts, which confound technology-related and cognitive factors. We hypothesized that the medium provides a contextual cue which leads to shallower processing on screen regardless of text length, particularly when task characteristics hint that shallow processing is legitimate. To test this hypothesis, we used briefly phrased yet challenging problems for solving on screen or on paper. In Experiment 1, the time frame for solving the problems was manipulated. As with lengthy texts, only time pressure resulted in screen inferiority. In Experiment 2, under a loose time frame, the same problems were now framed as a preliminary task performed before a main problem-solving task. Only the initial task, with reduced perceived importance, revealed screen inferiority similarly to time pressure. In Experiment 3, we replicated Experiment 1's time frame manipulation, using a problem-solving task which involved reading only three isolated words. Screen inferiority in overconfidence was found again only under time pressure. The results suggest that metacognitive processes are sensitive to contextual cues that hint at the expected depth of processing, regardless of the reading burden involved.     

Young Children's Attention to Textual Context when Reading

An attempt was made to measure children's attention to text when reading. Attention was measured by observing the children's ability to reflect on text during and after the reading task. Questions were asked to determine how far children had assimilated new information from the text. It was felt that perhaps children do not know why they have learned to read and regard competent performance rather than understanding or personal gain as the object of reading, especially when reading aloud to the teacher. Children were, therefore, also asked why they thought they had been taught to read. The possibility of a relationship between the degree of attention manifested and the number of purposes for reading nominated was also considered. The results revealed no relationship between individual's ability to attend to text and to nominate a greater number of purposes for learning to read. However, the evidence suggested that the ability to attend to text and the understanding of the uses of reading might both be improved.     

Eye-movement patterns and reader characteristics of students with good and poor performance when reading scientific text with diagrams

This study investigated the cognitive processes and reader characteristics of sixth graders who had good and poor performance when reading scientific text with diagrams. We first measured the reading ability and reading self-efficacy of sixth-grade participants, and then recorded their eye movements while they were reading an illustrated scientific text and scored their answers to content-related questions. Finally, the participants evaluated the difficulty of the article, the attractiveness of the content and diagram, and their learning performance. The participants were then classified into groups based on how many correct responses they gave to questions related to reading. The results showed that readers with good performance had better character recognition ability and reading self-efficacy, were more attracted to the diagrams, and had higher self-evaluated learning levels than the readers with poor performance did. Eye-movement data indicated that readers with good performance spent significantly more reading time on the whole article, the text section, and the diagram section than the readers with poor performance did. Interestingly, readers with good performance had significantly longer mean fixation duration on the diagrams than readers with poor performance did; further, readers with good performance made more saccades between the text and the diagrams. Additionally, sequential analysis of eye movements showed that readers with good performance preferred to observe the diagram rather than the text after reading the title, but this tendency was not present in readers with poor performance. In sum, using eye-tracking technology and several reading tests and questionnaires, we found that various cognitive aspects (reading strategy, diagram utilization) and affective aspects (reading self-efficacy, article likeness, diagram attraction, and self-evaluation of learning) affected sixth graders’ reading performance in this study.     

RESOLV: Readers' Representation of Reading Contexts and Tasks

We introduce RESOLV, a theoretical model to account for readers' construction and management of goals during text comprehension and use. RESOLV focuses on readers' experience of their physical, social, and communicative context prior to actually engaging with texts. RESOLV assumes that readers construct two types of mental models prior to reading: The context model is a representation of the physical and social reading context, whereas the task model is a set of goals and plans that drives readers' decisions and actions in reading. We first present the RESOLV model, and we articulate two core hypotheses. We then present and discuss evidence supporting these hypotheses, from past and more recent research conducted in our labs, as well as in others'. The data support the view that reading decisions and processes are guided by readers' perceptions and attributions regarding the task statement but also more implicit cues from the reading context.     

Genre effects on higher education students' text reading for understanding

This study provides empirical support for the proposalthat student learning in higher education is affectednot only by prior subject knowledge and by approachesto learning but also by ability to deal with textgenre. It first explores how 22 higher degreestudents on a course in education understood selectedtext extracts from different genres when reading attheir own pace under conditions expected to promotethorough understanding. After reading a text eachstudent wrote answers to a set of questions abouttheir reading and understanding of the text. Intape-recorded discussion with another reader of thesame text they then looked for similarities anddifferences between their responses. They laterreported on their feelings about the task and text. Understandings were found to vary between readers ofthe same text, difference being largely attributed bythe students to difficulty with the language andstructure of the texts in terms which pointed to thegenre. Considerable anxiety about difficulty anddifference in understanding was partly allayed throughdiscussion, but understanding was not much furthered. Most students felt that more work on the texts wouldbe fruitful.In a second part of the study a further 39 studentsfrom a new intake to the same course were asked abouttext types in their recommended and actual coursereading. The types they reported as most frequentlyrecommended were also felt to be the most difficultfor reasons to do with text genre. They were alsothose from which the earlier text extracts had beendrawn. Theoretical and practical issues arediscussed.     

Construct validity of the Nepalese school leaving english reading test

There has been a steady interest in investigating the validity of language tests in the last decades. Despite numerous studies on construct validity in language testing, there are not many studies examining the construct validity of a reading test. This paper reports on a study that explored the construct validity of the English reading test in the Nepalese school leaving examination. Eight students were asked to take the test and think-aloud, followed by retrospective interviews. Additionally, seven experts were asked to make judgments regarding the skills tested by the test. The findings provide grounded insights into students’ response behaviors prompted by the reading tasks, and indicate some threats to the construct validity of the test. Additionally, the study reports a low level of agreement among the experts, and a big gap between the skills used by the students and the skills that the experts thought were being examined by the test.     

Accuracy and Fluency in List and Context Reading of Skilled and RD Groups: Absolute and Relative Performance Levels

The purpose of this study was to examine (1) the performance levels and the magnitude of performance difference between students with reading disabilities (RD) and skilled readers when reading a typical classroom text; (2) the hypothesis that students with RD have specific difficulty using context in such a way that reading fluency is affected; and (3) whether RD subtypes may be differentiated according to performance on contextual and context‐free reading tasks. Two groups of fourth graders (85 skilled readers and 24 students with RD) completed a standardized test of reading comprehension, read aloud a folktale, and read aloud the folktale's words in a randomly sequenced list. Performance was scored as correct rate and percentage correct. Based on the number of words per idea unit in the passage, we also estimated the rate at which reader groups encountered and processed text ideas. Compared to the RD group, skilled readers read three times more correct words per minute in context, and showed higher accuracy and rates on all measures. Both context and isolated word‐reading rates were highly sensitive to impairment. We found no evidence for RD subtypes based on these measures. Results illustrate differences in reading levels between the two groups, the temporal advantage skilled readers have in linking text ideas, how word reading differs as a function of task format and performance dimension, and how limited word‐identification skills (not comprehension) produce contextual reading difficulties for students with RD.     

The role of question type and reading ability in reading comprehension

Performance on a standardized reading comprehension test reflects the number of correct answers readers select from a list of alternate choices, but fails to provide information about how readers cope with the various cognitive demands of the task. The aim of this study was to determine whether three groups of readers: normally achieving (NA), poor comprehenders (CD), with no decoding disability, and reading disabled (RD), poor comprehenders with poor decoding skills, differed in their ability to cope with reading comprehension task demands. Three task variables reflected in the question-answer relations that appear on standardized reading comprehension tests were identified.Passage Independent (PI) question can be answered with reasonable accuracy based on the reader's prior knowledge of the passage content.Inference (INFER) questions required the reader to generate an inference at the local or global test level.Locating (LOCAT) questions require the reader to match the correct answer choice to a detail explicitly stated in the text either verbatim or in paraphrase form. The relations among reader characteristics, cognitive task factors and reading comprehension test scores were analyzed using a structural relations equation with LISREL. It was found that the three reading groups differed with respect to the underlying relationship between their performance on specific question-answer types and their standardized reading comprehension score. For the NA group, a high score on PI was likely to be accompanied by a low score on INFER, whereas in the CD and RD groups, PI and INFER are positively related. The finding of a negative relationship between background knowledge and inference task factors for normally achieving readers suggests that even normal readers may have comprehension difficulties that go undetected on the basis of a standardized scores. This study indicates that current comprehension assessments may not be adequate for assessing specific reading difficulties and that more precise diagnostic tools are needed.     

2009年NMET阅读理解试题内容效度分析

文章依据高中英语教学大纲、课程标准和考试大纲,结合Bachman＆Palmer（1996）的任务特征框架,从语篇输入、预期回答和题项设计三个方面对2009年高考英语全国卷与各省市自主命题卷（NMET）（19套）阅读理解试题的内容效度进行分析。其结果表明,2009年NMET阅读理解试题总体符合两纲要求,并呈现体裁多样,覆盖面广,但篇章分布不均衡;文章题材涉及范围广,其中社会与文化类所占比例最大,其次是科普;阅读量多数符合大纲要求,但个别卷篇阅读量与大纲要求相比偏高或偏低;生词量符合大纲要求;篇章难度以中等题为主,但有些篇章选择缺乏梯度;考查技能全面,但集中于对理解文章具体信息的考查;题项设计合理,但还存在一些问题等特征。据此作者尝试为今后NMET阅读理解试题命制的改进提出意见与建议。     

Adults' prior exposure to print as a predictor of the legibility of text on paper and laptop computer

This study investigated the effects of variations in graphic elements that account for differences in speed and accuracy between reading text aloud from paper versus laptop computer. Variations in accurate reading-aloud performance are attributable to individual differences in the visual accessibility of information due to (1) the experimental manipulations of the independent variables, (2) the subjects' prior exposure to print within the culture, and (3) the educational attainment of the subject. A non-representative sample of 48 female survey interviewers (ages 38–72) were employed in the conduct of this study. Survey interviewers were selected because they gather information using laptop computers; the quality of the survey information collected may be directly associated with the legibility of computerized text on reader performance. Subjects completed a prior exposure to print questionnaire (Stanovich & West 1989) and a demographic data form (IDIQ). Repeated-measures analyses of variance were employed to examine individual differences in the speed and accuracy of reading-aloud performance for twenty-four conditions varying the levels of independent variables including: (1) Font (Times Roman, Helvetica, and Courier), (2) Justification (fully- justified versus left-justified), (3) Leading (single-spaced versus double-spaced), and (4) Mode of Presentation (paper versus laptop computer). Linear regression analyses found subjects' prior exposure to print significantly and positively related to predicting speed and miscue performance. The subjects' educational attainment significantly predicted miscue performance. Subjects' ages were not significantly related to performance. Results of this study inform computer programmers and designers who are responsible for developing standards and guidelines for legible computerized text for the effective access of accurate information.     

Types and sequences of self‐regulated reading of low‐achieving adolescents in relation to reading task achievement

This study examines the relationship between types and sequences of self‐regulated reading activities in task‐oriented reading with quality of task achievement of 51 low‐achieving adolescents (Grade 8). The study used think aloud combined with video observations to analyse the students' approach of a content‐area reading task in the stages of orientation, text reading and answering questions. Results show that in general these low‐achieving adolescents are infrequently monitoring their text comprehension or making connections with prior knowledge. Nevertheless, important differences are found between types and sequences of self‐regulated reading activities related to task achievement. The low‐achieving adolescents showing a straightforward linear approach to the task (orientation, reading of the whole text and finally answering of questions) yielded more success. In addition, readers demonstrating more activities directed at connections between text contents and prior knowledge during reading showed better task achievement. Implications of these findings for literacy education of this special group of adolescents are discussed.