“How do You Know this Answer?” – Children’s Use of Text Data and General Knowledge in Story Comprehension

This study investigates how, and to what extent, young readers (7–8 year-olds) use text information or their prior knowledge when answering comprehension questions about narrative texts. The children were asked to explain how they found out their responses by answering the following question: “how do you know this answer?” Their answers and justifications were analysed both qualitatively and quantitatively. The text proved to be the main source of information for these readers. However, sometimes the children seemed to ignore the text and over rely on their prior knowledge to answer the questions. The procedure of asking children to justify their answers was shown to be a good way of specifying more precisely some of their problems in text comprehension. It also seemed to encourage them to look back at the text and review their responses and, as such, it could be considered a useful tool to improve children’s reading comprehension.     

The role of paraphrased adjunct questions in facilitating learning from prose

Twenty-six college student read a 5000-word prose passage and answered either verbatim or paraphrased inserted questions while reading about international communication. Students either were or were not permitted to review the text while answering the inserted questions. On a subsequent post-test containing new paraphrases of the adjunct questions, students who received paraphrased adjunct questions outperformed students who received verbatim adjunct questions. This result supported the contention that paraphrased adjunct questions could facilitate meaningful learning from prose.     

如何从宽容的角度解读《老子》

在《老子》中,无论是直接的宽容思想如“报怨以德”、“不责于人”,“受国之垢”、“辅万物”,还是间接提供的人性基础和思想发源,均给我们重新思考个体自主和整体有序的统一提供了独特启示：如何在彰显个体权利的同时又能相互认同、相互宽谅？诸多有权利的个体在共处一体时该如何保持整体稳定和公共秩序？对这些问题的回答,不仅可以看到老子所具有的拓宽西方狭隘宽容理论的价值,而且还能拓展《老子》文本当今的创造空间。     

Learning from text: knowing the test format enhanced metacognitive monitoring

In an experiment with 56 young adults, the hypothesis was tested that information about the format of an anticipated test improves metacognitive monitoring. Half of the participants were informed about the format of the test before they started studying a text about human genetics. The other half of the sample received the same information after studying the text. All participants then answered 15 true-false inference items about the contents of the text and judged their confidence in the correctness of each answer. Whereas experimental and control group did not differ in the number of correct answers, the confidence judgments in the experimental group were more accurate and discriminated better between correct and incorrect answers than the control participants’ judgments. Furthermore, the informed participants’ discrimination performance correlated positively with their domain-related prior knowledge. The results extend earlier findings concerning the role of the test format for monitoring processes.     

Are sophisticated students always better? The role of topic-specific personal epistemology in the understanding of multiple expository texts

In a sample of 135 Norwegian education undergraduates, we examined the effects of topic-specific epistemic beliefs concerning the simplicity and source of knowledge on deep-level understanding of multiple expository texts about the same topic—climate change. The results showed that students holding sophisticated simplicity beliefs, viewing knowledge about climate change as complex, gained better multiple-text understanding than did students holding naïve simplicity beliefs, viewing knowledge about climate change as simple. However, students holding sophisticated source beliefs, viewing knowledge about climate change as personal construction, performed poorer than did students holding naïve source beliefs, viewing knowledge about climate change as transmitted from experts. Moreover, students believing knowledge to be complex and, simultaneously, relying on expert authors were at a particular advantage with respect to multiple-text understanding. Thus, in this complex reading-task context, source beliefs usually located at the sophisticated ends of epistemic belief continuums turned out to be maladaptive, presumably because they distracted from the building of a high-quality representation of author and text meaning.     

Oral versus written assessments: a test of student performance and attitudes

Student performance in and attitudes towards oral and written assessments were compared using quantitative and qualitative methods. Two separate cohorts of students were examined. The first larger cohort of students (n = 99) was randomly divided into ‘oral’ and ‘written’ groups, and the marks that they achieved in the same biology questions were compared. Students in the second smaller cohort (n = 29) were all examined using both written and oral questions concerning both ‘scientific’ and ‘personal development’ topics. Both cohorts showed highly significant differences in the mean marks achieved, with better performance in the oral assessment. There was no evidence of particular groups of students being disadvantaged in the oral tests. These students and also an additional cohort were asked about their attitudes to the two different assessment approaches. Although they tended to be more nervous in the face of oral assessments, many students thought oral assessments were more useful than written assessments. An important theme involved the perceived authenticity or ‘professionalism’ of an oral examination. This study suggests that oral assessments may be more inclusive than written ones and that they can act as powerful tools in helping students establish a ‘professional identity’.     

Schema effects in EFL reading comprehension

Schemata are known to play an important a role in reading comprehension. Comprehension is seen as the interaction between top-down processing from activated schemata and bottom-up processing from concepts expressed by the text. If readers activate an inappropriate schema, they may miss the meaning of the text. The present study examines the effects of faulty schemata on reading comprehension. At the end of an advanced English reading course at Haifa University, a test of reading comprehension was administered to 125 students. One section of the test contained an advanced level text about love and marriage, a text close to the personal experience of the examinees. Using dictionaries, students translated expressions and sentences and answered short-answer comprehension questions in English. Results indicated that 23% of the wrong answers to the comprehension questions were driven by schemata which differed substantially from the actual content of the text.     

Description and interactions of informative text structure knowledge and skills of French-speaking Grade 6 students

This study was conducted in the province of Québec, Canada, among French-speaking Grade 6 students (n?=?175) in the context of a school curriculum that does not clearly address text structure and main idea instruction. It aims to understand whether these students can identify informative text structures and main ideas in isolated paragraphs, comprehend main ideas and text structure in an informative text, and write a short structured informative text. It also describes relationships between these knowledge and skills coming from different reading and writing tasks. Three assessments relative to informative text structures were administered: a multiple-choice test on text structure knowledge and identification of main ideas, a reading comprehension test, and a short writing task. Results revealed that students performed better in the multiple-choice assessment compared to other assessments. Correlations between variables stemming from the three assessments were significant but their effect sizes were low to moderate. A hypothesized model was investigated via a path analysis suggesting that structure knowledge and main idea identification influence reading comprehension, which then influence writing.     

Locating information within extended hypermedia

New literacies researchers have identified a core set of strategies for locating information, one of which is “reading a Web page to locate information that might be present there” (Leu et al. in: Rush, Eakle, Berger (eds) Secondary school reading and writing: What research reveals for classroom practices, 2007, p. 46). Do middle-school, high school, and undergraduate students (N = 51) differ in effectiveness at locating information within extended hypermedia? Students completed a pretest measure of knowledge about the circulatory system. They then gave verbal answers to 10 researcher-developed questions about the circulatory system, which they answered by searching the environment and thinking aloud about the task. Consistent with large-scale national and international studies, students were only moderately successful at locating information. Successfully locating information was significantly associated with having more prior knowledge, efficient searching, and giving better quality answers to the researcher-posed questions. It was also associated with specific strategies only at the level of individual questions. That is, the “ideal” strategy depended on the question and how the answer was phrased in the text. Implications of the results for teaching students how to search in hypermedia are offered.     

Genre effects on higher education students' text reading for understanding

This study provides empirical support for the proposalthat student learning in higher education is affectednot only by prior subject knowledge and by approachesto learning but also by ability to deal with textgenre. It first explores how 22 higher degreestudents on a course in education understood selectedtext extracts from different genres when reading attheir own pace under conditions expected to promotethorough understanding. After reading a text eachstudent wrote answers to a set of questions abouttheir reading and understanding of the text. Intape-recorded discussion with another reader of thesame text they then looked for similarities anddifferences between their responses. They laterreported on their feelings about the task and text. Understandings were found to vary between readers ofthe same text, difference being largely attributed bythe students to difficulty with the language andstructure of the texts in terms which pointed to thegenre. Considerable anxiety about difficulty anddifference in understanding was partly allayed throughdiscussion, but understanding was not much furthered. Most students felt that more work on the texts wouldbe fruitful.In a second part of the study a further 39 studentsfrom a new intake to the same course were asked abouttext types in their recommended and actual coursereading. The types they reported as most frequentlyrecommended were also felt to be the most difficultfor reasons to do with text genre. They were alsothose from which the earlier text extracts had beendrawn. Theoretical and practical issues arediscussed.     

Assessing comprehension during reading with the Reading Strategy Assessment Tool (RSAT)

Comprehension emerges as the results of inference and strategic processes that support the construction of a coherent mental model for a text. However, the vast majority of comprehension skills tests adopt a format that does not afford an assessment of these processes as they operate during reading. This study assessed the viability of the Reading Strategy Assessment Tool (RSAT), which is an automated computer-based reading assessment designed to measure readers’ comprehension and spontaneous use of reading strategies while reading texts. In the tool, readers comprehend passages one sentence at a time, and are asked either an indirect (“What are your thoughts regarding your understanding of the sentence in the context of the passage?”) or direct (e.g., why X?) question after reading each pre-selected target sentence. The answers to the indirect questions are analyzed on the extent that they contain words associated with comprehension processes. The answers to direct questions are coded for the number of content words in common with an ideal answer, which is intended to be an assessment of emerging comprehension. In the study, the RSAT approach was shown to predict measures of comprehension comparable to standardized tests. The RSAT variables were also shown to correlate with human ratings. The results of this study constitute a “proof of concept” and demonstrate that it is possible to develop a comprehension skills assessment tool that assesses both comprehension and comprehension strategies.     

Not Read,but Nevertheless Solved? Three Experiments on PIRLS Multiple Choice Reading Comprehension Test Items

Multiple-choice (MC) reading comprehension test items comprise three components: text passage, questions about the text, and MC answers. The construct validity of this format has been repeatedly criticized. In three between-subjects experiments, fourth graders (N ₁ = 230, N ₂ = 340, N ₃ = 194) worked on three versions of MC items from the Progress in International Reading Literacy Study 2001 reading comprehension test with relevant components successively deleted: “original version” (text, questions, MC-answers), “version without text” (questions, MC-answers), “version without text and without questions” (only MC-answers). Answering correctly the MC items became more difficult as the relevant information was eliminated. In the two narrative fictional texts presented, the students' performance of the version without text was not better than chance. Conversely in the informational (fictional) text, the students' performance of the version without text was better than chance. In the third condition, students' performance was never better than chance.     

Fostering topic knowledge: essential for academic writing

Several researchers emphasize the role of the writer’s topic knowledge for writing. In academic writing topic knowledge is often constructed by studying source texts. One possibility to support that essential phase of the writing process is to provide interactive learning questions which facilitate the construction of an adequate situation model by initiating macro-strategies. In order to examine whether the provision of interactive learning questions during studying source texts leads to better results in academic writing both writing process and performance of a group supported by interactive learning questions was compared to a study-only group which read the source texts without learning questions. Results revealed that students provided with interactive learning questions wrote longer essays and spend significantly more time prewriting and writing/revising their essays than did the students of the study-only group. Studying source texts with learning questions resulted in text products of better readability and partly better accuracy and coverage of content. These findings suggest that engaging students in answering learning questions when reading source texts can positively affect both writing process and performance.     

Exploring the effect of background knowledge and text cohesion on learning from texts in computer science

In this study, we examine the effect of background knowledge and local cohesion on learning from texts. The study is based on construction–integration model. Participants were 176 undergraduate students who read a Computer Science text. Half of the participants read a text of maximum local cohesion and the other a text of minimum local cohesion. Afterwards, they answered open-ended and multiple-choice versions of text-based, bridging-inference and elaborative-inference questions. The results showed that students with high background knowledge, reading the low-cohesion text, performed better in bridging-inference and in elaborative-inference questions, than those who read the high-cohesion text. Students with low background knowledge, reading the high-cohesion text, performed better in all types of questions than students reading the low-cohesion text only in elaborative-inference questions. The performance with open-ended and multiple-choice questions was similar, indicating that this type of question is more difficult to answer, regardless of the question format.     

Only Familiar Information is a “Curse”: Children’s Ability to Predict What Their Peers Know

The ability to make inferences about what one’s peers know is critical for social interaction and communication. Three experiments (n = 309) examined the curse of knowledge, the tendency to be biased by one’s knowledge when reasoning about others’ knowledge, in children’s estimates of their peers’ knowledge. Four- to 7-year-olds were taught the answers to factual questions and estimated how many peers would know the answers. When children learned familiar answers, they showed a curse of knowledge in their peer estimates. But, when children learned unfamiliar answers to the same questions, they did not show a curse of knowledge. These data shed light on the mechanisms underlying perspective taking, supporting a fluency misattribution account of the curse of knowledge.     

‘Can you get pregnant when u r in the pool?’: young people's information seeking from a sexual health text line

Young people have questions about sex and development but may have trouble getting answers to them. Text messaging services can serve as a resource. This study analysed 1351 text messages sent to a sexual health text message service designed for young people in North Carolina to determine the types of questions asked of a confidential, accurate source of information. The majority of texts sought information about sexual health topics (89%). A smaller proportion asked for advice (7%) or reassurance (4%) that the sender was developing normally. Questions mainly asked about sexual acts (33.9%), unplanned pregnancy (20.2%), contraception (13.7%), physical or sexual development (12.9%) and sexually transmitted diseases (10.8%). Findings suggest that a confidential and accurate sexual health text message service could be an important resource for young people, especially if they seek advice before engaging in risky sexual behaviour.     

Content,domain, and word knowledge: Relationship to comprehension of narrative and expository text

The influence of background knowledge on reader's comprehension of text has become a fundamental assumption associated with text processing. Background knowledge has been represented in a variety of constructs including specific text knowledge, general world knowledge, discourse knowledge, and domain knowledge. Our interest in the present study was to examine background knowledge and strategy knowledge from a broad view to better understand how they contribute to elementary-age students' reading comprehension of both narrative and expository texts. Constructs of domain, content, and word knowledge were developed based on the extant literature. These knowledge constructs were measured with researcher-developed tests consisting of both narrative and expository passages ranging from 1000 to 2000 words (2–4 pages) in length. Comprehension of each passage was measured with a 25-item subtest. All students in Grades 3 through 6 were administered the knowledge and comprehension tests. A two-stage analysis procedure was used. Exploratory factor analysis was used to examine empirically the internal structures of the comprehension questions. Factor scores were constructed and then treated as a multivariate set, with a second set comprised of the background and strategy factors. A canonical analysis was then performed to relate the two sets and canonical functions and structure coefficients were interpreted substantively. Our findings indicated that background knowledge specific to the content of the text being read begins to diminish in importance at about Grade 4. Strategy knowledge and use in comprehension of text begin to play a more important role at Grade 4 and above. Grade 6 analyses support a conclusion that the reading development process in elementary grades has been completed, and that reading comprehension at this grade is primarily an indicator of students' proficiency in selecting efficient ways to gain meaning from text.     

Early prediction of student knowledge in game-based learning with distributed representations of assessment questions

Game-based learning environments hold significant promise for facilitating learning experiences that are both effective and engaging. To support individualised learning and support proactive scaffolding when students are struggling, game-based learning environments should be able to accurately predict student knowledge at early points in students' gameplay. Student knowledge is traditionally assessed prior to and after each student interacts with the learning environment with conventional methods, such as multiple choice content knowledge assessments. While previous student modelling approaches have leveraged machine learning to automatically infer students' knowledge, there is limited work that incorporates the fine-grained content from each question in these types of tests into student models that predict student performance at early junctures in gameplay episodes. This work investigates a predictive student modelling approach that leverages the natural language text of the post-gameplay content knowledge questions and the text of the possible answer choices for early prediction of fine-grained individual student performance in game-based learning environments. With data from a study involving 66 undergraduate students from a large public university interacting with a game-based learning environment for microbiology, Crystal Island , we investigate the accuracy and early prediction capacity of student models that use a combination of gameplay features extracted from student log files as well as distributed representations of post-test content assessment questions. The results demonstrate that by incorporating knowledge about assessment questions, early prediction models are able to outperform competing baselines that only use student game trace data with no question-related information. Furthermore, this approach achieves high generalisation, including predicting the performance of students on unseen questions.

Practitioner notes

What is already known about this topic

• A distinctive characteristic of game-based learning environments is their capacity to enable fine-grained student assessment.
• Adaptive game-based learning environments offer individualisation based on specific student needs and should be able to assess student competencies using early prediction models of those competencies.
• Word embedding approaches from the field of natural language processing show great promise in the ability to encode semantic information that can be leveraged by predictive student models.

What this paper adds

• Investigates word embeddings of assessment question content for reliable early prediction of student performance.
• Demonstrates the efficacy of distributed word embeddings of assessment questions when used by early prediction models compared to models that use either no assessment information or discrete representations of the questions.
• Demonstrates the efficacy and generalisability of word embeddings of assessment questions for predicting the performance of both new students on existing questions and existing students on new questions.

Implications for practice and/or policy

• Word embeddings of assessment questions can enhance early prediction models of student knowledge, which can drive adaptive feedback to students who interact with game-based learning environments.
• Practitioners should determine if new assessment questions will be developed for their game-based learning environment, and if so, consider using our student modelling framework that incorporates early prediction models pretrained with existing student responses to previous assessment questions and is generalisable to the new assessment questions by leveraging distributed word embedding techniques.
• Researchers should consider the most appropriate way to encode the assessment questions in ways that early prediction models are able to infer relationships between the questions and gameplay behaviour to make accurate predictions of student competencies.

     

The influence of question type,text availability,answer confidence and language background on student comprehension of an expository text

Reading is an essential activity for learning at university, but lecturers are not always experienced in setting appropriate questions to test understanding of texts. In other words, their assessments may not be ‘constructively aligned’ with the learning outcomes they hope their students to exhibit. In examination conditions, questions may be set with insufficient time for re-reading available texts, thus drawing more on students' powers of recall than on deeper learning and comprehension. Previous research has been undertaken on reading comprehension generally, but no research has yet explored the interaction of factors such as text availability (re-reading of texts), text layout, question type and respondents' language background. This study explores the correctness of 50 participants' responses to a set reading task based on an expository text, and participants' confidence in giving those answers, in relation to four factors: the effects of question type; text availability; text layout; and language background. The main findings are that non-native speakers of English have more difficulty and less confidence in answering implicit questions and that reviewing the text has a significant effect on response correctness for implicit questions. The form of text layout did not show a significant effect, however. Our results have implications for lecturers who set readings and questions for comprehension and others who use reading comprehension as part of their ‘hidden curriculum’. Further research in this area is required to determine more precisely the effects of language background.     

Scenario-Based Assessments in Writing: An Experimental Study

Writing from source text is critical for developing college-and-career readiness because it is required in advanced academic environments and many vocations. Scenario-based assessment (SBA) represents one approach to measuring this ability. In such assessment, the scenario presents an issue that the student is to read and write about. Before writing, lead-in exercises are presented to encourage the examinee to engage with the source materials and to model the process used in a classroom writing project. This study experimentally manipulated a middle-school assessment design to understand if (1) the lead-in/essay structure increased scores erroneously with a concomitant decrease in test technical quality, and (2) the presence of a single unifying scenario affected scores or score meaning. In general, the SBA design did not appear to artificially increase total-test or essay scores. As importantly, it functioned as well as, sometimes better than, the alternative designs in terms of the measurement characteristics examined.