Toward dynamic assessment of reading: applying metacognitive awareness guidance to reading assessment tasks

This paper focuses on research that aimed to provide a theoretical–practical framework to link literacy assessment practice with learning theory. An experimental study was designed with reference to three theoretical axes: ‘metacognitive awareness’ theory, ‘schema’ theory and the Vygotskian ‘zone of proximal fevelopment’. The study tested the effect of using written metacognitive awareness guidance (MCAG) as a tool for activating and engaging learners’Habits of Mind while processing authentic reading assessment tasks taken from Israeli kits of assessment tasks (Guterman, 2000). The study on 300 Grade–4 pupils used three modalities: a control group, which received no treatment; a placebo group, which received content instructions (CI); and a treatment group, which was given written MCAG. The findings confirmed that applying metacognitive awareness guidance to reading assessment tasks makes a difference in the learners’ level of performance and achievement on those tasks, and also increases learners’ chances of internalising the guidance components.     

Assessing Metacognitive Awareness

We constructed a 52-item inventory to measure adults′ metacognitive awareness. Items were classified into eight subcomponents subsumed under two broader categories, knowledge of cognition and regulation of cognition. Two experiments supported the two-factor model. Factors were reliable (i.e., α = .90) and inter-correlated (r = .54). Experiment 2 reported the knowledge of cognition factor was related to pre-test judgments of monitoring ability and performance on a reading comprehension test, but was unrelated to monitoring accuracy. Implications for educational assessment and future research were discussed.     

Structural enhancement of learning

Structural learning aids, such as interactive overviews (IOs), have previously been shown to facilitate text comprehension and recall. In this study, we examined the effects of structural aids on learners’ structural knowledge and their performance on a procedural transfer task. In Experiment 1, 90 college students were presented definitions of computer programming concepts in an IO that was structured alphabetically, randomly, or in accordance with experts’ structural knowledge. Learning was assessed by examining students’ acquired knowledge structures and performance on a procedural transfer task that required integration and elaboration of the learned material in ways that went beyond the strictly definitional information provided during training. Students taught with the expert-structured IO acquired better knowledge structures and performed better on the transfer task than those using the other IOs. Experiment 2, utilizing 101 college students, showed that the visible structure, rather than the navigational constraints, of the expert IO was largely responsible for this advantage.     

Comparison of Taiwanese and Canadian Students' Metacognitive Awareness of Science Reading,Text, and Strategies

This study used a Chinese-language version of the Index of Science Reading Awareness to explore the science reading metacognition and comprehension of Taiwanese students. Structural equation modelling results confirmed the underlying model comprised three clusters of metacognitive knowledge: beliefs and confidence in science reading, knowledge of structure of science text, and knowledge of science reading strategies. The main contribution of the current research was to provide evidence about the relationship between metacognitive awareness and comprehension of science text. In addition, data comparisons to Canadian (British Columbia) benchmarks from the original development revealed that metacognitive awareness of science reading deteriorated from elementary to middle school in both countries, and there were no significant differences of metacognitive awareness of science reading between Canadian and Taiwanese students. Instructional suggestions for raising students' metacognitive awareness on science reading were discussed.     

Scaffolding Peer-questioning Strategies to Facilitate Metacognition During Online Small Group Discussion

Meaningful discussion that facilitates reflective thinking can be initiated when learners raise thoughtful questions or provide critical feedback; however, generating effective questions requires a certain level of domain knowledge and metacognitive skills of the question-askers. We propose a peer-questioning scaffolding framework intended to facilitate metacognition and learning through scaffolding effective peer-questioning in online discussion. This framework assumes that novice students who lack domain and metacognitive knowledge can be scaffolded to generate meaningful interactions at an early stage of learning and the resulting peer-generated adaptive questions can facilitate learners’ metacognition. Thus, this study investigated the effects of providing online scaffolding for generating adaptive questions to peers during online small group discussion. A field experimental time-series control-group design was employed as a mixed model for the research design. Thirty-nine college students from an online introductory class on turfgrass management participated in the study. The findings revealed that the scaffolds were useful to increase the frequency of student questioning behavior during online discussion. For some students, the online guidance reportedly served as “a starting point” to generate questions when they had difficulty asking questions. However, the guidance did not improve the quality of questions and thus learning outcomes. The interview data indicated that peer-generated adaptive questions served a critical role in facilitating learner’s reflection and knowledge reconstruction. Further study should focus on the quality improvement of peer-generated questions while considering adaptive and dynamic forms of scaffolding and intermediate factors such as prior knowledge, metacognition, task complexity, and scaffolding type.     

Index of science reading awareness: An interactive-constructive model,test verification,and grades 4–8 results

The purpose of this study was to develop and verify the Index of Science Reading Awareness (ISRA) based on a model of an efficient, successful interactive-constructive science reader and three independent metacognitive awareness domains. Several researchers have noted the need for efficient, reliable, and valid measures of metacognition. ISRA data collected on 532 students (Grades 4–8) were analyzed using factor analyses, linear structural modeling, and analyses of variance (ANOVAs) to help verify the model and the test. The factor analyses and linear structural modeling indicated that these data did not support the assumption about the three independent metacognitive awareness domains, but suggested that the model and the test were structured around the design features of science reading, science text, and science reading strategies. One-way ANOVAs indicated significant, predicted reading ability and gender differences but unexpected grade-level differences. The composite metacognitive awareness data indicated that most Grade 4–8 students have surface knowledge about science reading, science text, and science reading strategies, and indicated specific targets for explicit science reading instruction. © 1998 John Wiley & Sons, Inc. J Res Sci Teach 35: 27-51, 1998.     

Components of reading comprehension and scholastic achievement

The aim of this study was to understand whether the reading comprehension process is better explained by a single or by multiple factors. 184 students (9 to 13 years old) were presented with a recently devised battery of tests, that measure ten aspects of reading comprehension. Structural equation modelling showed that a two factors model better accounts for the data compared to a one or a three factors model. Results confirmed the hypothesis which distinguishes between ‘basic’ and ‘complex’ aspects of reading comprehension. The second goal of the study was to analyze the relationship between the two-components model and scholastic achievement. Our results highlighted that the more ‘complex’ aspects of reading comprehension, reflecting some metacognitive knowledge and control processes, are the better predictors of scholastic achievement.     

元认知意识与英语阅读

基于对60名英语学习者的日记分析,发现许多学习者在英语阅读过程中学习者的主体知识、任务知识和策略知识。三个方面的元认知知识均有所认识。讨论了这些发现对英语教学的启示,认为教师应该帮助学习者提高英语阅读方面的元认知意识,从而达到培养学习者自主学习能力的目的。     

The benefits of metacognitive reading strategy awareness instruction for young learners of English as a second language

This paper presents a small‐scale study examining the effects of metacognitive reading strategy instruction on English language learners' reading comprehension in a Hong Kong international school. Twenty‐five primary school (Grade 5) students who learn English as a second language participated in this study. Metacognitive instruction was incorporated into 10 process‐based reading lessons. Data were collected from notes learners took during reading, post‐reading reflection reports, teacher‐facilitated group discussions and two types of reading tests. Results revealed that the young learners could articulate several knowledge factors that influenced their reading. In addition, learners reported a better understanding of the nature and demands of reading, a deeper awareness of metacognitive knowledge in improving reading comprehension and increased confidence in handling reading exercises. The learners also showed enhanced reading performance compared to those in a control group without metacognitive intervention. This study highlights the potential of metacognitive instruction to enhance primary school English learners' reading literacy.     

Using informed strategies for learning to enhance the reading and thinking skills of children with learning disabilities

The purpose of this study was to improve the reading performance of children with learning disabilities using a classroom-based metacognitive reading program. The participants were third- and fourth-level 8- and 9-year-olds (13 boys and 5 girls) at a school for children with learning disabilities. A cross-sectional time series design was used. During pre- and posttesting participants were assessed on two measures of strategy awareness and a measure of perceived self-competence. During the instructional phase the metacognitive reading program was introduced. Participants were assessed on measures of reading performance throughout the study. In general, the findings were positive: Students increased in performance and awareness of strategies from pre- to posttesting. In addition, participants were placed in subgroups based on pretesting assessments. The subgroups differed on their improvements over time. This study provides preliminary evidence that a metacognitive reading program can be used with children with learning disabilities to improve their awareness about reading and their comprehension skills.     

不同民族中学生英语阅读的元认知意识研究

采用英语阅读的元认知意识问卷对汉族、纳西族和彝族中学生进行测查。结果表明 :( 1 )3种民族中学生英语阅读的元认知意识水平随着年级的升高呈现出提高的趋势 ;( 2 )汉族与纳西族中学生英语阅读的元认知意识水平整体上具有一致性 ,彝族中学生英语阅读的元认知意识水平呈现出优于汉族和纳西族的趋势。基于此 ,教师在英语阅读教学中应加强对学生元认知意识的培养。     

The academic writing of community college remedial students: Text and learner variables

Community college remedial reading studentscomposed an informational report from sources,a task commonly assigned in higher educationthat integrally combines reading and writingprocesses. Productivity, use of source text,reproductions, accuracy, and inclusion of keyideas were analyzed as a function of twoexogenous variables, text density (high, low)and domain (health, business), and twoendogenous variables, prior knowledge andgeneral literacy skill. Text densityinteracted with literacy skill forproductivity, and with prior knowledge foraccuracy. Individuals with lower literacy orprior knowledge scores were at a disadvantagewith high but not low density source text. Domain affected productivity, use of source,and inclusion of key ideas: health text wasassociated with superior performance. Domaineffects may be explained by metacognitive ormotivational variables, or differences in thereadability of the source texts. The resultshave implications for instruction aimed atimproving community college students'preparedness for the academic demands of highereducation.     

Contribution of morphological awareness and lexical inferencing ability to L2 vocabulary knowledge and reading comprehension among advanced EFL learners: testing direct and indirect effects

Within the Structural Equation Modeling framework, this study tested the direct and indirect effects of morphological awareness and lexical inferencing ability on L2 vocabulary knowledge and reading comprehension among advanced Chinese EFL readers in a university in China. Using both regular z-test and the bootstrapping (data-based resampling) methods, the study found that morphological awareness contributed to L2 vocabulary knowledge directly and indirectly through the mediation of learners’ lexical inferencing ability. It was also observed that morphological awareness made no significant unique or direct contribution to reading comprehension after adjusting for vocabulary knowledge; its indirect effects on reading comprehension, however, were significant, both through the mediation of vocabulary knowledge alone, and the multiple mediations of lexical inferencing ability and vocabulary knowledge.     

非英语专业大学生英语阅读元认知意识培训的研究

文章通过英语阅读课程教学,从元认知知识、元认知体验和元认知调节等三个方面入手对111名大二学生进行了元认知意识的调查及培训,旨在探索建立适合中国大学生特点及认知策略习得一般规律的、切实可行的英语阅读的元认知意识培训模式,以丰富元认知理论,为英语阅读教学提供参考意见。     

Hypermedia design as learner scaffolding

A number of available resources offer guidance about hypermedia design strategies, many of which rely on principles of user-centered design. Many recent efforts, however, have focused more on developing learner-centered hypermedia. Learner-centered hypermedia is designed to help learners achieve their educational goals, rather than offer mere usability. Unfortunately, this endeavor is hamstrung by a lack of empirical research on the topic. Research conducted in my laboratory and others has provided some insight, however. It is now understood that several system and user characteristics influence outcomes of hypermedia-assisted learning (HAL). Among the most relevant factors are learners’ levels of metacognition and prior knowledge, and the interaction between these factors and hypermedia structure. By capitalizing on this research, it is possible to create hypermedia that scaffolds learners in their quest to build knowledge and understanding. The present article draws from empirical findings to suggest hypermedia design strategies aimed at scaffolding learners engaged in HAL. These guidelines target learners’ knowledge and metacognitive ability to structure hypermedia that maximizes learning potential.

Managing cognitive load during document-based learning

Designers of interactive learning environments face the issue of managing the learner's cognitive load, reducing irrelevant sources while optimizing useful sources of load. I propose a conceptual framework aimed at organizing the contributions of the papers presented in this special issue. The framework identifies three main dimensions, namely individual, task and environment, which may have specific or combined effects on the amount and type of cognitive load experienced during learning activities. I summarize some of the findings presented in the special issue with respect to each of these dimensions. Then I discuss some limitations of the studies and some perspectives for further research in the domain. I emphasize the need to control learners’ level of familiarity with the task setting and environment features, not just their prior knowledge of the content area, in order to obtain reliable assessments of cognitive load and learning outcomes.     

Teaching Phonological Awareness and Metacognitive Strategies to Children with Reading Difficulties: A comparison of two instructional methods

This paper describes an applied training study which investigated the differential effects of two instructional methods on the reading performance of primary school children with reading difficulties. Sixty five children aged 7-10 participated. Twenty five children were assigned to each of two experimental groups: direct instruction in phonological awareness and the alphabetic principle, or the same direct instruction in phonological awareness in conjunction with training in specific metalinguistic concepts and metacognitive strategies. Fifteen children were selected as controls. Reading performance from baseline was measured within a pretest, posttest, and delayed posttest experimental design. Data and results are presented on 60 children owing to attrition. Results showed that direct instruction in phonological awareness improved the reading performance of children with reading difficulties over time. However, direct instruction in phonological awareness in conjunction with explicit training in specific metalinguistic concepts and metacognitive strategies was more advantageous overall.     

Morphological awareness and learning to read Chinese

This study investigated the nature of morphological awareness and its relation to learning to read Chinese characters among 46 Chinese-speaking preschool children. The children took a morphological awareness task, which varied in semantic transparency and morpheme position. Children’s vocabulary knowledge and extant character reading ability were measured. Additionally, a character learning task was administered. Results showed that children’s performances on morphological awareness were affected by semantic transparency but not by morpheme position. Morphological awareness was related to vocabulary knowledge when partialling out character reading ability but not to character reading ability after partialling out vocabulary knowledge. The results of the character learning task further revealed that morphological awareness was related to character identification in the words that were just taught but not to character identification in the words that were not taught or in pseudowords. The relation between morphological awareness and character identification ceased to be significant when partialling out the variance in children’s prior knowledge of the characters to be learnt. Taken together, the findings suggested that vocabulary knowledge may play a more important role than reading ability in the initial development of morphological awareness and that the facilitative effect of morphological knowledge in reading does not seem to be significant in the very initial stages of reading acquisition.     

Deaf students’ metacognitive awareness during language comprehension

This study explored deaf and hearing university students’ metacognitive awareness with regard to comprehension difficulties during reading and classroom instruction. Utilising the Reading Awareness Inventory (Milholic, V. 1994. An inventory to pique students’ metacognitive awareness of reading strategies. Journal of Reading 38: 84–6), parallel inventories were created to tap metacognitive awareness during comprehension of sign language (deaf students) and spoken language (hearing students). Overall, both deaf and hearing students appeared to have greater metacognitive awareness of ongoing comprehension and repair strategies during reading than during instruction in the classroom, but deaf students scored lower than hearing students in both modalities. Deaf students were no more likely than hearing students to report adopting inappropriate strategies, but both groups indicated they were more likely to do so in classroom contexts than during reading.