Do illustrations help or harm metacomprehension accuracy?

The present research examined the effect of illustrations on readers' metacomprehension accuracy for expository science text. In two experiments, students read non-illustrated texts, or the same texts illustrated with either conceptual or decorative images; were asked to judge how well they understood each text; and then took tests for each topic. Metacomprehension accuracy was computed as the intra-individual correlation between judgments and inference test performance. Results from both studies showed that the presence of decorative images can lead to poor metacomprehension accuracy. In the second study, an analysis of the cues that students reported using to make their judgments revealed that students who used comprehension-relevant cues showed more accurate metacomprehension. A self-explanation instruction did not alter either comprehension-relevant cue use or metacomprehension accuracy, although some advantages were seen when readers were prompted to self-explain from texts illustrated with conceptual images. These results suggest that students may need more explicit instruction or support to promote the use of valid cues when engaging in comprehension monitoring with illustrated text, and that seductive information such as decorative images may undermine comprehension monitoring.     

Metacomprehension effects situated within an anchoring and adjustment framework

Low accuracy levels are often obtained when readers are asked to predict test performance over reading materials. Three investigations further explore the information readers use to make predictions during metacomprehension. Our results show that readers’ estimates are influenced by factors such as their initial impression of the reading task, based in part on their perceptions surrounding text genre and test item type. To explain these and other published results, a new framework for investigating metacomprehension using Tversky and Kahneman’s (Science, 185:1124–1131, 1974) anchoring and adjustment heuristic as a guide is proposed. We argue that readers anchor comprehension test performance on factors such as self-perceptions of reading ability and/or perceptions of the reading task and then insufficiently adjust their predictions to reflect the demands of the specific reading task at hand such as text difficulty.     

Anchoring effects on prospective and retrospective metacomprehension judgments as a function of peer performance information

Two experiments were conducted to investigate anchoring effects on metacomprehension judgments as a function of fictitious information participants received about past peer performance. In Experiment 1 participants were randomly assigned to one of the three anchor groups that, in some cases, provided past peer performance averages in terms of a percentage: a high anchor (85%) a low anchor (55%), and no anchor. Results showed that relative to the no anchor group, the low anchor group made significantly lower prospective judgments whereas the high anchor group did not make significantly higher prospective judgments. With a high anchor of greater magnitude (95%), Experiment 2 demonstrated more pronounced anchoring effects on prospective judgments: Relative to the no anchor group, the low anchor group made significantly lower judgments and the high anchor group made significantly higher judgments. In addition, Experiment 2 showed that anchoring effects occurred even on retrospective judgments. Overall, the results showed that peer performance information can serve as an anchoring point for metacomprehension judgments and the anchoring effects are tenacious given that they can even affect retrospective judgments of performance.     

The effect of metacomprehension judgment task on comprehension monitoring and metacognitive accuracy

The authors investigated differences in the processes underlying two types of metacomprehension judgments: judgments of difficulty and predictions of performance (JOD vs. POP). An experiment was conducted to assess whether these two types of judgments aligned with different types of processing cues, and whether their accuracy correlated with different factors such as sensitivity to processing ease and reading ability. Participants (n?=?72) read an extended text about brain structure and after each sentence made either a JOD or POP. Results suggested that JODs and POPs were made based on different sets of cues because different factors correlated with the accuracy of metacomprehension judgments. JOD accuracy correlated with sensitivity to processing ease and POP accuracy most strongly correlated with reading ability. Engaging in different metacomprehension judgments during reading may alter the information sources to which a reader attends and which factors influence metacognitive accuracy.     

Individual differences in relative metacomprehension accuracy: variation within and across task manipulations

In recent decades, increasing numbers of studies have focused on metacomprehension accuracy, or readers’ ability to distinguish between texts comprehended more vs. less well. Following early findings that suggested readers are fairly poor at doing so, a number of studies have identified specific tasks to supplement a single reading of text that have resulted in greater metacomprehension accuracy. One assumption underlying these studies is that, in the absence of such tasks, metacomprehension accuracy is uniformly poor, and given their implementation, readers uniformly improve. Here we describe the individual variation that occurs both in the absence (e.g., within a single text reading manipulation) and presence (e.g., within a rereading or selective rereading task manipulation) of these supplementary tasks (N = 214), in order to make a case for greater attention to individual differences in metacomprehension accuracy. We also introduce a new manipulation in metacomprehension research, selective rereading, and argue that certain types of tasks may be more likely to reveal individual differences in metacomprehension accuracy due to the nature of the task being more or less demanding on working memory capacity.     

Drawing to improve metacomprehension accuracy

We examined the effect of drawing sketches on metacomprehension accuracy of science texts for 5th grade (ages 10–11) students (Study 1: N = 60, Study 2: N = 62). Students either received instruction on drawing organizational sketches, which focused on capturing the relationships described in texts, or representational sketches, which focused on capturing details described in texts. They then read and drew sketches for texts, predicted their performance, and completed tests. They completed this procedure for five texts. Metacomprehension accuracy was greater for students instructed to draw organizational sketches than for students instructed to draw representational sketches or those in a control group who did not draw (Study 2). Performance on comprehension tests was also greater for students instructed to draw organizational sketches than for students in other groups. The superior metacomprehension accuracy was explained in terms of the cue-utilization framework of metacognitive monitoring (Koriat, 1997).     

Metacognitive judgments and disfluency – Does disfluency lead to more accurate judgments,better control,and better performance?

Theories of metacognition assume that better monitoring leads to better control and performance. Furthermore, monitoring accuracy is often low because students are overconfident (absolute accuracy) and unable to discriminate comprehension of different text-passages (relative accuracy). Fluency seems to be a cue for metacognitive judgments, and therefore, reducing fluency should lead to less automatic processing, lower judgments, and better absolute and relative accuracy. Because the accuracy of metacognitive judgments is the basis of the control of learning, disfluency should lead to more appropriate control and thus to better performance. To test these assumptions, students (N = 83) learned either with disfluent or with fluent text-passages. The results show that disfluency led to better absolute and relative accuracy but not for all types of judgments. Moreover, students hardly implemented monitoring in control, resulting in lack of improved performance. Further research is required to investigate why students did not base control on monitoring.     

Explanation generation, not explanation expectancy, improves metacomprehension accuracy

The ability to monitor the status of one’s own understanding is important to accomplish academic tasks proficiently. Previous studies have shown that comprehension monitoring (metacomprehension accuracy) is generally poor, but improves when readers engage in activities that access valid cues reflecting their situation model (activities such as concept mapping or self-explaining). However, the question still remains as to which process, encoding or retrieving, causes the improvement of metacomprehension accuracy, and the findings of previous research on this matter have been inconsistent. This study examined whether college students’ metacomprehension accuracy improves when they expect, at the time of reading, that they will explain the content later (active encoding) or when they actually generate an explanation (encoding plus active retrieving). In the experiments, college students read five texts. During reading, some students expected that they would generate explanations but did not actually generate them. In contrast, some students actually generated an explanation of the text after reading. All students then rated their comprehension of each text. Finally, they completed tests on the materials. Results of both studies revealed that metacomprehension accuracy, operationalized as the association between comprehension ratings and test performance, was greater for the group that actually generated explanations than for the expectancy or control groups.     

A review on the accuracy of teacher judgments

In everyday school life, teachers need a wide range of judgment competencies to accurately assess student characteristics, learning and task requirements. The purpose of this literature review is to synthesize the methodological, empirical, theoretical, and practical knowledge from 40 years of research on the accuracy of teacher judgments. We define the accuracy of teacher judgments and differentiate the term from other related constructs. We explain the methodological approaches and summarize the main research findings on the accuracy of teacher judgments of student characteristics and task difficulties. Furthermore, we empirically demonstrate that teachers tend to overestimate student achievement on standardized tests. We discuss possible moderators of teachers’ judgment accuracy and show the effects on teaching and the learning of students. We present the main theoretical approaches that can explain the empirical findings and describe ways to improve teacher judgment accuracy. In the discussion, we address important implications for research and practice.     

Functional Measurement Assessment of Young Children''s Ability to Predict Future Behavior

The reported experiments demonstrate that young children's ability to use previous behavioral information to predict future behavior emerges on quantitative, but not dichotomous, judgment tasks. In a first experiment, kindergartners, second graders, and fourth graders made quantitative liking judgments and predictions for peers after being presented 2 pieces of behavioral information. Children of all 3 age groups used both pieces of information in their liking judgments and predictions. In a second experiment, kindergartners were presented with 2 types of tasks; one was a quantitative prediction, comparable to the task in Experiment 1, and the second were dichotomous predictions, comparable to judgment tasks typically used in other experiments. Children's predictions were significantly more consistent with the behavioral information on the quantitative task than on either of the dichotomous tasks. These results suggest that young children believe in the ability of interpersonal behavior, but have difficulty dealing with the complexity of some prediction tasks.     

Concept mapping improves metacomprehension accuracy among 7th graders

Two experiments explored concept map construction as a useful intervention to improve metacomprehension accuracy among 7th grade students. In the first experiment, metacomprehension was marginally better for a concept mapping group than for a rereading group. In the second experiment, metacomprehension accuracy was significantly greater for a concept mapping group than for a control group, while a group of students who were given already constructed concept maps had accuracy between these two groups. In both experiments, control groups had poor metacomprehension accuracy. That is, they performed worse on tests they predicted better performance and performed better on tests they predicted worse performance. Although constructing concept maps did not produce the same high level of accurate monitoring previously reported in the literature, it still reduced the illusion of knowing.     

Teachers’ judgment accuracy concerning consistent and inconsistent student profiles

Research on teachers' judgment accuracy has focused on teacher judgments of single student characteristics. We analyzed differences in teachers' judgment accuracy concerning students with consistent and inconsistent cognitive and socio-emotional profiles (i.e., profiles of cognitive abilities, self-concept, motivation, and anxiety). Based on test scores and self-reports of N = 743 students, we identified one inconsistent and two consistent profiles. Judgments of N = 43 teachers yielded only three consistent student profiles, indicating that teachers perceived student profiles to be more consistent than they really were. Contrary to our expectations, teachers' judgments were not more accurate for consistent student profiles.     

Accuracy of metacognitive judgments as a moderator of learner control effectiveness in problem-solving tasks

A possible explanation for why students do not benefit from learner-controlled instruction is that they are not able to accurately monitor their own performance. The purpose of this study was to investigate whether and how the accuracy of metacognitive judgments made during training moderates the effect of learner control on performance when solving genetics tasks. Eighty-six undergraduate students solved self-selected genetics tasks using either a full learner control or a restricted learner control. Results indicated that learner control effectiveness was moderated by the absolute accuracy (i.e., absolute bias) of metacognitive judgments, and this accuracy was a better predictor of learning performance for full learner control than for restricted learner control. Furthermore, students’ prior knowledge predicted absolute accuracy of both ease-of-learning judgments (EOLs) and retrospective confidence judgments (RCJs) during training, with higher prior knowledge resulting in a better absolute accuracy. Overall, monitoring guided control, that is, EOLs predicted time-on-task and invested mental effort regardless of the degree of learner control, whereas RCJs predicted the total training time, but not the number of tasks selected during training. These results suggest that monitoring accuracy plays an important role in effective regulation of learning from problem-solving tasks, and provide further evidence that metacognitive judgments affect study time allocation in problem solving context.     

Elementary mathematics teachers’ judgment accuracy and calibration accuracy: Do they predict students’ mathematics achievement outcomes?

In this study we investigated whether elementary mathematics teachers’ knowledge of their students, as reflected in both the accuracy and confidence with which they are able to estimate their students’ task-specific performance on sets of mathematics problems, predicted students’ overall mathematics achievement. Thirty-nine teachers made predictions about the performance of a random sample of target students (n = 150) in their classrooms on sets of “easy” and “difficult” multiplication and division problems. Teachers also provided confidence ratings for those judgments. From these data, indicators of teachers’ judgment accuracy, judgment confidence and calibration accuracy (a measure of metacognitive monitoring) were then related to all of their students’ (n = 834) performance on year-end standardized mathematics achievement tests. Multilevel analyses indicate that teachers’ calibration accuracy, but not their task-specific judgment accuracy, significantly predicted students’ mathematics achievement. Implications for future research on teacher knowledge as well as professional development programs are discussed.     

The double curse of misconceptions: misconceptions impair not only text comprehension but also metacomprehension in the domain of statistics

Research shows that misconceptions are usually detrimental to text comprehension. However, whether misconceptions also impair metacomprehension accuracy, that is, the accuracy with which one self-assesses one’s text comprehension, has received far less attention. We conducted a study in which we examined students’ (N = 47) comprehension and metacomprehension accuracy (prediction accuracy and postdiction accuracy) of a statistics text as a function of their statistical misconceptions. Text comprehension and metacomprehension accuracy referred to both conceptual and procedural aspects of statistics. The results showed that students who had more misconceptions achieved poorer conceptual text comprehension and, at the same time, provided more overconfident predictions of their conceptual and procedural text comprehension than students who had fewer misconceptions. In contrast, postdiction accuracy of conceptual and procedural text comprehension was not affected by misconceptions.     

Teachers' judgments of students' early literacy skills measured by the Early Literacy Skills Assessment: Comparisons of teachers with and without assessment administration experience

Teacher judgments about students' academic abilities are important for several reasons, including their day‐to‐day instructional decision making. Not surprisingly, previous studies have investigated the accuracy of teachers' judgments about their students' reading abilities. Previous research, however, has not investigated teachers' judgments about students' early literacy skills, nor has previous research systematically examined how teachers' training and use of an objective assessment instrument impact their judgments of students' performance on that instrument. This exploratory study offers the first investigation of teachers' judgment accuracy of pre‐kindergarten students' early literacy skills, and compares the judgment accuracy of teachers with and without opportunities to administer the Early Literacy Skills Assessment (ELSA). Findings suggest that teachers with opportunities to administer the ELSA had a significantly higher percentage of accurate judgments across half of the ELSA subtests, but their judgment accuracy was still no better than 50%–60% on all but one ELSA subtest. Implications for school psychologist practitioners and researchers are presented. © 2010 Wiley Periodicals, Inc.     

The effects of labeling bias on prognostic outlook for children as a function of diagnostic label and profession

Labeling bias refers to expectations that others might have for a person given a particular label. This study investigated the effects of labeling bias on prognostic outlook for children as a function of diagnostic labels and professional group membership. School psychologists, regular and special education teachers, and introductory psychology students read a vignette that described an elementary-school-aged boy with behavior problems. The vignette was held constant for all participants, but one of four labels (conduct disordered, socially maladjusted, serious emotionally disturbed, no exceptionality) was varied at the end of the vignette. Respondents then estimated the child's likelihood of future success in interpersonal relationships, the likelihood of further behavioral difficulties, and overall adjustment of the child. There was a significant effect noted for diagnostic label across all professionals on the interpersonal relations variable; the serious emotionally disturbed label resulted in judgments of significantly poorer outlook than any other diagnostic label except socially maladjusted. There were no other significant labeling bias effects for the other dependent measures, but a main effect for diagnostic label on judgment of overall adjustment did approach significance. There were no significant effects noted for professional group membership. Professionals should consider the use of a label other than SED to minimize labeling bias effects on judgment of interpersonal adjustment in children with behavioral difficulties. © 1996 John Wiley & Sons, Inc.     

Lost in narrative? The effect of informative narratives on text comprehension and metacomprehension accuracy

Informative narratives are enriched expository texts that provide to-be-learned conceptual information within a storyline with the aim to foster comprehension. However, research casts doubt on such a benefit for comprehension. Additionally, it is an open question how informative narratives impact metacomprehension accuracy. The results of two experiments (N₁ = 63 and N₂ = 70 university students) showed that informative narratives were less or not at all beneficial to text comprehension compared with expository texts. Moreover, informative narratives often led to more overestimation of comprehension in terms of predictions, postdictions, and response confidence than expository texts. This seemed to be particularly true, as Experiment 2 revealed, for readers with a lower need for cognition because they were more transported into the storyline of informative narratives. The findings suggest that informative narratives prime the activation of a narrative-specific reading goal and, thus, distract readers from learning and accurately monitoring the to-be-learned conceptual information.     

Measuring deep, reflective comprehension and learning strategies: challenges and successes

There is a heightened understanding that metacognition and strategy use are crucial to deep, long-lasting comprehension and learning, but their assessment is challenging. First, students’ judgments of what their abilities and habits and measurements of their performance often do not match. Second, students tend to learn and comprehend differently depending on the subject matter, contexts, goals, and tasks. As a consequence, a student may appear to use deep, reflective strategies in one situation, and fail to do so in other circumstances. Third, it is generally assumed that strategy use (metacognition, metacomprehension) are separable constructs from the underlying skills germane to the target task. The studies described in this issue draw attention to the potential challenges to developing a pure (separable) measure of strategy use that is also reliable, valid, and contextualized.     

Improving the judgment of task difficulties: prospective teachers’ diagnostic competence in the area of functions and graphs

To teach adaptively, teachers should be able to take the students’ level of knowledge into account. Therefore, a key component of pedagogical content knowledge (PCK) is the ability to assume the students’ perspectives. However, due to the so-called expert blind spot, teachers tend to misestimate their students’ knowledge, such as when estimating the difficulty of a given task. This empirically well-documented estimation bias is predicted by Nickerson’s anchoring and adjustment model, which generally explains how people take on other people’s perspectives. In this article, we present an intervention study that aims to improve the accuracy of prospective teachers’ judgments of task difficulty in the area of functional thinking. Two types of treatments are derived from Nickerson’s model. In the first condition (PCK group), participants acquire knowledge about task characteristics and students’ misconceptions. The second condition (sensitizing group) serves to control the idea that potential improvements in the PCK group are not merely based on a pure sensitization of the expert’s estimation bias. Accordingly, these participants are only informed about the general tendency of overestimating task difficulties. The results showed that the PCK group improved both in terms of the accuracy of the estimated solution rates and the accuracy of rank order, whereas the sensitizing group only improved in regard to the former. Thus, the study shows that prospective teachers’ diagnostic judgments can be improved by teaching them relevant PCK in a short amount of time.