Using first-order cognitive analysis to understand problem solving behavior: An example from accounting

We report a study of human problem solving in the accounting domain. This study has three characteristics. First, it is explicitly intended to aid in the development of instructional materials. Second, it uses a powerful task analyzing scheme as the basis for its conclusions. Third, we use a technique called First-Order Cognitive Analysis that combines the advantages of task analysis to overcome some of the traditional methodological problems posed to protocol analysis. Our representational scheme, KO, seems to be largely valid as a basis for reasoning about cognition, although only three subjects were studied.     

The effects of cognitive and metacognitive strategy instruction on the mathematical problem solving of middle school students with learning disabilities.

This study investigated the effects of cognitive and metacognitive strategy instruction on the mathematical problem solving of six middle school students with learning disabilities. Conditions of the multiple baseline, across-subjects design included baseline, two levels of treatment, setting and temporal generalization, and retraining. For Treatment 1, subjects received either cognitive or metacognitive strategy instruction. Treatment 2 consisted of instruction in the complementary component of the instructional program so that all subjects eventually received both cognitive and metacognitive strategy instruction. This design allowed a componential analysis of the content as well as sequence of instruction. Generally, subjects improved their mathematical problem solving as measured by performance on one-, two-, and three-step word problems. Discussion focused on effectiveness of treatment, acquisition and application of strategic knowledge, error pattern analysis, and the need to tailor instruction to the learner's individual characteristics.     

A computational model of student cognitive processes while solving a critical thinking problem in science

Critical thinking when engaged in science problem solving around even simple tasks such as the Piagetian volume conservation task is a complex endeavor. Tasks such as the conservation task often require the interaction of multiple cognitive systems. Parity judgment, retrieval, and lateral thinking are three examples of such systems interacting with critical thinking during a student’s attempt to solve the Piagetian task. The purpose of this computational ablation study is to establish the role of critical thinking as a necessary component of a system of cognition used for the completion of the Piagetian volume conservation task. This ablation study consists of three phases. The confidence interval between the ablation model and the elementary students do not overlap, indicating they are not statistically significantly different. This provides evidence that the model successfully emulates aspects of human cognition and the model can provide a robust picture of science student cognitive processes.     

The dynamic roles of cognitive reappraisal and self-regulated learning during mathematics problem solving: A mixed methods investigation

Emotion regulation (ER) and self-regulated learning (SRL) are crucial to learners’ academic achievements. To date, little research has considered the dynamic relations between cognitive reappraisal (as a form of ER) and SRL in middle-to-upper-elementary-aged children. To address this gap, we conducted an explanatory mixed-methods study to examine relations between cognitive reappraisal, the four macro phases of SRL (task definition, planning/goal setting, enactment of learning strategies, monitoring/evaluation), and mathematics problem-solving outcomes in a sample of 134 elementary students from grades 3 through 6. Path analysis revealed that cognitive reappraisal positively predicted all four phases of SRL, but that the four phases of SRL did not predict cognitive reappraisal. Moreover, both task definition and planning/goal setting positively predicted enactment and monitoring/evaluation. Results from path analyses further revealed that task definition mediated relations between cognitive reappraisal and enactment, and reappraisal and monitoring. Enactment mediated relations between reappraisal and mathematics problem-solving outcomes. Finally, enactment predicted mathematics problem-solving outcomes. Further, quantitative results were cross-validated by results from trend analyses; results converged regarding the weakly sequenced nature of SRL and with regard to cognitive reappraisal serving as an important antecedent for effective SRL.     

Children's emotions in math problem solving situations: Contributions of self-concept,metacognitive experiences,and performance

This study examined 1) experiences of six discrete emotions ‒ joy, pride, contentment, worry, shame, hopelessness ‒ after solving a math problem of students aged 10–12 years, and 2) the contribution of self-concept, metacognitive experiences (feeling of difficulty and feeling of success) and performance on emotions experienced after the task, controlling for gender and emotions experienced before the task. Results indicated a decrease in joy and contentment after problem solving. Performance did not contribute to emotions apart from hopelessness. The influence of performance on hopelessness was mediatized by metacognitive experiences. Self-concept contributed to joy, pride and shame but its influence became non-significant when we controlled for metacognitive experiences. Feeling of success mediatized the effect of self-concept on joy, pride and shame. Metacognitive experiences were also found to be important predictors of all emotions except worry. The need for new paradigms to study emotions in education is discussed.     

Language-related longitudinal predictors of arithmetic word problem solving: A structural equation modeling approach

We investigated the longitudinal relations between cognitive skills, specifically language-related skills, and word-problem solving in 340 children (6.10–9.02 years). We used structural equation modeling to examine whether word-problem solving, computation skill, working memory, nonverbal reasoning, oral language, and word reading fluency measured at second grade were associated with performance on measures of word-problem solving in fourth grade. Results indicated that prior word-problem solving, computation skill, nonverbal reasoning, and oral language were significantly associated with children’s later word-problem solving. Multi-group modeling suggested that these relations were not significantly different for boys versus girls. Implications of these findings are discussed.     

Cognitive holding power,fluid intelligence,and mathematical achievement as predictors of children's realistic problem solving

The present study explored whether first and second order cognitive holding power perceived by children in mathematical classrooms, fluid intelligence, and mathematical achievement predicted their performance on standard problems, and especially realistic problems. A sample of 119 Chinese 4–6th graders were administered the word problem test, the cognitive holding power questionnaire, and Raven's standard progressive matrices. Results showed that: (1) children's fluid intelligence and general mathematical achievement significantly predicted their performance on both realistic and standard problems, however, second order cognitive holding power predicted their performance on realistic problems but not standard problems; (2) the relationship between first order cognitive holding power and children's correct answers to realistic problems was mediated by second order cognitive holding power; (3) children's performance on standard problems was significantly better than that on realistic problems, and children's performance on both types of problems improved with their grades.     

Spreadsheets as cognitive tools: A study of the impact of spreadsheets on problem solving of math story problems

In this study, we investigated the impact of computer spreadsheets on the problem solving practices of students for math story problems, and more specifically on the transition from arithmetic to algebraic reasoning, through the construction of algebraic expressions. We investigated the relationships among the students’ prior knowledge and skills, the verification processes, and the effectiveness of the problem solving tasks. For identifying the factors involved in the problem solving process and their role, in our analysis we employed the Structural Equation Modeling (SEM) approach. We mainly focused on math story problems and on students of tertiary education with little prior experience on the use of computers and spreadsheets. Analysis of the data indicates that spreadsheets can support the transition from arithmetic to algebraic reasoning and this transition is influenced by prior skills of the students relevant to the interaction with the interface (enter formula skills), and the students’ frequency of verification of the solution.     

Teaching science problem solving: An overview of experimental work

The traditional approach to teaching science problem solving is having the students work individually on a large number of problems. This approach has long been overtaken by research suggesting and testing other methods, which are expected to be more effective. To get an overview of the characteristics of good and innovative problem‐solving teaching strategies, we performed an analysis of a number of articles published between 1985 and 1995 in high‐standard international journals, describing experimental research into the effectiveness of a wide variety of teaching strategies for science problem solving. To characterize the teaching strategies found, we used a model of the capacities needed for effective science problem solving, composed of a knowledge base and a skills base. The relations between the cognitive capacities required by the experimental or control treatments and those of the model were specified and used as independent variables. Other independent variables were learning conditions such as feedback and group work. As a dependent variable we used standardized learning effects. We identified 22 articles describing 40 experiments that met the standards we deemed necessary for a meta‐analysis. These experiments were analyzed both with quantitative (correlational) methods and with a systematic qualitative method. A few of the independent variables were found to characterize effective strategies for teaching science problem solving. Effective treatments all gave attention to the structure and function (the schemata) of the knowledge base, whereas attention to knowledge of strategy and the practice of problem solving turned out to have little effect. As for learning conditions, both providing the learners with guidelines and criteria they can use in judging their own problem‐solving process and products, and providing immediate feedback to them were found to be important prerequisites for the acquisition of problem‐solving skills. Group work did not lead to positive effects unless combined with other variables, such as guidelines and feedback. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 442–468, 2001     

Scaffolding cognitive and metacognitive processes in low verbal ability learners: Use of diagrams in computer-based training environments

This study investigated how instructionalstrategies can support learners' knowledgeacquisition and metacomprehension of complexsystems in a computer-based trainingenvironment, and how individual characteristicsinteract with these manipulations. Incorporating diagrams into the trainingfacilitated performance on measures ofintegrative knowledge (i.e., the integrationand application of task-relevant knowledge),but had no significant effect on measures ofdeclarative knowledge (i.e., mastery of basicfactual knowledge). Diagrams additionallyfacilitated the development of accurate mentalmodels (as measured via a card sorting task)and significantly improved the instructionalefficiency of the training (i.e., higher levelof performance was achieved with less mentaleffort). Finally, diagrams effectivelyscaffolded participants' metacognition,improving their metacomprehension accuracy(i.e., their ability to accurately monitortheir comprehension). These beneficial effectsof diagrams on learners' cognitive andmetacognitive processes were found to bestrongest for participants with low verbalability. Results are discussed in terms ofimplications for the design of adaptivelearning systems.     

Decoding strategies as predictors of reading skill: A follow-on study

A follow-on study was conducted on first- and third-grade children who were tested on the Word Identification subtest of the Woodcock Reading Mastery Test. Errors were phonetically transcribed and proportions of total errors were assigned to one of four strategy types based upon the type of decoding analysis: whole word, part word, and either probable (legal) or improbable (illegal) phonetic decoding. The type of strategy employed was highly correlated with concurrent reading test scores and predicted 30 to 37% of the variance in word recognition 19 months later. Use of phonetic decoding was a strong positive predictor and whole word decoding a negative predictor at both first and third grade. Part word decoding became a negative predictor at third grade. Nearly all children used more than one strategy and there was a developmental trend to shift to a phonetic strategy. But this shift was not inevitable and had not occurred for 31% of the children at the close of the study. Children who stayed with the most inefficient strategies had significantly higher vocabulary scores and equivalent phonemic processing ability when compared to readers with more efficient decoding strategies.     

Field dependence/independence cognitive style and problem posing: an investigation with sixth grade students

Field dependence/independence cognitive style was found to relate to general academic achievement and specific areas of mathematics; in the majority of studies, field-independent students were found to be superior to field-dependent students. The present study investigated the relationship between field dependence/independence cognitive style and problem-posing ability among sixth grade students. The 94 students’ sample was clustered into three groups, according to the cognitive-style field dependence/independence (field dependents, field mixed and field independents). The results suggest that field-independent participants outperformed field-mixed and field-dependent ones in both problem-posing ability and the complexity of the problems posed. It was also found that the content of the task influenced the differences between the three groups of students as regards the ability in problem posing and the complexity of the problems posed; while in the first (informal context), second and fourth task differences were found between the groups, in the third task (formal context), no differences were evident. The results are interpreted in consideration to the relevant literature findings. Some educational implications are discussed and directions for future research are drawn.     

Behavioral and adjustment correlates of problem solving: validational analyses of interpersonal cognitive problem-solving measures

The interpersonal cognitive problem-solving (ICPS) skills (i.e., means-ends thinking, identified obstacles, alternative solutions, consequential thinking) of 150 families (father, mother, and child 6-11 years old) were assessed via written tests and problem-solving behavioral performance. The interrelationships of ICPS written and behavioral problem-solving skills were examined, as were the relationships of each of these measures of problem solving to both parent and teacher indices of child adjustment. IQ, as measured by the age-appropriate Wechsler scale, was partialed out. Results indicated some ecological validity of written alternatives and consequential tests for children and means-ends tests for parents. Neither parents' nor children's written ICPS scores nor problem-solving behavior were systematically related to either teacher or parent ratings of child adjustment. However, a behavioral index of parental facilitation of child problem solving was significantly related to all problem-solving behaviors and some written ICPS measures. Results are discussed in terms of the role of ICPS skills in child adjustment, the potential limits of ICPS measures in therapy outcome, and the manner in which children learn interpersonal cognitive problem solving.     

Computer technology and complex problem solving: Issues in the study of complex cognitive activity

Goals and plans organize much of complex problem solving behavior and are often inferable from action sequences. This paper addresses the strengths and limitations of inferring goals and plans from information that can be derived from computer traces of software used to solve mathematics problems. We examined mathematics problem solving activity about distance, rate, time relationships in a computer software environment designed to support understanding of functional relationships among these variables (e.g., distance =rate × time; time=distance/rate) using graphical representations of the results of simulations. Ten adolescent-aged students used the software to solve two distance, rate, time problems, and provided think-aloud protocols. To determine the inferability of understanding from the action traces, coders analyzed students' understanding from the computer traces alone (Trace-only raters) and compared these to analyses based on the traces plus the verbal protocols (Traceplus raters). Inferability of understanding from the action traces was related to level of student understanding how they used the graphing tool. When students had a good understanding of distance, rate, time relationships, it could be accurately inferred from the computer traces if they used the simulation tool in conjunction with the graphing tool. When students had a weak understanding, the verbal protocols were necessary to make accurate inferences about what was and was not understood. The computer trace also failed to capture dynamic exploration of the visual environment so students who relied on the graphing tool were inaccurately characterized by the Trace-only coders. Discussion concerns types of scaffolds that would be helpful learning environment for complex problems, the kind of information that is needed to adequately track student understanding in this content domain, and instructional models for integrating learning environments like these into classrooms.Members of the Cognition and Technology Group at Vanderbilt who have contributed to this project are (in alphabetical order) Helen Bateman, John Bransford, Thaddeus Crews, Allison Moore, Mitchell Nathan, and Stephen Owens. The research was supported, in part, by grants from the National Science Foundation (NSF-MDR-9252990) but no official endorsement of the ideas expressed herein should be inferred.     

Beyond cognitive and metacognitive tools: the use of the Internet as an 'epistemological' tool for instruction

This paper argues that Internet‐based instruction should not be only perceived as a cognitive tool or a metacognitive tool; rather, it can be perceived and used as an epistemological tool. When the Internet is used as an epistemological tool for instruction, learners are encouraged to evaluate the merits of information and knowledge acquired from Internet‐based environments, and to explore the nature of learning and knowledge construction. This paper further asserts that Internet‐based instruction is perceived as a way to help learners develop advanced epistemologies. On the other hand, developmentally advanced epistemological beliefs can facilitate the practice of Internet‐based instruction.