Evaluating mental models in mathematics: a comparison of methods

Cognitive scientists investigate mental models (how humans organize and structure knowledge in their minds) so as to understand human understanding of and interactions with the world. Cognitive and mental model research is concerned with internal conceptual systems that are not easily or directly observable. The goal of this research was to investigate the use of Evaluation of Mental Models (EMM) to assess the mental models of individuals and groups in solving complex problems and to compare novices and experts models as bases for providing feedback to learners. This study tested a qualified web-based assessment tool kit, Highly Interactive Model-based Assessment Tools and Technologies (HIMATT), in an as yet untested domain—mathematics. In this study, university students and their mathematics instructors used two tools in HIMATT, Dynamic Evaluation of Enhanced Problem Solving (DEEP) and Text-Model Inspection Trace of Concepts and Relations (T-MITOCAR). The research questions include: Do novice participants exhibit common patterns of thoughts when they conceptualize complex mathematical problems? Do novices conceptualize complex mathematical problems differently from experts? What differences in DEEP and T-MITOCAR patterns and responses exist according to the measures of HIMATT? Findings suggest that EMM and HIMATT could effectively support formative assessment in a complex mathematical domain. Finally, this study confirms a common assumption of cognitive scientists that the tool being used could affect the tool user’s understanding of the problem being solved. In this case, while DEEP and T-MITOCAR led to somewhat different expert models, both tools prove useful in support of formative assessment.