What do tests of “formal” reasoning actually measure?

Tests of formal operational reasoning derived from Piagetian theory have been found to be effective predictors of academic achievement. Yet Piaget's theory regarding the underlying nature of formal operations and their employment in specific contexts has run into considerable empirical difficulty. The primary purpose of this study was to present the core of an alternative theory of the nature of advanced scientific reasoning. That theory, referred to as the multiple-hypothesis theory, argues that tests of formal operational reasoning actually measure the extent to which persons have acquired the ability to initiate reasoning with more than one specific antecedent condition, or if they are unable to imagine more than one antecedent condition, they are aware that more than one is possible; therefore conclusions that are drawn are tempered by this possibility. As a test of this multiple-hypothesis theory of advanced reasoning and the contrasting Piagetian theory of formal operations, a sample of 922 college students were first classified as concrete operational, transitional, or formal operational, based upon responses to standard Piagetian measures of formal operational reasoning. They were then administered seven logic tasks. Actual response patterns to the tasks were analyzed and found to be similar to predicted response patterns derived from the multiple-hypothesis theory and were different from those predicted by Piagetian theory. Therefore, support was obtained for the multiple-hypothesis theory. The terms intuitive and reflective were suggested to replace the terms concrete operational and formal operational to refer to persons at varying levels of intellectual development.