Practised intelligence testing based on a modern test conceptualization and its reference to the common intelligence theories

The question is to what extent intelligence test-batteries prove any kind of empirical reference to common intelligence theories. Of particular interest are conceptualized tests that are of a high psychometric standard – those that fit the Rasch model – and hence are not exposed to fundamental critique. As individualized testing, i.e., a psychologist and a testee face to face, is often preferred by many practitioners, a Wechsler-like test-battery will be dealt with here: The Adaptive Intelligence Diagnosticum (AID 2; [Kubinger, K. D. &, Wurst, E. (2000). Adaptives Intelligenz Diagnostikum—Version 2.1(AID 2). [Adaptive intelligence diagnosticum 2.] Weinheim: Beltz.]). Using the standardization sample, confirmatory factor analyses were performed with respect to intelligence theories and models, respectively, as concerns Spearman, Wechsler, Thurstone, Cattell, Jäger, and Carroll. Additionally, a confirmatory factor analysis was performed with respect to a simplified neuropsychological model of specific learning disorders, which proved to fit the data best, even better than the (exploratory) four factor solution as given in the AID 2-manual. This model is based on the three interdependent factors “perception”, “retrieval”, and “utilization”. The answer is that if modern test conceptualizations attempt to fulfill pragmatic purposes they hardly have any relation to pertinent intelligence theories, but rather create their own kind of informal, heuristic model of “intelligence”.     

Les tests d’apprentissage comme alternative ou complément aux tests d’intelligence: Un bilan de leur evolution

For a number of reasons the learning test approach (also known under the expressions «dynamic assessment» or «assessment of learning potential») can be considered as a promising alternative and a complement to conventional procedures of intelligence testing. But learning test procedures have also been criticized for their lack of psychometric standardization, their time-consuming application and for their only modest increase in predictive validity compared to conventional intelligence tests with regard to normal populations. The author traces the history of the learning test concept and critically discusses its main theoretical problems and practical implications. He further presents an overview of the different learning test procedures that have been developed in his research group since the early seventies. One of the main goals of this research is to combine the learning test approach with psychometric standardization i.e. to maximize individualization of the assessment and pursue the goal of a more qualitative evaluation without losing the necessary objectivity or the possibility to quantify results in order to make interindividual comparisons. Other major concerns are to enhance content validity of learning tests by using better analyzed items and to invest more into research regarding construct validity.     

Intelligent Job Aids For Professionals: Lessons Learned From Medicine

With the rapid advancements in artificial intelligence, computer science, and information management seen today, it is becoming increasingly feasible to improve the performance of professional white collar workers with intelligent job aids (i.e., expert systems). Already, performance technologists are advocating and predicting their wide spread adoption. Nevertheless, while such high tech performance interventions are relatively new to business, they have existed in medical research labs and clinics for nearly 20 years. Because of the almost ideal match between medical decision making and the emergent capabilities of knowledge engineering, artificial intelligence researchers made medicine one of their major testing grounds. In this article, I summarize the use of intelligent job aids in medicine as a source of insights for anticipating how they can and should be used by professionals in other areas. These insights include three generalizations which form the conclusion of the paper: (a) intelligent job aids must address a real need, (b) they must be compatible with professionals' larger “performance systems,” and (c) the adoption of intelligent job aids will depend as much on future trends as on the professionals' needs or performance systems.     

Individual differences in speed of mental processing and human cognitive abilities: Toward a taxonomic model

Extensive research within the field of learning and individual differences focuses upon the relationship between general intelligence and process measures derived from elementary cognitive tasks (ECTs). This emphasis has ignored data indicating that cognitive abilities are best described by three levels (or strata). It has also been suggested that mental speed is a unitary construct, although it is more likely to have a complex structure. To address shortcomings evident in this literature, a multivariate investigation (N = 179) was conducted. Factor analysis of 25 psychometric indices gave seven factors postulated under the theory of fluid (G_f) and crystallized (G_c) intelligence. Correlations between cognitive abilities and parameters derived from 11 ECTs indicated that G_f (alone) was related to processing speed. This relationship is seemingly dependent upon experimental manipulations of task complexity. Regarding the factorial structure of mental speed, the results were unequivocal: Broad second-order factors may be derived from both ECTs and psychometric tests. These constructs are independent from abilities defined by accuracy scores and collectively define a general cognitive speed factor. Implications of these findings are discussed. It would appear that mental speed is more intricate than proposed, and that cognitive complexity (reflected in stimulus-response compatibility effects) plays a crucial role in its ontogenesis. In addition, several explanatory models linking intelligence to processing speed are untenable. It is likely that the search for a basic process of intelligence by means of mental speed frameworks (alone) is misguided.Recently, within the field of individual differences, there has been “an explosion of experimental studies into the speed of mental processes” (H.J. Eysenck 1995, p. 225). Various tasks, ranging from those paradigms assessing simple, psychomotor movements and on up through to measures of complex problem solving and psychometric test performance, have been employed (Stankov & Roberts 1997). The present study was designed to explore speed of processing constructs within a structural model of human cognitive abilities. Utilizing the evidence presented in Carroll's (1993) extensive reanalysis of the main data sets collected within the psychometric discipline this century, the structural model of cognitive abilities adopted is that known as the theory of fluid and crystallized intelligence (see Horn & Noll 1994; Stankov et al. 1995). In contrast, the mental speed measures selected for investigation in this study were chosen on the basis of both experimental and psychometric findings that rely on disparate accounts (e.g., information theory). Notably, mental speed constructs are not presently encapsulated within a single unifying model. Another major aim of the present study was to redress this imbalance by establishing a rapprochement between conceptual models of mental speed and human cognitive abilities.     

The Knowledge Arena as an Extension to the Concept Map: Reflection in Action

The concept map is becoming a ubiquitous tool in education. In recent years there has been a growing interest in “diagramming” or “mapping” ideas to be learned (e.g., Jonassen et al., 1998). The approach has been championed by study skills proponents such as Buzan (1993). Maps of concepts and relationships have been used by many researchers and practitioners to help diagnose misunderstanding, improve study methods and glimpse how learners come to know. In other areas, the representation of knowledge in formalisms such as the Net greatly assisted the development of intelligent tutoring systems (e.g., Sowa, 1983). In order to better understand the claims made for its efficacy, reference to how concept maps have been used and defined will lead to a plausible explanation of the process of “off–loading” of concepts during learning or study (McAleese, 1994, 1998). In order to demonstrate the widespread application of supporting learn ers with external “learning spaces” (c.f. ISLEs/ and REALs—Grabinger et al., 1998; Grabinger & Dunlap, 1995; MacFarlane, 1993), consideration will be given in this paper to techniques that formally represent knowledge in Concept–Relationship–Concept “instances” (Fisher et al., 1990). Other techniques such as reflection journals draw on the same process but are not considered in detail here. The importance of self regulation (Zimmerman, 1990) and self-confrontation (McAleese, 1985) is highlighted in this framework. The map as a mirror or an assistant to learners is well documented (e.g., Vizcarro & Leon, 1998); the question of an external representation of thinking, in the form of a map determining what can be learned as well as what might be learned, is not yet fully known.     

An analysis of research studies in which “homemade” achievement instruments were utilized

This article will have as its focus a problem described by Anderson, namely, the quality of the construction and reporting of research that contains “homemade” achievement tests. The current status of homemade achievement tests was examined in this study. Research reports in two science education journals were analyzed, using Anderson's eight categories of information that a high quality research report should include. The journals examined were Journal of Research in Science Teaching and Science Education from January 1975 to January 1980. The findings indicate that reliability estimates and the procedures for selecting test items were mentioned more frequently in the science education journals than in Anderson's study. However, there was little or no improvement in describing the relation of test items to instruction. These findings should be of interest both to researchers who are utilizing “homemade” achievement tests in their studies and to journal panels who are reviewing studies for publication.     

Minimizing the Influence of Item Parameter Estimation Errors in Test Development: A Comparison of Three Selection Procedures

In test development, item response theory (IRT) is a method to determine the amount of information that each item (i.e., item information function) and combination of items (i.e., test information function) provide in the estimation of an examinee's ability. Studies investigating the effects of item parameter estimation errors over a range of ability have demonstrated an overestimation of information when the most discriminating items are selected (i.e., item selection based on maximum information). In the present study, the authors examined the influence of item parameter estimation errors across 3 item selection methods—maximum no target, maximum target, and theta maximum—using the 2- and 3-parameter logistic IRT models. Tests created with the maximum no target and maximum target item selection procedures consistently overestimated the test information function. Conversely, tests created using the theta maximum item selection procedure yielded more consistent estimates of the test information function and, at times, underestimated the test information function. Implications for test development are discussed.     

AN INVESTIGATION OF THE CONTEXT EFFECT IN MATRIX SAMPLING1

Practical use of the matrix sampling (i.e. item sampling) technique requires the assumption that an examinee's response to an item is independent of the context in which the item occurs. This assumption was tested experimentally by comparing the responses of examinees to a population of items with the responses of examinees to item samples. Matrix sampling mean and variance estimates for verbal, quantitative, and attitude tests were used as dependent variables to test for differences between the “context” and “out-of-context” groups. The estimates obtained from both treatment groups were also compared with actual population values. No significant differences were found between treatments on matrix sample parameter estimates for any of the three types of tests.     

The Significance of Physical Education Content: “Sending the Message” in Physical Education Teacher Education

Mind-body dualism has likely influenced how many view human beings and their behavior—mind (i.e., thinking) is elevated over body (i.e., performing)—even in Physical Education Teacher Education. The problem is that such a perspective makes physical education content (i.e., dance, games, play, and sport) subsidiary to more “intellectual” or “academic” content. This article invites Physical Education Teacher Education faculty to consider how to effectively “send the message” to future physical educators that physical education content is valuable and respectable and in many ways epitomizes intelligent human behavior. Specifically, Physical Education Teacher Education faculty are invited to (a) challenge traditional definitions of intelligence and introduce alternative definitions, (b) participate regularly in their own playgrounds, (c) provide a sufficient dose of activity courses in their curricula, (d) teach activity courses, (e) establish a performance-oriented culture, and (f) administer performance and/or game play evaluations.     

Do Lions Have Manes? For Children,Generics Are About Kinds Rather Than Quantities

Generic statements (e.g., “Lions have manes”) make claims about kinds (e.g., lions as a category) and, for adults, are distinct from quantificational statements (e.g., “Most lions have manes”), which make claims about how many individuals have a given property. This article examined whether young children also understand that generics do not depend purely on quantitative information. Five‐year‐olds (n = 36) evaluated pairs of questions expressing properties that were matched in prevalence but varied in whether adults accept them as generically true (e.g., “Do lions have manes?” [true] vs. “Are lions boys?” [false]). Results demonstrated that children evaluate generics based on more than just quantitative information. Data suggest that even young children recognize that generics make claims about kinds.     

AN EMPIRICAL INVESTIGATION OF COMPLEXITY AND PROCESS IN MULTIPLE-CHOICE ITEMS1,2

In this study hierarchical syndrome analysis was used to investigate empirically the clustering of tests each of which consisted of items based upon the rationale of a particular category of the Bloom Taxonomy (i.e., an “application” test, an “analysis” test, etc.). Each test included items related to eight physical science principles. Thus an attempt was made to hold the content as constant as possible and systematically vary process in the formation of the tests. The data indicate possible ways of combining Taxonomy item types based on the psychological distance between the categories.     

Examining factors associated with elevated Lie Scale responding on the Child Abuse Potential Inventory

Child maltreatment remains a serious public health issue in the United States. Therefore, it is important to engage in quality control of the assessment, prevention, and treatment services for families affected by maltreatment. Parenting capacity assessments (PCAs) are typically an integral part of service delivery for families affected by maltreatment and can carry serious consequences for the referred parent. The Child Abuse Potential Inventory (CAPI) is a measure that is widely used in PCAs; however, socially desirable responding on the CAPI can serve to invalidate the important information derived from this assessment, as well as lead to negative impressions of the parent. Using data collected via multiple methods (including a non-face valid behavioral measure, intelligence screening, and self-report) from a predominantly at-risk sample of parents, the aim of this study was to better understand factors that may predict socially desirable responding on the CAPI. Results indicated that lower parental intelligence, a “positivity bias” (i.e., the tendency to learn and attend to positive over negative information during the non-face valid behavioral task), and lower reported depressive symptoms were associated with higher socially desirable responding. These findings suggest that assessors should thoughtfully consider the possibility that invalid CAPI scores may be more related to low intelligence and a positivity bias than to psychopathy and manipulation (e.g., purposefully trying to present oneself in a positive light to gain favor in a PCA).     

Finding the missing Einsteins: Expanding the breadth of cognitive and noncognitive measures used in academic services

Education researchers, policymakers, and practitioners are concerned with identifying and developing talent for students with fewer opportunities, especially students from historically marginalized groups. An emerging body of research suggests “universally screening” or testing all students, then matching those students with appropriate educational challenges, is effective in helping marginalized students. However, most tests have focused on two areas: math and verbal reasoning. We leverage three nationally representative samples of the U.S. population at different time points that include both novel cognitive measures (e.g., spatial, mechanical, and abstract reasoning) and non-cognitive measures (e.g., conscientiousness, creativity or word fluency, leadership skill, and artistic skill) to uncover which measures would improve proportional representation of marginalized groups in talent identification procedures. We find that adding spatial reasoning measures in particular—as well as other non-cognitive measures such as conscientiousness, leadership, and creativity—are worthwhile to consider for universal screening procedures for students to narrow achievement gaps at every level of education, including for gifted students. By showing that these nontraditional measures both improve proportional representation of underrepresented groups and have reasonable predictive validity, we also broaden the definition of what it means to be “gifted” and expand opportunities for students from historically marginalized groups.     

人机协同化与学生发展核心素养——基于社会智能三维模型的分析

人机协同系统正在全面渗入人类的工作与生活,社会正快速地人机协同化。在弱人工智能时代,人工智能没有“意向性”,但其智能程度在迭代进化中。人机协同系统存在人类与机器两类智能主体,以及人类智能、人工智能和协同智能三种智能元素,其中协同智能的主体是人类,人机协同系统的智能是分布式的。受国际关系“软”“硬”“巧”实力三分理论启示,本研究构建了人机协同时代社会智能的三维模型。“三维”指硬智能、软智能和巧智能,机器拥有更多硬智能,并向人类“学习”软智能,人类拥有更强的软智能和巧智能,并把更多的硬智能“让渡”给机器。基于社会的快速人机协同化,教育所培育的学生核心素养应以发展学生的软素养和巧素养为重点,在硬素养维度着力培育学生的计算思维。未来人机关系将决定个体的社会存在,因此强调以人机协同的价值观、意识、能力等为重点的巧素养培育具有前瞻性。     

Explorations of crystallized intelligence: Completion tests,cloze tests,and knowledge

An attempt is made to reconcile two historically important tools for the assessment of intelligence and the prediction of academic achievement with extant theories of verbal–crystallized–knowledge aspects of adult abilities. A study of 167 adults ranging in age from 18 to 69 reasserts the importance of individual differences in completion test and cloze test performance in accounting for both measures of crystallized intelligence (Gc) and four scales of knowledge (biology, U.S. history, U.S. literature, and technology). The completion tests were found to account for all of the variance in Gc and knowledge that the cloze tests accounted for, and resulted in incremental predictive validity for both domains. In addition, completion and cloze tests were found to have a suppressor effect on the relationship between Gc and Age. We note that C. Spearman's [The nature of “Intelligence” and the principles of cognition. New York: MacMillan (1927).] assertion, namely that the completion test had higher correlations with intelligence than any other measure. Our results suggest that abstract reasoning may be far less useful in predicting learning and performance than the completion test is.     

Critically Reflecting on the Origins,Evolution, and Impact of the Cattell-Horn-Carroll (CHC) Model

ABSTRACT

The Cattell-Horn-Carroll (CHC) model presently serves as a blueprint for both test development and a taxonomy for clinical interpretation of modern tests of cognitive ability. Accordingly, the trend among test publishers has been toward creating tests that provide users with an ever-increasing array of scores that comport with CHC. However, an accumulating body of independent research on modern intelligence tests has questioned many instruments’ alignment with the CHC model. To shed potential insight on these discrepancies, we review the developmental history of CHC and its numerous modifications from 1997 to the present. Next, we identify and discuss several potential limitations in the CHC literature that may be responsible for this discrepancy. Finally, we encourage clinicians to consider the extant evidence currently available for engaging in CHC-inspired assessment applications (e.g., XBA, PSW).     

Bias among Professionals Who Erroneously Declare Students Eligible for Special Services

A computer simulation program was developed for the purpose of studying the processes diagnostic personnel in the schools engage in when making psychoeducational decisions about students. Initially, the program collected demographic data on 159 participants and assessed their knowledge base in assessment. Bogus referral information for one of sixteen “cases” was then provided and subjects were instructed that they were to make eligibility and classification decisions for the child. They were told that scores and other information (e.g., observations) were available to them on a variety of tests from among seven domains. Although the referral data indicated the student might be evidencing academic or behavior problems, all assessment data indicated a normal or average performance. Fiftytwo percent of the subjects (i.e., 83) declared the “average” child eligible for special education services; their classification decisions were further analyzed. Decisions about the 16 different children were found to be more a function of referral information than child performance data. The outcomes are discussed with regard to implications for the practice of psychoeducational assessment.     

Asking Children to “Be Helpers” Can Backfire After Setbacks

Describing behaviors as reflecting categories (e.g., asking children to “be helpers”) has been found to increase pro-social behavior. The present studies (N = 139, ages 4–5) tested whether such effects backfire if children experience setbacks while performing category-relevant actions. In Study 1, children were asked either to “be helpers” or “to help,” and then pretended to complete a series of successful scenarios (e.g., pouring milk) and unsuccessful scenarios (e.g., spilling milk while trying to pour). After the unsuccessful trials, children asked to “be helpers” had more negative attitudes. In Study 2, asking children to “be helpers” impeded children's helping behavior after they experienced difficulties while trying to help. Implications for how category labels shape beliefs and behavior are discussed.     

道德人工智能:基础、原则与设计

随着技术的快速进步,人工智能的决策和行为开始从外部设定转向内在发展,智能主体逐渐具有不同程度的自适应能力、决策能力和行为能力,它的自主能力也不断增强。出于道德考量,自主决策和行为的人工智能开始被视为道德主体。因此,传统道德如何在智能技术中自主发挥作用就成为一个必须面对的问题;后果论、道义论和美德论都具有解决此问题的潜力。要构建能够进行道德决策的人工智能系统,需要确保人工智能的自主推理能够受人类社会道德和价值观约束,与人类价值观保持一致,担负起决策“责任”。实现有“道德”的人工智能是一个复杂的系统工程,价值敏感设计、透明度设计及道德学习设计是开发道德人工智能的三种重要方法。