Applying an IRT-Based Cognitive Diagnostic Model to Diagnose Students' Knowledge States in Multiplication and Division with Exponents

The study applied a psychometric model-the rule-space model-to diagnose students' states of knowledge about how the exponents behave in multiplication and division of quantities with exponents. A 38-item test was administered to 431 Grade 10 students. Each item was characterized by a list of task attributes required for answering the item correctly, and each student was classified, based on his or her item-score pattern, into the most likely knowledge, state (i.e., attribute-mastery pattern) corresponding to an ideal item-score pattern. The following outcomes of the rule-space model were presented: (a) the results of the classification of examinees to knowledge states at the group level along with individual examples, (b) the mastery level of the underlying task attributes as evaluated at three different test-score groups, and (c) a tree diagram of the transitional relationships among the knowledge states that can guide the design of effective remediation. Implications for utilizing the feedback, provided by the rule-space model in the context of instruction and assessment are discussed.     

An NCME Instructional Module on Exploring the Logic of Tatsuoka's Rule-Space Model for Test Development and Analysis

K. Tatsuoka's rule-space model is a statistical method for classifying examinees' test item responses into a set of attribute-mastery patterns associated with different cognitive skills. A fundamental assumption in the model resides in the idea that test items may be described by specific cognitive skills called attributes which can include distinct procedures, skills, or processes possessed by an examinee. The rule-space model functions by collecting and ordering information about the attributes required to solve test items and then statistically classifying examinees' test item responses into a set of attribute-mastery patterns, each one associated with a unique cognitive blueprint. The logic of Tatsuoka's rule-space model, as it applies to test development and analysis, is examined an this module. Controversies and unresolved issues are also presented and discussed.     

“How”—beyond the “what”, towards the “why”: A rule-assessment approach to achievement testing

The advantages of a rule assessment approach to the interpretation of achievement test results have been demonstrated using an S-P chart with coded error types. The problems of similar total test scores resulting from completely different misapprehensions, as well as correct answers resulting from incorrect rules of operation, were addressed using a simulated data-set.Although the overall quality of the test used here as measured by conventional psychometric indices proved satisfactory, it was shown that the traditional interpretation, which refers to total test scores, can be misleading, especially when adaptive remediation is sought. It is well known in medical sciences that a disease has several symptoms yet several diseases can share the same symptoms (i.e. high fever). Consequently, no responsible physician would prescribe the same medicine for two patients suffering from different diseases just because they both share high fever as one of their symptoms. Similarly, when two students with different misapprehensions get the same total test score, should the teacher prescribe the same remediation for correcting their misapprehension?Although the method for diagnostic test construction was out of the scope of this paper, it should be noted that test design is a crucial matter which eventually determines the quality of the diagnosis. One has to, therefore, carefully choose the items for the diagnosis in order to maximize the information about the rules of operation underlying the students' responses. A task specification chart (Birenbaum & Shaw, 1985) may serve as a useful tool in the process of test construction. As was illustrated in the chart, when an item yields the same results as a result of various “bugs”, its contribution to rule assessment is in question.Although in reality test results are contaminated by noise resulting from careless errors or strategy changes during the test, the overall identification rate achieved by diagnostic tests ranges between 70%–80% (Tatsuoka, 1984). Similarly, current AI diagnostic systems such as DEBUGGY and DPF are reported as being capable of identifying 80%–90% of student errors (VanLehn, 1981; Ohlesson & Langley, 1985). It seems that such a rate justifies the tedious work involved in constructing a diagnostic tool.     

Studaxology: the expertise students need to be effective in higher education

Students in higher education have to develop two types of expertise; the first refers to the mastery they want to acquire within a well defined occupational or disciplinary domain; the second relates to the deep level learning needed to achieve that mastery as an expert student or studax. Research has indicated that in solving a problem any expert simultaneously has to draw on four types of knowledge. Where the personal organisation of these four leads to effectiveness, this brings about the quintessence of expertise - experiencing problem solving behaviour as intrinsically motivating, or rewarding in itself. This intrinsic motivation integrates experiences of competence (through declarative knowledge), causality (through procedural knowledge), creativity (through situational knowledge) and self regulation (through strategic knowledge). The same will then necessarily hold for the student who proves, by experiencing this very same effectiveness, to be the studax or deep level learner higher education needs.This paper describes a theory - studaxology - which explains to the student, on the basis of what is being experienced while studying, how to become organized as a person within the study environment, so as to succeed in the required task. Studaxology's core is a 3 × 3 matrix of study experiences, based on that number of sources of variance, empirically identified by means of factor analysis of Likert-type items in study inventories. Its central experience of intrinsic motivation brings together four pairs of complementary experiences (ability vs. difficulty, effort vs. relevance, intention vs. demand and time perspective vs. discipline), with each pair constituting a basic component of intrinsic motivation, and as such reflecting a specific form of metacognitive knowledge. Adequate interpretation and use of the 3 × 3 scores on a similar study inventory enable the studax effectively to meet deep level learning that optimal functioning in higher education demands. Factor analyses of students evaluations of lecturing behaviours can also be fitted into a 3 × 3 matrix equivalent to that of the studax. It is argued from these analyses that the essential prerequisites for achieving studaxological expertise stem from an appropriate initial vocational choice (which will help to produce an internally well-cohering 3 × 3 matrix of experiences) and are further enhanced by an equivalent matrix of lecturing behaviours designed to support students' own study experiences.     

The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka's Rule-Space Approach

A cognitive item response theory model called the attribute hierarchy method (AHM) is introduced and illustrated. This method represents a variation of Tatsuoka's rule-space approach. The AHM is designed explicitly to link cognitive theory and psychometric practice to facilitate the development and analyses of educational and psychological tests. The following are described: cognitive properties of the AHM; psychometric properties of the AHM, as well as a demonstration of how the AHM differs from Tatsuoka's rule-space approach; and application of the AHM to the domain of syllogistic reasoning to illustrate how this approach can be used to evaluate the cognitive competencies required in a higher-level thinking task. Future directions for research are also outlined.     

Making Diagnostic Inferences About Cognitive Attributes Using the Rule-Space Model and Attribute Hierarchy Method

The purpose of this paper is to describe the logic and identify key assumptions associated with making cognitive inferences using two attribute-based psychometric methods. The first method is Kikumi Tatsuoka's rule-space model. This model provides a strong point of reference for studying the nature of diagnostic inferences because it is important in the evolution of skills diagnostic testing and it is well documented. The second method is a new procedure called the attribute hierarchy method that was developed from the rule-space approach. Although the attribute hierarchy method shares many commonalities with rule space, it represents an extension by including an attribute hierarchy that serves as an explicit cognitive model of task performance designed to link psychometric practices with contemporary cognitive theories. In this paper, we describe and compare these two attribute-based psychometric methods and identify new directions for research and practice in skills diagnostic testing.     

Negative expertise: comparing differently tenured elder care nurses' negative knowledge

Negative expertise is conceptualised as the professional's ability to avoid errors during practice due to certain cognitive agencies. In this study, negative knowledge (i.e. knowledge about what is wrong in a certain context and situation) is conceptualised as one such agency. This study compares and investigates the negative knowledge of elder care nurses at three different levels of professional experience. Thereby, various forms of negative knowledge—declarative, procedural, self-reflective and vicarious—are investigated. Moreover, the specificity of negative knowledge is compared across different levels of professional experience. Verbal data was collected from a prompting task study of 37 elder care nurses. These were prompted 20 diagnoses of varying typicality for the professional field. The nurses were asked what was critical to pay attention to and what ought to be avoided in case of these diagnoses. The study's results reveal a significant superiority of highly tenured elder care nurses in vicarious negative knowledge, as well as in highly specified negative knowledge. Procedural and procedural negative knowledge (PNK)—which are the most important facets in quantitative terms—show an approximately parallel developmental pattern among the different groups. For declarative knowledge, a U-shaped intermediate effect was discovered. It is concluded that PNK has the most immediate error-preventive function. With regards to future research, the great specificity of highly tenured elder care nurses' negative knowledge discovered in this study is discussed here as one possible explanation as to why they are able to intentionally avoid errors (i.e. their negative expertise).     

Quantifying participation in,and the effectiveness of,remediating assessment in a university mathematics module

In Maths for Business, a large first-year mathematics module, the continuous assessment component comprises 10 weekly quizzes which combine to contribute 40% of the final module mark. If students did not receive the full five marks on their weekly quiz, they were provided with the opportunity to resubmit their corrected weekly quiz with an explanation of their error(s) for one additional mark. We refer to this process as ‘remediation’. Of the students who had the opportunity to remediate, ～70% did. Through examining learning management system data, we show that the remediation process encouraged students to access module resources. Furthermore, by using a Bayesian hierarchical model to account for students’ level of participation, achievement and prior knowledge, we show that participation in the remediation process positively impacted the final examination marks of moderate to high-achieving students (based on initial continuous assessment marks). However, participation in the remediation process provided limited benefit to low-achieving students. We conjecture this is because these students had not achieved a level of understanding whereby participation in the remediation process could progress their knowledge.     

Differential Item Functioning Resulting From The Use of Different Solution Strategies

The present study investigates the degree to which item "bias" techniques can lead to interpretable results when groups are defined in terms of specified differences in the cognitive processes involved in students' problem-solving strategies. A large group of junior high school students who took a test on subtraction of fractions was divided into two subgroups judged by the rule-space model to be using different problem-solving strategies. It was confirmed by use of Mantel-Haenszel (MH) statistics that these subgroups showed different performances on items with different underlying cognitive tasks. We note that, in our case, we are far from faulting items that show differential item functioning (D1F) between two groups defined in terms of different solution strategies. Indeed, they are "desirable" items, as explained in the discussion section     

Preparing Primary Teachers to Teach Art & Design Effectively

This paper is an account of an action research project funded by the Teacher Training Agency and co‐ordinated by the Institute of Education, University of London. It critically considers the features, methodology and main findings of an enquiry into the nature of art and design components of courses of primary initial teacher education in nine institutions of higher education in the U.K. The project provides evidence to support ways in which primary teachers can effectively transform their subject knowledge in art and design into subject‐specific pedagogical knowledge. The value — for the participants — of a collective case study approach to research, along with some wider implications for art and design courses in initial teacher education are also identified.     

Evidence-based diagnosis and treatment for specific learning disabilities involving impairments in written and/or oral language

Programmatic, multidisciplinary research provided converging brain, genetic, and developmental support for evidence-based diagnoses of three specific learning disabilities based on hallmark phenotypes (behavioral expression of underlying genotypes) with treatment relevance: dysgraphia (impaired legible automatic letter writing, orthographic coding, and finger sequencing), dyslexia (impaired pseudoword reading, spelling, phonological and orthographic coding, rapid automatic naming, and executive functions; inhibition and rapid automatic switching), and oral and written language learning disability (same impairments as dyslexia plus morphological and syntactic coding and comprehension). Two case studies illustrate how these differential diagnoses can be made within a conceptual framework of a working memory architecture and generate treatment plans that transformed treatment nonresponders into treatment responders. Findings are discussed in reference to the importance of (a) considering individual differences (diagnosis of impaired hallmark phenotypes) in planning and evaluating response to instruction and modifying instruction when a student is not responding; (b) recognizing that teaching may change epigenetic gene expression at one stage of schooling, but not the underlying gene sequences that render individuals still vulnerable as curriculum requirements increase in nature, complexity, and volume in the upper grades; and (c) using evidence-based diagnoses of specific learning disabilities that are consistent across states for free and appropriate education K to 12 and for accommodations throughout higher education and professional credentialing.     

Integration of new domain-related states and events from texts and illustrations by subjects with high and low prior knowledge

The aim of this article is to investigate with high and low knowledge subjects in the scientific domain of the neuron, the way information should be presented and illustrated to promote the integration of new information. This fundamental process for learning was examined in two experiments using a primed recognition task. In the first study, the nature of domain-specific information depicted (states or events) was manipulated while in the second, the temporal position of illustrations (before or after textual information) was also considered. The main result showed that when presented before the text, illustrations allow easier retrieval from memory (a) when they represent “states” for beginners, and (b) when they represent “events” for experts. Within the theoretical framework of text and picture comprehension (Schnotz, W., & Bannert, M. (2003). Construction and interference in teaming from multiple representation. Learning and Instruction, 13, 141–156), our results offer a more precise definition of the conditions under which the addition of illustrations for a text is beneficial to the learning process.     

Constructing knowledge: An effective use of educational technology for teaching Islamic studies in the UK

The 21st century as a digital age is characterized by the increased accessibility of information and knowledge through the medium of sophisticated technological tools. The main aim of this article is to show how educational technology can be used effectively to help students construct knowledge when teaching Islamic studies in the UK. The first part of this paper summarizes the differences between information sharing and knowledge construction with reference to the essence of knowledge as Aristotelian ‘episteme’ (theoretical knowledge) and technê (practical knowledge: know-how); and the extent which the former process is created by the use of Information Technology (IT) while the latter is enhanced by Educational Technology (ET). The second part explores how ET can be used effectively to ‘provide training in critical and creative thinking skills of students’ as an integral part of producing useful tools and generating practical benefit during their learning process (Felder et al. 2000, p. 26; Callaos 2009, p. 3). The third, then, explains why a student centred and research-based teaching is preferred to traditional research-led method in order to support the construction of knowledge. The paper concludes by presenting some reflections and limitations on how effective use of ET and research-based teaching can help students to become critical thinkers while studying Islam and Middle Eastern politics as part of international politics curriculum in the UK.     

Perceived Benefits of Anatomy Coursework Prior to Medical and Dental School

Student struggles in gross anatomy coursework at the professional level can result in hours of remediation along with a need to allot time and other resources by both the student and the faculty. Since this course typically occurs in the first semester of the first year, programs can turn to admissions data to try to determine which of these students may struggle. This study looked at two years of medical (n = 280) and dental (n = 78) students to determine if there is a relationship between pre-admissions anatomy coursework and performance in gross anatomy at the professional school level. Students provided data regarding their past anatomy coursework and final grades in professional school gross anatomy courses were obtained. In addition, students responded to questions regarding their feelings of preparation and how they valued the prior anatomy coursework as it related to the professional course. Statistical analysis showed no difference in final course grade between students with and without prior anatomy in either program. Counter to the numerical data, 96.6% of the students in the study recommended an anatomy course prior to pursuing a health science degree. The primary reasons given for this recommendation were the benefits of repeated content exposure, knowledge of the anatomy terminology, and decreased stress regarding the course. The results from this study suggest that the benefits of prior anatomy may be seen more in the students’ stress and quality of life rather in the numerical performance of course grades.     

Estimating Classification Consistency and Accuracy for Cognitive Diagnostic Assessment

This article introduces procedures for the computation and asymptotic statistical inference for classification consistency and accuracy indices specifically designed for cognitive diagnostic assessments. The new classification indices can be used as important indicators of the reliability and validity of classification results produced by cognitive diagnostic assessments. For tests with known or previously calibrated item parameters, the sampling distributions of the two new indices are shown to be asymptotically normal. To illustrate the computations of the new indices, we apply them to the real diagnostic data from a fraction subtraction test (Tatsuoka). We also use simulated data to evaluate their performances and distributional properties.     

Epidemiology of abusive head trauma in West Virginia children <24 months: 2000–2010

BackgroundFew states have published statewide epidemiology of abusive head trauma (AHT).ObjectiveTo examine the statewide epidemiology of AHT in West Virginia (WV), with the primary objective of establishing AHT incidence for comparison to national data, and to use as a baseline for comparison to incidence post-implementation of a statewide AHT prevention program.Participants and settingAHT cases in children less than 2 years old were identified from the 3 tertiary pediatric centers in WV.MethodsCases were identified by using ICD-9 codes for initially identifying those with injuries which might be consistent with AHT, followed by medical record review to determine which of these met the criteria for inclusion as a case. Medical examiner data was used to find additional cases of AHT. Using the number of cases identified along with relevant census data, incidence of AHT was calculated.ResultsThere were 120 cases of AHT treated in WV hospitals from 2000 to 2010, 100 of which were WV residents. The incidence was 36.1/100,000 children <1 year of age and was 21.9 cases per 100,000 children <2 years of age. Incidence in infants increased during the latter years (2006–2010) of the study to 51.8/100,000 compared to the incidence during 2000–2005, which was 24.0/100,000 (p < .01).ConclusionsCompared to US national, state and regional figures, the WV incidence of AHT was among the highest. In addition, the incidence of AHT increased significantly over the study period. Possible factors contributing to the rise in incidence are discussed.     

Exact Computation of Parallel Robot''''s Generalized Inertia Matrix

According to the definition of the new hypothetical states which have obvious physical sig-nificance and are termed as no-gravity static and accelerated states, a method for exact computa-tion of the parallel robot's generalized inertia matrix is presented. Based on the matrix theory, the generalized inertia matrix of the parallel robot can be computed on the assumption that the robot is in these new hypothetical states respectively. The approach is demonstrated by the Delta robot as an example. Based on the principle of the virtual work, the inverse dynamics model of the robot is formulized after the kinematics analysis. Finally, a numerical example is given and the element dis-tribution of the Delta robot's inertia matrix in the workspace is studied. The method has computational advantage of numerical accuracy for the Delta robot and can be parallelized easily.     

From builders to their tools: crafting ‘poverty knowledge’ from “the Moynihan Report” to today

ABSTRACT

Critics often cite the “Moynihan Report” to illustrate how cultural diagnoses of poverty blame the victims and ignore the structural roots of inequality. This paper considers how the tendency to attribute such dangerous analyses to researchers’ individual shortcomings leaves the underlying analytical elements that generate cultural diagnoses. Rather than focus on those who build “poverty knowledge,” we advocate for more attention to the analytical tools that continue to encourage cultural diagnoses today. We use the “Moynihan Report” as a mirror in which we can see how static theories of culture and atomistic units of analysis encourage cultural diagnoses. They obscure the ongoing, collective work of reproducing inequality, including the work of race-making and racialization which help sustain economic exploitation, making it dangerously easy to locate blame and responsibility on particular persons or groups of persons. Transgressing the roots of inequity requires a relentless confrontation with these analytical tools.