Modeling the student in intelligent tutoring sytems: The promise of a new psychometrics

This paper reviews a number of relatively new and promising psychometric approaches to the problem of modeling student achievement (the student model) within intelligent tutoring systems (ITS). A shared characteristic of most ITSs is their need to estimate a model of the student's understanding of the domain, and use this model to modify and adapt subsequent instructional content and sequence. Sound cognitive diagnosis and the need to advance ITS technology require the development of student models that are integrated with cognitive theory and instructional science. A number of cognitively oriented psychometric approaches — including latent-trait models, statistical pattern recognition methods, and causal probabilistic networks — are described and discussed within the current ITS framework. As measurement-based student models are refined, we anticipate their compatibility with future generations of intelligent tutoring systems.     

Database Problem Behavior: A Literature Retrieval System for Professionals Dealing with Problem Behaviours of Individuals with Intellectual Disabilities

Knowledge of the literature among professionals in the area of problem behaviours and developmental disabilities is most likely soon out of date as several hundred useful studies appear each year. A literature database may be a useful tool for clinicians and researchers to conveniently retrieve needed information, preventing them from collecting, reading, processing, and storing all the available material. This report describes a database (i.e., Database Problem Behavior) encompassing information that goes far beyond presenting abstracts only. The third version of the database encompasses 885 empirical studies on the (functional) analysis, and behavioural and pharmacological treatment of problem behaviours (e.g., self‐injurious behaviour, aggressive behaviour, depression) for persons with developmental disabilities.     

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.     

Cross‐survey analysis of school violence and disorder

School psychologists and administrators are often asked to respond to student violence and disorder based on incomplete or inaccurate information about the nature and scope of these problems in their schools. Records of disciplinary actions and incidents may reflect only the tip of the proverbial iceberg. In national surveys, school administrators report rates of violence and disorder that are only a small fraction of the rates reported by students in national self‐report surveys. Student self‐report surveys on school violence and disorder may offer school officials a means of more accurately appraising the prevention needs of their students. This article compares the methods and findings of three national surveys of students in an effort to understand what methodological characteristics have the most salient impact on their findings. The article examines measures of school‐related weapon carrying and fear from all three national surveys contrasting their modes of administration and question phrasing. Estimates from even the most expertly designed and administered survey will include some error. However, the stability and comparability of the national surveys across time and across surveys suggest that student self‐report surveys are valuable tools for school‐level needs assessment. © 2001 John Wiley & Sons, Inc.     

A New Method for Assessing the Statistical Significance in the Differential Functioning of Items and Tests (DFIT) Framework

A new item parameter replication method is proposed for assessing the statistical significance of the noncompensatory differential item functioning (NCDIF) index associated with the differential functioning of items and tests framework. In this new method, a cutoff score for each item is determined by obtaining a (1 −α) percentile rank score from a frequency distribution of NCDIF values under the no-DIF condition by generating a large number of item parameters based on the item parameter estimates and their variance-covariance structures from a computer program such as BIILOG-MG3. This cutoff for each item can be used as the basis for determining whether a given NCDIF index is significantly different from zero. This new method has definite advantages over the current method and yields cutoff values that are tailored to a particular data set and a particular item. A Monte Carlo assessment of this new method is presented and discussed.     

Creative Endeavors: Inspiring Creativity in a First Grade Classroom

With an emphasis on high-stakes testing and a focused curriculum, it would seem at times, the joy of creativity is missing from the classroom. This article describes a curricular approach the children named “Creative Endeavors”, as implemented by a first grade teacher. The approach is described in three phases. In the exploratory stage the children were presented with various media, made requests for additional materials, and explored different forms of expression. In the second phase the children made choices based on interests, and typically worked with others in a spirit of collaboration with an emphasis on the process of the project as well as affective responses. In the final phase of intentional planning, children began to independently design, construct, and take responsibility for their project working alone or with others. Phases are intended to provide a blueprint for other teachers to explore creative endeavors in the classroom. Children in this project communicated through various media including sewing, photography, and drama. Through collaborating with peer and choosing projects, the children became authentically engaged in the various classroom experiences. Although the curricular approach was intended to foster creativity, in each phase writing about their projects became a natural and critical element in the process. Children wrote about their experiences, the steps they took to take part in the process, why they chose it, and reflected upon the projects undertaken.     

Contemporary Classics? The Early Onset of Influence of Articles Published in Criminology and Criminal Justice Journals, 2010–2015

Citation analysis is a valuable method for tracking the growth and transmission of scientific knowledge. In criminology and criminal justice, such analyses tend to be focused on “classic” works or on highly cited scholars over many years. As an alternative approach, we focus on work that has been cited rather quickly – what we term the “early onset” of impact. In doing so, we identified 221 of the most highly cited peer-reviewed articles, authored by 480 scholars, which were published in criminology and criminal justice journals between 2010 and 2015. Analyses reveal wide variation in substantive topics and methodological approaches, as well as a substantial gender gap with respect to authorship. Additional analyses provide insight into both journal and institutional/program prestige. We conclude with directions for future research and stress the importance of tracking over time articles that appear to be off to an early start with respect to scholarly impact.