An NCME Instructional Module on Latent DIF Analysis Using Mixture Item Response Models

The purpose of this ITEMS module is to provide an introduction to differential item functioning (DIF) analysis using mixture item response models. The mixture item response models for DIF analysis involve comparing item profiles across latent groups, instead of manifest groups. First, an overview of DIF analysis based on latent groups, called latent DIF analysis, is provided and its applications in the literature are surveyed. Then, the methodological issues pertaining to latent DIF analysis are described, including mixture item response models, parameter estimation, and latent DIF detection methods. Finally, recommended steps for latent DIF analysis are illustrated using empirical data.     

Assessing reflective thinking in solving design problems: The development of a questionnaire

Reflection is a critical factor in solving design problems. Using good methods to observe designers' reflection is essential to inform the design of the learning environments that support the development of design problem‐solving skills. In this study, we have developed and validated a novel self‐reporting questionnaire as an efficient instrument to explore reflection, called Assessing Reflective Thinking in Solving Design Problems (ARTiD). This questionnaire has been developed based on the three‐dimensional model for reflective thinking: the timing, the objects and the levels of reflection. A total of 294 undergraduate and graduate students in the fields of engineering, computer science and instructional design were recruited to participate in one of four iterative, formative tests through which reliability and validity analyses were performed to revise and confirm the questionnaire. Examples of participants' reflection patterns were demonstrated to show the practical value of this questionnaire. In conclusion, the final version of ARTiD was presented as a valid instrument to explore students' reflection for research and educational practice. In addition, we found that ARTiD can be used as an effective learning tool to guide student designers' reflections toward better learning and performance.     

Assessing university students' perceptions of their Physics instructors' TPACK development in two contexts

Technological Pedagogical and Content Knowledge (TPACK) has been gaining traction among educational researchers; however, studies documenting university students' perceptions of their teachers' TPACK remain limited. This study intends to investigate the professional development of two physics instructors through the lens of the TPACK framework. Moreover, this study spans an 18‐week semester within both the contexts of Taiwan and China. Multiple data were collected and analyzed, including the pretest and posttest TPACK surveys, instructor interviews, in‐class observations and students' feedback and opinions. The results revealed that John's instructional representations and strategies and technology integration and application scores increased significantly, as well as Mike's knowledge of students' understanding score showing a significant increase from the middle to the end of semester. John (Taiwan) emphasized life examples and the use of multimedia while Mike (China) chose to emphasize students' knowledge and evaluation. Such results showed different teaching characteristics in the two contexts. Implications and suggestions are put forward based on the results of this study.     

Creation of reusable open textbooks: Insights from the Connexions repository

Open textbook initiatives have appeared as an alternative to traditional publishing. These initiatives for the production of alternatively copyrighted educational resources provide a way of sharing materials through the Web. While the open model of peer‐produced materials enables the global reuse of textbooks, the combination of fragments to produce new textbooks tailored to particular needs is not easy. The heterogeneity of topics and target audiences, as well as the cultural and institutional differences of educational systems across the globe, raises the need for new compositional features, fostering—at the same time—the reuse of the newly produced combinations. This paper reviews the current models of reuse and sharing, and presents a quantitative analysis of the Connexions repository which provides evidence about the culture of reuse in open textbook initiatives.     

Variable‐Length Computerized Adaptive Testing Using the Higher Order DINA Model

Cognitive diagnosis models provide profile information about a set of latent binary attributes, whereas item response models yield a summary report on a latent continuous trait. To utilize the advantages of both models, higher order cognitive diagnosis models were developed in which information about both latent binary attributes and latent continuous traits is available. To facilitate the utility of cognitive diagnosis models, corresponding computerized adaptive testing (CAT) algorithms were developed. Most of them adopt the fixed‐length rule to terminate CAT and are limited to ordinary cognitive diagnosis models. In this study, the higher order deterministic‐input, noisy‐and‐gate (DINA) model was used as an example, and three criteria based on the minimum‐precision termination rule were implemented: one for the latent class, one for the latent trait, and the other for both. The simulation results demonstrated that all of the termination criteria were successful when items were selected according to the Kullback‐Leibler information and the posterior‐weighted Kullback‐Leibler information, and the minimum‐precision rule outperformed the fixed‐length rule with a similar test length in recovering the latent attributes and the latent trait.