Assessing the Utility of Item Response Models: Computerized Adaptive Testing

How has Item Response Theory helped solve problems in the development and use of computer-adaptive tests? Do we need to balance item content with computer-adaptive tests? Could we use IRT to evaluate unusual responses to computer-delivered tests?     

A multivariate analysis of task dimensions related to science concept learning difficulties in primary school children

The purpose of this study was to determine if achievement on a series of problems was affected by differences among task values within a dimension. Values within inference pattern, goal object, and percept dimensions were studied. One hundred and sixty primary school children participated in the study. An analysis of the results of their achievement on the tasks indicate that task solution is affected by the dimension values used in this study. The results of the study suggest that a multidimensional scale of task dimension values should be constructed in order to assist the curriculum specialist in the construction and sequencing of concept tasks for specific grade levels.     

The effect of intra- versus post-interview feedback during simulated practice interviews about child abuse

OBJECTIVE: This study compared the effectiveness of two types of instructor feedback (relative to no feedback) on investigative interviewers' ability to adhere to open-ended questions in simulated practice interviews about child abuse. METHOD: In one condition, feedback was provided at the end of each practice interview. In the other, the instructor stopped the interviewer at various stages during the practice interviews to provide feedback. The relative effect of these conditions was examined by measuring interviewers' performance in a standardized mock interview paradigm immediately prior to, immediately after, and 12 weeks after the practice and feedback sessions. RESULTS: Prior to and 12 weeks after the practice sessions were administered, there was no significant difference in participants' adherence to open-ended questions irrespective of the nature of the feedback, or whether feedback was received. At the immediate post-practice assessment interval, however, the participants who received feedback during the practice interviews performed better (M proportion of open-ended questions=.85, SD=.13) than the other participants (post-interview feedback M=.67, SD=.18, p<.001; no feedback M=.56, SD=.16, p<.001). This heightened use of open-ended questions was associated with a greater tendency among the interviewees to provide abuse-related details in response to open-ended questions (M=.91, SD=.11) compared to the other participants (post-interview feedback M=.77, SD=.15, p<.05; no feedback M=.69, SD=.16, p=.001). CONCLUSIONS: Different types of feedback can be differentially effective in training child abuse investigators to adhere to open-ended questions. The benefits of any training program, however, are likely to be short-lived without ongoing practice. PRACTICE IMPLICATIONS: A single study compared the relative effectiveness of two types of instructor feedback (relative to no feedback) on investigative interviewers' ability to adhere to open-ended questions in simulated practice interviews about child abuse. This research is relevant to trainers of investigative interviewers because there is currently large variability in the type of feedback employed in training programs. This study is one of the first to empirically demonstrate that different types of feedback may be differentially effective in improving the performance of investigative interviewers.     

The validity and reliability of focus groups as a research method in adult education

The article examines the theory and practice of focus groups in adult education research. Three theoretical positions are described: radical hermeneutic, moderate interpretative and pragmatic realistic position trying to bridge the gap with positivist research. This last position has been chosen as the departure point for a further analysis of the focus group elements, which relate to validity and reliability. The research examines four European research projects in the field of adult education. The most important and surprising data are presented with reference to both socio-psychological and technical problems in the use of this method. The results highlight the importance and dynamics of all the elements discussed.