Application of Think Aloud Protocols for Examining and Confirming Sources of Differential Item Functioning Identified by Expert Reviews

This paper demonstrates and discusses the use of think aloud protocols (TAPs) as an approach for examining and confirming sources of differential item functioning (DIF). The TAPs are used to investigate to what extent surface characteristics of the items that are identified by expert reviews as sources of DIF are supported by empirical evidence from examinee thinking processes in the English and French versions of a Canadian national assessment. In this research, the TAPs confirmed sources of DIF identified by expert reviews for 10 out of 20 DIF items. The moderate agreement between TAPs and expert reviews indicates that evidence from expert reviews cannot be considered sufficient in deciding whether DIF items are biased and such judgments need to include evidence from examinee thinking processes.     

Forensic child sexual abuse evaluations: assessing subjectivity and bias in professional judgements

Objectives

Evaluators examining the same evidence often arrive at substantially different conclusions in forensic assessments of child sexual abuse (CSA). This study attempts to identify and quantify subjective factors that contribute to such disagreements so that interventions can be devised to improve the reliability of case decisions.

Methods

Participants included 1106 professionals in the field of child maltreatment representing a range of professional positions or job titles and years of experience. Each completed the Child Forensic Attitude Scale (CFAS), a 28-item survey assessing 3 forensic attitudes believed to influence professional judgments about CSA allegations: emphasis-on-sensitivity (i.e., a focus on minimizing false negatives or errors of undercalling abuse); emphasis-on-specificity (i.e., a focus on minimizing false positives or errors of overcalling abuse); and skepticism toward child and adolescent reports of CSA. A subset of 605 professionals also participated in 1 of 3 diverse decision exercises to assess the influence of the 3 forensic attitudes on ratings of case credibility.

Results

Exploratory factor analysis identified 4 factors or attitude subscales that corresponded closely with the original CFAS scales: 2 subscales for emphasis-on-sensitivity and 1 each for emphasis-on-specificity and skepticism. Attitude subscale scores differed significantly by sample source (in-state trainings vs. national conferences), gender, years of experience, and professional position, with Child Protective Service workers unexpectedly more concerned about overcalling abuse and more skeptical of child disclosures than other professionals—a pattern of scores associated with an increased probability of disbelieving CSA allegations. The 3 decision exercises offered validation of the attitude subscales as predictors of professional ratings of case credibility, with adjusted R²s for the three exercises ranging from .06 to .24, suggesting highly variable effect sizes.

Conclusions

Evaluator disagreements about CSA allegations can be explained, in part, by individual differences in 3 attitudes related to forensic decision-making: emphasis-on-sensitivity, emphasis-on-specificity, and skepticism toward child reports of abuse. These attitudes operate as predispositions or biases toward viewing CSA allegations as likely true or likely false. Several strategies for curbing the influence of subjective factors are highlighted including self-awareness of personal biases and team approaches to assessment.     

An approach to web-based Personal Health Records filtering using fuzzy prototypes and data quality criteria

Nowadays, new ways of managing and accessing to health-care information are continuously appearing. Web-based Personal Health Records (web PHRs) have the potential to make data about health-care available to clinicians, researchers and students in different medical contexts and applications. Therefore, the amount of web PHRs accessible through Internet has grown enormously and as a result health-care professionals are currently burdened with more and more data. It’s probable that these data, unfortunately, have not always the adequate levels of quality, making that their work cannot always be as successful as expected. As a way of alleviating this fact, the present work is focused on improving the document filtering results in the context of web PHRs management. To achieve this goal, a new kind of document filtering model is proposed. This model is based on fuzzy prototypes which are defined by means of conceptual prototypes. These prototypes are obtained by using a data quality analysis of documents. This analysis guarantees that filtered information will be relevant enough for the information user. The complete model provides an efficient strategy of document filtering that can be very useful when it is necessary to deal with a constant flow of new information.     

Soft approaches to information access on the Web: An introduction to the special issue

Soft Computing (SC) tools present a great potential in real-life problems related with engineering, industrial applications, medicine, finances, etc. In this special issue we present a set of seven papers that report original research about the use of SC techniques to solve the problems in the field of information access on the Web.     

Telematics for Teacher Education: issues from a European conference

ABSTRACT

The world Information Superhighway is under discussion in many countries. Within Europe the discussion is also very active as is the promotion of the use of telematics for trans‐European communication and co‐operation in various areas. For education and training, representatives of European teacher trainers, teachers, students, parents, policy and decision makers discussed their needs and views on a trans‐European network at a conference held in May 1994 in Luxembourg. This paper presents a selection of various issues raised at the conference regarding the use of telematics for teacher education and illustrates the discussion and the method of discussion. It also describes and discusses the conference programme model, as it offered innovative on‐line activities to facilitate appreciation of telematics. The case is argued that teacher education can benefit greatly from telematics and some points are put forward for further discussion.     

Relationships Among Measures as Empirical Evidence of Validity: Incorporating Multiple Indicators of Achievement and School Context

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine the relationships of the assessment with the Stanford Achievement Test, 9th Edition and the California High School Exit Examination. The results indicate that the measures are related, that students demonstrate a degree of transfer, and that the language arts PA is relatively more fair than comparison assessments. The results are robust to various model specifications and demonstrate that benefits do not accrue to all students equally.     

A Longitudinal Analysis of the Relations Among Professional Development,Educators’ Beliefs and Perceived Skills,and Response-to-Intervention Implementation

ABSTRACT

Professional development is a critical systems-level intervention thought to facilitate Response-to-Intervention (RtI) implementation. The current study examined the relations between professional development, educator outcomes, and problem-solving implementation within an RtI framework using growth curve modeling. School leadership teams from pilot schools (n = 34) participated in 3 years of training. Pilot schools also received job-embedded coaching. Comparison schools (n = 27) provided a referent group. Results indicated that problem-solving implementation increased faster at pilot schools (β = 0.10, SE = 0.05, t = 2.03, p < .05). In addition, beliefs regarding data-based decision-making (β = 0.36, SE = 0.17, t = 2.13, p < .05) and perceived problem-solving skills applied to academics (β = 0.30, SE = 0.10, t = 3.07, p < .01) positively related to implementation. Implications include the needs to further explore professional development activities and for consultants to utilize evidence-based professional development principles when supporting RtI implementation.     

Gene editing in plants: progress and challenges

The clustered regularly interspaced short palindromic repeat (CRISPR)-associated protein 9 (Cas9) genome editing system is a powerful tool for targeted gene modifications in a wide range of species, including plants. Over the last few years, this system has revolutionized the way scientists perform genetic studies and crop breeding, due to its simplicity, flexibility, consistency and high efficiency. Considerable progress has been made in optimizing CRISPR/Cas9 systems in plants, particularly for targeted gene mutagenesis. However, there are still a number of important challenges ahead, including methods for the efficient delivery of CRISPR and other editing tools to most plants, and more effective strategies for sequence knock-ins and replacements. We provide our viewpoint on the goals, potential concerns and future challenges for the development and application of plant genome editing tools.     

Revisiting the cluster-based paradigm for implicit search result diversification

To cope with ambiguous and/or underspecified queries, search result diversification (SRD) is a key technique that has attracted a lot of attention. This paper focuses on implicit SRD, where the subtopics underlying a query are unknown. Many existing methods appeal to the greedy strategy for generating diversified results. A common practice is using a heuristic criterion for making the locally optimal choice at each round. As a result, it is difficult to know whether the failures are caused by the optimization criterion or the setting of parameters. Different from previous studies, we formulate implicit SRD as a process of selecting and ranking k exemplar documents through integer linear programming (ILP). The key idea is that: for a specific query, we expect to maximize the overall relevance of the k exemplar documents. Meanwhile, we wish to maximize the representativeness of the selected exemplar documents with respect to the non-selected documents. Intuitively, if the selected exemplar documents concisely represent the entire set of documents, the novelty and diversity will naturally arise. Moreover, we propose two approaches ILP4ID (Integer Linear Programming for Implicit SRD) and AP4ID (Affinity Propagation for Implicit SRD) for solving the proposed formulation of implicit SRD. In particular, ILP4ID appeals to the strategy of bound-and-branch and is able to obtain the optimal solution. AP4ID being an approximate method transforms the target problem as a maximum-a-posteriori inference problem, and the message passing algorithm is adopted to find the solution. Furthermore, we investigate the differences and connections between the proposed models and prior models by casting them as different variants of the cluster-based paradigm for implicit SRD. To validate the effectiveness and efficiency of the proposed approaches, we conduct a series of experiments on four benchmark TREC diversity collections. The experimental results demonstrate that: (1) The proposed methods, especially ILP4ID, can achieve substantially improved performance over the state-of-the-art unsupervised methods for implicit SRD. (2) The initial runs, the number of input documents, query types, the ways of computing document similarity, the pre-defined cluster number and the optimization algorithm significantly affect the performance of diversification models. Careful examinations of these factors are highly recommended in the development of implicit SRD methods. Based on the in-depth study of different types of methods for implicit SRD, we provide additional insight into the cluster-based paradigm for implicit SRD. In particular, how the methods relying on greedy strategies impact the performance of implicit SRD, and how a particular diversification model should be fine-tuned.