Modeling Partial Knowledge on Multiple‐Choice Items Using Elimination Testing

Single‐best answers to multiple‐choice items are commonly dichotomized into correct and incorrect responses, and modeled using either a dichotomous item response theory (IRT) model or a polytomous one if differences among all response options are to be retained. The current study presents an alternative IRT‐based modeling approach to multiple‐choice items administered with the procedure of elimination testing, which asks test‐takers to eliminate all the response options they consider to be incorrect. The partial credit model is derived for the obtained responses. By extracting more information pertaining to test‐takers’ partial knowledge on the items, the proposed approach has the advantage of providing more accurate estimation of the latent ability. In addition, it may shed some light on the possible answering processes of test‐takers on the items. As an illustration, the proposed approach is applied to a classroom examination of an undergraduate course in engineering science.     

The Value of Choice: An Experiment Using Multiple‐Choice Tests

This article presents a novel experimental methodology in which groups of students were offered the option to choose between two equivalent scoring rules to assess a multiple‐choice test. The effect of choosing the scoring rule on marks is tested. Two major contributions arise from this research. First, it contributes to the literature on the value of choice. Second, it also contributes to the literature on the educational measurement of knowledge. The results suggest that choice could positively affect students' scores. However, students need to learn to choose the assessment method. Moreover, women seem to obtain greater benefits from the option of choosing the scoring rule.     

Comparing Teachers’ Job Satisfaction across Countries: A Multiple‐Pairwise Measurement Invariance Approach

There is much interest in comparing latent traits, such as teacher job satisfaction, in large international surveys. However, different countries respond to questionnaires in different languages and interpret the questions through different cultural lenses, raising doubts about the psychometric equivalence of the measurements. Making valid comparisons depends on the latent traits displaying scalar measurement invariance. Unfortunately, this condition is rarely met across many countries at once. Different approaches that maximize the utility of such surveys, but remain faithful to the principles of measurement invariance testing, are therefore needed. This article illustrates one such approach, involving multiple‐pairwise comparisons. This enables us to compare teacher job satisfaction in England to 17 of the countries that participated in TALIS 2013. Teacher job satisfaction in England was as low, or lower, than all of the 17 comparable countries.     

An Alternative Multiple‐Choice Question Format to Guide Feedback Using Student Self‐Assessment of Knowledge

Management science professors who teach large classes often assess students with multiple‐choice questions (MCQs) because it is efficient. However, traditional MCQ formats are ill‐fitted for constructive feedback. We propose the reward for omission with confidence in knowledge (ROCK) format as an original formative assessment technique to help guide feedback associated with MCQs in an introductory undergraduate management science course. Our study contributes to theory by empirically showing that students can self‐assess their state of knowledge, signal it to the professor, and use proper answering options. In practice, ROCK is an easily implementable MCQ format that allows professors to gain information on student learning based on answers selected. ROCK identifies lack of knowledge or misinformation at both individual and collective levels thus providing opportunities for better feedback in class and during office hours. Limitations of the application of ROCK are also discussed.     

Evaluating Type I Error and Power Rates Using an Effect Size Measure With the Logistic Regression Procedure for DIF Detection

The logistic regression (LR) procedure for differential item functioning (DIF) detection is a model-based approach designed to identify both uniform and nonuniform DIF. However, this procedure tends to produce inflated Type I errors. This outcome is problematic because it can result in the inefficient use of testing resources, and it may interfere with the study of the underlying causes of DIF. Recently, an effect size measure was developed for the LR DIF procedure and a classification method was proposed. However, the effect size measure and classification method have not been systematically investigated. In this study, we developed a new classification method based on those established for the Simultaneous Item Bias Test. A simulation study also was conducted to determine if the effect size measure affects the Type I error and power rates for the LR DIF procedure across sample sizes, ability distributions, and percentage of DIF items included on a test. The results indicate that the inclusion of the effect size measure can substantially reduce Type I error rates when large sample sizes are used, although there is also a reduction in power.     

Controlling Bias in Both Constructed Response and Multiple‐Choice Items When Analyzed With the Dichotomous Rasch Model

Even though guessing biases difficulty estimates as a function of item difficulty in the dichotomous Rasch model, assessment programs with tests which include multiple‐choice items often construct scales using this model. Research has shown that when all items are multiple‐choice, this bias can largely be eliminated. However, many assessments have a combination of multiple‐choice and constructed response items. Using vertically scaled numeracy assessments from a large‐scale assessment program, this article shows that eliminating the bias on estimates of the multiple‐choice items also impacts on the difficulty estimates of the constructed response items. This implies that the original estimates of the constructed response items were biased by the guessing on the multiple‐choice items. This bias has implications for both defining difficulties in item banks for use in adaptive testing composed of both multiple‐choice and constructed response items, and for the construction of proficiency scales.     

2.5‐Year‐Olds Express Suspense When Others Approach Reality With False Expectations

The study investigated if 2.5‐year‐olds are susceptible to suspense and express tension when others' false expectations are about to be disappointed. In two experiments (N = 32 each), children showed more tension when a protagonist approached a box with a false belief about its content than when she was ignorant. In Experiment 2, children also expressed more tension when the protagonist's belief was false than when it was true. The findings reveal that toddlers affectively anticipate the “rude awakening” of an agent who is about to discover unexpected reality. They thus not only understand false beliefs per se but also grasp the affective implications of being mistaken. The results are discussed with recourse to current theories about early understanding of false beliefs.     

Creating Connections: Using a Narrative Approach in Career Group Counseling With College Students From Diverse Cultural Backgrounds

Changes in the world of work as well as demographic shifts toward a more diverse college population and workforce require changes in the way career counseling is provided on college campuses. The authors propose using a narrative group counseling approach to serve college and university students from various cultural backgrounds. Through this approach, students develop their own career/life stories, incorporating their cultural values, family and community considerations, and life roles.     

Examining Differential Rater Functioning Using a Between‐Subgroup Outfit Approach

When practitioners use modern measurement models to evaluate rating quality, they commonly examine rater fit statistics that summarize how well each rater's ratings fit the expectations of the measurement model. Essentially, this approach involves examining the unexpected ratings that each misfitting rater assigned (i.e., carrying out analyses of standardized residuals). One can create plots of the standardized residuals, isolating those that resulted from raters’ ratings of particular subgroups. Practitioners can then examine the plots to identify raters who did not maintain a uniform level of severity when they assessed various subgroups (i.e., exhibited evidence of differential rater functioning). In this study, we analyzed simulated and real data to explore the utility of this between‐subgroup fit approach. We used standardized between‐subgroup outfit statistics to identify misfitting raters and the corresponding plots of their standardized residuals to determine whether there were any identifiable patterns in each rater's misfitting ratings related to subgroups.     

Using a Group‐Centered Approach to Observe Interactions in Early Childhood Education

This study examined the value of using a group‐centered approach to evaluate process quality of early childhood education and care (ECEC). Is observed support of group processes a different aspect of classroom quality, and does it predict children's collaborative play in ECEC in the Netherlands? In two play situations, 37 teachers and 120 two‐ to four‐year‐old children were observed with the Classroom Assessment Scoring System (CLASS) Toddler and two new measures. In a two‐level structural equation model, teachers’ support of group processes was positively related to the CLASS domains and to children's collaborative play, over and above the effect of children's cognitive ability and social competence. These findings suggest that ECEC quality evaluation could be enriched by adding group‐centered indicators of classroom quality.     

Effects of Mindfulness‐Based Versus Interpersonal Process Group Intervention on Psychological Well‐Being With a Clinical University Population

This quasi‐experimental study compared a group mindfulness‐based intervention (MI) with an interpersonal process (IP) group intervention and a no‐treatment (NT) control condition in reducing psychological distress among 112 students at 2 universities. At postintervention, IP and MI group participants exhibited significant reductions in anxiety, depression, and interpersonal problems compared with the NT group. At the 6‐month follow‐up, only MI participants maintained the reduction in anxiety, depression, and academic problems; conversely, only IP participants maintained reductions in interpersonal problems.     

Creative Disruption — A Task‐Based Approach to Engaging With Original Works of Art

This paper examines the value of a task‐based approach to engaging with original works of art and focuses in particular upon the experiences of a group of PGCE Art and Design trainees when they visited an exhibition entitled, Air Guitar: Art Reconsidering Rock Music, to carry out given tasks. The extent to which a task‐based approach might creatively disrupt trainee teachers' thinking and the nature and significance of such thinking within an educational context is considered, as is the impact of the tasks on their developing understanding of the need for a diversity of approaches to critical engagement. It draws extensively upon selected responses to the tasks and the trainees' subsequent reflections on the experience. Comparisons are made between trainees' responses to particular works and references to these works that are detailed in the exhibition catalogue. This is undertaken with the intention of examining the interrelationship between understanding and knowledge when reading and responding to works of art in the gallery context. A further concern is to generate debate and challenge assumptions regarding the nature and dynamics of gallery based activities and contribute to a process of development and innovation in which all participants can play an active part.     

Enhancing Father Involvement in Low‐Income Families: A Couples Group Approach to Preventive Intervention

To address the problem of fathers’ absence from children's lives and the difficulty of paternal engagement, especially among lower income families, government agencies have given increasing attention to funding father involvement interventions. Few of these interventions have yielded promising results. Father involvement research that focuses on the couple/coparenting relationship offers a pathway to support fathers’ involvement while strengthening family relationships. Relevant research is reviewed and an exemplar is provided in the Supporting Father Involvement intervention and its positive effects on parental and parent–child relationships and children's outcomes. The article concludes with policy implications of this choice of target populations and the need to develop new strategies to involve fathers in the lives of their children.     

Implementation of Real‐World Experiential Learning in a Food Science Course Using a Food Industry‐Integrated Approach

Success skills have been ranked as the most important core competency for new food science professionals to have by food science graduates and their employers. It is imperative that food science instructors promote active learning in food science courses through experiential learning activities to enhance student success skills such as oral and written communication, critical thinking, problem solving, and team work. The aim of this study was to incorporate “real‐world” experiential learning into a food product development course. Undergraduate students enrolled in a food product development course worked on a semester‐long product development case study developed by the Instructor of the course and the Manager from ACH Food Companies, Inc. The case study was presented to students in the form of a product development competition. Students were placed into groups and given the task to develop a cake mix with specified parameters. At the end of the semester, student groups participated in a case study competition to showcase their product concepts. Each student group gave a PowerPoint presentation and was evaluated using selected criteria. Students rated the course overall as being very good. Students perceived the case study to be beneficial and informative regarding employer (ACH's) expectations. Students recommended that the Instructor minimize restrictions/specifications for product concepts and increase the quantity of course/laboratory meetings per week. This approach will continue to be used and further evaluated as an approach to incorporate active learning and provide food science undergraduates with a sense of employer expectations.     

Designing SQL Database Modules Using Correlation Coefficients and Linear Regression: A Content‐Centered Approach

Information systems educators are increasingly incorporating business intelligence and analytics topics into information systems training. Educators often create modules independently, which can be difficult to effectively design given most information systems trainers have no formal training in instructional design. This article incorporates a content‐centered design model, pebble‐in‐the‐pond (Merrill, 2002 ), and provides an example of the design process using the topic of predictive analytics that information systems educators can follow to create instructional modules using sound instructional design considerations.     

Detection of Differential Item Functioning with Nonlinear Regression: A Non‐IRT Approach Accounting for Guessing

In this article we present a general approach not relying on item response theory models (non‐IRT) to detect differential item functioning (DIF) in dichotomous items with presence of guessing. The proposed nonlinear regression (NLR) procedure for DIF detection is an extension of method based on logistic regression. As a non‐IRT approach, NLR can be seen as a proxy of detection based on the three‐parameter IRT model which is a standard tool in the study field. Hence, NLR fills a logical gap in DIF detection methodology and as such is important for educational purposes. Moreover, the advantages of the NLR procedure as well as comparison to other commonly used methods are demonstrated in a simulation study. A real data analysis is offered to demonstrate practical use of the method.     

Hope,Friends, and Subjective Well‐Being: A Social Network Approach to Peer Group Contextual Effects

Research on adolescence has previously shown that factors like depression and burnout are influenced by friendship groups. Little research, however, has considered whether similar effects are present for variables such as hope and subjective well‐being. Furthermore, there is no research that considers whether the degree of hope of an adolescent's friends is associated with well‐being over the individual's level of hope. Data were collected in 2012 from a sample of 15‐year‐olds (N = 1,972; 62% Caucasian; 46% identified as Catholic; 25% had professional parents) from the East Coast of Australia. Findings suggest that individuals from the same friendship group were somewhat similar in hope and well‐being. Multilevel structural equation modeling indicated that friendship group hope was significantly related to psychological and social well‐being.