The Effect of the Most-Attractive-Distractor Location on Multiple-Choice Item Difficulty

Abstract

The arrangement of response options in multiple-choice (MC) items, especially the location of the most attractive distractor, is considered critical in constructing high-quality MC items. In the current study, a sample of 496 undergraduate students taking an educational assessment course was given three test forms consisting of the same items but the positions of the most attractive distractor varied across the forms. Using a multiple-indicators–multiple-causes (MIMIC) approach, the effects of the most attractive distractor's positions on item difficulty were investigated. The results indicated that the relative placement of the most attractive distractor and the distance between the most attractive distractor and the keyed option affected students’ response behaviors. Moreover, low-achieving students were more susceptible to response-position changes than high-achieving students.     

Choosing competency-based,humanistic and self-directed programs: The contribution of personality and reasoning

Fifty-five college women enrolled in competency-based, humanistic or self-directed education programs completed their California Personality Inventory and listed five reasons for selecting their program. CPI results indicated that self-directed students scored lower than other students on the Femininity Scale (p.001). Using Chickering's seven vectors of change as a framework, the study found competency-based students identifying purpose and competence, humanistic students identifying inter-personal relationships and integrity, and self-directed student identifying autonomy and purpose as reasons for enrolling in their nontraditional programs. The study used these results to question the mythology that adherents to different programs are of different personality types and to argue that differences in perceptions of purpose in education distinguish students in the three programs.     

Least Squares Models to Correct for Rater Effects in Performance Assessment

This study describes three least squares models to control for rater effects in performance evaluation: ordinary least squares (OLS); weighted least squares (WLS); and ordinary least squares, subsequent to applying a logistic transformation to observed ratings (LOG-OLS). The models were applied to ratings obtained from four administrations of an oral examination required for certification in a medical specialty. For any single administration, there were 40 raters and approximately 115 candidates, and each candidate was rated by four raters. The results indicated that raters exhibited significant amounts of leniency error and that application of the least squares models would change the pass-fail status of approximately 7% to 9% of the candidates. Ratings adjusted by the models demonstrated higher reliability and correlated slightly higher than observed ratings with the scores on a written examination.     

University-level teaching of Anthropogenic Global Climate Change (AGCC) via student inquiry

This paper reviews university-level efforts to improve understanding of anthropogenic global climate change (AGCC) through curricula that enable student scientific inquiry. We examined 152 refereed publications and proceedings from academic conferences and selected 26 cases of inquiry learning that overcome specific challenges to AGCC teaching. This review identifies both the strengths and weaknesses of each of these case studies. It is the first to go beyond examining the impact of specific inquiry instructional approaches to offer a synthesis of cases. We find that inquiry teaching can succeed by concretising scientific processes, providing access to global data and evidence, imparting critical and higher order thinking about AGCC science/policy and contextualising learning with places and scientific facts. We recommend educational researchers and scientists collaborate to create and refine curricula that utilise geospatial technologies, climate models and communication technologies to bring students into contact with scientists, climate data and authentic AGCC research processes. Many available science education technologies and curricula also require further research to maximise trade-offs between implementation and training costs and their educational value.     

The Efficacy of Policy Statements on Plagiarism: Do They Change Students' Views?

Concern about academic dishonesty has led to studies designed to explore its nature, prevalence, and causes. Nevertheless, to date there has been little empirical work designed to test the efficacy of measures to reduce cheating behavior. Many authors agree that there should be institutional statements about academic dishonesty that give definitions and state the penalties for cheating. The purpose of this study was to examine the efficacy of such statements in terms of their influence on perceived severity and perceived incidence of plagiarism in undergraduates. We found that having the students read a carefully worded statement about plagiarism was an effective way to change the perceptions of how seriously plagiarism breaches academic guidelines. Our data further suggest that providing guidance about avoiding plagiarism that encourages students to take a more serious view of the issue is likely to have positive effects on future behavior.     

Language and Williams Syndrome: How Intact Is "Intact"?

It has been claimed that Williams syndrome (WS), a rare neurodevelopmental disorder, is characterized by serious cognitive deficits alongside intact language. The syndrome is often used as a prime example of the modularity of an innate faculty for morphosyntactic rules. We challenge this claim and hypothesize that morphosyntax, although surprisingly good given WS level of mental retardation, is by no means intact. We make an initial test of this hypothesis through an analysis of the receptive language of a group of English-speaking WS individuals on a standardized morphosyntactic test. We then present an experimental study of expressive language that examines grammatical gender assignment in French-speaking WS patients. Despite a Verbal Mental Age selected to be higher than the chronological age of the young control group, these people with WS continue even in adulthood to show clear-cut deficits in their production of an aspect of morphosyntax that normal children acquire effortlessly very early. The results of the 2 studies, one focusing on receptive language and the other on expressive language, challenge the notion that comprehension and use of morphosyntactic rules in WS individuals are intact. The within-domain dissociations regarding the use of grammatical gender assignment across several sentence elements and their difficulties in understanding embedded sentences—two quintessentially linguistic skills—suggest that we must rethink the notion of spared, modular, language capacities in Williams syndrome. We conclude that WS language follows a different path to normal acquisition and may turn out to be more like second language learning.     

Development,validation, and factorial comparison of the McGill Self-Efficacy of Learners For Inquiry Engagement (McSELFIE) survey in natural science disciplines

Sociocognitive theory [Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall; Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44, 1175–1184. doi:10.1037/0003-066x.44.9.1175; Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248–287. doi:10.1016/0749-5978(91)90022-L] accords high importance to the mechanisms of human agency and how they are exercised through self-efficacy. In this paper, we developed and validated the McGill Self-Efficacy For Inquiry Engagement (McSELFIE) instrument with undergraduate students in natural science disciplines. We defined inquiry engagement as carrying out the practices of science (POS) that are supported by students’ personality characteristics (SPCs) and that result in achieving inquiry-learning outcomes (ILOs). Based on these theoretical perspectives, the McSELFIE is a 60-item, learner-focused survey that addresses three components that are theoretically important for engaging in scientific inquiry: (a) SPCs, (b) ILOs, and (c) POS. Evidence for construct and content validity were obtained by using experts’ judgments and confirmatory factor analysis with a sample of 110 undergraduate students enrolled in science disciplines. Internal consistency of the factors and instrument was also examined. The McSELFIE instrument is a reliable and valid instrument for measuring science undergraduate students’ self-efficacy for inquiry engagement. Matched pairs analyses were conducted among the instruments’ factors. Students reported the highest self-efficacy for openness, applying knowledge, and carrying out investigations. Students reported the lowest self-efficacy for extraversion, understanding metacognitive knowledge, and planning investigations. Theoretical and practical implications are discussed.     

Factor structures of Taiwanese teachers' ratings of ADHD: a comparison with U.S. studies

The purpose of this study was to compare factor structures from Taiwanese teachers' ratings with diagnostic definitions of attention-deficit/hyperactivity disorder (ADHD) as described in the DSM-III-R, DSM-IV, and ICD-10; to examine how factor structures of Taiwanese teachers' ratings of ADHD symptoms compare with factor structures reported in research using school-based American participants; and to examine gender differences. One hundred and twenty-one homeroom teachers from six public elementary schools in Taipei County, Taiwan, rated two boys and two girls randomly selected from their homerooms using the ADHD checklist. Findings from this study support the concurrent validity of the DSM-IV ADHD factor structures of hyperactivity-impulsivity and inattention. Comparability of these findings with school-based U.S. studies suggests the cross-cultural congruency of behaviors associated with ADHD.     

Making practical work work: using discussion to enhance pupils’ understanding of physics

Background: Practical work is widely seen as a necessary part of a good physics education, but convincing evidence that it impacts positively on pupils’ learning is scarce. Recent work suggests the use of talk and discussion might hold the key to making practical work more educationally productive.

Purpose: The research question that this study aims to answer is: Can the use of targeted discussion improve learning through practical work?

Sample: The study took place in a medium-sized (700-pupil, Years 7–13 (ages 11–18)) mixed-gender independent (fee-paying) day school in southern England. It is academically selective and public examination results are above the national average. The activities were carried out with three teaching groups (n = 73) with a fourth being used occasionally for comparison purposes (giving n = 97).

Design and methods: Pupils were given a number of practical tasks – some from the usual scheme of work, some devised for this project – and given specific time to discuss with a partner the various aspects of the practical such as predictions, devising methods, problem-solving, conclusions or explanations of phenomena. Questionnaires and booklets were used to gather pupil responses. Some audio recording and analysis of pupil-pupil discussion was also done.

Results: There was evidence from booklets pupils filled in during the practical that prescribing the use of discussion at various points in a practical task led to improved outcomes, particularly on the more cognitively challenging tasks. In particular, discussion was seen to turn incorrect responses into correct ones much more frequently than the reverse. In a later exam, pupils who had participated in the discussion activities performed better than a control group on questions which directly related to practical concepts, but performed similarly on the remaining exam questions.

Conclusion: Evidence gathered suggests a more nuanced pattern than that reflected in previous literature: that discussion during practical work can impact positively on pupils’ understanding of ideas related to practical work, but does not necessarily have a positive impact on their wider understanding.     

Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha

Abstract

Contract cheating happens when students outsource their assessed work to a third party. One approach that has been suggested for improving contract cheating detection is comparing students’ assignment submissions with their previous work, the rationale being that changes in style may indicate a piece of work has been written by somebody else. This approach is time consuming, but recent advances in machine learning and natural language processing suggest that it may be well suited to computerization. We trialed an early alpha version of Turnitin’s Authorship Investigate tool, which compares students’ submissions against their previous work. Twenty-four experienced markers from five units of study were asked to make decisions about the presence of contract cheating in bundles of 20 student assignments, which included 14 legitimate assignments and six purchased from contract cheating sites. We asked markers to determine if each assignment was contract cheating, then provided them with an Authorship Investigate report and let them change their decision. Marker accuracy at detecting contract cheating increased significantly, from 48% to 59% after using the report, with no significant difference in false positives. These findings suggest that software may be an effective component of institutional strategies to address contract cheating.