Multiple‐choice and true/false tests: myths and misapprehensions

Examiners seeking guidance on multiple‐choice and true/false tests are likely to encounter various faulty or questionable ideas. Twelve of these are discussed in detail, having to do mainly with the effects on test reliability of test length, guessing and scoring method (i.e. number‐right scoring or negative marking). Some misunderstandings could be based on evidence from tests that were badly written or administered, while others may have arisen through the misinterpretation of reliability coefficients. The usefulness of item response theory in the analysis of academic test items is briefly dismissed.     

The limits of school choice: some implications for accountability of selective practices and positional competition in Australian education

The application of market principles to educational provision continues to attract governments across the globe, despite an international body of literature suggesting that marketisation can exacerbate inequalities. In light of a renewed policy push in Australia towards accountability via a market model, this paper analyses the impact of existing school choice policies in the state of Victoria, with particular reference to educational provision in an area of social disadvantage in Melbourne's north. This analysis challenges the claims of the now normalised market model, but also points to the need to expand research into this theme, which has attracted relatively little critical attention in Australia. I argue that both the operation of existing policies and the direction of new proposals imply an uneven system of accountability which applies different standards to increasingly polarised ‘closed’ and ‘open’ schooling sectors.     

Developmentally invariant dissociations in children's true and false memories: not all relatedness is created equal

The role of categorical versus associative relations in 5-, 7-, and 11-year-old children's true and false memories was examined using the Deese-Roediger-McDermott (DRM) paradigm and categorized lists of pictures or words with or without category labels as primes. For true items, recall increased with age and categorized lists were better recalled than DRM lists. For false items, recall increased with age except for picture lists, there were no differences between categorized and DRM lists and no effect of priming, and there were fewer false memories for pictures than words. Like adults, children's false memories are based on associative not thematic relations, whereas their veridical memories depend on both. This new, developmentally invariant dissociation is consistent with knowledge- and resource-based models of memory development.     

Truth induction in young maltreated children: the effects of oath-taking and reassurance on true and false disclosures

OBJECTIVE: Two studies examined the effects of the oath or reassurance ("truth induction") on 5- to 7-year-old maltreated children's true and false reports of a minor transgression. METHODS: In both studies an interviewer elicited a promise to tell the truth, reassured children that they would not get in trouble for disclosing the transgression, or gave no instructions before questioning the child. In Study 1, children were encouraged to play with an attractive toy by a confederate, who then informed them that they might get in trouble for playing. In Study 2, a confederate engaged children in play, but did not play with the attractive toy. RESULTS: In Study 1, the oath and reassurance increased disclosure among children who would qualify as competent to take the oath. In Study 2 neither the oath nor reassurance increased false reports among children who would qualify as competent, whether yes/no questions or tag questions were asked. Among non-competent children, reassurance (but not the oath) increased false reports. Children were more likely to accuse the confederate of the transgression than to implicate themselves. CONCLUSIONS: The results suggest that a promise to tell the truth may increase true disclosures without increasing false allegations. Reassurance that specifically mentions the target activity also increases true disclosures, but may increase acquiescence among some children. PRACTICE IMPLICATIONS: A child-friendly version of the oath may be a useful addition to child interviews.     

＂虚实结合＂的自动控制原理教学方式研究

对＂虚实结合＂教学方法在自动控制原理课程教学中的应用进行了探索研究,将Matlab仿真融入理论教学中,＂虚＂拟的仿真结果使抽象的理论知识简单化,同时结合＂实＂际的控制装置设立项目任务,以任务驱动学生自主学习。实践证明这种教学方式既提高了教学效率,也培养了学生学习的兴趣和实践能力。     

The influence of feedback of diagnosis and executive function skills on rates of false positive and false negative outcomes for ADHD

This study examined executive function (EF) skills and self-reported symptoms of ADHD. EF skills were measured to determine whether skills were different between groups that reported clinical levels of ADHD symptoms (clinical group) and nonclinical levels of ADHD symptoms (nonclinical group). EF skills in the nonclinical group were also investigated to determine whether differences existed for those who received false positive and negative feedback about an ADHD diagnosis. Results indicated statistically and clinically significant differences in EF skills between the clinical and nonclinical group. Participants in the nonclinical group who reported elevated scores on the ADHD screener after receiving false positive feedback had statistically significantly different EF function scores, but not clinically significant scores, compared to the participants in nonclinical group who received negative feedback.     

Bones,boys, bombs and booze: an exploratory study of the reliability of marking dissertations across disciplines

The primary aim of this study was to evaluate the reliability of the University’s Masters’ level (M‐level) generic assessment criteria when used by lecturers from different disciplines. A further aim was to evaluate if subject‐specific knowledge was essential to marking these dissertations. Four senior lecturers from diverse disciplines participated in this study. The University of Teesside’s generic M‐level assessment criteria were used and formatted into a grid. The assessment criteria related to the learning outcomes, the depth of understanding, the complexity of analysis and synthesis and the structure and academic presentation of the work. As well as a quantitative mark, a qualitative statement for the reason behind the judgement was required. Each lecturer provided a dissertation that had previously been marked. All participants then marked each of the four projects using the M‐level grid and comments sheet. The study found very good inter‐rater reliability. For any one project, the variation in marks from the original mark was no more than 6% on average. This study also found that subject‐specific knowledge was not essential to marking when using generic assessment criteria in terms of the reliability of marks. The authors acknowledge the exploratory nature of these results and hope other lecturers will join in the exploration to test the robustness of generic assessment criteria across disciplines.     

An alternative method of answering and scoring multiple choice tests

A simple modification to the method of answering and scoring multiple choice tests allows students to indicate their estimates of the probability of the correctness of the multiple choice options for each question, without affecting the validity of the assessment. A study was conducted using a test that investigated common misconceptions in mechanics. The study showed that for assessment purposes this method gives results that are very similar to results obtained by students who answer in the traditional manner. Year 12 Physics students (N=85) were randomly allocated to two treatment groups: one received a standard format multiple choice test, the other a test format allowing students to select more than one response in a multiple choice test, and to distribute their marks among their chosen optionsl An analysis of the students' uncertainties is used to argue that not only can students appeal to different conceptions in different contexts, but that they can also hold conflicting conceptions with respect to a single context.     

Intrinsic motivation and school misbehavior: some intervention implications

The intrinsic motivational concepts of self-determination, competence, and relatedness to others are highlighted to underscore their value in identifying motivational underpinnings for deviant and devious behavior. These constructs are woven into a working schema to differentiate intentional misbehavior in terms of whether it is reactive or proactive. Finally, from the perspective of such motivational thinking, implications for formal intervention research and experimental practice are illustrated.     

Inter-method reliability of school effectiveness measures: a comparison of value-added and regression discontinuity estimates

Value-added (VA) measures are currently the predominant approach used to compare the effectiveness of schools. Recent educational effectiveness research, however, has developed alternative approaches including the regression discontinuity (RD) design, which also allows estimation of absolute school effects. Initial research suggests RD is a viable approach to measuring school effectiveness. The present study builds on this pioneering work by using RD and VA designs to estimate school effects at system and school level, comparing estimates from several measurement designs. The study uses a large English dataset (N = 148,135) spanning 342 schools, 10 local authorities, 6 consecutive school year groups (UK Years 3–9) across 3 years. RD is found to be a suitable approach for system-level absolute school effect estimates. Cross-sectional and longitudinal measures are found to lead to markedly different estimates when comparing individual schools. The results also reinforce the need to treat measures based on a single cohort with extreme caution.     

Evaluating individualized educational programs: A recommendation and some programmatic implications

The character of urban schools and the practices of school personnel have been influenced by a variety of sociopolitical factors. Currently, the Education for All Handicapped Children Act of 1975 (P.L. 94–142) is having impact on schools in general and especially on urban schools, which typically have many students affected by this law. This paper describes certain aspects of the law which have posed problems for schools, and makes suggestions about how such problems can be resolved. Schools are required to identify goals and to evaluate educational progress on anindividual student basis. This requirement can be met by using the Goal Attainment Scaling (GAS) technique. Using an illustrative GAS-modified individualized educational program (IEP), advantages and disadvantages of this technique are discussed. Certain potential problems are identified and suggestions made so that administrators and other school personnel can be successful in their attempts to comply with requirements to evaluate educational progress in conjunction with IEPs.     

Multiple voices in tests: towards a macro theory of test writing

Tests, especially high-stakes ones, occupy a prominent role in education and impact on students’ personal and professional trajectory. Therefore, it is crucial that they are well understood. Enhancing understanding of the workings of tests requires a multidisciplinary approach, one that treats tests not only as assessment instruments, but also as sociolinguistic and sociopolitical products. Using data collected by means of non-participant observation, stimulated recall and one-to-one interviews with writers of GCSE and IGCSE tests, this paper attempts to provide a macro-analysis of the process of test writing. It does so by identifying the various actors and discourses that directly or indirectly take part in test writing. It uses Bakhtin’s framework of voice as a tool to facilitate this. The paper concludes by proposing a macro model of test writing, one that explains the macro process of test writing by capturing the main voices involved in it and their interrelationships.     

Effective teaching and values: some implications for research and teacher appraisal

Teacher effectiveness research has tended to neglect the analysis of values in two senses: the general values associated with the processes of education, and the more specific values underlying effective teaching. The possibilities for re‐conceptualising teacher effectiveness, by incorporating a values dimension, are illustrated through two examples: effectiveness in developing independent learning and effectiveness in achieving a classroom climate characterised by inclusiveness. The potential contribution to teacher effectiveness outside, as well as inside, the classroom is explored through a discussion of the strengths and problems inhering in the English government's adoption of the Hay McBer model of teacher effectiveness for the assessment of teacher performance. The contribution of teacher self‐evaluation to the process of the identification of values underlying effectiveness is discussed.     

A question of choice: The implications of assessing expressive writing in multiple genres

The Maryland School Performance Assessment Program (MSPAP) tests include an expressive writing task in which students at grades 3, 5, and 8 can choose to write about any topic they wish in the form of either a story, poem, or play. This test design feature provided the opportunity to investigate what factors contribute to students' choice of genre, how scorers apply a single expressive writing rubric to a range of genres, and whether these genres constitute equivalent tasks for measurement and reporting purposes. Our study, which combined analysis of statewide score data, 300 randomly selected student texts, questionnaires given to teacher-scorers, and interviews with students, argues strongly for the validity of this choice task as a measure of expressive writing and demonstrates that choice of genre both increases writers' engagement and enhances the fairness of the assessment by giving all students the best opportunity to demonstrate proficiency in this learning outcome. By highlighting several features of student texts that complicate scoring, the study also suggests that accuracy and consistency might be improved by

• 1.1) providing additional sample papers during training,
• 2.2) attending to scorers' assumptions regarding several key concepts, especially “originality,” and
• 3.3) adjusting the ways that training for focused holistic scoring generally takes place.

The study concludes that the perceptions of students, scorers, and classroom teachers are critical to the ongoing development of writing assessments that offer students increasing control and choice.