Methodological quandaries in studying process and outcomes in peer assessment

Peer assessment is very various in its implementation. Six studies of peer assessment are reviewed, four of them in higher education. A literature review is followed by five empirical studies. Strengths and weaknesses of each study are considered and issues are raised. Variables in peer assessment needing further exploration are extricated – in sample definition, specification of type of peer assessment, organisational nature of the intervention, and the measures which are used to evaluate change. It is suggested that further experimental and quasi-experimental studies are necessary to contrast one or two of these variables at a time.     

Scaling: An ITEMS Module

Scaling is the process of constructing a score scale that associates numbers or other ordered indicators with the performance of examinees. Scaling typically is conducted to aid users in interpreting test results. This module describes different types of raw scores and scale scores, illustrates how to incorporate various sources of information into a score scale, and introduces vertical scaling and its related designs and methodologies as a special type of scaling. After completion of this module, the reader should be able to understand the relationship between various types of raw scores, understand the relationship between raw scores and scale scores, construct a scale with desired properties, evaluate an existing score scale, understand how content and standards information are built into a scale, and understand how vertical scales are developed and used in practice.     

Reflective peer coaching: Crafting collaborative self-assessment in teaching

Reflective peer coaching is a formative model for improving teaching and learning by examining intentions prior to teaching, then reflecting upon the experience. The goal of reflective peer coaching is to promote self-assessment and collaboration for better teaching and ultimately better learning. There are obvious benefits to colleagues collaborating and sharing ideas, thoughts, and observations. However, many models of assessing teaching effectiveness focus on summative evaluation in which colleagues observe each other once or twice a year and fill out institutional evaluation forms. Rarely do colleagues engage in formative conversations about teaching that are guided by the instructor's personal goals and objectives. Reflective peer coaching necessitates a ten-minute planning conversation prior to the actual lesson and a ten-minute reflective conversation after the lesson. These conversations happen regularly and frequently to build self-awareness and self-assessment of the personal craft of teaching. The following article outlines the dynamics of the reflective peer coaching process as a formative assessment model that leads to better learning through improved teaching.     

Complacent and conflicting scientific expertise in British and American drug regulation: clinical risk assessment of triazolam

This paper presents a case study in the production and interpretation of regulatory science, focusing on the conflicting British and American clinical risk assessments of triazolam (Halcion) - the most controversial sleeping pill in the world. The regulation of triazolam is shown to be more permissive in the USA than in the UK. Six principal socio-political factors are put forward to explain this; differential regulatory trust; regulators' socio-technical data selections; medico-scientific disciplinary influences; organizational and professional interests; conflicts of interest of expert advisers; and the growth of the neo-liberal regulatory state. The risk assessments of both the British and American regulatory agencies are shown to be complacent relative to technical standards which the agencies themselves later accepted. It is suggested that, when the interests of pharmaceutical manufacturers and patients diverge, regulatory assessments are crucially affected by whether regulators are predisposed to award the benefit of the scientific doubts to the manufacturers or patients, and by the balance of such predispositions both within and between regulatory institutions of scientific expertise. The triazolam case indicates that the amount of trust placed in the pharmaceutical industry by the British American regulatory systems may hamper detection of flaws in manufacturers' medical data in a timely manner and, as a consequence, compromise patients' interests. Some policy implications for drug regulation are sketched.     

Four-year changes in college athletes' ethical value choices in sports situations.

Positive values for fairness in competition are supposed to undergird the behavior of athletes engaged in sport. Whether athletes' values actually develop over 4 years in a college that emphasizes character development is the focus of this study. Athletes' (N = 631) use of deontological ethics (Hahm, Beller & Stoll, 1989) in 21 sports value dilemmas were evaluated. At entrance, as well as near graduation, intercollegiate athletes' value scores were lower than intramural athletes' scores. Both groups' scores declined while they were in college. Individual-sport athletes had higher scores than team-sport athletes but manifested a greater decline over 4 years. The findings are consistent with other studies that show decreases in "sportsmanship orientation" and an increase in "professional" attitudes associated with participation in sport.