Avoiding Bias in Randomised Controlled Trials in Educational Research

Randomised controlled trials (RCTs) are often seen as the 'gold standard' of evaluative research. However, whilst randomisation will ensure comparable groups, trials are still vulnerable to a range of biases that can undermine their internal validity. In this paper we describe a number of common threats to the internal validity of RCTs and methods of countering them. We highlight a number of examples from randomised trials in education and health care where problems of execution and analysis of the RCT has undermined their internal validity. However, awareness of these potential biases can lead to careful planning to avoid or reduce their occurrence. If good quality randomised trials are to inform policy and practice in education then rigorous trials need to be designed that are the least susceptible to threats to their validity.     

Successive reversals of a discriminated preference for signaled tailshock

Three rats received unmodifiable tailshock at random intervals in a shuttlebox. In a continuous-choice situation, Ss could choose between an auditory signal immediately preceding or immediately following the tailshock. Over repeated daily 3-h sessions, each S acquired a spatial discrimination indicating a strong preference for the signal preceding tailshock. This preference continued undiminished through two successive reversals of the position associated with signaled shock. This demonstration precludes explanations of the preference-for-signaled-shock phenomenon based upon primary reinforcement value or acquired value of the signal, position preferences, and overt modification of the aversiveness of the reinforcer through such means as postural adjustments. An explanation of recent failures to obtain the preference-for-signaled-shock effect is offered.     

Depth of processing and the quality of learning outcomes

Problems associated with earlier research on the relationship between depth of processing and quality of learning outcomes are first discussed. Then I present an interview study with 60 second year tertiary students which supports the hypothesis that depth of processing is related to the quality of learning outcomes.     

The effects of isolation and handling on reactivity to footshock and tactile dorsal stimulation

The effects of isolation and handling on the reactivity of rats to two tactile stimuli were assessed in two experiments. In the first experiment, reactivity to more intense footshock was increased in rats raised in isolation but was not affected by handling by an experimenter. In the second experiment, increased reactivity to airpuff stimuli presented to the dorsal surface of isolation-reared rats was reduced by handling. The data suggest that the effects of handling on shock-motivated behaviors are not due to altered reactivity to shock and that the effect of housing or handling must be considered in behavioral tests in which reactivity to tactile stimulation might play a role as dependent or independent variables.     

Secondary science teachers' knowledge base when teaching science courses in and out of their area of certification

This study examined the similarities and differences in experienced secondary science teachers' planning, teaching, and reflecting on their teaching, when teaching in their science area of certification and when teaching in another science area. The study also focused on the influence of these teachers' content knowledge, pedagogical knowledge, and pedagogical content knowledge on their planning, teaching, and reflecting. Experienced teachers were observed and interviewed while teaching classes in their science area of certification, and in another science area they were teaching for the first or second time. Both similarities and differences in teaching were found in the two areas for all three teachers. For example, their planning and postlesson reflections were similar in both areas. In the interactive phase of teaching more differences were observed. Many aspects of their teaching resembled that of expert teachers in other studies. In the unfamiliar science area, the teachers sometimes acted like novice teachers. However, they were able to draw upon their pedagogical knowledge to provide a framework for their teaching in both science areas. Their wealth of pedagogical knowledge, and pedagogical content knowledge for general science topics, seemed to sustain them in whatever content they were teaching. Recommendations for further study and implications for teacher education are discussed.