Reassessing RCTs as the ‘gold standard’: synergy not separatism in evaluation designs

ABSTRACT

Randomised controlled trials (RCTs) are increasingly used to evaluate educational interventions in the UK. However, RCTs remain controversial for some elements of the research community. This paper argues that the widespread use of the term ‘gold standard’ to describe RCTs is problematic, as it implies that other research methods are inferior. The usefulness of RCTs can be greatly enhanced when used in conjunction with implementation-specific measures (e.g. observation tools, attitude/engagement surveys and interviews). The proposal is advanced through case studies of two evaluations. One relates to the development of science subject leader skills and expertise at primary school level and the other to co-operative learning of primary maths. Both evaluations randomised schools to the intervention or the business-as-usual control, and compared impact using subject knowledge tests. Integral to each study was a process evaluation which looked at evidence from classroom practice along with feedback from the teachers and pupils themselves. We contend that this enabled much more holistic and richly interpretative pieces of research. The paper concludes that privilege for particular paradigms should be set aside when designing effective evaluations of educational interventions, and that it is insufficient to ask ‘what works?’ without also asking ‘why?’, ‘where?’ and ‘how?’.     

Career plateauing: Implications for career development specialists

Conclusion While career plateauing is a major challenge facing contemporary organizations, possible solutions are available. The career development specialist possessing a variety of skills geared toward both proaction and reaction can be instrumental in designing interventions which minimize the negative impact of plateauing. While career plateauing can be a devastating problem for many high achieving, self-sacrificing baby boomers, for others it may present an opportunity to escape the internal pressures for success. Much of what happens to a plateaued employee is determined by the resources possessed by the employee as well as the organization's response to the employee. The activities of the career development specialist constitute much of what is considered the organization's response. A career plateau may mark the beginning of a period characterized by frustration and depression or it may present the opportunity for a new challenge or reappraisal of life direction and goals.     

Politics,Poverty and Education in Rural South Africa

This article examines the relationship between poverty and education in South Africa, and how its conceptualisation has changed historically. By analysing two major inquiries into poverty conducted by the Carnegie Corporation (in 1929‐32 and 1982‐84), it highlights the political nature of poverty and also its racialisation in South Africa. Using material from a peri‐urban research study, it extends the analysis to include the underprovision of schooling, gender relationships of poverty and also child labour. The article illustrates how the relationship between poverty and education has been differently constructed in different discourses, and concludes by considering the challenges of developing policies to address the education/poverty nexus in the rural areas of post‐apartheid South Africa.     

The Role of Drawing in Graphic Design and the Implications for Curriculum Planning

Over a number of years a programme of research has been carried out into the role of drawing in design processes, primarily the graphic design process, and on the possible implications for curriculum planing for design students. From the findings of an original in-depth analysis of the use of drawing in the commercial environment of the design consultant, various initiatives in educational planning and assessment in England and Scotland have been fostered and a contribution has been made to the ongoing debate in drawing studies     

Mixed methods,mixed outcomes? Combining an RCT and case studies to research the impact of a training programme for primary school science teachers

A randomised controlled trial (RCT) and a series of case studies were used to determine the impact of two variants of an intervention (a professional development programme) aimed at improving primary school science teachers’ subject and pedagogic content knowledge, and enhancing their subject leadership ability. Ninety-six schools were randomly assigned to full or partial treatment groups or a ‘business-as-usual’ control group. Quantitative data were collected from teachers and pupils through an assessment of scientific knowledge based on standardised assessment items. Qualitative data were collected through interviews and lesson observation initially in thirty case study schools. There were three data collection points: pre- and post-intervention, and one year later.

[Guskey, T. (1986). Staff development and the process of teacher change. Educational Researcher, 15(5), 5–12.] Levels of Professional Development Evaluation model was used as the analysis framework. The quantitative data from the teachers’ subject knowledge assessment indicated neither the full nor the partial training programmes had a statistically significant impact on teachers’ performance. In contrast, the qualitative data suggested that many teachers in the full treatment group believed that their subject knowledge had improved and reported increased confidence in their teaching of science. Lesson observations provided corroborating evidence of change in teachers’ practice, and some modest evidence of wider change in schools. There was no statistically significant improvement in pupil performance in subject knowledge assessments when teachers had participated in the intervention. In the context of research methods, the study suggests that a mixed-methods approach to evaluation is likely to yield a more rounded and nuanced picture of the overall impact of an intervention.     

Becoming and being a talented undergraduate student

This article reports on a case study of one New Zealand university faculty involved in the second phase of a three-phase study investigating the experiences of talented undergraduate students. Talented undergraduate students are a largely forgotten group in research. The current study sought to investigate who the talented students were, and then what their lived experiences as talented undergraduate students were. The study involved 128 undergraduate students who provided information about their experiences as high achieving students in an undergraduate degree program. Approximately 10% of all students enrolled across five different undergraduate degree programs in the faculty were defined as talented undergraduate students. These students were ethnically diverse and largely older than we had anticipated. The majority had not previously been identified as talented and many had been largely unsuccessful educationally, prior to embarking on their undergraduate studies. Several students experienced challenging personal circumstances, such as financial hardship and extensive family responsibilities. The grit or resilience demonstrated by these students seemed to explain the essence of the phenomenon of being a talented undergraduate student in this faculty.