Mitigating Illusory Results through Preregistration in Education

Like performance-enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the production of illusory results. In this article, we examine the processes that lead to illusory findings and describe their consequences. We borrow from an approach used increasingly by other disciplines—the norm of preregistering studies. Specifically, we examine how this practice of publicly posting documentation of one's prespecified hypotheses and other key decisions of a study prior to study implementation or data analysis could improve scientific integrity within education. In an attempt to develop initial guidelines to facilitate preregistrations in education, we discuss the types of studies that ought to be preregistered and the logistics of how educational researchers might execute preregistrations. We conclude with ideas for how researchers, reviewers, and the field of education more broadly might speed the adoption of this new norm.     

Questionnaires as interventions: can taking a survey increase teachers’ openness to student feedback surveys?

Administrators often struggle in getting teachers to trust their school’s evaluation practices – a necessity if teachers are to learn from the feedback they receive. We attempted to bolster teachers’ support for receiving evaluative feedback from a particularly controversial source: student-perception surveys. For our intervention, we took one of two approaches to asking 309 teachers how they felt about students evaluating their teaching practice. Control participants responded only to core questions regarding their attitudes towards student-perception surveys. Meanwhile, treatment participants were first asked whether teachers should evaluate administrators in performance reviews and were then asked the core items about student-perception surveys. Congruent with cognitive dissonance theory, this juxtaposition of questions bolstered treatment teachers’ support for using student surveys in teacher evaluations relative to the control group. We discuss the implications of these findings with respect to increasing teacher openness to alternative evaluation approaches, and consider whether surveys show promise as a vehicle for delivering interventions.     

One Hundred Children: baseline assessment of literacy in the early years of education

One Hundred Children is an Australian study of literacy learning in the early years of education. The larger study (Hill, Comber, Louden, Reid and Rivalland, 1998) includes detailed case studies of children’s literacy learning in their homes, pre-schools, day-care centres and schools. This paper reports on the assessment framework developed during the study and provides a brief overview of children’s performance on items in that framework. The study demonstrates that most children made rapid and substantial progress in literacy learning and identifies the literacy domains in which children learned most during their first year at school. A number of advantages of the use of baseline measures for assessing young children’s progress in literacy are also highlighted.