The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different programme assessment patterns

Audits of 23 degree programmes in eight universities showed wide variations in assessment patterns and feedback. Scores from Assessment Experience Questionnaire returns revealed consistent relationships between characteristics of assessment and student learning responses, including a strong relationship between quantity and quality of feedback and a clear sense of goals and standards, and between both these scales and students’ overall satisfaction. Focus group data helped to explain students’ learning responses but also identified ambivalent responses to the use of formative-only assessment, particularly when it was optional. Frequently, students were unclear about goals and standards, and found feedback unhelpful when assessment demands differed across modules, and when marking standards and approaches varied widely, making it difficult for feedback to feed forwards. The methodology underpinning the Transforming the Experience of Students through Assessment study described here has been used in more than 20 universities worldwide and is helping teachers to redesign assessment regimes, so that teachers’ efforts support learning better.     

Comparing global judgments and specific judgments of teachers about students' knowledge: Is the whole the sum of its parts?

Teachers' judgments about students' knowledge and skills can be global or specific depending on the diagnostic situation during teaching. We test the relationship between these judgments, their accuracy, and whether global judgment (GJ) accuracy can be measured by aggregating specific judgments (SJ). Judgments of 52 primary school teachers about their students' achievement in a standardized mathematics test were assessed. SJs and GJs correlated high. However, SJs were slightly more accurate than GJs. Additionally, teachers' GJ accuracy is not similar to the accuracy of aggregated SJs. We conclude that teachers use different judgment strategies for GJs and SJs.     

How reliable is marking in GCSE English?

In a re-marking exercise to investigate the reliability of marking in GCSE English examinations, the Principal Examiners' marks were adopted as 'true' marks for comparison with the actual or 'live' marks that had been awarded by the appointed examiners. A variety of approaches to investigating mark/re-mark reliability collectively led to the conclusion that marking reliability was particularly strong in the Higher tier of the specification that differentiates by outcome.     

Metacognitive activity in the physics student laboratory: is increased metacognition necessarily better?

In this study natural-in-action metacognitive activity during the student laboratory in university physics is explored, with an aim towards quantifying the amount of metacognition used by the students. The study investigates whether quantifying natural-in-action metacognition is possible and valuable for examining teaching and learning in these contexts. Video recordings of student groups working during three types of introductory physics laboratories were transcribed and then coded using a coding scheme developed from related research on mathematical problem solving. This scheme identifies a group’s general behaviour and metacognitive activity. The study recognizes that reliably identifying metacognition is challenging, and steps are taken to improve reliability. Results suggest that a greater amount of metacognition does not appear to improve students’ success in the laboratory—what appears to matter is whether the metacognition causes students to change behaviour. This study indicates that it is important to consider the outcome of metacognition, not just the amount.

Implications of the Introduction of the Doctor of Education Degree in British Universities: can the EdD reach parts the PhD cannot?

Abstract

At a time of continuing concern about the nature and meaning of doctoral education in the United Kingdom, this paper outlines the moves by an increasing number of universities towards ‘part‐taught’ doctorates. Focusing on the new degree of Doctor of Education now offered by over seven British universities, the paper outlines the differences between EdD and PhD in education, and questions the need for any differentiation. Arguing that the ‘statistics of failure’ in doctoral education in the United Kingdom continue to mount a challenge to universities in the United Kingdom, the paper argues for a refocus on process as well as product, where the goal should be on learner empowerment and transformation, rather than on the misassumptlons that doctoral study is primarily an ‘academic apprenticeship’. Finally, the paper indicates an agenda to establish doctoral programmes and structures capable of developing and fostering creative talent whether the individuals undertaking doctoral study are going to become professional researchers or researching and scholarly professionals.     

Real school improvement: is it in the eye of the beholder?

The outcomes of a school improvement programme were measured in 26 primary schools over one year. Authorised pupil and staff absences and staff turnover decreased, and unauthorised pupil absences significantly increased (p < 0.05). Fixed-term exclusions significantly decreased (p < 0.001). The self-esteem of 316 pupils aged between 4 and 11 before and after programme implementation showed a significant increase (p < 0.001). Staff self-esteem also increased significantly (p < 0.05) and provided a significant indicator of programme sustainability (p < 0.05). Thematic analysis of data from staff interviews demonstrated that the effectiveness of school improvement programmes rested on staff participation in terms of capacity, perceptions of change and stability of the school before programme implementation. These elements were incorporated into a model of school improvement. Goal Attainment Scores were significantly correlated with fidelity of programme participation (p < 0.05). Feedback indicated that there was more to school improvement programmes than simply buying one “off the shelf.”     

For want of a better word: unlocking threshold concepts in natural sciences with a key from the humanities?

The process of identifying threshold concepts invites experts to reflect on their discipline in a new way with the ultimate goal of improving learning and teaching. During a workshop to identify threshold concepts in biochemistry, we asked a group of natural scientists to explore ‘signification,’ a threshold concept from the humanities, as a mechanism to push them out of their comfort zones and recall how it feels to experience learning from a student’s perspective. In addition to accomplishing this goal, we subsequently realized that signification could also help us uncover and remedy ways in which use of scientific terminology impedes learning in biochemistry. Using the results of a survey of university teachers that aimed at refining a list of possible threshold concepts, we present three scenarios to illustrate the challenges that teachers and students encounter when attempting to cement a label (the signifier) and a concept (the signified) into a coherent sign. Based on these findings, we propose that teachers can better explore threshold concepts if they carefully consider the role terminology plays in learning and teaching. Thus we describe the ‘terminological canyon’ through which university teachers must journey in order to produce effective learning and teaching activities related to the threshold concepts. While the work described here pertains to biochemistry, we believe the process and findings can be generalized to a broad range of disciplines.     

Interactive or inactive? a consideration of the nature of interaction in whole class teaching

Recent learning theories and the suggested importance of ‘interactive’ approaches in national initiatives, (NLS, NNS) led to an ESRC‐funded action research project involving the University of Exeter and teachers in three schools—‘Using Talk to Activate Learners’ Knowledge' (TALK). This investigated how ‘interactive’ whole class teaching was and whether teacher‐pupil interactions permitted recoding of information and understanding. The findings corroborate those from Galton (1999 Galton M Hargreaves L Comber C Wall D Pell T (1999a) Inside the primary classroom—20 years on (London, Routledge) [Crossref] [Google Scholar]a, 1999 Galton, M, Hargreaves, L, Comber, C, Wall, D and Pell, T. (1999b). Changes in patterns of teacher interaction in primary classrooms: 1976–1996. British Educational Research Journal, 25(1): 23–37. [Taylor & Francis Online] [Google Scholar]b), Mroz (2000 Mroz, M, Smith, F and Hardman, F. (2000). The discourse of the Literacy Hour. Cambridge Journal of Education, 30(3): 379–390. [Taylor & Francis Online] [Google Scholar]) and English (2002 English, E, Hargreaves, L and Hislam, J. (2002). Pedagogical dilemmas in the National Literacy Strategy: primary teachers' perceptions, reflections and classroom behaviour. Cambridge Journal of Education, 32(1): 9–26. [Taylor & Francis Online] [Google Scholar]): that teacher‐led questioning and explanation still dominate, as do teachers' objectives. Interaction as participation was differentially experienced by higher and lower abilities, boys and girls, with few opportunities for pupil initiation or extended response. The study, however, provides a model for more fully analysing the varying forms and functions of teachers' questions and statements which may lead to a greater clarity in recognising ways to provide more effective discourse for learning.     

Confidence: A better predictor of academic achievement than self-efficacy,self-concept and anxiety?

In this paper we report the results from a study that assessed confidence together with scales measuring self-belief – i.e., self-efficacy, different kinds of self-concepts, and anxiety – among the 15-year old students from Singapore. A distinct confidence factor was identified in the domains of mathematics (N = 1940) and English (N = 1786). Our results show that confidence is: a) a robust individual differences dimension; b) that can be combined with accuracy information to obtain bias scores that may be useful for group comparisons and for identification of misconceptions about particular topics. Confidence as studied in our work to date has been c) the best predictor of achievement in both mathematics and English; d) is related to both cognitive and self-belief measures; and e) it captures much of the predictive variance of other self-beliefs that are, in turn, among the best known predictors of achievement.     

Making the grade: a history of the A–F marking scheme

This article provides a historical interpretation of one of the defining features of modern schooling: grades. As a central element of schools, grades—their origins, uses and evolution—provide a window into the tensions at the heart of building a national public school system in the United States. We argue that grades began as an intimate communication tool among teachers, parents, and students used largely to inform and instruct. But as reformers worked to develop a national school system in the late nineteenth century, they saw grades as useful tools in an organizational rather than pedagogical enterprise—tools that would facilitate movement, communication and coordination. Reformers placed a premium on readily interpretable and necessarily abstract grading systems. This shift in the importance of grades as an external rather than internal communication device required a concurrent shift in the meaning of grades—the meaning and nuance of the local context was traded for the uniformity and fungibility of more portable forms.     

“Hell is Other People”: the Interpretation of Inter-human Relationship in AClockwork Orange from the Perspective of Existentialism

Anthony Burgess’s novel A Clockwork Orange explores human being’s predicament of freedom—the contradiction between Libertarianism and Totalitarianism. The issues Burgess explores in this fiction share affinities with what Sartrean existentialism elucidate. This paper attempts to discuss the inter-human relationship between the protagonist—Alex and other two important characters from the perspective of Sartre’s elaboration of being-with-others, to find the evidence for Sartre’s famous dictum hell is the other people.     

Learning from reviewing peers’ concept maps in an inquiry context: Commenting or grading,which is better?

In peer assessment, both receiving feedback and giving feedback (reviewing peers’ products) have been found to be beneficial for learning. However, the different ways to give feedback and their influence on learning have not been studied enough. This experimental study compared giving feedback by writing comments and by grading, to determine which contributes more to the feedback providers’ learning. Secondary school students from Russia (n = 51) and the Netherlands (n = 42) gave feedback on concept maps during a physics lesson. The lesson was given in an online inquiry learning environment that included an online lab. Students gave feedback in a special Peer Assessment tool, which also provided assessment criteria. Findings indicate that post-test knowledge scores were higher for students from the commenting group. The difference between the groups was largest for the low prior knowledge students. Possible educational implications and directions for further research are discussed.     

Othello: What is the Position of Race in a Multicultural English Classroom?

Making literature and learning relevant to students is inextricably linked to giving them ownership in an English classroom. This essay looks at the Eurocentric nature of the study of literature and the problems that creates when trying to make canonical texts relevant through the paradigm of race. The ambiguous role of literature as an empowering and oppressive force is discussed from the perspective of the student and teacher alike. Race relations in the classroom are explored through the public and private dialogue initiated through the teaching of Othello in an East End comprehensive school.     

Ways of knowing in doctoral examination: How well is the doctoral regime?

The article draws on findings from the PhD Examination Project at the SORTI Research Centre of The University of Newcastle, Australia. It focuses on an analysis of the roles of examiner and supervisor, in relation to the candidate, as seen through the lens of Habermas's ‘Ways of Knowing’ theory. On the basis of this, it has been postulated that the dominant text in the PhD examination process may work to constrain the generation of new knowledge rather than encourage it. The paper explores practical implications for research training and questions the current well being of the doctoral regime.     

Theory of mind in dogs: is the perspective-taking task a good test?

Udell, Dorey, and Wynne (in press) have reported an experiment in which wolves, shelter dogs, and pet dogs all showed a significant preference for begging from a person who faced them (seer) over a person whose back was turned to them (blind experimenter). On tests with the blind person’s eyes covered with a bucket, a book, or a camera, pet dogs showed more preference for the seer than did wolves and shelter dogs. We agree with the authors’ position that most of these findings are best explained by preexperimental learning experienced by the subjects. We argue, however, that the perspective-taking task is not a good test of the domestication theory or of the theory of mind in dogs. The problem we see is that use of the perspective-taking task, combined with preexperimental learning in all the subjects, strongly biases the outcome in favor of a behavioral learning interpretation. Tasks less influenced by preexperimental training would provide less confounded tests of domestication and theory of mind.     

Research League Table: is there a better way?

The results of the 1996 Research Assessment Exercise were presented in several places, notably in The Higher, in the form of a league table. The article discusses how this table was compiled and points to a number of fundamental problems in the way in which the data were handled. It goes on to discuss some more general issues in the calculation of league tables and shows the extreme sensitivity of the RAE data to certain arbitrary assumptions that have to be made. It concludes by urging caution on any who may be tempted to draw conclusions from such tables as well as on those who might compile such tables in the future.