Examining how teachers judge student writing: an Australian case study

This paper reports a 3‐year (1999–2001) Australian study of teacher judgement of student writing. It analyses teachers' talk to discover how they arrive at such judgements. It focuses on the processes teachers use as they read and appraise student writing, as distinct from judgements recorded as numerical or letter grades. It identifies and discusses a set of data‐based indexes the teachers rely on to constitute their judgement. In so doing, the ‘global’ standard‐setting of external assessment (judging the quality of student work against stated standards), and the ‘local’ of teacher judgement (based on the richness of what teachers bring to the task) are reconsidered. This study notes how teacher judgement of student coursework may be intertwined with and shaped both by officially authorized curriculum materials, syllabus documents, and assessment practices, and by other essentially private, local ways of knowing.     

The Value of a Programme-wide Approach to Assessment

Reliability is important in the academic judgements that higher education institutions pass to employers, graduate schools and other stakeholders. However, there are well-known limits to the levels of reliability that can be reached at reasonable cost and without sacrificing validity. It is argued that a systemic approach to the assessment of student learning shows how resources may be freed up to invest in securing more reliable assessments where they are desirable and might reasonably be had.     

A Framework for Explaining Teachers’ Diagnostic Judgements by Cognitive Modeling (DiaCoM)

Research on diagnostic competencies of teachers nowadays raises the question which person or situational characteristics moderate judgement accuracy. Beside this correlational approach, a stronger interest in understanding the cognitive processes involved in the genesis of diagnostic judgements has emerged. To address the theoretical gap regarding cognitive processes underlying diagnostic judgements, we propose a framework, called DiaCoM (Explaining Teachers’ Diagnostic Judgements by Cognitive Modeling). It aims at supporting (existing or envisioned) research that strives to test cognitively oriented explanations for processes and products of diagnostic judgements of teachers.     

Teachers and tests: assessing pupils' reading achievement in primary schools

The purpose of the present study was to examine validity aspects of teachers’ judgements of pupils’ reading skills. Data come from Sweden's participation in the Progress in International Reading Literacy Study (PIRLS) 2001, for Grades 3 and 4. For pupils at the same achievement levels, as measured by PIRLS 2001 test, teachers’ judgements of pupils’ achievement levels varied from one teacher to another. Moreover, there were significant differences between teachers’ judgements in Grades 3 and 4. Teachers in Grade 3 who had taught their pupils for almost 3 years showed higher correspondence between their judgements and pupil achievement within classrooms than 4th-grade teachers who, typically, had only taught their pupils for approximately 1 semester at the time of the data collection. The results indicate that teachers’ judgements and tests can be useful within classrooms, but that teachers may need external assessments to calibrate judgements over classrooms.     

The analytical lens: developing undergraduate students’ critical dispositions in undergraduate EAP writing courses

ABSTRACT

This paper comprises findings from three parallel case studies within the broad framework of English for Academic Purposes (EAP). These provide results from classroom-based action research conducted over two years working with Semantics, Specialisation and axiological cosmologies from Legitimation Code Theory (LCT). Each author shares how one or both of these LCT dimensions have guided their practice in the teaching of English for academic writing and specifically, the development of their students’ critical dispositions by teaching them how to apply lenses to analyse texts so that they may make informed judgements. Data from classroom interventions and student writing are provided. It is hoped that this paper elicits further discussion in the growing field of applying LCT and may draw attention to the role of the EAP teacher as expert in linguistics and the need to focus more on knowledge in EAP writing courses.     

(Mis)appropriations of criteria and standards‐referenced assessment in a performance‐based subject

This paper draws on semi‐structured interview data and participant observations of senior secondary Physical Education (PE) teachers and students at two school sites across 20 weeks of the school year. The data indicated that the teachers in this study made progressive judgements about students’ level of achievement across each unit of work without explicit or overt reference to the criteria and standards represented in the schools’ work programmes and in the Senior PE syllabus. The teachers’ justification for such an approach was that the criteria and standards had become for them sufficiently ‘internalised’. Determining students’ levels of achievement was for the teachers somewhat ‘intuitive’, being reliant on their memory of students’ performances, and influenced by the construct‐irrelevant affective characteristics of the students. It is argued in this paper that such construct‐irrelevance compromised the construct validity and possible inter‐rater reliability of the decisions made and advantaged some students and marginalised others on the basis of characteristics that were not specifically related to the learning expected from following the syllabus. The potential inequities of such an approach are discussed and suggestions are made for the consolidation of the validity and reliability of teachers’ judgements.     

Write on or write off? An exploration of Asian international students’ approaches to essay writing at an Australian university

This paper investigates the approaches taken to essay writing by five Asian international students at an Australian university. Analysis of their in‐depth interviews reveals links between their perceptions of learning, their perceptions of essay writing, their motivation for completing the task, and their awareness of the structural conventions of the essay as a genre. These links are apparent, irrespective of whether the students were taking a deep or a surface approach to the task. In addition, analysis of the interviews shows that the students who were taking a deep approach to writing had developed high levels of cross‐cultural awareness, specifically in relation to the learning cultures, and the discursive structures of academic writing in Australia and their home country. Implications for the practice of teaching academic writing within the context of increasing cultural diversity in higher education are discussed.     

Mapping the landscape of writing instruction in New Zealand primary school classrooms

Writing instruction in New Zealand occurs in a context with potential for variability in curriculum and delivery. The national curriculum is broad; self governing schools are to interpret and apply as appropriate to their local context. There are no mandated tests, nor external examinations until the last three years of school. Schools report to the Ministry about achievement in Years 1–8 against national standards in writing, based on overall teacher judgements. The nature of this context supports the notion of drawing on several sources to describe the current landscape of writing instruction: policy documents, specifically the curriculum and standards; national tools and resources for professional learning; the limited existing research base relating to writing in New Zealand, and a study designed to extend this latter body of work by surveying teachers about their practices. The existing research largely concerns the practices of exemplary teachers or relates to investigating the effects of professional development interventions. The survey reported provides the most direct evidence of actual practice. Survey responses (N = 118) came from, on average, moderately experienced teachers who reported a relatively high level of confidence regarding aspects of teaching writing. Results suggest that surveyed teachers reflected the themes identified in the New Zealand teacher based and intervention research, which in turn reflected policy initiatives. The combined evidence indicates the influence of the ‘teaching as inquiry’ approach on writing pedagogy.     

论行为表现评估在英语过程写作教学中的运用

以过程为中心的英语写作教学法已成为近年来非常流行的改革英语写作教学的思潮。改变教学方法就必然先要改变传统的终结性评价方式。本文从英语写作教学实践的角度出发,探讨行为表现评估在英语过程写作教学中的具体运用,并指出采用这种全新评估方式的过程写作教学必然会促进英语写作教学的质的飞跃。     

The application of pairwise comparisons to form scaled exemplars as a basis for setting and exemplifying standards in teacher education

ABSTRACT

This paper details a two-stage process for standard setting using an extended pairwise comparison method. It describes the application of a new process of establishing a set of scaled and ordered performances as a basis for exemplifying standards. In stage one, the method of pairwise comparison was used to generate data from pairwise judgements made at the criterion level and overall grade. In stage two, judges identified where the pass/fail boundary lay in relation to a selected subset of ordered performances generated from stage one. The combined evidence from the two-stage process demonstrates the method’s application to establish exemplified standards using on-balance judgements across multiple criteria. The approach combines expert judgement and statistical data to provide a basis for comparable cross-institution assessments against standards in the case of a summative assessment of professional competence in teacher education. It may be relevant to any discipline with professional standards and complex performance assessments.     

Learning through collaborative writing

This article looks again at the dynamics of reading, writing and telling stories. It argues that working collaboratively can motivate writers in ways which encourage them to redraft their work purposefully and explicitly in pursuit of particular creative effects. It exemplifies ways in which knowledge about language at word, sentence and whole text level can effectively be brought to bear as part of a holistic approach to writing.     

Intervention in teachers’ differential scoring judgments in assessing L2 writing through communities of assessment practice

In writing assessment, the inconsistency of teachers’ scorings is among the frequently reported concerns regarding the validity and the reliability of assessment. The study aimed to find out to what extent participating in a community of assessment practice (CAP) can impact the discrepancies among raters’ scorings. Adopting one group pretest-posttest design, patterns in the teachers’ scoring judgments were explored based on both quantitative and qualitative data. The results indicate significant increase in the degrees of agreement in the teachers’ differential scorings showing changes in their severity tendencies for structural variety, lexical accuracy, organization and mechanics criteria while their scoring judgements on structural accuracy, task achievement, and lexical variety criteria had low levels of agreement.     

项目教学法结合语块优化英语应用文写作教学

英语应用文写作教学以培养学生实际运用语言的书面交际能力为目标,强调教学的实践性和针对性。项目教学法旨在培养学生的实际能力,是改进英语应用文写作教学的有效途径。语块理论认为语块的掌握有助于学习者语言产出的流利性、准确性和语篇连贯性。探讨了如何将项目教学法与语块理论相结合应用于英语应用文写作课程中,以促进写作教学。     

Disciplinary discourse in doctoral theses

This paper discusses stylistic conventions within broad disciplinary groupings in the language of doctoral theses. It argues that the language conventions in doctoral thesis texts reflect spohisticated learning of key disciplinary norms governing the conception, production and reporting of knowledge in particular fields. The paper shows that many conventions are subtle; they may not be readily identifiable to experienced scholars, yet doctoral students are expected to learn and master them, suggesting that discipline-specific writing norms and conventions are learned largely by tacit means during doctoral study. The paper reviews the nature of linguistic forms in doctoral theses and identifies the underlying epistemological and cultural influences which shape the writing. Doctoral theses from a range of disciplines are examined closely. Attention particularly is directed to the overall structure of argument and the techniques for coherence; the conventions for citing, acknowledging and making judgements about previous research; and the nature of the technical language of the field.     

英语写作理论与实践探究

根据英语写作理论及其教学方法的发展脉络,英语写作理论可概述为6种:古典修辞学理论、过程写作理论、语境写作理论、社交-认知写作理论、体裁理论和图式理论。在此6种写作理论指导下,经过教学实践和提炼,学界总结出8种英语写作教学方法:成果教学法、过程教学法、语境教学法、社交-认知教学法、图式教学法、体裁教学法、过程体裁教学法及中英文差异对比法。英语写作教学实践表明,这8类英语写作教学法各有特色,也有其不足之处,客观地说,不同的教学法可以用在不同的英语写作学习阶段或不同的学习对象。     

过程写作和结果写作在大学英语教学中的比较研究

在过去的二十年中,中国的非英语专业大学生在英语学习方面取得了巨大的进步。然而英语写作方面的现状却不容乐观,这种写作能力必然影响他们在学习和工作中的成功交际活动。为改变这一现状,文章对两种普遍的写作教学法（结果法和过程法）进行了对比实验研究,旨在探究一种适用于英语综合水平较差的非英语专业大学生的写作教学法。     

The Impact of Students’ Approaches to Essay Writing on the Quality of their Essays

This paper presents a detailed analysis of the essay writing processes of three third year Bachelor of Education students. The case studies, which were drawn from a larger research project, indicate that students writing essays of differing levels of conceptual structure approach the essay writing task very differently at every stage of the process, and these differences appear to emerge from how students conceptualise the task—whether the intention is to reproduce an ordered sequence of information on the topic, or to construct an argument and develop a coherent thesis. The results suggest that attempts to improve students’ essay writing abilities need to shift from a focus on discrete skills to an emphasis on the relationship between students’ understanding of the content and their ability to write about it.