Challenges in assessing the development of writing ability: Theories,constructs and methods

This article discusses three sets of challenges involved in the assessment of writing from a developmental perspective. These challenges include defining a workable theory of development, developing a suitable construct, and overcoming limitations in technocentric approaches to writing assessment.In North America in recent years, a burgeoning number of scholars have begun to explore the development of writing ability through the lens of knowledge transfer research. This paper discusses limitations in current and traditional conceptions of transfer, proposing a bioecological model of transfer to resolve these issues. Related to issues of transfer are challenges in defining the construct under investigation in the assessment of writing development. Beaufort's (2007) model of expertise in writing is discussed as a promising framework for defining this construct. Related to these challenges are limitations in current assessment technologies. Drawing on Huot's (2002) concept of assessment-as-research, this article discusses possibilities for broadening the range of assessment practices available to assessors of writing.     

Teachers as writers: learning together

This paper reports upon the insights gained through working with teachers as writers at their own level. As part of a two‐year research project into the development of children's voice and verve in writing, a group of fourteen teachers' reflective journeys as writers were documented. Two other groups of teachers and one group of student teachers also took part in writers' workshops across the same period. The data encompassed: questionnaires, observations and teacher commentaries on their own writing, as well as interviews. A number of issues emerged, including: the tension between public and private writing and the security of the writing environment; authenticity in modelling writing; the importance of re‐reading writing at the point of composition; the significance of choice and autonomy in writing and the potency of drama as an ideational and reflective tool. The consequences for classroom practice are also considered. It is argued that in order to enhance the teaching of writing, teachers and student teachers need real opportunities to write at their own level and reflect upon the process.     

Learning to Write: Plowing and Hoeing,Labor and Essaying

In this paper Amanda Fulford addresses the issue of student writing in the university, and explores how the increasing dominance of outcome‐driven modes of learning and assessment is changing the understanding of what it is to write, what is expected of students in their writing, and how academic writing should best be supported. The starting point is the increasing use of what are termed “technologies” of writing — “handbooks” for students that address issues of academic writing — that systematize, and smooth the work of writing in, Fulford argues, an unhelpful way. This leads to a reconsideration of what it means to write in the university, and what it is to be a student who writes. Fulford explores etymologically the concept of “writing” and suggests that it might be seen metaphorically as physical labor. Writing as physical labor is explored further through the agricultural metaphors in Henry David Thoreau's Walden and through Stanley Cavell's reading of that text. In making a distinction between writing‐as‐plowing and writing‐as‐hoeing, Fulford argues that some technologies of writing deny voice rather than facilitate it, and she concludes by offering a number of suggestions for the teaching and learning of writing in the university that emphasize the value of being lost (in one's subject and one's work) and finding one's own way out. These “lessons” are illustrated with reference to Thoreau's text Walden and to American literature and film.     

The development of large-scale portfolio placement assessment at the University of Michigan: 1992–1998

The first section of this article examines in detail the mechanics of a large-scale writing portfolio assessment at the University of Michigan, including its impact on matriculation and placement; students' reactions to the requirement; and instructors' evaluation of the efficacy of placements under the new system. The second section of the article examines the scoring process used by readers to assess portfolios, describes its evolution over a 4-year period, and examines the results produced by various changes made to this process over time. The evolution of this portfolio placement assessment has not been traditional, in that it has enacted an understanding that people learn to read and write within specific social contexts, at the same time that it has enacted a commitment to connecting the intellectual and practical work of university teaching with the work of teaching that comes before it in the high schools. The assessment process also has enacted the authors' belief that issues of assessment are intimately linked to issues of curriculum and pedagogy; thus, the assessment has become an occasion for sustained talk about the purposes of our required writing courses, about the fit between our lower division and upper division writing requirements, and about the degree to which teachers share criteria for assessing students' writing within courses.     

issues in Portfolio Assessment: Assessing Writing Processes From Their Products

We investigated the ways that portfolio evidence of students' competencies with writing processes was created and interpreted in 4 classrooms. Our study was conducted during preliminary classroom trials of California Learning Assessment System portfolios, when teachers and students were challenged with the new task of preparing portfolios that demonstrated students' competency with the "dimensions of learning." Drawing data from teacher and student interviews as well as portfolios, we considered three issues regarding the meaning of portfolio indicators of writing processes (a) Students' opportunities to learn to use a range of resources, processes, and standards in ways that enhance the effectiveness of their writing; (b) students' opportunities to produce "hard copy" evidence of their uses of processes; and (c) students' capacities to analyze their writing processes. Further research is needed to understand how participants in a large-scale portfolio assessment program develop shared understandings of the ways that evidence of writing processes is considered in the scoring and how the programmatic needs for comparability of evidence can be reconciled with the personal needs of young writers, whose uses of processes will vary with the purposes and contexts of their writing.     

Using Case Study Analysis and Case Writing to Structure Clinical Experiences in a Teacher Education Program

This study reports on the design and results of a two-semester study on the use of case study analysis and case writing in clinical experiences in an undergraduate teacher education program. Findings indicated that structured experiences with case studies and case writing increase preservice teachers' informed decision making on educational issues.     

Reading beneath the lines of children's stories

This paper is drawn from a study of 10-12 year-old children's stories, the specific purpose of which was to provide a means of investigating the influence of television and videos on children's imagination. Faced with the need to understand nearly 500 stories from their authors' points of view, that is interpreting the authors' meanings and intentions rather than constructing my own meaning as a reader, I found myself wondering how the 'reading' of these texts was to be undertaken. The paper describes the technique I devised for this task and what I found it could reveal about the subtlety, complexity and multiple meanings that can often be discovered in children's stories.The potential power of such an approach to reading children's writing raises issues about the assessment of school writing.     

Exploring the Metafictive in Elementary Students' Writing

Although some researchers have examined students' literary understandings of and responses to books with metafictive characteristics, few have explored how elementary students incorporate metafictive devices into their writing. In this article I analyse the stories and books created by a class of Grade 5 students and discuss the metafictive devices evident in their work. In addition to considering how the literature students read and how the classroom interpretive community influenced the students' stories/books, I discuss some broader issues about the significance of students reading and writing metafictive texts.     

Reviews

Cox, Professor Brian Cox on Cox: an English curriculum for the 1990's (1991) Nursery and Infant teachers in the Midlands Start With A Story; supporting young children's exploration of issues. (1991) Graham, Judith, Pictures on the Page. (1991) Littlefair, A. Reading all types o writing. (1991)     

Thinking critically: a look at students' critiques of a research article

The ability to reason, analyse and evaluate issues critically is a valued skill and ranks highly in the list of attributes expected of graduates. Much has been written about the importance and application of critical thinking in various domains, but studies on the actual manifestation of such skills in students' writing have attracted only modest interest. Even less has been written about critiques in relation to critical thinking. This study sought to investigate the form and nature of issues raised by 119 second-year biology undergraduates in their critiques of the introduction section of a research article. The study revealed that the vast majority of students tended to raise surface issues in their critiques, focusing on visible textual features such as rhetorical structure and language-related issues. The minority who raised depth issues addressed the arguments used in the reading and their significance. In light of the skewed results, a two-stage process – involving (1) summary writing and (2) the use of evaluative criteria and the Toulmin model as an overarching framework – is recommended to enhance the teaching of critical thinking within the curriculum. This study offers a glimpse into the outcomes of critical thinking, as represented by the students' critiques. It provides a bottom-up approach to our understanding of the issues raised by students in a task centred on critical thinking and so focuses our attention on specific areas for further consideration or remediation.     

Developing Reflection on Practice Through Journal Writing: Impacts of variations in the focus and level of feedback

Journal writing is a popular technique for encouraging student-teachers to reflect on their professional practice during field experience placements. This paper explores the role and importance of journal feedback in developing students' reflective skills. Weekly journal entries were submitted by 35 student-teachers during a 6-week field placement. Students received individual feedback on each journal entry that focused on either the level of reflection attained in their writing or the particular issues that their entries addressed. Within these groups, the type of feedback provided was further varied according to the level of questioning and challenge with which students were confronted (high versus low). The relative effectiveness of the four different types of feedback in improving student journal writing and facilitating reflection on practice is examined. Although students in all conditions reported positive aspects of the feedback they received, feedback that focused on the level of reflection attained was more effective in bringing about improvement in journal writing than feedback that focused on teaching issues. Such feedback, combined with issue-related questions and comments designed to challenge the student and encourage consideration of alternative perspectives, would appear to offer the most effective strategy for enhancing the effectiveness of journal writing as a learning tool.     

Linguistic Features of Middle School Environmental Education Texts

The language used in environmental education texts has linguistic features that affect students' comprehension of concepts and their ability to envision solutions to environmental education problems. This paper shows what 143 middle school students focused on and what they failed to focus on when reading texts about biodiversity issues and identifies linguistic features of the texts that can account for this. Since much evaluation of students' knowledge of academic subjects, including environmental education, is done through reading and writing assignments, it is important to examine how environmental education texts present issues to students and what students write after reading environmental education texts. We suggest that features of texts such as abstract nouns and lack of explicit agents impede students' full comprehension of complex issues and obscure the causes and solutions to environmental problems. We present guidelines for teachers and materials writers to make abstract concepts more accessible to middle school students.     

Differences in African‐American and European‐American students' engagement with nanotechnology experiences: Perceptual position or assessment artifact?

This study examined middle and high school students' perceptions of a weeklong science experience with nanotechnology and atomic force microscopy. Through an examination of student self assessments and their writing, the study allowed us to examine some of the issues that may contribute to discrepancies that are seen between European‐American and African‐American students in science. The results of the study showed that after instruction, African‐American students were significantly more likely to agree with the statement that “science involves mostly memorizing things and getting the right answer,” than European‐American students. In addition, European‐American students were significantly more likely to write their newspaper stories from a first person perspective than their African‐American peers. The results are discussed in light of the assessment task, students' interpretations of formal writing, cultural differences in the use of language in writing, and possible cultural differences in students' perceptions of the science experience. © 2007 Wiley Periodicals, Inc. J Res Sci Teach 44: 787–799, 2007     

Reflections on being female in school science: Toward a praxis of teaching science

This essay interrogates the phenomenon and the implications of being female in school science through girls' and young women's stories interwoven with my own narrative as a woman teacher/researcher in science education. The intent is to raise awareness of issues related to being female in conventional science teaching, and to suggest some alternative perspectives and approaches for action and reflection. I discuss several directions within my teaching which announce my own interpretation of bell hook's “engaged pedagogy,” which emphasizes a commitment to self-actualization and well-being for both teacher and student. My choice to integrate disparate writing styles (conversational and scholarly) was a deliberate effort to resonate issues between practice and theory and interrupt this separation. J Res Sci Teach 35: 463–471, 1998.     

Large-scale assessment,locally-developed measures,and automated scoring of essays: Fishing for red herrings?

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater^® and the Criterion^® Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the criticisms leveled at AES are reasonable, the more important, underlying issues relate to the aspects of the writing construct of the tests AES can rate. Because these tests underrepresent the construct as it is understood by the writing community, such tests should not be used in writing assessment, whether for admissions, placement, formative, or achievement testing. Instead of continuing the traditional, large-scale, commercial testing enterprise associated with AES, we should look to well-established, institutionally contextualized forms of assessment as models that yield fuller, richer information about the student's control of the writing construct. Such tests would be more valid, as reliable, and far fairer to the test-takers, whose stakes are often quite high.     

Paradoxical tensions in creating a teaching and learning space within a graduate education course on spirituality

Much has been written recently about inclusive learning environments in higher education. Most of this writing has unproblematically assumed a simple link between inclusion and access to institutions. In this paper I suggest that inclusive learning environments need to be reconceptualised as contexts within which a 'democratisation of knowledge' occurs. I draw upon my experience as a tutor on a professional development course for university lecturers to explore this reconceptualisation, looking in particular at issues to do with experiential knowledge, gender and sexuality.     

Toward Expert Thinking: How curriculum case writing prompts the development of theory-based professional knowledge in student teachers

The present paper explores what, and how, student teachers may learn about theory and practice from writing cases, and examines some pedagogical features that may contribute to these results. Drawing on data collected from our course "Principles of Learning for Teaching", including student cases from outline to final drafts and students' course reflections, we found that students' successive case drafts demonstrated a development from naïve generalizations to sophisticated, theory-based explanations of the issues at play in their cases. In particular, we suggest that students' cases demonstrated some of the moves that Berliner (1986, 1991) identified as characteristic of more "expert" thinking about teaching. We propose that reading theory in context with writing cases, that sharing cases with peer readers, that specific, theoretically grounded, and concrete feedback from instructors, and that providing multiple opportunities for revision may have been most useful in helping student teachers learn to think like a teacher.     

The Effects of an Environmental Education Program on Students,Parents, and Community

Abstract

This study evaluated the impact of an environmental education program on students, parents, and the community. The program, in place for 5 years in 5th and 6th grade classes, was designed to help learners take an in-depth look at environmental issues in their community, make data-based decisions about those issues, and participate in resolving those issues. Qualitative and quantitative methodologies were employed (including student, parent, school personnel, and community member interviews). The authors discuss program connections to students' critical thinking, environmental literacy, and participation in the community; reading, writing, and speaking skills; student and teacher characteristics; and parent and community viewpoints.