Integrating Quantitative Literacy into Technical Writing Instruction

The authors argue that skills in quantitative literacy (QL) and quantitative reasoning (QR) augment students’ communicative effectiveness. This article offers a pedagogical framework and model for how QR can be productively interwoven with the rhetorical know-how of technical writing pedagogy. The authors describe their course redesign, present preliminary assessment data, and conclude by highlighting some implications not only for student learning, but also for the QL movement itself.     

Looking beyond judging and ranking: Writing assessment as a generative practice

Establishing the score or the placement as the first priority in a writing assessment leads to more reductive forms of writing assessment. However, if the prompts used in a direct test of writing were generative – that is, if they asked test-takers to analyze their own experiences as writers or learners, for example – the resulting texts would be useful data beyond the act of producing a ranking or a judgment. Washington State University developed and trialled such a prompt, one that asks students to reflect on their curricular and extra-curricular learning opportunities in relation to the university's Six Learning Goals for the Baccalaureate. The results were texts that demonstrate, among other things, which goals are (and are not) effectively distributed across the curriculum. Using these texts to address outcomes assessment on a university-wide level makes the assessment more valuable than it would be if it merely produced a set of placements. In addition, the richness of the student texts has provided a valuable resource for graduate-level research that is broader and more meaningful than simply training future raters of writing. Further, the raw data have proved to be accessible to researchers with wide-ranging theoretical lenses, meaning that the data yielded by an assessment can become a significant resource for research beyond the needs of the assessment program alone. Given the need for university assessment programs to compete for ever-scarcer resources, exploring the potential of the generative prompt seems in our enlightened self-interest.     

Everything is illuminated: What big data can tell us about teacher commentary

What happens to writing instructors’ feedback when they use a common rubric and an online tool to respond to student papers in a first-year composition course at a large state university in the United States? To investigate this question, we analyze the 118,611 comments instructors made when responding to 17,433 student essays. Using concordance software to quantify teachers’ use of rubric terms, we found instructors were primarily concerned with global, substantive, higher-order concerns—such as responding to students’ rhetorical situations, use of reason, and organization—rather than lower-order concerns about grammar or formatting. Given past research has determined teachers overemphasize lower-order concerns such as grammar, mechanics, and punctuation (Connors and Lunsford, 1988, Lunsford and Lunsford, 2008, Moxley and Joseph, 1989, Moxley and Joseph, 1992, Schwartz, 1984, Sommers, 1982, Stern and Solomon, 2006), these results may suggest the possibility of a generational shift when it comes to response to student writing. Aggregating teacher commentary, student work, and peer review responses via digital tools and employing concordance software to identify big-data patterns illuminates a new assessment practice for Writing Program Administrators—the practice of Deep Assessment.     

Exploring the theories and consequences of self-assessment through ethical inquiry

The authors argue that writing studies has not explored the ethical implications of using self-evaluations in classroom and large-scale writing assessment. Postmodern ethics does not attempt to provide moral answers to practices or activities but rather strives to explore and uncover hidden assumptions and unarticulated implications. Its focus is on the process of inquiry, not on determining the rightness or wrongness of individual practices. Using this postmodern ethical framework, the authors explore portfolio cover letters, reflective essays, self grading, and self-placement as depicted in college composition literature. They demonstrate a process of ethical inquiry that dovetails with validity inquiry, arguing that both types of research are necessary—although often missing—in the literature on self-assessment.     

Assessing writing: Are we bound by only one method?

The present study describes the way one institution operationalized the notion of process-oriented writing assessment in its entry placement testing context. In an attempt to enhance how it assesses academic writing ability, a workshop-based essay test was proposed in place of the existing, traditional timed essay test. The study looked at how the tests from the two different approaches in writing assessment (product-oriented and process-oriented) affected examinees’ test performance by comparing both the textual quality of the test essays and the placement results. Content-area faculty evaluations of the examinees’ writing ability and the examinees’ view on the tests were also used in evaluating the quality of the tests. The results suggested that assessment methods have an impact on the examinees’ test performance. Even though further evidence is needed for a better understanding of the result, a traditional method of direct writing assessment seemed to put examinees at a disadvantage. The new test was implemented in the given research context and implications of writing assessment in general are discussed.     

Students’ perspectives of early assessment tasks in their first-year at university

Assessment plays an important role in first-year students’ transition to university, influencing how they learn. Poorly designed assessment can be demotivating and may even cause students to withdraw from university. An early low-stakes assessment task is recommended in first year, to provide information to students and staff on student progress, provide students with a positive experience and help them prepare for future assessments. This paper reports on first-year students’ experiences of two different early assessment tasks for two disciplines, one using an online quiz and the other a ‘research challenge’ that served as a precursor to writing an essay. The paper also presents lecturers’ reflections on what they learned, what they would change and the implications for others planning early assessment tasks for first years. Finally, ideas for future research are provided.     

Students’ writing ‘in transition’ from A-levels to university: how assessment drives students’ understandings,practices and discourses

The lament that ‘students can’t write’ remains loud and defiant, even after years of research pointing to the myriad factors that make students’ writing challenging, particularly when they move into university. This paper reports on a longitudinal, ethnographic study which explored students’ writing ‘in transition’, from A-levels to university in the UK, through the critical lens offered by the academic literacies conceptual framing. This paper offers critical analysis of the ways that students, teachers and institutions position writing at A-level and university, exploring the assumptions and beliefs that underpin their understandings and practices using Ivani?’s framework of discourses of writing. The analysis proposes that the centrality of assessment in the treatment of language at both levels creates an ‘assessment discourse of writing’, which originates in school, and becomes a defining and restrictive frame for students’ writing as they move into higher education. The analysis further suggests that assessment is the principal cause for the students’ challenges with adapting to the writing requirements of university. Moreover, assessment is used as a metalanguage for discussing writing at A-levels, and can become an unhelpful ‘anchor of continuance’ for students as they move into university.     

因特网辅助外语写作测评

在英语的各项能力中,写作能力的培养一直是学者关注的焦点,但是学生写作能力的提高一直是教学中的难题。因特网技术的发展,为实现网络写作测评提供了平台。利用学生所熟悉的QQ相关软件对学生的写作进行评估,可以使学生更好地了解写作的过程和进步,提高写作水平。     

Dangerous liaisons: Reflections on a pilot project for state-mandated outcomes assessment of written communication

This study details the development and results of a campus-based writing assessment plan that was mandated by a state-wide university system in order to explore the “value-added” from a writing program curriculum to undergraduate students’ competence with written expression. Four writing samples (two timed essays and two conventional essays) that are produced before, during and at the close of enrollment in the writing program are scored for critical thinking, genre knowledge, rhetorical skills, and mechanics. The results indicate that local campus student achievement related to writing is comparable with performance captured by nationally normed technologies and larger assessment projects at similar state universities. The data also suggest significant performance difference related to the institutional position of students’ instructors as well as students’ own enrollment status. The article concludes that while the data produced by this research project are compelling, the policy and political implications of the undertaking suggest greater sensitivity to the institution's culture and program direction as well as its impact on instructor morale, student learning environment and the place of composition studies in the university.     

District Accountability Without a State Assessment: A Proposed Model

Most states have adopted assessment and accountability systems that involve common measures of student performance. A state assessment system that allows school districts to choose the specific strategies they use to measure student performance on state-adopted content standards presents a unique state accountability challenge. The authors propose an accountability model that addresses this challenge using a combination of student performance, technical quality, and noncognitive indicators of performance. They also describe a study that evaluated the proposed model using data from all school districts in a southern state.     

Developing professional skills in journalism through blogs

The curriculum for journalism is being forced to change because the traditional print‐based and broadcast modes are being challenged by wide and easy access to online mass communication. Primarily, students need to develop proficiency in writing, editing and publishing. However, they are also expected to be skilled in the Web medium as they venture into their careers, and weblogs offer a dynamic platform to develop these skills. Based on a statistical summary of student comments on the performance criteria for a blog project, a focus group interview and survey results, this paper will discuss the findings from the introduction of a course‐based blog to an undergraduate course in Hong Kong, including: (1) the technical challenge of making Web features; (2) the perception of proficiency in online publishing; (3) the evaluative skills developed through a peer‐review process; and (4) the development of a learning community through writing in blogs.     

Students' conditioned response to teachers' response: portfolio proponents, take note!

Portfolio advocates argue that teacher commentary becomes more meaningful for students with the use of portfolio assessment, particularly because the commentary is unaccompanied by a grade. However, my own study of portfolio classrooms suggests that students continue to regard teacher responses as directives that leave them few options in terms of revisions. My study involved six writing classrooms from the middle school to the university level and included classroom observations, interviews of students and teachers, and examination of student writing and teacher response. The students generally looked to their teachers to show them the “correct” way to write and resisted the notion of making independent judgments about their writing and the necessary revisions, primarily because they could not ignore the ultimate reality of the grade. Yet the teachers generally avoided being directive in their responses. In one instance, the teacher's best efforts to adopt a more open style of response backfired when a sensitive student read the commentary as demeaning. I argue, therefore, that even though portfolios represent a more enlightened approach to assessment, students have difficulty escaping their conditioned obeisance to teacher authority. While this finding should not be read as a condemnation of portfolio assessment, it does indicate that teachers need to be aware of how students read their responses. Otherwise, portfolios alone may not substantially alter the teacher–student dynamics.     

Student perspectives on formative feedback as part of writing portfolios

Despite the crucial role that students play in formative assessment practices, student perspectives on such practices are relatively under-researched. Through a qualitative analysis of 128 reflection notes written by student teachers of English, this article investigates the students’ perceptions of formative feedback as part of portfolio assessment at two teacher education institutions in Norway. As such, it contributes to bridging the gap between research and practice. Students received peer and teacher feedback on assignments and wrote reflection notes during the semester. Findings show that students are positive towards teacher feedback and highlight the significance of teacher praise. Main objections raised against peer feedback concern the lack of constructive criticism. However, positive attitudes towards peer discussion groups suggest that they may be a more effective way of implementing peer assessment than formalised written peer commentary. Student reflections suggest that a failure to understand the task and the feedback is a possible hindrance to successfully revising assignments. Overall, students’ positive attitudes towards the portfolio process, which includes multiple drafting, suggest that students in higher education would benefit from more opportunities to revise and resubmit their work, yet they need adequate practice in providing peer feedback, and interpreting and implementing feedback in general.     

Elaborating a framework for communicating assessment aims in higher education

Assessment is a central feature of student learning in higher education and has a strong influence on the student experience. Accordingly, the appropriate communication of assessment aims is a priority for all higher education institutions. This study proposes an analytical framework for the interpretation and creation of assessments across higher education disciplines. The framework suggests that assessments can be categorised according to rhetorical purposes, formats, modes and group arrangements. Assessments from three degree programmes at one Australian university are analysed using the framework to show its usefulness in classifying and evaluating task components and generating broad images of degrees based on assessment regimes. We draw on these practical applications to explain and compare discipline-specific qualities of each degree, and argue that the framework might enhance the communication of assessment aims to benefit higher education stakeholders.     

Supporting writing for assessment in online learning

This article explores some specific issues involved in online learning and assessment. It draws on data from a postgraduate course for professional educators, delivered globally online, and highlights the relationship between students’ online discussion and their written assessed work, arguing that we need to focus on both of these in terms of the writing demands they make on students. In so doing it utilizes a theoretical framework which conceptualizes writing as contextualized social practice. The paper illustrates the complexity of the rhetorical demands being made on students in these new environments of teaching and learning and, in focusing on writing, complements present approaches to online learning which have, to date, tended towards collaborative and constructivist perspectives. The article highlights the relationship between pedagogy, technology and assessment. It concludes with a discussion of the design of an online writing resource to support student writers on this particular masters programme.     

Do we need a single standard of value for institutional assessment? An essay response to Asao Inoue's “community-based assessment pedagogy”

Inoue [Inoue, A. B. (2005). Community-based assessment pedagogy. Assesing Writing: An International Journal, 3, 208–238] sets up a radically experimental writing class as a kind of laboratory of assessment. He seeks to avoid the standard situation where a teacher unilaterally assesses and grades student writing, using only his or her own criteria or standard. His premise is that value in writing is socially constructed, and so he gets students to enact a social, communal process to work out the criteria for effective writing. In this way he gets students to take full responsibility for assessing and even grading each other.But when he requires his students to agree on a single model of good writing (a “rubric” with various dimensions), I think he misunderstands how value is actually socially constructed. In fact we live in a world with various models of good or effective writing. Given, however, that he is working in an institutional setting, his approach seems readily understandable. How can feedback be coherent or final grades be fair if they are based on multiple and competing models of goodness? This sounds like a rhetorical question, but in fact I use this essay to argue that it is not only feasible but desirable to use multiple models of value in an institutional setting.     

Expanding the dialogue on culture as a critical component when assessing writing

This article reports on the findings of a study that illustrates the value of including the voices of teachers from diverse backgrounds in discussions concerning writing assessment. The inclusion of such voices can help, not only to inform, but to re-shape current assessment practices, research priorities, and policy debates that focus on finding solutions to problems of assessment, particularly as they relate to diverse populations. In Part one of the article, I review the findings of an investigation in which four European-American teachers assess students' written texts. These teachers rated essays written by ethnically diverse fifth- and sixth-grade students using an array of rhetorical and linguistic measures—including overall quality, coherence, sentence-level mechanics, and use of an organizational structure. In Part two, I describe a replication of this investigation in which four African-American teachers assessed the writings of the same students' written essays using the same rhetorical and linguistic measures. Findings suggest that European-American and African-American teachers hold some consistently different views about the assessment of diverse students' written texts. Motivated by these observed differences and a desire to expand the dialogue on culture as a critical component in assessing writing, I present in the third and final section of the article the voices of the African-American teachers who participated in this study. Through candid and reflective comments, these teachers share deep felt concerns and specific suggestions concerning writing assessment. In this section, I also discuss how the voices of teachers from diverse backgrounds can be useful for broadening debates about the reform of writing assessment in general, and the assessment of writing for culturally and linguistically diverse students in particular.     

Enhancing the work placement experience of international students: towards a support framework

This article reports the findings from an institutional case study into the work placement experience of undergraduate international students studying at a UK-based university. Given the centrality and importance of the employability and internationalisation agendas in higher education, little published literature considers both and how they impact upon the work placement experience of international students. With past increases in international student recruitment numbers and the drive for students, regardless of their nationality, to gain work experience, there is a need for institutions to better understand work placement from the international students’ perspective. Findings revealed that challenges exist in finding and securing work placement, with critical themes emerging to include placement information, visa rules, culture differences, prior work experience, academic preparation for placement and academic writing ability. A support framework is proposed to assist institutions to consider how they could align their practices to meet international students’ work placement and employability needs.     

Multiple voices in tests: towards a macro theory of test writing

Tests, especially high-stakes ones, occupy a prominent role in education and impact on students’ personal and professional trajectory. Therefore, it is crucial that they are well understood. Enhancing understanding of the workings of tests requires a multidisciplinary approach, one that treats tests not only as assessment instruments, but also as sociolinguistic and sociopolitical products. Using data collected by means of non-participant observation, stimulated recall and one-to-one interviews with writers of GCSE and IGCSE tests, this paper attempts to provide a macro-analysis of the process of test writing. It does so by identifying the various actors and discourses that directly or indirectly take part in test writing. It uses Bakhtin’s framework of voice as a tool to facilitate this. The paper concludes by proposing a macro model of test writing, one that explains the macro process of test writing by capturing the main voices involved in it and their interrelationships.     

Formative assessment of writing in English as a foreign language

Recognizing the importance of formative assessment, this mixed-methods study investigates how four teachers and 100 students respond to the new emphasis on formative assessment in English as a foreign language (EFL) writing classes in Norway. While previous studies have examined formative assessment in oral classroom interactions and focused on either studying students or teachers, little research has been conducted on formative assessment of writing where both students and teachers are studied. As such, this study provides new insight. The findings mostly indicate that contradictions are prevalent amongst teachers’ and students’ perceptions of formative assessment of writing. The contradictions revolve around feedback, grades, text revision, self-assessment, and student involvement. The identified contradictions suggest the need for developing a mutual understanding of formative assessment in order to make it useful and meaningful.