Uneasy writing: The defining moments of high-stakes literacy testing

The authors report on an exploratory study that investigated Grade 10 students’ responses to a large-scale, high-stakes literacy test, the results of which determine high school graduation in Ontario, Canada. Through focus-group and individual interviews, the authors found that students perceived the test as evaluating their English skills rather than their cross-curricular literacy as is the stated purpose by the test administrators. Furthermore, the test contributed to students’ narrow definition of writing. Rather than promoting writing as a powerful tool for thinking and expressing ideas, students understood writing to be a fill-in-the-blanks response to particular genres. The authors conclude by pointing out the power of such tests to define unintended learning in schools.     

When a “sloppy copy” is good enough: Results of a state writing assessment

Students in grades 5 and 8 completed a state writing assessment, and their first and final drafts on the extended writing portion of the test were copied and scored using the state writing rubric. The rubric consisted of three primary traits: Content and Organization, Style and Fluency, and Language Use. Scorers were blind to the study purpose and scored either a student's first or final draft. No significant difference was found between the first and final drafts written by students in special education at both grade levels. Likewise, no significant difference was found for the writing of general education students in grade 8. A significant difference was found, however, between first and final drafts written by fifth-grade students in general education. Cross tabulations conducted at grades 5 and 8 revealed that over 50% of the first drafts received the same score or a better score than what was earned on the final draft.     

The impact of bilingual dictionaries on lexical sophistication and lexical accuracy in tests of L2 writing proficiency: A quantitative analysis

Writing assessment essentially juxtaposes two elements: how ‘good writing’ is to be defined, and how ‘good measurement’ of that writing is to be carried out. The timed test is often used in large-scale L2 writing assessments because it is considered to provide reliable measurement. It is, however, highly inauthentic. One way of enhancing authenticity in a way that might also promote good writing is to include access to support resources in the test. This article reports a study into whether allowing a bilingual dictionary enhances good writing in German writing tests. Participants (n = 47) took two tests, one with and one without a bilingual dictionary. The essays of sub-sets of the participants were compared in terms of lexical sophistication, lexical accuracy and test score. Lexical sophistication was measured by means of the Lexical Frequency Profile (Laufer & Nation, 1995). Lexical accuracy was determined through a quantitative analysis of words looked up in the dictionary. The results revealed an increase in lexical sophistication in ‘with dictionary’ tests but frequent misuse of look-ups. There was also no improvement in test scores. Subsequent issues for the use of dictionaries in writing tests are raised.     

Self and teacher assessment as predictors of proficiency levels of Turkish EFL learners

Although self-assessment of foreign language skills is not a new topic, it has not yet been widely explored in the Turkish English as a Foreign Language (EFL) context. The current study investigates the potential of self-assessment of foreign language skills in determining proficiency levels of Turkish learners of EFL: 239 learners participated in the study. Their receptive language skills were tested with an objective placement test, and the results were compared with the grades assigned by their instructors and the results of criterion-referenced self-assessment scores. Multiple regression analysis revealed that teacher and self-assessment scores were significantly correlated with each other; however, teacher assessment, compared to self-assessment, appeared to be a much stronger predictor of the actual proficiency levels of the participants. In addition to this, participants at lower level of proficiency revealed a common tendency of overestimating their language skills, while with the higher level learners there were clear signs of underestimation.     

Remembering places: Student reliance on place in timed essays

This is the story of a research journey that follows the trail of a novel evaluand—place. I examine place as mentioned by rising juniors in timed exams. Using a hybridized methodology—the qualitative approach of a hermeneutic dialectic process as described by Guba and Lincoln (1989), and the quantitative evidence of place mention—I query student writing to uncover motivations for reliance on place-based experience despite the absence of an explicit mention of place in the writing prompt. Finally, I consider implications for using generative prompts to elicit evidence of student learning.     

Writing and pseudo‐writing from Internet‐based sources: implications for learning and assessment

In recent years, plagiarism has been on the increase across the Western world. This article identifies Internet access as a contributory cause of this trend and addresses the implications of readily available Internet sources for the teaching and assessment of writing in schools. The basis for the article is a previous study showing a wide incidence of plagiarism in the Internet‐based writing of students in three classes at upper secondary school level in Norway. I relate the students' choices to writing as a cognitive process and as a cultural practice. My basic assumption is that the students' writing is work. It is this work we have in mind when we relate writing to learning and when we assess students' skills on the basis of their written texts. Access to the Internet changes the premises for this work because writing can be replaced by ‘pseudo‐writing’. ‘Pseudo‐writing’ is a work reducing writing practice, which neither excludes nor coincides with what we traditionally associate with plagiarism in schools. The main point in this article is that when students have access to the Internet during essay writing, the result is unavoidably a product of both writing and pseudo‐writing. Internet access thus leads to greater uncertainty about the role writing plays in student learning and makes it more difficult to take written assignments into account in assessing students' school results and effort.     

Collaborative writing in engineering: Perspectives from research and implications for undergraduate education

In engineering, like in many other disciplines, collaborative writing (CW) has been identified as a central practice in both the academy and industry. A number of studies have shown that both students and professionals in this field write most discipline-specific genres collaboratively. Despite its centrality, CW in engineering is still an under-researched area with very few empirical studies on how it happens as situated practice. This article reports on a study that used a variety of data sets as its empirical base to examine CW in four schools of the faculty of engineering at a university in the UK. It also reports on the views of CW in the workplace that lecturers involved in industry hold. The article aims at contributing to the understanding of CW in engineering by identifying its role, nature and dynamics. It also examines pedagogical implications resulting from the study with a view to making a contribution to developments in writing in engineering education. The article finishes by making a number of recommendations for future research.     

The quality assurance of a national English writing assessment: Policy implications for quality improvement

The purpose of this study was to examine the quality assurance issues of a national English writing assessment in Chinese higher education. Specifically, using generalizability theory and rater interviews, this study examined how the current scoring policy of the TEM-4 (Test for English Majors – Band 4, a high-stakes national standardized EFL assessment in China) writing could impact its score variability and reliability. Eighteen argumentative essays written by nine English major undergraduate students were selected as the writing samples. Ten TEM-4 raters were first invited to use the authentic TEM-4 writing scoring rubric to score these essays holistically and analytically (with time intervals in between). They were then interviewed for their views on how the current scoring policy of the TEM-4 writing assessment could affect its overall quality. The quantitative generalizability theory results of this study suggested that the current scoring policy would not yield acceptable reliability coefficients. The qualitative results supported the generalizability theory findings. Policy implications for quality improvement of the TEM-4 writing assessment in China are discussed.     

Employing Wikipedia for good not evil: innovative approaches to collaborative writing assessment

Wikipedia is an open educational resource that connects writers and editors to diverse discourse communities around the world. Unwarranted stigma is attached to the use of Wikipedia in higher education due to fears that students will not pursue rigorous research practices because of the easy access to information that Wikipedia facilitates. In studies referred to in this paper, undergraduate writing students are taught about the need to interrogate any information they find on Wikipedia just as they would other online source material. They are inducted into fact checking, editing and creating Wikipedia articles as a means to analyse source material critically and to advance their research, writing and digital literacy. Meanwhile, in a postgraduate course in magazine studies, instead of writing essays, students are promoting Australian magazines and print culture by writing Wikipedia entries about Antipodean magazines and their editors. These courses experiment with new approaches to formative and summative assessment; promote group research, collaborative and participatory writing, writing across networks and negotiating discourse communities; and challenge students’ perceptions about peer review and the legitimacy of Wikipedia.     

A closer look at integrated writing tasks: Towards a more focussed definition for assessment purposes

The use of integrated tasks is increasingly common in second language assessment in both high stakes and classroom contexts. However, because of the vague definition of what constitutes an integrated task, the types of tasks included in this area and the assessment of the resulting writing is highly varied. In this paper, we argue for a better definition of the term ‘integrated writing task’. We start by reviewing current definitions available in the literature, discuss the construct underlying integrated tasks and then propose a number of features which we feel need to be present in integrated tasks. We then propose a new, narrower definition of the term. We hope that this will result in better task design, improved assessment outcomes for test takers, more detailed test specifications, better replicability of research and a shared research agenda.     

Student threats of violence against teachers: Prevalence and outcomes using a threat assessment approach

Internationally, student aggression against teachers is a prevalent problem in schools. Student threat assessment is an emerging violence prevention practice, but its use for threats against teachers has not been investigated. This study examined use of threat assessment for a statewide sample of student threats against teachers (n = 226) compared to threats against other students (n = 1,228). Results indicated that threats against teachers were less prevalent (15.5%) than threats against peers (84.5%). Of threats against teachers, 30% were classified as serious by the school’s threat assessment team and 5.8% were attempted. Implications for school policy and practice and teacher safety are discussed.     

Incorporating formative assessment in Iranian EFL writing: a case study

Undergraduate students' experience of assessment in universities is usually of summative assessment which provides only limited information to help students improve their performance. By contrast, formative assessment is informative and forward-looking, possessing the leverage to inform students of their day-to-day progress and inform teachers of how to better tailor their instruction to students' immediate learning needs. Despite these potentials, studies carried out on the use of formative assessment in English as a foreign language (EFL) contexts are somehow rare. The current study reports on incorporating formative assessment in an L2 writing course in Iran. The analysis of data from pre- and post-study writing tasks, pre- and post-study questionnaires, and semi-structured interviews revealed that first-year undergraduate students were offered opportunities to improve various aspects of their writing and to develop positive attitudes toward writing as well as formative assessment. However, the students reported several challenges that could have implications for the further implementation of formative assessment in similar contexts.     

Bilingual education through a pluri-centric approach: A case study of the effects of simultaneously learning two languages on L1 and L2 reading and writing proficiency

Language education has often been conducted based on monoglossic ideologies that keep L1 and L2 separate, a practice that receives growing criticism as out of sync with today’s multilingual globalized world. Informed by interdependence hypotheses and translanguaging theory, this research study explored the effects of a pluri-centric approach to learning L1 and L2 simultaneously. Through a bilingual intervention course with data collected from interviews, parents’ logbooks, observation field notes, questionnaires, and L1 and L2 reading and writing tests, this case study (n = 9) revealed that the approach effectively improved learners’ L1 and L2 proficiency, learning behaviour outside the classroom, self-efficacy, and confidence. Overall, the participants also perceived the bilingual course as effective, efficient, and innovative. The study provides a basis to reflect upon transferrable skills and core underlying proficiency as powerful concepts to draw upon in enhancing bilingual proficiency and effecting positive changes in learners’ behaviour and psychological well-being.     

Paradigms and Pedagogy: revisiting D’Arcy’s critique of the teaching and the assessment of writing

This paper draws on research funded by the Bedford Charity (Harpur Trust) in which the narrative writing of reluctant writers in 10 Lower Schools (pupils aged 5–9) was levelled using, firstly, the Assessment of Pupil Progress (APP) criteria used in English primary and secondary schools and, secondly, criteria devised by the researcher. The latter were specifically designed for the assessment of narrative writing, whereas the APP criteria are generic. Findings resonate with the contrasting paradigms critiqued in D’Arcy’s paper on the teaching and assessment of writing (D’Arcy 1999). Evidence suggests that levels assigned to individual texts can vary between one assessment criteria and another. In many cases, pupils achieved higher levels against criteria designed to evaluate creativity and the ability to tell a story. Appropriate assessment criteria are critical not only to summative evaluations but also to the planning of future teaching and learning. However, it is suggested that the dominant assessment paradigm and concomitant approaches to the pedagogy of writing in England have been influenced by discourses over which teachers have had little control. However, the future for the teaching and assessment of writing necessitates a paradigmatic shift. A retrospective of D’Arcy’s paper may provide teachers and academics with the impetus for change.     

A scaffolded approach to Discussion Board use for formative assessment of academic writing skills

Mastery of academic writing skills remains one of the greatest challenges for university students, especially in the first year. Amongst the reasons offered for the challenges are lack of clarity about the university’s expectations and low levels of teacher feedback on work submitted, a failure to engage, and low levels of contact with teaching staff and other students. Academic staff are challenged by increased class numbers and increased student diversity in classes and university policies to adopt a wide range of information technologies into teaching modes. In this paper, we offer one attempt at addressing these three important contemporary academic challenges: use of information technology to provide timely feedback through formative assessment to help students with a range of abilities to acquire the academic writing skills necessary to succeed in higher education. Early indications show that students find this form of formative assessment very useful. They value the timely and focused feedback from the lecturer and are developing collegiality as they learn from each others’ writing and feedback. We believe that this strategy is sustainable and can be adapted to facilitate academic and social integration for students across many disciplines.     

Implementing interactive courseware into EFL business writing: computational assessment and learning satisfaction

The implementation of interactive courseware within a task-based learning (TBL) approach was conducted for English Writing for Business to Chinese students in an applied foreign languages department housed in a university of technology. The development of the interactive courseware is based on Mayer’s multimedia learning theory, which allows learners to construct verbal and visual cognitive representations and integrate them; its language learning focus is keyed to Chapelle’s suggested criteria for development of multimedia CALL (computer assisted language learning). According to the curriculum, students had to accomplish three tasks rooted in the real world workplace: inquiry, negotiation, and complaint. Evaluation of the interactive ESP courseware into TBL instruction was based upon data from students’ pre- and post-writing performance measured by three different types of online computational assessments. In addition, a questionnaire survey about learning satisfaction was administered. The results indicate that students had significantly better post-writing performance and showed greater satisfaction after self-studying with the courseware-implemented instruction, which effectively integrated content knowledge, integrative English skills and writing practices embedded in the courseware. This instruction could be a potentially useful for teaching business writing by providing both content knowledge and its related integrative language practice.     

Accounting for Student Success: An Empirical Analysis of the Origins and Spread of State Student Unit-record Systems

This event history analysis explores factors driving the emergence over recent decades of comprehensive state-level student unit-record [SUR] systems, a potentially powerful tool for increasing student success. Findings suggest that the adoption of these systems is rooted in demand and ideological factors. Larger states, states with high proportions of students of traditional college-going age, and states subject to federal civil-rights monitoring, were more likely to adopt SUR systems, suggesting influences of demands posed by size and legal constraints. In addition, states with more liberal citizen ideology were more likely to adopt the systems. Interestingly, the strength of private colleges and universities in a state worked against the adoption of SUR systems, suggesting that privacy and autonomy concerns were important deterrents to adoption. The results of this analysis illuminate the factors that inhibit and enhance SUR systems’ organizational and philosophical acceptance, and thus ideally can contribute to future policymaking in this arena.

Children writing: Multimodality and assessment in the writing classroom

Multimedia has had a widely recognised impact on society, but it is still under‐represented in the literacy pedagogies of many schools. This may relate to the way we view assessment for literacy, which is still almost wholly monomodal. Students assessed at a low level as producers of verbal text may respond positively when working multimodally, but the only assessment instruments we have do not reveal this. In a class of 26 ten‐year‐olds in Victoria, Australia, with rich access to computers, five children performed at a very low level when working with verbal expression only, but responded remarkably when invited to work multimodally. A programme of text production was monitored and the results analysed, progressing from handwritten monomodal work, through stages of multimodality to full multimodal expression. The results suggest strongly that some children need multimodal scaffolding in order to communicate complex ideas effectively. This, however, requires an acceptance of multimodal texts as part of the primary literacy curriculum. It is therefore suggested that assessment of multimodal composition, both narrative and other texts, should be developed to help teachers accept the value of introducing multimodal literacies into the classroom.