A closer look at integrated writing tasks: Towards a more focussed definition for assessment purposes

The use of integrated tasks is increasingly common in second language assessment in both high stakes and classroom contexts. However, because of the vague definition of what constitutes an integrated task, the types of tasks included in this area and the assessment of the resulting writing is highly varied. In this paper, we argue for a better definition of the term ‘integrated writing task’. We start by reviewing current definitions available in the literature, discuss the construct underlying integrated tasks and then propose a number of features which we feel need to be present in integrated tasks. We then propose a new, narrower definition of the term. We hope that this will result in better task design, improved assessment outcomes for test takers, more detailed test specifications, better replicability of research and a shared research agenda.     

Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study

This study explores whether linguistic features can predict second language writing proficiency in the Test of English as a Foreign Language (TOEFL iBT) integrated and independent writing tasks and, if so, whether there are differences and similarities in the two sets of predictive linguistic features. Linguistic features related to lexical sophistication, syntactic complexity, cohesion, and basic text information were investigated in relation to the writing scores for both integrated and independent samples. The results of this study show that linguistic features can be used to significantly predict essay scores in the integrated as well as the independent writing. When comparing across the two writing tasks, there are both similarities and differences in the two sets of predictive features. For instance, lexical sophistication was found to be a significant predictor for both tasks while features such as verbs in 3rd person singular form and semantic similarity were only significant predictors for the integrated task. These findings demonstrate that evaluation of the two writing tasks rely on similar and distinct features, and are at least partially assessed using different linguistic criteria. Implications of these findings for the assessment of second language (L2) writing are also discussed.     

Using emotional intelligence in an EFL integrated writing assessment

The use of integrated writing tasks instead of traditional impromptu writing-only tasks is gaining ground in the teaching and assessment of EFL/ESL writing. Source text as the distinct feature of the integrated writing and the concern to improve its quality for the enhanced students' engagement has been a fertile line of inquiry. In this regard, the present study used a body of 67 Iranian intermediate EFL learners in three modalities: emotionalized reading-to-write (ERW) group which received emotionally-loaded source texts along with the emotional activities based on Goleman's Emotional Intelligence framework, reading-to-write (RW) group which developed their essays based on the texts with no focus on their emotional content and a writing-only (WO) group as the control group which followed the traditional writing procedure. The findings showed that ERW group outperformed RW group on the integrated writing tasks. In addition, ERW group performed better than the RW and WO on the overall independent writing task. Furthermore, the emotional intelligence of the students in the ERW group improved more than that of the RW and WO groups in the study. The study discusses that the emotional content of the source text followed by the emotionalized activities would significantly enhance the learners’ engagement with the source text in the reading-to-write tasks and consequently improve their writing. The implications and suggestions for further research are next discussed.     

Incorporating formative assessment in Iranian EFL writing: a case study

Undergraduate students' experience of assessment in universities is usually of summative assessment which provides only limited information to help students improve their performance. By contrast, formative assessment is informative and forward-looking, possessing the leverage to inform students of their day-to-day progress and inform teachers of how to better tailor their instruction to students' immediate learning needs. Despite these potentials, studies carried out on the use of formative assessment in English as a foreign language (EFL) contexts are somehow rare. The current study reports on incorporating formative assessment in an L2 writing course in Iran. The analysis of data from pre- and post-study writing tasks, pre- and post-study questionnaires, and semi-structured interviews revealed that first-year undergraduate students were offered opportunities to improve various aspects of their writing and to develop positive attitudes toward writing as well as formative assessment. However, the students reported several challenges that could have implications for the further implementation of formative assessment in similar contexts.     

When a “sloppy copy” is good enough: Results of a state writing assessment

Students in grades 5 and 8 completed a state writing assessment, and their first and final drafts on the extended writing portion of the test were copied and scored using the state writing rubric. The rubric consisted of three primary traits: Content and Organization, Style and Fluency, and Language Use. Scorers were blind to the study purpose and scored either a student's first or final draft. No significant difference was found between the first and final drafts written by students in special education at both grade levels. Likewise, no significant difference was found for the writing of general education students in grade 8. A significant difference was found, however, between first and final drafts written by fifth-grade students in general education. Cross tabulations conducted at grades 5 and 8 revealed that over 50% of the first drafts received the same score or a better score than what was earned on the final draft.     

“This test makes no freaking sense”: Criticism, confusion, and frustration in timed writing

Large-scale writing programs can add value to the traditional timed writing assessment by using aspects of the essays to assess the effectiveness of institutional goals, programs, and curriculums. The “six learning goals” prompt in this study represents an attempt to provide an accurate writing assessment that moves beyond scores. This paper focuses on student challenges to the prompt and testing situation to reveal that many students successfully challenge the task as a deliberate strategy, while less savvy test-takers clearly resist in response to anger, confusion or frustration. While only a small minority of test-takers openly protest the prompt or testing situation, the paper suggests that all students could be better prepared to reflect upon their university experience in timed essays through more coaching and experience with reflective impromptu questions. This finding offers both encouragement and caution for writing programs seeking a single test that can generate both placement scores and valuable feedback.     

Discursively complex identities of New Zealand teachers of writing in times of contestation and change

Changes in political, social and educational curriculum policies over the past four decades have created discursive shifts in writing theory and practice for New Zealand primary teachers. While these policies have historically privileged a particular view of writing over others, very little is known as to how teachers engage with experienced discourses of writing.Three broad conceptual metaphors, taking a ‘writer’, ‘text’ and ‘social’ perspective, frame the writing theories and practices and provide a context for the development of heuristic markers used to analyse the teachers’ interviews. Discourse analysis revealed teachers’ complex identities and knowledge or lack of, available writing discourses.     

The impact of bilingual dictionaries on lexical sophistication and lexical accuracy in tests of L2 writing proficiency: A quantitative analysis

Writing assessment essentially juxtaposes two elements: how ‘good writing’ is to be defined, and how ‘good measurement’ of that writing is to be carried out. The timed test is often used in large-scale L2 writing assessments because it is considered to provide reliable measurement. It is, however, highly inauthentic. One way of enhancing authenticity in a way that might also promote good writing is to include access to support resources in the test. This article reports a study into whether allowing a bilingual dictionary enhances good writing in German writing tests. Participants (n = 47) took two tests, one with and one without a bilingual dictionary. The essays of sub-sets of the participants were compared in terms of lexical sophistication, lexical accuracy and test score. Lexical sophistication was measured by means of the Lexical Frequency Profile (Laufer & Nation, 1995). Lexical accuracy was determined through a quantitative analysis of words looked up in the dictionary. The results revealed an increase in lexical sophistication in ‘with dictionary’ tests but frequent misuse of look-ups. There was also no improvement in test scores. Subsequent issues for the use of dictionaries in writing tests are raised.     

Evaluating the reliability of a detailed analytic scoring rubric for foreign language writing

The demand for valid and reliable methods of assessing second and foreign language writing has grown in significance in recent years. One such method is the timed writing test which has a central place in many testing contexts internationally. The reliability of this test method is heavily influenced by the scoring procedures, including the rating scale to be used and the success with which raters can apply the scale. Reliability is crucial because important decisions and inferences about test takers are often made on the basis of test scores. Determining the reliability of the scoring procedure frequently involves examining the consistency with which raters assign scores. This article presents an analysis of the rating of two sets of timed tests written by intermediate level learners of German as a foreign language (n = 47) by two independent raters who used a newly developed detailed scoring rubric containing several categories. The article discusses how the rubric was developed to reflect a particular construct of writing proficiency. Implications for the reliability of the scoring procedure are explored, and considerations for more extensive cross-language research are discussed.     

身体写作在中国:别有一番滋味的感性呈现

在西方20世纪60年代以埃莱娜·西苏、波伏娃等为代表的女权主义者提出的身体写作是从意识形态的角度以一种男女平等的信念,旨在反对包括性别歧视在内一切不平等的社会现象。在这种浪潮的影响下,我国女性文学迅速发展,充分的挖掘了女性躯体的革命性,同时也让身体写作成为一股女性文学的重要潮流。     

On the basis of source: Impacts of individual differences on multiple-document integrated reading and writing tasks

Few studies have explored how general skills in both reading and writing influence performance on integrated, source-based writing. The goal of the present study was to consider the relative contributions of reading and writing ability on multiple-document integrative reading and writing tasks. Students in the U.S. (n = 94) completed two tasks in which they read text sets about a socioscientific issue, generated constructed responses while reading, and then composed integrated essays. They also completed individual difference measures (general knowledge, reading skill, reading strategy use) and wrote independent essays to assess their writing ability. Mixed effect models revealed that general knowledge and reading skills contributed to integrated essay performance, but that once general writing ability was entered into the model, it became the strongest predictor of integrated writing scores. These results suggest the need for deeper consideration of the role of writing skills in integrated reading and writing tasks.     

心灵的感悟--谈王安忆的"三恋"创作

王安忆在1986年连续发表了<荒山之恋><小城之恋><锦绣谷之恋>,写作手法独特,突破了她固有的创作模式,受到评论界的极大关注,该文通过对"三恋"的分析,认为"三恋"的创作是作者从心灵深处对人生的感悟,是用小说语言谱写的人性乐章,而且"三恋"的整体创作带有一种较明显的音乐特征.     

Evaluating the impact of first-year writing course grades on college-student persistence

Informed by Tinto’s model of persistence, this large-scale evaluation study (N = 8916) centers on intersections of first-year composition (FYC), grades, and persistence at a career-focused, polytechnic university in a midwestern U.S. state. It presents a replicable design exploring a pathway through Composition 1 and Composition 2 to graduation and suggests theoretical and practical implications for early-alert, student-support intervention. Results of binary logistic regression indicate that, accounting for demographic and enrollment factors, grades significantly predict whether students take Composition 2 immediately after Composition 1, whether they persist to Composition 2 at all, and whether they graduate. We propose that the linked sequence of writing courses may allow students to build skills over a year-long sequence, enhancing self-efficacy and grade-reinforced senses of belonging, while factors related to ethnic identity and first-generation status pose barriers at critical points along the first-year sequence. We conclude with suggested points of student-support intervention and future research on grades, first-year writing coursework, and college-student persistence related to writing self-efficacy and senses of belonging.     

Self-assessment of writing in independent language learning programs: The value of annotated samples

The availability of multi-media technologies in education has made the option of independent learning increasingly attractive. Whilst independent learning presents learners with a more flexible learning context, it also presents new challenges in assessment in that the onus is placed upon the learners themselves to monitor and evaluate their own learning.This paper reports on the development of an approach to learner self-assessment which aims to facilitate the self-assessment of performance on extended-response writing tasks. Whilst such tasks are typical within current approaches to communicative language learning, a review of the self-assessment literature reveals no precedent for how actual task performance might be effectively and reliably self-assessed. A methodology was therefore developed which used annotated samples of learner-produced texts, where the annotations would elucidate both the criteria by which performance should be judged and the standards expected, thus enabling learners to assess their own performances by comparing their work with these samples. The methodology was trialled on a small sample of students. It was found to be both reliable and useful, allowing students not only to accurately evaluate their own performance but also to learn new language skills from the samples.     

Historical view of the influences of measurement and writing theories on the practice of writing assessment in the United States

The purpose of this study is to examine the interactions among measurement theories, writing theories, and writing assessments in the United States from an historical perspective. The assessment of writing provides a useful framework for examining how theories influence, and in some cases fail to influence actual practice. Two research traditions are described to classify measurement theories (test-score and scaling), and three research traditions are proposed for classifying writing theories (form, idea and content, and sociocultural context). The results of this study trace the impact of measurement and writing traditions on writing assessment practices within selected time periods during the 20th century in the United States. One of the major findings of this historical analysis is that measurement theory has had a strong influence on writing assessments, while writing theory has had minimal influence on writing assessments. We also found support for the idea that a new discipline of writing assessment has emerged. This new discipline combines multiple fields including the writing, composition, and measurement communities of scholars, and it has the potential to set the stage for the future of writing assessment in the 21st century.     

Examining the use of multiple writing and discourse tasks in 5th grade mathematics

Abstract

The purpose of this study is to examine how fifth grade students were impacted by the infusion of multiple writing tasks in mathematics. In this study, writing tasks provided opportunities for students to communicate prior knowledge, share ideas to construct and justify arguments, for reflection, and assessment. In this deductive qualitative study, students’ work samples were analyzed. Findings indicated that students grew in their understanding of mathematics and ability to self-reflect and self-evaluate through multiple opportunities to write for a variety of purposes. The opportunities for constructing mathematical understanding with activities that included writing and discourse also fostered learning between peers. The findings suggest a variety of opportunities to write and engage in mathematics discourse encouraged reflection, evaluation, and learning. Implications for future research include the need to examine the impact of these activities on students’ mathematics understanding as measured by assessments or an analysis of student work samples.     

National Assessment of Writing Proficiency Among Norwegian Students in Compulsory Schools

The paper reports and discusses a government‐initiated nationwide assessment of writing proficiency among Norwegian compulsory school students. A sample‐study of 7th and 10th grade students are discussed and reported with regard to challenges in measuring writing skills in a valid and reliable manner. For the 7th graders the results showed a greater proportion of narrative texts, and in contrast to more scientific oriented texts, was assessed as “lower than expected”; however, for the 10th graders the tendency was opposite with respect to central linguistic components. Low correlations between the raters were ascertained at both levels, indicating different views among teachers as to what can be expected of students' writing proficiency. The results are discussed in relation to the usefulness of the theoretical model as a basis for assessment of writing proficiency, as well as other obstacles to constructing valid and reliable writing tests.     

Keyboarding compared with handwriting on a high-stakes writing assessment: Student choice of composing medium, raters’ perceptions, and text quality

This article examines the influence of keyboarding versus handwriting in a high-stakes writing assessment. Conclusions are based on data collected from a pilot project to move Old Dominion University's Exit Exam of Writing Proficiency from a handwritten format into a dual-option format (i.e., the students may choose to handwrite or keyboard the exam). This test case is used to demonstrate the complex implications for a writing assessment tool when universities aim to accommodate students’ various methods of physically creating a text. Discussion includes speculation about the format of the writing component on the SAT. It is argued that by analyzing the differences in high-stakes exams when students keyboard and when they handwrite, researchers can better understand how the material conditions for composing influence students and raters. It is concluded that if handwriting and keyboarding represent even slightly different composing skills and cognitive processes, then high-stakes writing assessments should aim to accommodate students by providing methods of textual production with which students are most comfortable, even as this accommodation produces new challenges for the creators and managers of large-scale writing assessments.     

Conceptualizations of writing in early years curricula and standards documents: international perspectives

Abstract

In this article, researchers in the field of early writing identify underlying beliefs and values about writing and learning to write for the beginning years of formal schooling in four jurisdictions: the American state of Connecticut, New Zealand, the Canadian province of Ontario, and Sweden, as reflected in the respective curricula and standards documents that guide instruction. Using Ivani?’s Discourses of Writing and Learning to Write to guide our text analysis, we found that curriculum developers have primarily been influenced by views of writing as a set of skills, processes, and genres. We found few references to the sociopolitical discourse which indicates a view among curriculum developers that sociopolitical literacy is not suitable for this age group. We argue, with support in previous research, that young children’s writing does not have to be politically neutral and that it can be developed under age-appropriate circumstances. Implications for policy and curriculum development include a need for greater consideration of the complexities of writing shown in research conducted across five decades. We propose a change to the model for early years, recognising that young children’s socio-political understandings lie within their home and school lives, rather than the broader community.