Students' conditioned response to teachers' response: portfolio proponents, take note!

Portfolio advocates argue that teacher commentary becomes more meaningful for students with the use of portfolio assessment, particularly because the commentary is unaccompanied by a grade. However, my own study of portfolio classrooms suggests that students continue to regard teacher responses as directives that leave them few options in terms of revisions. My study involved six writing classrooms from the middle school to the university level and included classroom observations, interviews of students and teachers, and examination of student writing and teacher response. The students generally looked to their teachers to show them the “correct” way to write and resisted the notion of making independent judgments about their writing and the necessary revisions, primarily because they could not ignore the ultimate reality of the grade. Yet the teachers generally avoided being directive in their responses. In one instance, the teacher's best efforts to adopt a more open style of response backfired when a sensitive student read the commentary as demeaning. I argue, therefore, that even though portfolios represent a more enlightened approach to assessment, students have difficulty escaping their conditioned obeisance to teacher authority. While this finding should not be read as a condemnation of portfolio assessment, it does indicate that teachers need to be aware of how students read their responses. Otherwise, portfolios alone may not substantially alter the teacher–student dynamics.     

issues in Portfolio Assessment: Assessing Writing Processes From Their Products

We investigated the ways that portfolio evidence of students' competencies with writing processes was created and interpreted in 4 classrooms. Our study was conducted during preliminary classroom trials of California Learning Assessment System portfolios, when teachers and students were challenged with the new task of preparing portfolios that demonstrated students' competency with the "dimensions of learning." Drawing data from teacher and student interviews as well as portfolios, we considered three issues regarding the meaning of portfolio indicators of writing processes (a) Students' opportunities to learn to use a range of resources, processes, and standards in ways that enhance the effectiveness of their writing; (b) students' opportunities to produce "hard copy" evidence of their uses of processes; and (c) students' capacities to analyze their writing processes. Further research is needed to understand how participants in a large-scale portfolio assessment program develop shared understandings of the ways that evidence of writing processes is considered in the scoring and how the programmatic needs for comparability of evidence can be reconciled with the personal needs of young writers, whose uses of processes will vary with the purposes and contexts of their writing.     

Portfolios as evidence of reflective practice: what remains ‘untold’

Addressing recent calls for investigating the specific quality of reflection associated with the uses of portfolios in teacher education, this paper describes and interprets the ‘practice of portfolio construction’ as revealed in the construction and presentation of two kinds of portfolio in two in-service courses for mentors of teachers in Israel: a ‘process’ portfolio and a ‘product’ portfolio. The study revealed that the language of practice and form of reflection bore striking similarities across the two practices of portfolio construction, regardless of their differences in content, purpose, organization and the degree of intervention of the course instructors in its construction. In both types of portfolios, the mentors described their learning mostly at technical levels of reflection. This tendency raises the question of whether the genre of portfolio writing, inevitably bound by institutional constraints, is generically conducive to reflecting on controversial experiences at interpretative, critical levels. The study suggests that within a centralized educational system, as in the case of Israel, the documentation of critical reflection is problematic.     

Student teachers’ patterns of reflection in the context of teaching practice

This study clarifies the basic structure of student teachers’ reflective thinking. It presents a constructivist account of teacher knowledge through a detailed analysis of various patterns of reflection in student teacher portfolios. We aim to gain a greater understanding of the process and outcomes of portfolio writing in the context of teaching practice. By closely analysing portfolio texts, we defined six main starting points for reflective episodes and several patterns under each of them. Also, the patterns of reflective episodes were analysed according to their deductive and inductive dimensions, together with their static and dynamic features. According to our results, it is possible that student teachers can reflect beyond solely practical issues on teaching, articulate multiple concerns about practice and elaborate them in an integrative manner as well as learn both from theory and from practice as a result of reflection for their future profession.     

It was the best of times. it was a waste of time: university of kentucky students' views of writing under KERA

A survey was administered to University of Kentucky freshmen enrolled in introductory composition courses. Former Kentucky high school students were asked to describe their high school writing experiences since the 1990 passage of the Kentucky Education Reform Act (KERA) and the implementation of its controversial testing and accountability component. The survey revealed much that is positive. Almost three-quarters of the students reported writing daily in high school in a variety of disciplines. Approximately one-half rated their writing abilities as “above average” or “excellent” and felt prepared or somewhat prepared to write in college. Most reported regularly employing writing process strategies. Few, however, mentioned consideration of purpose and audience as part of their approach to writing, even though these are the two most important criteria for scoring the portfolio. Some two-thirds of the students responded that compiling the portfolio was not a useful activity. These findings suggest that KERA's accountability system may be undermining the instructional improvements it was intended to foster. In its haste to implement mandated reform, the state failed to take into account students' willingness to assume new roles as “creators, authors, and owners” of writing portfolios. Students' negative attitudes toward writing under KERA may reflect a more widespread misunderstanding among teachers and administrators of the theoretical bases for the writing portfolio. Professional development might help to dispel some of this negativity, but classroom teachers alone cannot bear responsibility for ensuring that students see “real world” value in the writing portfolio. Public schools, post-secondary institutions, and employers must work together to find authentic uses for the portfolio.     

Digital portfolios in pre-service teacher education

With the pre-service student portfolio process and product well in hand in a paper-based format, in the Bachelor of Education (Primary) (B.Ed. Primary) at University of Western Sydney (UWS), new horizons have presented themselves. These new possibilities are facilitated but not driven by developments in Information and Communication Technology (ICT). The impetus for this study comes from the changing context in which the students will work and from the rapidly evolving mediums of communication employed by the society these future teachers will serve. With new technologies being developed the use of digital portfolios could give an extra dimension to student learning as well as giving choice and variety to the reporting and presenting of that learning. This paper aims to report on the process of the development of digital portfolios as an alternative method of reporting and presenting student learning, as opposed to the current paper-based portfolios used in Pre-service Teacher Education at UWS. A study of the processes employed by nine students who elected to develop their portfolios digitally was carried out over a three-semester period. This resulted in a procedure that assisted students in using ICT to showcase their learning. The advantages and limitations of the use of such technology and the results of this process during its initial implementation are discussed.     

The Application of Student Portfolios in Primary-Intermediate and Self- Contained-Multiage Team Classroom Environments: Implications for Instruction,Learning, and Assessment

Portfolios have gained wide acceptance as a learning and assessment tool. Yet, little research has been reported on the practices of teachers who are actually using portfolios within their classrooms and how those practices are moderated by contextual variables. This research examined the instructional, learning, and assessment roles of student portfolios and explored, from the perspective of the classroom teacher, variations in portfolio applications associated with teaching level (primary vs. intermediate) and classroom environment (self-contained vs. multiage-teaming).

Kindergarten through Grade 5 teachers in 13 elementary schools completed a survey questionnaire regarding the instructional and assessment uses to which portfolios are put within their classrooms. To further examine for patterns of portfolio use, a subset of teachers was interviewed to explore the perceptions that teachers hold about the impact of student portfolios on themselves and on their students. The results suggest that Kindergarten through Grade 5 teachers make deliberate decisions regarding uses of their students' portfolios, decisions that appear heavily impacted by the maturity or skill level of the child, the purposes of the application, and the classroom environment within which the application occurs. They also depend on whether the portfolio product is in a formative state (working portfolio) or final state (performance portfolio).     

Teaching portfolios: A strategy for developing learning and teaching in preservice education

Teaching portfolios have been used in the preservice teacher education program at Monash University to help student teachers to reflect on their learning about learning and teaching and to help them to convey this to others. The portfolio is an open-ended and un-graded task designed to explore teaching from many different vantage points. It is organised as a dynamic assessment task, not simply a static end product. This is done by considering teaching portfolios as comprising two important aspects, one is the process the other is the product. The process involves learning from the variety of experiences offered in the preservice education program and encouraging student teachers to reflect on these. The product is the development of the individual portfolio items that are used to demonstrate this learning to others. The portfolio comprises a number of individual items which act as a prompt to “tap” the creator's understanding of what it means to be a (science) teacher. This paper reports on the effectiveness and value of portfolios from the student-teachers' perspective by exploring how their understanding of the task evolved as they completed their preservice teacher education program.     

Addressing Reliability Problems in the Portfolio Assessment of College Writing

In research and development designed to assess the writing skills of third-year college students, the University of Wisconsin Verbal Assessment Project developed and tested procedures for assessing writing portfolios from students in courses representing each college in the university. Following the work of Britton (1970) and 1:he National Assessment of Education Progress, we defined expository writing as sustained reflection in which the writer focuses and processes information to various degrees. Basing our work on this construct, we assessed writing samples in each portfolio in terms of both degree of reflection and extent of text elaboration. Results of two studies are presented. In Study 1, raters scored each text from a given portfolio before rating texts in the next portfolio. Reliability estimates were low to moderate for both scores. In follow-up Study 2, involving a comparable group of students, several changes were made to improve reliability: (a) Raters scored all texts written in response to a given prompt or assignment within a class before moving to the next set of texts; and (b) each time readers dealt with a new task, they read several examples together, coming to agreement about how various texts were to be rated. Estimates of reliability for both scores were somewhat higher and suggest that the modifications improved reliabilities. Results demonstrate that adequate reliability should be expected if texts are rated by task across portfolios within classes. Based on these findings, we contend that, because writing normally varies by topic, genre, and other variables, writing portfolios are better characterized by scores for each piece than by a single writing-skill score.     

Rhetoric and the writer's profile: Problematizing directed self-placement

Open enrollment and walk-in advising at two-year colleges make placement assessment a continual, year-round process, effectively prohibiting the use of placement processes like entrance portfolios. On the grounds of expediency, many two-year institutions have turned to computerized editing tests such as COMPASS for placing entering students into writing courses, even though such tests do not directly measure writing. Forgoing placement assessment entirely through directed student self-placement, such as that described by Royer and Gilles, has also become an attractive alternative for some institutions. Beginning with the premise that assessment is a rhetorical act, the authors describe their reasons for resisting computer editing tests and suggest possible problems with using only directed student self-placement in open access institutions. They then describe a placement process, the Writer's Profile, which they developed. A sample student profile is presented to illustrate the interaction and negotiation among writing teachers as they read profiles and reach an agreement about their placement recommendation. The authors argue that the form of assessment chosen is important because as a rhetorical act assessment affects curriculum, pedagogy, faculty development, and even the surrounding community's expectations and perceptions of college writing.     

Knowledge management for multi-assessment portfolios

This paper presents the results of a project conducted under the Greenfield Coalition concerning the development of an e-portfolio (a digital version of portfolios) assessment system. The goal of the system is to support student, course, instructor and curriculum assessment to enable continuous improvement of the educational process through the development of portfolios for each of them. In designing this system two areas of concern were identified: the development of an architecture for an easily maintainable system for structuring assessment portfolios and whether the system would be easy enough for first-year students to use. Two experiments were conducted to resolve these issues and verify that the architecture under development is proceeding appropriately. Toassure that the resulting system can properly structure and display the portfolios, a set of student and course data was entered into the system. Pages demonstrating the students' progress toward a course objective were automatically generated. To affirm that the mechanism identified for collecting the data is appropriate, a year-long digital portfolio collection was undertaken. Students submitted their coursework and supporting documents to a web server and used it to create their portfolio. Student reaction to the system was favourable. This project leads us to believe that the suggested architecture holds promise and merits continued development.     

The development of large-scale portfolio placement assessment at the University of Michigan: 1992–1998

The first section of this article examines in detail the mechanics of a large-scale writing portfolio assessment at the University of Michigan, including its impact on matriculation and placement; students' reactions to the requirement; and instructors' evaluation of the efficacy of placements under the new system. The second section of the article examines the scoring process used by readers to assess portfolios, describes its evolution over a 4-year period, and examines the results produced by various changes made to this process over time. The evolution of this portfolio placement assessment has not been traditional, in that it has enacted an understanding that people learn to read and write within specific social contexts, at the same time that it has enacted a commitment to connecting the intellectual and practical work of university teaching with the work of teaching that comes before it in the high schools. The assessment process also has enacted the authors' belief that issues of assessment are intimately linked to issues of curriculum and pedagogy; thus, the assessment has become an occasion for sustained talk about the purposes of our required writing courses, about the fit between our lower division and upper division writing requirements, and about the degree to which teachers share criteria for assessing students' writing within courses.     

Assessing Agreement: An Examination of the Interrater Reliability of Portfolio Assessment in Rochester,New York

This article reports on the interrater reliability of language arts portfolio assessment in the primary grades of the Rochester, New York, school system. This assessment was based on a sample of approximately 400 primary-grade language arts portfolios that were rated by two raters: the student's classroom teacher and an external reviewer. Reliabilities for student portfolios ranged from .58 for Grade 1 reading portfolios to .79 for kindergarten reading portfolios. Many of the reading portfolios could not be rated by the external reviewer, revealing problems both with the portfolio policies of the district and with the classroom-level implementation. The findings showed that primary-grade classroom teachers can reliably rate their own students' work, but tensions exist if a single portfolio system is intended both to provide classroom teachers with information about their students and to supply accountability information to administrators.     

Portfolios: A tool for reflective thinking in teacher education?

Portfolios have often been promoted as a tool for reflective thinking, yet few studies have examined the use of portfolios in reflective teacher education programs. This exploratory study uses interviews, essays, and survey data to examine 212 teacher education students' efforts to think reflectively through the process of constructing portfolios based on their experiences in a community service-learning program. Findings revealed that the portfolio process prompted reflective thinking in many, but not all, students. Recommendations for using portfolios in teacher education programs include: focusing attention on students' initial understanding of the process and its purpose, encouraging student ownership and individual expression, providing some structured aspects to balance the open-ended nature of portfolios, and evaluating the portfolio process and students' responses.     

The Development and Use of Literacy Portfolios for Students,Classes, and Teachers

Portfolio assessment is one of the most interesting and widely discussed of the new alternative assessments. In this paper we explore the portfolio metaphor as it is applied to literacy portfolios. We suggest that portfolios provide a powerful tool for the enhancement of instruction and assessment, addressing educators' concerns about authentic assessment, documentation of academic progress, and teacher and student involvement. However, we caution that there are many important and unresolved issues that must be confronted if portfolio assessment is to succeed. Fundamental issues of validity and reliability must be addressed as well as practical issues of implementation, standard setting, sufficient resources, and teacher expertise. We argue that the ultimate success of portfolios will rest on our ability to communicate portfolio-based assessment information to others. We offer the teacher's class portfolio as a promising strategy for aggregating and reporting information, while preserving the integrity of individual student portfolios and teacher judgment. Whether the emphasis is on individual student portfolios or classroom portfolios, educators must be committed to the staff development and additional research that portfolio assessment demands.     

Stories of reading: Inside and outside the texts of portfolios

This essay employs Barbara Herrnstein Smith's notion of “contingencies of value,” the idea that evaluations of text vary because our readings take place in specific contexts and are shaped by cultural and historical exigencies. In this study, we apply this notion to the reading of student texts in a college composition portfolio assessment. Through an analysis of taped teacher discussions of students' writing and an examination of student responses to the grading process, we conclude that in every reading of a text (but especially in the reading of the multiple texts of a portfolio) readers posit an “implied author.” That is, based on their reading of a single text or portfolio, teacher-readers construct a persona that represents the author, and this projection can strongly influence the reader's evaluation of student work. Group discussions of portfolios allow teachers to expose and explore the value-laden nature of these judgments.     

Two portfolio systems: EFL students’ perceptions of writing ability,text improvement,and feedback

Research into portfolio assessment (‘PA’) typically describes teachers’ development and implementation of different portfolio models in their respective teaching contexts, however, not much attention is paid to student perceptions of the portfolio approach or its impact on the learning of writing. To this end, this study aims to investigate how two groups of Hong Kong EFL pre-university students (Groups A and B) perceived and responded to two portfolio systems (with each group experiencing one portfolio system either working portfolio or showcase portfolio) in one academic writing course. The case study approach was adopted and data sources included semi-structured interviews, student reflective journals, classroom observations, and analysis of text revisions. Findings indicated that students from the showcase portfolio group (Group B) were less enthusiastic about the effectiveness of PA, and queried whether it could promote autonomy in writing, while the working portfolio group (Group A) was more receptive to the experience, and considered that a feedback-rich environment in the working portfolio system could facilitate writing improvement. The paper concludes with a discussion of how PA can be used to promote self-regulation in the learning of writing.     

Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios

Student portfolios are increasingly used for assessing student competences in higher education, but results about the construct validity of portfolio assessment are mixed. A prerequisite for construct validity is that the portfolio assessment is based on relevant portfolio content. Assessment criteria, are often used to enhance this condition. This study aims to identify whether assessment criteria can improve content, argumentation and communication during teacher moderation while judging student portfolios. Six teachers scored 32 student portfolios in dyads with and without assessment criteria. Their judgement processes were qualitatively analysed. Results indicated that the quality of their judgement processes was low, since teachers based their judgements mainly on their own personal opinion and less on evidence found in the portfolio. Teachers barely paid attention to quality checks and easily agreed with each other. When teachers used assessment criteria, the quality of their judgements slightly improved. They based their judgements more on relevant evidence, used less personal experiences and more often checked the quality of their judgement processes. It is concluded that the quality of teacher portfolio judgement is low, and that the use of assessment criteria can enhance its quality.     

Portfolio assessment in an American Engineering College

Estimates indicate that as much as 80% of an engineer's work time is spent on communicating. Studies done by the Department of Electrical and Computer Engineering (ECE) at the University of Arizona show that engineering firms, as well as ECE graduates, rank writing ability as the most important skill in determining engineers’ success, even above the much more obvious technical skills that are the focus of much engineering education. Arizona's ECE Department began teaching its senior-level capstone design courses as “writing emphasis” courses to assist students in developing their writing skills prior to graduation, as well as to link communication skills and coursework to create graduates proficient in both written and oral communication. This paper describes a portfolio assessment that has been developed specifically for this context and population, and argues that this portfolio assessment demonstrates several advantages frequently called for in theory but rarely made operational. This program has provided an unusually rich practical application, showing how portfolio assessment changes assessment from a mere rating of limited and often costly tests to a continuous, integrated, and appropriate activity directly related to learning itself, while at the same time, developing the data about student.     

Advancement towards the scholarship of teaching and learning through the writing of teaching portfolios

This study explores the impact academic teachers’ writing of a teaching portfolio can have on their professional learning. Through an open-ended questionnaire, 26 academics from three faculties reported on insights, effects on teaching practice, and effects on collegial exchange that the portfolio writing entailed. We discuss how the experienced impact relates to the three competence levels excellence, expertise and scholarship of teaching and learning. With regard to academic development, we conclude that the writing of reflective teaching portfolios has the potential of contributing to an emerging academic community of practice characterised by a scholarly approach to teaching and learning.