Interrater reliability in a california middle school english/language arts portfolio assessment program

This article examines statistical evidence for the reliability of a locally developed portfolio assessment system across three separate portfolio scoring sessions over the course of a year and finds that the English teachers at this California site were able to demonstrate “strong” levels of interrater agreement. Further, the statistical evidence reported indicates that levels of agreement improved with each scoring session without mandating a fixed task portfolio menu or resorting to piece-by-piece scoring procedures. The study argues that defensible local portfolio assessment systems can be developed which enhance, rather than diminish, teacher professionalism while still providing dependable data for external purposes.