How to capture growth? – Video narratives as an instrument for assessment in teacher education

Portfolios are widely used as instruments for assessment in initial teacher education courses. They are claimed to present a comprehensive picture of student teachers' knowledge and performance. But what type of evidence is needed to safely say that an aspiring teacher has not only grasped essential notions and concepts from the teacher education course, but is also able to implement them in real world classroom situations? This paper reports on the design process of one portfolio assignment that has been developed specifically to capture students' classroom performance and development in their portfolio. Data from the portfolio entries of one student from a small-scale pilot conducted with the assignment are analyzed in detail followed by a discussion of the implications of the case study.     

Assessing Agreement: An Examination of the Interrater Reliability of Portfolio Assessment in Rochester,New York

This article reports on the interrater reliability of language arts portfolio assessment in the primary grades of the Rochester, New York, school system. This assessment was based on a sample of approximately 400 primary-grade language arts portfolios that were rated by two raters: the student's classroom teacher and an external reviewer. Reliabilities for student portfolios ranged from .58 for Grade 1 reading portfolios to .79 for kindergarten reading portfolios. Many of the reading portfolios could not be rated by the external reviewer, revealing problems both with the portfolio policies of the district and with the classroom-level implementation. The findings showed that primary-grade classroom teachers can reliably rate their own students' work, but tensions exist if a single portfolio system is intended both to provide classroom teachers with information about their students and to supply accountability information to administrators.     

The Application of Student Portfolios in Primary-Intermediate and Self- Contained-Multiage Team Classroom Environments: Implications for Instruction,Learning, and Assessment

Portfolios have gained wide acceptance as a learning and assessment tool. Yet, little research has been reported on the practices of teachers who are actually using portfolios within their classrooms and how those practices are moderated by contextual variables. This research examined the instructional, learning, and assessment roles of student portfolios and explored, from the perspective of the classroom teacher, variations in portfolio applications associated with teaching level (primary vs. intermediate) and classroom environment (self-contained vs. multiage-teaming).

Kindergarten through Grade 5 teachers in 13 elementary schools completed a survey questionnaire regarding the instructional and assessment uses to which portfolios are put within their classrooms. To further examine for patterns of portfolio use, a subset of teachers was interviewed to explore the perceptions that teachers hold about the impact of student portfolios on themselves and on their students. The results suggest that Kindergarten through Grade 5 teachers make deliberate decisions regarding uses of their students' portfolios, decisions that appear heavily impacted by the maturity or skill level of the child, the purposes of the application, and the classroom environment within which the application occurs. They also depend on whether the portfolio product is in a formative state (working portfolio) or final state (performance portfolio).     

issues in Portfolio Assessment: Assessing Writing Processes From Their Products

We investigated the ways that portfolio evidence of students' competencies with writing processes was created and interpreted in 4 classrooms. Our study was conducted during preliminary classroom trials of California Learning Assessment System portfolios, when teachers and students were challenged with the new task of preparing portfolios that demonstrated students' competency with the "dimensions of learning." Drawing data from teacher and student interviews as well as portfolios, we considered three issues regarding the meaning of portfolio indicators of writing processes (a) Students' opportunities to learn to use a range of resources, processes, and standards in ways that enhance the effectiveness of their writing; (b) students' opportunities to produce "hard copy" evidence of their uses of processes; and (c) students' capacities to analyze their writing processes. Further research is needed to understand how participants in a large-scale portfolio assessment program develop shared understandings of the ways that evidence of writing processes is considered in the scoring and how the programmatic needs for comparability of evidence can be reconciled with the personal needs of young writers, whose uses of processes will vary with the purposes and contexts of their writing.     

The Development and Use of Literacy Portfolios for Students,Classes, and Teachers

Portfolio assessment is one of the most interesting and widely discussed of the new alternative assessments. In this paper we explore the portfolio metaphor as it is applied to literacy portfolios. We suggest that portfolios provide a powerful tool for the enhancement of instruction and assessment, addressing educators' concerns about authentic assessment, documentation of academic progress, and teacher and student involvement. However, we caution that there are many important and unresolved issues that must be confronted if portfolio assessment is to succeed. Fundamental issues of validity and reliability must be addressed as well as practical issues of implementation, standard setting, sufficient resources, and teacher expertise. We argue that the ultimate success of portfolios will rest on our ability to communicate portfolio-based assessment information to others. We offer the teacher's class portfolio as a promising strategy for aggregating and reporting information, while preserving the integrity of individual student portfolios and teacher judgment. Whether the emphasis is on individual student portfolios or classroom portfolios, educators must be committed to the staff development and additional research that portfolio assessment demands.     

The Performance Assessment System: A Portfolio Assessment Model for Evaluating Beginning Teachers

The Performance Assessment System (PAS) is a portfolio assessment mechanism used to evaluate beginning teacher performance as a function of the teachers approach and effectiveness at increasing student learning and growth. The PAS was originally developed by Teach For America, the national teacher corps, as an internal assessment tool for its teachers. The PAS has evolved in response to input from participants, administrators, and external evaluations over the course of four pilot implementations. Specifically, developers have streamlined requirements, focused efforts to incorporate measures of student performance, and clarified scoring rubrics. The PAS continues to be piloted as an assessment resource that eventually will help districts to retain only beginning teachers who are effective in the classroom and will empower states to license only teachers whose performance meets approved standards.     

Impact of a Content Selection Framework on Portfolio Assessment at the Classroom Level

Among the various alternative approaches to student evaluation, portfolio assessment has been shown to have considerable potential in terms of its use in the evaluation of higher-order, cross-curricular skills. Despite this favourable outlook, it may nevertheless remain a passing fad due to the lack of a conceptual content selection framework and difficulties surrounding its implementation within the classroom setting. The present study reports on the conceptual issues that arose when a generic content framework was used to guide the selection of entries for assessment purposes. The framework proposed the systematic collection of evidence of the cognitive, affective, behavioural, metacognitive and developmental dimensions of students' mastery of a specific competency. Data were drawn from 11 case-studies of portfolio applications by teachers from eastern Ontario, Canada. It was found that the effective integration of the portfolio into teaching and assessment practices depended largely on the flexibility of the framework and, to some extent, on the type of training or support provided to the teachers.     

Digital Portfolios

Abstract

Educators are faced with developing performance indicators for reporting to state and national agencies on pre-service teacher candidates' ability to teach using innovative pedagogy as well as content-specific standards. Portfolio assessment has gained momentum as a standard in assessing student-learning outcomes. This paper is a discussion of digital portfolio development as a stimulus to the intellectual involvement of the learner (a Type II application). Digital portfolio development is an activity that naturally involves intellectual involvement as the learner identifies the problem (developing the digital portfolio) and then problem solves as he or she creates the digital portfolio. Each student is in control of portfolio development and manipulates the Web-based software to create the professional portfolio.     

Student teacher portfolios: A portrait of the beginning teacher

Abstract

This article describes a student teacher portfolio for use as an interviewing tool. A review of the research on hiring practices indicated that prospective employers reviewed the following competencies of teacher candidates: planning skills, classroom management philosophy, reflective abilities, knowledge of appropriate assessment methods, and the initiative to do more than is expected. Therefore, the recommended sections of the student teacher portfolio are Self‐Assessment; Philosophy of Classroom Management; Planning, Implementation, and Assessment Skills; and Special Projects and Activities. Because prospective employers indicated that they would devote limited time to reading a portfolio, student teachers are instructed to make all sections only six to eight pages long. Professionally arranged appendixes can then be used to provide the employer with specific evidence for claims made in the main body of the portfolio.     

Changes in secondary teachers' perceptions of barriers to portfolio assessment

Portfolio assessment has become a popular medium for merging classroom assessment with large-scale testing, but adoption of portfolios in the classroom for external assessment purposes may be difficult because the use of such portfolios may require changes in the curriculum, instructions, and assessments used by teachers. As a result, there are numerous potential barriers to the adoption of portfolios that can be used for large-scale assessment purposes. This study investigates how secondary teachers' perceptions of portfolio implementation barriers changed when teachers participated in a 1-year portfolio implementation effort. Survey results are analyzed with a Rasch rating scale model. Results suggest that teachers' apprehension about portfolio barriers increased slightly, but that this increase can be attributable to teachers with little portfolio experience. Furthermore, teachers' concerns about the amount of time required to develop and score portfolios increased substantially while concerns about the availability of resources and resistance from parents decreased.     

Authentic assessment: creating a blueprint for course design

Authenticity has been identified as a key characteristic of assessment design which promotes learning. Authentic assessment aims to replicate the tasks and performance standards typically found in the world of work, and has been found to have a positive impact on student learning, autonomy, motivation, self-regulation and metacognition; abilities highly related to employability. Despite these benefits, there are significant barriers to the introduction of authentic assessment, particularly where there is a tradition of ‘testing’ decontextualised subject knowledge. One barrier may be the lack of conceptualisation of the term authentic assessment sufficient to inform assessment design at the individual course level. This article tackles that omission by a systematic review of literature from 1988 to 2015. Thirteen consistent characteristics of authentic assessment are identified leading to the classification of three conceptual dimensions: realism, cognitive challenge and evaluative judgement. These dimensions are elaborated and used to propose a step-based model for designing and operating authentic assessment in individual higher education subjects.     

Evaluating students' science notebooks as an assessment tool

The idea of using science notebooks as a classroom assessment tool is not new. There is general agreement that science notebooks allow teachers to assess students' conceptual and procedural understanding and to provide the feedback students need for improving their performance. In this study we examined the use of science notebooks as an unobtrusive assessment tool that can also be used by individuals outside the classroom (for example, school district personnel), and as a means for obtaining information about students' learning and their opportunities to learn. More specifically, in this study students' science notebooks were used as a source of data about the (a) implementation of a curriculum's intended activities, (b) students' performance, and (c) quality of teachers' feedback. Our results indicated that: (1) Students' science notebooks can be reliably scored. Unit implementation, student performance, and teacher feedback scores were highly consistent across raters and units. (2) High and positive correlations with other performance assessment scores indicated that the student performance score can be considered as an achievement indicator. And (3) low performance scores across the two units revealed that students' communication skills and understanding were far away from the maximum score and did not improve over the course of instruction during the school year. This result may be due, in part, to the fact that no teacher feedback was found in any of the students' notebooks across the six classrooms studied. This may reflect some characteristics of the teachers' assessment practices that may require further professional development.     

基于翻转课堂的“工程地质”课程教学设计与评价

在总结"工程地质"课程特点的基础上，将翻转课堂引入"工程地质"课程教学中，开展基于翻转课堂的"工程地质"课程教学设计，构建形成性学习评价和考核体系。翻转课堂教学模式将线上与线下教学有效融合，提高了课堂效率，增强了学生学习的主动性和课堂的参与度，有助于培养学生解决工程地质问题的综合能力。学生成绩对比分析结果表明，翻转课堂教学班期末考核成绩明显高于传统教学班；学生满意度问卷调查结果表明，学生对翻转课堂教学环节、考核体系和教学效果的满意度均超过90%，对翻转课堂教学模式给予了充分的肯定。     

Teacher Questioning: The Epicenter of Instruction and Assessment

In this article we examine some sequences of teacher–student interaction in which a teacher generates and acts on formative assessment data. We look at the teacher's practices of question construction and her decisions about in situ next pedagogical steps made in real time to support and further student learning. Our observations are guided by the following research questions: (a) What are the interactional practices that constitute formative assessment? (b) Are there observable classroom routines and organization that support these interactional practices? Our observations suggest that open and respectful pedagogical questioning is a key resource in eliciting students' current learning status, and for making decisions about next steps in student learning. Stable classroom routines and mutually understood interactional goals and practices are significant supports for these processes.     

Learning to teach: enhancing pre‐service teachers’ awareness of the complexity of teaching–learning processes

Abstract

Interest in story in teaching has been linked to teacher research (Carter, 1993; Elbaz, 1991), to teacher education (Connelly &; Clandinin, 1994), to curriculum (Britz‐man, 1989; Gudmundsdottir, 1991c), and to school change (Giltin, 1990). I wish to argue here for a link between story and one form of teacher reflection, for portfolio construction, unlike more conventional forms of teacher development, encourages teachers to tell the story of their classrooms and to frame that story in particular ways. I wish to argue here for a view that constructing a portfolio shifts the ownership of learning to the portfolio‐maker and that in this constructing, we can trace a teacher's developing understanding of pedagogy. Specifically, my aim is to illustrate the narrative dimensions of a self‐generated portfolio question — its interpretations, the reflections upon its meaning, and its transformations of pedagogical understanding — as this text becomes pedagogy and pedagogy becomes text. This interpretive process is illustrated through a case study of Ellen Nicol, a secondary English as a Second Language teacher, in her graduate teacher education year and her first 2 years of classroom teaching. Ellen's pedagogical text, her question, is reinterpreted with major changes each time she comes to understand more completely the richness and complexity of her classroom. Each new transformation and reinterpretation serve as guide for selection of materials, for selection of pedagogy, and for assessment of success. Each new collection of pedagogical information serves as impetus for possible reframing and transformation of the text.     

Children’s Preferences In Picture Story Book Variables

Abstract

Many assessment procedures used in the classroom fail to measure the student's comprehension of text materials. Using constructing- algorithms to develop the testing instruments, three quizzes and a posttest measuring both critical and incidental content were given to three fourth grade and three sixth grade classes. Three levels of student comprehension were tested: verbatim, paraphrased, and transformed paraphrased. Significant increases in performance over tests, and superior incidental recall by sixth graders indicated that as the learner's sophistication increases, deeper and more comprehensive instructional encoding can be measured by rule-based assessment systems.     

In-Service Improvement of Teaching

We examine the validity of peer observation of classroom instruction for purposes of faculty evaluation. Using both a multi-section course sample and a sample of different courses across a university's School of Business and Economics we find that the results of annual classroom observations of faculty teaching are significantly and positively correlated with student learning outcome assessment measures. This finding supports the validity of classroom observation as an assessment of teaching effectiveness. The research also indicates that student ratings of teaching effectiveness (SETEs) were less effective at measuring student learning than annual classroom observations by peers.     

Enhancing Student Performance Using Tablet Computers

Tablet PCs have the potential to change the dynamics of classroom interaction through wireless communication coupled with pen-based computing technology that is suited for analyzing and solving engineering problems. This study focuses on how tablet PCs and wireless technology can be used during classroom instruction to create an Interactive Learning Network (ILN) that is designed to enhance the instructor's ability to solicit active participation from all students during lectures, to conduct immediate and meaningful assessment of student learning, and to provide needed real-time feedback and assistance to maximize student learning. This interactive classroom environment is created using wireless tablet PCs and a software application, NetSupport School. Results from two separate controlled studies of the implementation of this model of teaching and learning in a sophomore-level Introductory Circuit Analysis course show a statistically significant positive impact on student performance. Additionally, results of student surveys show overwhelmingly positive student perception of the effects of this classroom environment on their learning experience. These results indicate that the interactive classroom environment developed using wireless tablet PCs has the potential to be a more effective teaching pedagogy in problem-solving intensive courses compared with traditional instructor-centered teaching environments.     

Teacher beliefs and teacher behaviour in portfolio assessment

This study concerns the relation between teachers’ beliefs towards teaching behaviour and their actual teaching behaviour in teacher portfolio assessment. We analysed the beliefs and behaviour of 18 teachers as described in their portfolios. In addition, each portfolio was independently assessed by two trained raters on eight content standards and the teachers’ classroom behaviour was assessed by their own students in a questionnaire (n=317). Linear multilevel analysis showed that part of the raters’ assessments of the teachers’ beliefs and their behaviour as described in their portfolios was significantly related to the students’ assessments of their teachers’ behaviour. Teachers with high raters’ assessments on the content standard about ‘choosing and arguing for teaching strategies that meet students’ knowledge, abilities and experience’ had significantly higher student assessments than teachers who were judged low on this standard. Implications of the results and suggestions for further research are discussed.     

An Analysis of Some New Statistical Methods for Selecting Test Items

In this study, the relationship between student affective performance and classroom physical environment, social climate, and management style were investigated in a sample of classes in Hong Kong primary schools. The results of Pearson and canonical correlation analyses indicated that among the measures of classroom environment, perceived quality of physical environment and class master's expert power, personal power, and coercive power were the strongest predictors of affective performance. This finding supports the importance of class master's management style in the classroom environment. Students' attitudes toward school and teachers appeared to be most sensitive to variation in the classroom environment, and self-concept was the least sensitive among the seven student affective measures. Students' self-efficacy of learning and intention to drop out were moderately sensitive to classroom environment. Profiles of effective and ineffective classroom environments were also mapped. In effective classrooms, class masters care for students, pay attention to teaching, do not use force or punishment but do create a good classroom climate with their professional knowledge, personal morality, and personality. Physical environment and psychological environment are both important; a good classroom environment is highly correlated with student affective performance.