Assessing the interpersonal competence of beginning teachers: The quality of the judgement process

A procedure to assess the interpersonal competence of teachers was developed and validated. The procedure includes the use of the Questionnaire on Teacher Interaction (QTI), an observation instrument and a self-evaluation instrument. The concepts of construct-irrelevant variance and construct under-representation were used to explore the validity of the assessment procedure with regard to the noting of evidence and combining of evidence to attain an overall judgement by four separate assessors for the same beginning teacher. The validity of the assessment procedure was found to be satisfactory and the use of multiple assessment instruments to clearly have added value.     

Teacher perceptions and individual differences: How they influence rural teachers’ motivating strategies

This study examined the influence of high school teachers’ perceptions and individual difference characteristics on teachers’ use of motivating strategies in their classrooms. Participants were 75 teachers in 19 rural, public high schools. A mixed method approach was used. Quantitative measures included demographics, individual differences, perceptions, and motivating strategies, analyzed as correlations and regressions. Qualitative measures included generative self-report of motivating a student, and semi-structured interviews, which were open coded and then axial coded to identify themes and issues. Peer-related environment stood out among teacher perceptions predicting student motivation. Teacher support and efficacy predicted motivating strategies, but teacher perceptions of student goals and causes of lack of motivation did not. Teachers admit that they lack knowledge and efficacy for motivating students. Findings can inform teacher education, professional development, and administrative support for teaching effectiveness.     

Synergistic scaffolds as a means to support preservice teacher learning

This study investigates the role of synergistic scaffolds in supporting preservice teachers’ knowledge of self-as-teacher. Data include preservice teacher papers written before and after the introduction of scaffolds, surveys, and interviews. Quantitative and qualitative methods were used to analyze differences in the quality of the papers, the relationship between the scaffolds and the preservice teachers’ revisions, and the preservice teachers’ perceptions of which scaffolds best supported their learning. We conclude that carefully designed synergistic scaffolds can support preservice teachers in their exploration of self-as-teacher. Implications of our findings for teacher education and suggested areas for further research and development are discussed.     

Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios

Student portfolios are increasingly used for assessing student competences in higher education, but results about the construct validity of portfolio assessment are mixed. A prerequisite for construct validity is that the portfolio assessment is based on relevant portfolio content. Assessment criteria, are often used to enhance this condition. This study aims to identify whether assessment criteria can improve content, argumentation and communication during teacher moderation while judging student portfolios. Six teachers scored 32 student portfolios in dyads with and without assessment criteria. Their judgement processes were qualitatively analysed. Results indicated that the quality of their judgement processes was low, since teachers based their judgements mainly on their own personal opinion and less on evidence found in the portfolio. Teachers barely paid attention to quality checks and easily agreed with each other. When teachers used assessment criteria, the quality of their judgements slightly improved. They based their judgements more on relevant evidence, used less personal experiences and more often checked the quality of their judgement processes. It is concluded that the quality of teacher portfolio judgement is low, and that the use of assessment criteria can enhance its quality.     

Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning?

While much of L2 teacher feedback research has focused on the effectiveness of feedback and its impact on student revision and writing, little has been done to examine teachers’ feedback in the larger classroom context of teaching and learning to ascertain the functions teacher feedback serves from an assessment-for-learning perspective. Using multiple sources of data from 26 secondary teachers’ written feedback to 174 student texts, interviews with six of the teachers and 18 students, the present study investigates the nature of teacher feedback and the functions it serves in the teaching-learning-assessment process in the writing classroom. The findings show that teacher feedback focuses largely on assessing writing summatively, primarily serving the purpose of assessment of learning, rather than assessment for learning – i.e., using feedback as a pedagogical tool for improving the teaching and learning of writing. The study calls for greater attention to the implementation of assessment for learning in the writing classroom, and specifically the use of feedback for formative purposes.     

The longitudinal effects of teacher judgement and different teacher treatment on students’ academic outcomes

The self-fulfilling prophecy model of Brophy and Good was applied to the area of teacher judgement in order to disclose the processes of how teacher judgement of student achievement influences students’ future academic outcomes. It was assumed that achievement and achievement motivation might be affected through the mediating processes of student-perceived positive and negative teacher treatment, which represents students’ perceptions of teachers’ actions towards them. A sample of 294 Chinese fifth-grade students and their eleven English language teachers were tracked over one school year. Structural equation modelling showed that after controlling for prior achievement and achievement motivation, teacher judgement was directly related to students’ future achievement, expectancy for success and level of aspiration. These relations were partially mediated by perceived negative teacher treatment. Teacher judgement was related to students’ academic self-concept and pride in English learning, and fully mediated by both perceived positive and negative teacher treatment. The study supported the model of Brophy and Good and identified the specific processes of how teacher judgements can become self-fulfilled. Positive and negative teacher treatment had different mediating effects. Compared with positive teacher treatment, perceived negative treatment was related to student academic outcomes more extensively.     

Navigating the complex cognitive task of classroom assessment

We investigated how a fifth-grade literacy teacher with classroom assessment expertise integrated assessment into his teaching practice. We employed a qualitative case study methodology to examine and document the processes this teacher used to make instructional decisions informed by classroom assessment events. We identified a repertoire of assessment relevant management routines and interpretation strategies that supported his assessment practice and instructional decision-making. Findings can be used to inform policy and organize content for teacher education by identifying a repertoire of potential management routines and interpretation strategies for engaging in classroom assessment and documenting how to enact these practices.     

Rethinking teachers’ goal orientations: Conceptual and methodological enhancements

The article provides a theoretical extension of the goal orientation approach for teaching by proposing three different competence facets of learning goals and four types of addressees for performance approach and avoidance goals. On the basis of responses from 495 teacher trainees and 224 in-service teachers, the development and validation of an advanced goal orientation measure is presented. Internal consistencies of the goal dimensions were good, and confirmatory factor analyses verified the postulated model for teacher trainees and in-service teachers. In both groups, learning goal orientation positively predicted self-efficacy for teaching and perceived benefits of help-seeking. Performance approach goals positively predicted self-efficacy for teaching and performance avoidance goals negatively predicted self-efficacy for teaching and positively predicted perceived threats of help-seeking. The use of goal orientation facets provided a more thorough analysis of the predictions regarding perceptions of help-seeking. Results supported the convergent and divergent validity of the new measure.     

Portfolio appraisal: In search of criteria

Two inherently contradictory forces are pushing for reform in portfolio assessment. On the one hand there is a felt need for creating more rigid standards that operate to promote uniformity of ratings in appraisal practice to certify achievement. However, on the other hand, critical questions are being raised about separating acclaimed portfolio goals aimed both at appraising achievement while also improving quality of student learning and development. The position of portfolio assessment, which is widely used nowadays in teaching and teacher education, comes into debate. In our study, we look for actual practices in portfolio appraisal in search of criteria used for rating the quality of portfolio materials. It is our interest to find out how appraisal criteria are selected and used to evaluate achievement or to improve the quality of development and learning. In the context of teacher education, we have examined both the espoused criteria of both assessors and collectors of portfolios as well as the actual appraisal practices by looking at the judgmental orientations and supervision styles used by portfolio assessors. In addition, we offered an authentic portfolio document to be rated by different assessors to gauge and compare their quality of rating and criteria use. The actual processes we detected point to a most common practice of employing judgmental, usually normative evaluations based on assessor dependent, more or less pre-decided criteria which permit a “checkbox” approach to appraisal.     

Teachers' perceptions and attitudes about Response to Intervention (RTI) in their schools: A qualitative analysis

As Response to Intervention (RTI) models are increasingly implemented in United States' schools, it is important to consider the perceptions of those directly involved. The current study assessed teachers' perceptions of RTI and utilized qualitative methodology and a computer-based text search program to explore teachers' perceptions of RTI. Constant comparison analysis yielded four emergent themes related to teacher perception of RTI: (a) overall understanding of RTI, (b) teachers' perceptions of barriers to RTI in their schools, (c) teachers' suggestions to improve RTI, and (d) teachers' suggestions for making paperwork more efficient. We conclude with implications for training and RTI implementation.     

Pre-service and beginning teachers’ professional identity and its relation to dropping out of the profession

This study explores different perceptions of pre-service and beginning teachers’ professional identity in relation to their decisions to leave the profession. Teachers’ professional identity was further broken down into six factors: value, efficacy, commitment, emotions, knowledge and beliefs, and micropolitics. This study employed mixed-methods which included 84 participant surveys, and 27 interviews from four groups of participants at different stages of teaching. The findings of this study showed that pre-service teachers tended to have naïve and idealistic perceptions of teaching, and dropout teachers showed most emotional burnout. Implications for improvement of teacher education and retention of beginning teachers are discussed.     

The Perceptions of Success Inventory for Beginning Teachers: Measuring its psychometric properties

This study investigated the construction and evaluation of an instrument called the Perceptions of Success Inventory for Beginning Teachers (PSI-BT) intended to measure factors documented in research that contribute to beginning teachers’ perceptions of success. The PSI-BT was found to assess the following factors using exploratory factor analysis: (1) Administrative Support, (2) Classroom Climate, (3) Mentor Support, (4) Colleague and Instructional Resource Support, (5) Commitment, and (6) Assignment and Workload. Internal reliability, content validity, and concurrent validity were also measured in the validation process. Our findings suggest that the PSI-BT provides a reliable and valid instrument that can provide schools with valuable feedback to ensure the success of their beginning teachers.     

Evaluating instructional quality

ABSTRACT

Building on the papers in this special issue, this article uses modern conceptions of validity theory to provide a framework for considering the evaluation of teaching quality. The 3 facets of teaching quality focused on are domain conceptualization, evidence and inferences, and their evaluation. Domain definitions vary in their specificity with tradeoffs in their range of applicability and specificity of inference. Evidence collection can range from highly standardized assessments to observations that must attend to evidence from a myriad of classroom interactions. For all assessments, however, even the most standardized, different interpretations of assessment tasks can threaten the validity of score interpretations. The papers consider a range of processes that are designed to generate, support, and interrogate the validity of inferences based on assessment scores. A fundamental question underlying this type of measurement is whether differences in the quality of teaching that students experience can be causally attributed to the teacher.     

Electronic Portfolios in Teacher Education: A Case Study of Early Childhood Teacher Candidates

The use of electronic portfolios is increasingly becoming popular in teacher education programs. The electronic portfolio allows learners to express their work and take ownership over their own ability to plan and assess and reflect upon their learning during a specific period of time. While many teacher education programs have adopted electronic portfolios in order to meet accreditation requirements, others use electronic portfolios to develop and improve on students’ technology skills—train students on how to better use technology in instruction and student assessment. Irrespective of the reasons for adoption, there is the need to assess students’ perceptions of electronic portfolios, and especially to explore the benefits and challenges students face in using electronic portfolios in their course work. Using a qualitative research design, the current study explored teacher candidates’ attitudes toward electronic portfolios and the differences that electronic portfolios bring to their profession. Such a study is necessary to help administrators identify students’ needs to ensure that they are gaining the maximum benefits from the use of electronic portfolios in their programs of study.     

Predictive validity evidence for an assessment program based on the Work Sampling System in mathematics and language and literacy

This study examined the predictive nature of teacher ratings of student performance on the mathematical thinking and language and literacy domains of a state mandated curriculum-embedded performance assessment for children in first grade to student achievement on a criterion-referenced test in third grade in mathematics and English language arts, using a multilevel modeling approach. The sample included 1,281 elementary school children nested within 132 classrooms in an urban school district in South Carolina. Results showed positive associations between first grade teacher ratings and students’ third grade achievement. However, first grade teacher ratings were not found to uniquely contribute to the proportion of explained variance in third grade performance on a high-stakes statewide assessment, over and above student demographic variables.     

Do teachers’ wages matter for proficiency? Evidence from a funding reform in Brazil

This paper uses the 1998 reform in the funding of fundamental education in Brazil (FUNDEF) to identify the effect of teachers’ wages on the proficiency of public school pupils. Wages are set by legislation at the local level and this reform established a floor on the share of expenditures on teacher wages out of total revenues in each Brazilian State and Municipality. The evidence using micro-data suggests that increases in relative wages within the public system brought about by the reform had a positive impact on the students’ proficiency.     

A comparative judgement approach to teacher assessment

We report one teacher’s response to a top-down shift from external examinations to internal teacher assessment for summative purposes in the Republic of Ireland. The teacher adopted a comparative judgement approach to the assessment of secondary students’ understanding of a chemistry experiment. The aims of the research were to investigate whether comparative judgement can produce assessment outcomes that are valid and reliable without producing undue workload for the teachers involved. Comparative judgement outcomes correlated as expected both with test marks and with existing student achievement data, supporting the validity of the approach. Further analysis suggested that teacher judgement privileged scientific understanding, whereas marking privileged factual recall. The estimated reliability of the outcome was acceptably high, but comparative judgement was notably more time-consuming than marking. We consider how validity and efficiency might be improved and the contributions that comparative judgement might offer to summative assessment, moderation of teacher assessment and peer assessment.     

Teachers learn about student learning assessment through a teacher education process

This study aims to understand the extent to which university professors adopt new pedagogical voices in their learning assessment practices through a teacher education process. Participants (N = 32) were interviewed before and after the teacher education process, and data were analysed using qualitative and quantitative methods. The results of the study demonstrated, first, that teachers renamed their educational discourse about learning assessment significantly, increasing it in assessment for learning practices, particularly in the themes of timing and agents, and reducing it in all themes referred to the assessment of learning practices. And second, three clusters of faculty were identified, which differed in terms of the way they merge both learning assessment practices: professors with a slight prevalence of the assessment for learning conceptual voice, professors with a slight prevalence of the assessment for learning practical voice, and professors with a strong prevalence of the assessment for learning voice.     

Working as partners for classroom reform

This paper discusses a partnership involving teachers and a teacher educator working together to develop teaching and learning practices in Pakistan. The partnership was established in response to the teachers’ need for support in developing their professional practices within the actual contexts and realities of their schools, after they had attended an in-service training course at a university. Providing case studies, the paper analyses the processes and outcomes of this co-learning partnership. The findings suggest that the collaborative process advances the understanding of teachers’ problems, resolves them on the spot, and provides teachers with the confidence and courage to transcend the constraints of their workplaces while developing professionally.     

Assessment of teacher competence using video portfolios: Reliability, construct validity, and consequential validity

The richness and complexity of video portfolios endanger both the reliability and validity of the assessment of teacher competencies. In a post-graduate teacher education program, the assessment of video portfolios was evaluated for its reliability, construct validity, and consequential validity. Although video portfolio facilitated a reliable and valid assessment of teacher competencies, procedures to improve assessment quality were also revealed and are therefore discussed: more explicit grounding of assessment results in the data, peer debriefing, prolonged engagement with the assessment data, cross-checking to find confirmatory or counter examples.