Observer ratings of instructional quality: Do they fulfill what they promise?

Despite considerable interest in the topic of instructional quality in research as well as practice, little is known about the quality of its assessment. Using generalizability analysis as well as content analysis, the present study investigates how reliably and validly instructional quality is measured by observer ratings. Twelve trained raters judged 57 videotaped lesson sequences with regard to aspects of domain-independent instructional quality. Additionally, 3 of these sequences were judged by 390 untrained raters (i.e., student teachers and teachers). Depending on scale level and dimension, 16–44% of the variance in ratings could be attributed to instructional quality, whereas rater bias accounted for 12–40% of the variance. Although the trained raters referred more often to aspects considered essential for instructional quality, this was not reflected in the reliability of their ratings. The results indicate that observer ratings should be treated in a more differentiated manner in the future.     

Quality of instruction and achievement outcomes

This chapter reviews the fundamental assumptions, main results, and major problems of research on instructional quality and cognitive student outcomes. Although considerable progress has been made in this field of research, the present theoretical state, encompassing mostly simple process-product relations, is far from satisfactory. Findings from our own research demonstrate that it is both necessary and fruitful to expand the simple process-product paradigm by including cumulative, interactive, compensatory, and contextual effects.     

Value orientations and motivational interference in school-leisure conflict: The case of Vietnam

The experience of motivational interference during studying and leisure in a school-leisure conflict and its relationship to pupils' value orientations were investigated in a self-report study of Vietnamese pupils (N = 346, mean age = 15 years). The results were compared with a combined sample consisting of pupils from nine other countries (N = 2155). From a cross-cultural perspective, the mean values of the variables show that Vietnamese pupils differ with regard to their values and motivational interference from pupils in other countries. Within the Vietnamese sample, modern value orientation was positively related to decision for a studying activity in a school-leisure conflict and to experience of motivational interference during a leisure activity and negatively related to interference during studying. For post-modern value orientation, the relationships were mostly reversed. Despite differences between sample means, the consistency in the relationships between the variables across the samples indicates that the pattern of relationships can be generalized.