A comparison of student feedback obtained from three methods of formative evaluation of instructional materials

The success of formative evaluation techniques to improve instructional materials is well documented. Researchers have found that materials revised on the basis of one-to-one sessions are equally effective as materials revised on the basis of small group sessions, and that both are superior to the unrevised, original versions. However, there is a paucity of research comparing the types of data obtained from these two recommended approaches to formative evaluation. This study compares the quantitative and qualitative nature of the problems identified by students in three conditions. Specifically, a one-to-one active, a small group active, and a small group passive were the conditions employed. Explanations for the observed differences are attributed to and described in terms of the instructional materials themselves and differences in the procedures or conditions employed. Finally, a number of more practical issues are raised and discussed in terms of their impact on the selection of a particular method or combination of methods for the formative evaluation of instructional materials.     

The role of written formative feedback in inducing non-formal learning among masters students

This article adds to the existing literature on nonformal learning in education in general and the role played by written feedback in providing the space for such learning in particular. The study was conducted in an English as a Foreign Language context in which the participating students received written feedback on their English writings. Employing qualitative interviews and stimulated recall methodology, we explored interstudent interactions, cross-cultural understanding, and initiation into communities of practice, and categorized these as reactive and deliberative types of nonformal learning. Drawing upon Eraut's typology of nonformal learning, we would argue that formative written feedback provided on students' writings could lead to nonformal learning because it encourages them to act upon it.     

Historical overview of formative evaluation of instructional media products

Though the term “formative evaluation” was coined only in 1967, formative evaluation activities, by different labels, have been going on for a long time. This article traces those activities—from study of World War I training films in the 1920s to detailed models and microcomputer technology currently employed.     

Informal formative assessment: The role of instructional dialogues in assessing students’ learning

This paper focuses on an unceremonious type of formative assessment - informal formative assessment - in which much of what teachers and students do in the classroom can be described as potential assessments that can provide evidence about the students’ level of understanding. More specifically, the paper focuses on assessment conversations, or dialogic interactions or exchanges, which continuously happen in the classroom and that are at the center of informal formative assessment. It is argued that assessment conversations make students’ thinking explicit in an unobtrusive manner, and when students’ thinking is explicit, it can be examined, questioned, and shaped as an active object of constructive learning. The paper conceptualizes informal formative assessment at the center of effective instructional activities with the use of instructional dialogues as assessment conversations, a typical informal formative assessment practice. The paper then presents a discussion about the evidence on the effect of assessment conversations on student learning.     

The use of observational data for formative evaluation of an instructional model

This paper describes a study investigating the characteristics of teacher behavior under two different instructional management models and their differential effects on students under an individualized early learning program. The study investigated the effectiveness and feasibility of using observational techniques in classroom settings to obtain information about student learning processes and outcomes necessary for implementing an adaptive early education model. Observational data, records of student progress in he curricula, and records of student learning activities were collected systematically in order to characterize the classroom behaviors of students and teachers.Learning Research and Development Center University of PittsburghThe research reported herein was supported in part by a grant from the Ford Foundation, and in part by a grant from the United States Office of Education to the Learning Research and Development Center. The opinions expressed do not necessarily reflect the position or policy of the sponsoring agencies and no official endorsement should be inferred.     

The priorities established among data sources when instructional designers revise written materials

The objective of the study was to investigate the priorities instructional designers establish among data sources when they revise written materials and the relationship of their practice to standard models of formative evaluation. Two modules of printed instructional material were revised by each of eight experienced instructional designers under three treatment conditions, using a counterbalanced design. Data were collected using a think-aloud procedure. The think-aloud protocol was segmented, coded and analysed. Results show that instructional designers were significantly more likely to incorporate their own knowledge into revisions than to use feedback data, and when they did use feedback data, they preferred learner comments. They did not accurately assess their use of their own knowledge or of feedback data; they thought they used feedback data more than the results showed. These instructional designers reflected some aspects of the standard instructional design model when they revised instructional materials, but their experience has caused many of the procedures to become internalized, so they are invisible in the protocols. This research was funded in part by a grant from the Social Sciences and Humanities Research Council of Canada.     

Some instructional strategies for improved learning from distance teaching materials

In distance teaching we are usually concerned with indirect methods of interacting with learners, using for the most part, the written word. Quality of learning in distance education ought therefore to be closely related to the quality of textual materials provided to students. In this paper we examine some ways of facilitating learning at a distance. We begin by noting the current dearth of models for teaching through text and make a plea for diversity and experimentation in this area. Some traditional instructional strategies (including advance organizers, overviews, pretests, objectives and inserted questions) from the dominant approach to distance teaching are examined along with devices in typoraphy and graphics. Guidelines for their use are developed, based on research, everyday rationality, and from an embryonic conceptualization of distance learning.     

Promoting learning by leveraging the collaborative nature of formative peer assessment with instructional scaffolds

Over the past two decades, formative peer assessment has become a popular instructional approach. Initially, it was more readily applied in higher education but has since expanded to other educational levels, including primary and secondary education. The popularity is understandable given the increased amount of feedback by multiple peers and enhanced awareness of performance criteria. Although it is increasingly acknowledged by the research community that formative peer assessment is inherently a social endeavour, the collaborative nature is simultaneously the least-explored mechanism. The contributions in this special issue address this gap conceptualising peer assessment and peer feedback as both an individual and a collaborative learning practice. Furthermore, we highlight core learning conditions: learner characteristics, domain and task characteristics, and, finally, instructional scaffolds.     

The influence of socio-cultural factors on leadership practices for instructional improvement in Indonesian schools

Empirical studies have shown that although leadership shares similar practices across East and West, some practices have inherently distinguished socio cultural characteristics. Understanding these characteristics is important in Asian contexts since socio-cultures are a major power in determining the success or failure of a change process. This study sought to explore and identify local leadership practices influenced by socio-cultural factors through in-depth interviews with principals and teachers from six different senior secondary schools in Malang Regency, Indonesia. Key local practices were identified: monitoring learning hours, improving the welfare of teachers, focusing on students’ character building, encouraging students’ participation in extracurricular programmes, increasing standards, and building partnership with stakeholders. Although some of the practices are similar to those recognised by empirical studies done in Western school context, others reflect local leadership practices influenced by school conditions, social practices, and economic factors. The findings imply that in Indonesian schools, practices of leadership are directly and indirectly dictated by the policies regulated by the government. This strong influence of government can be the impediment for school leaders to make necessary initiatives to promote instructional improvements in their schools.     

The influence of visual cognitive style when learning from instructional animations and static pictures

In a 2 × 2 design, we examined the role of visual cognitive style in two multimedia-based learning environments (text plus static pictures/animations). A statistically significant interaction was obtained for deeper comprehension: Highly developed visualizers (HDV) who learned with static pictures performed better than HDV who learned with animations, and less developed visualizers (LDV) performed the same with static pictures or animations.For factual knowledge, there was a main effect in favor of HDV. Subsequent tests revealed that HDV outperformed LDV only when learning from static pictures, but not when studying animations. There were no overall differences between animations and static pictures. The assumption is made that HDV benefit from their cognitive style when they have to construct a mental animation from static pictures.Concluding, we did not find any rationale for converting static pictures to animations — HDV learned better with static pictures, while for LDV, it made no difference.