Formative evaluation of instructional development

Summary In this paper we have tried to describe the research pertinent to the activity of formative evaluation. While the number of good examples of formative evaluation is expanding, the level of research into the process is relatively limited. Perhaps a compromise to the difficult task of accumulating research data on formative evaluation might be suggested. When formative evaluation activities have been successful in terms of program effects and staff satisfaction, then detailed technical reports might be made available to the evaluation public. Formative evaluation might improve as a consequence of the technology developed in the course of finding solutions to development data problems. Both the increase in technology as a result of dissemination of successful procedures and the pursuit of experimental research on the process of evaluation might ultimately remove formative evaluation recommendations from the realm of seers and clairvoyants and thrust it closer to, if not into, the domain of scientific application. This article is the second ERIC/AVCR Annual Review Paper, preparation of which was supported by the ERIC (Educational Resources Information Center) Clearinghouse on Media and Technology, Stanford University, Stanford, California. The authors acknowledge with appreciation the assistance of Laura Spooner and Bea Kass.     

Formative evaluation applied to a learning hierarchy

A formative evaluation method applicable to learning hierarchies is described which simplifies data collection by employing results from a posttest. Relations among pairs of lower and higher elements of a portion of a previously formulated hierarchy for graphic skills in kinematics were tested by means of data obtained from a posttest given to 148 secondary-school students. Results were compared with those previously obtained for the same hierarchy by a rigorous validation method. Correspondence of the two sets of results was high. Examination of test responses for discrepant instances yielded further information of value. Results additionally indicate hierarchical relationships to be the same in retention as they were for learning.     

Formative and summative evaluation in the faculty peer review of teaching

If the process of the faculty peer review of teaching is to overcome institutional marginalization, then its formative and summative components must employ rules, criteria, and standards for the identification of effective teaching that have been agreed to within a peer conversation among the faculty members of a scholarly unit. This conversation serves to collectively clarify the unit's expectations for its curriculum, teaching, and student learning. Only such a process can produce the credibility necessary to regularly effect the faculty development and personnel decisions of a unit.Dr. Ronald R. Cavanagh holds a doctorate from the Graduate Theological Union in Berkeley, California, a Master of Divinity from Moravian College and the Moravian Theological Seminary in Bethlehem, Pennsylvania, and a B.A. in English. Since coming to Syracuse University in 1967, Dr. Cavanagh has served as a faculty member in and chairperson of the Department of Religion, Associate Dean, and Dean of the College of Arts and Sciences, and he is presently the University's Vice President for Undergraduate Studies. In 1995, he was a participant in the Institute for Educational Management at Harvard Univesity, Boston, Massachusetts.     

Design of an interactive informational program: Formative evaluation and experimental research

The design and evaluation of an interactive informational program is outlined in this article. Also reported is an integrated research study which examines the effects of differing levels of orientation to an interactive informational program on user confidence and attitudes. Research results indicate that a full orientation providing information, examples, practice, and feedback for various program options can lead to greater levels of user comfort than a more limited type of orientation providing brief information and single examples, and with equally positive user attitudes. However, subjects receiving the full orientation spent less time in the actual program and accessed a smaller proportion of available program segments than did their limited-orientation counterparts. No differences in efficiency were noted between the two groups in the use of the program. Sex differences noted in attitudes and time expended are discussed, with reference to a possible preference by females for greater levels of program support. Recommendations are offered for interactive program developers and future research directions are suggested. This work was supported in part by Summer Research Awards provided by the Curry School of Education at the University of Virginia. I gratefully acknowledge the assistance of Richard Berdel, Cheryl Wissick, and Maureen Glasser in the data collection, and of theETR&D editor and anonymous reviewers for their comments on a draft of this article.     

Formative evaluation of a collaborative community-based child abuse prevention project

OBJECTIVE: Together for Kids is a child abuse prevention project that serves children and families in two neighboring communities in a mid-sized Canadian city. The project, a collaborative endeavor of various agencies in the health, social services, and law enforcement sectors, focuses on preventing child abuse and neglect through family support and programming. This article presents the results of a formative evaluation of the project focusing on client and team member views on project implementation. METHOD: The evaluation strategy was primarily qualitative. In-person interviews following a semi-structured format were conducted with 17 clients and 10 team members by an external evaluator. In addition a review of all client records was conducted. RESULTS: The community-based approach, the multidisciplinary composition of the team, the ability to seek services when needed, the immediacy of the response time and the availability of support during stressful times were all aspects of the project that clients found beneficial. The most beneficial aspect of the project, however, was the informal support received from team members who were accepting, non-threatening, and non-judgmental. Team members found the collaborative approach made access to services easier for clients, particularly for those who were more socially isolated. CONCLUSIONS: Multidisciplinary, community-based models of service delivery contribute to a more effective and compassionate response to vulnerable families. Attention to the variables identified as important aspects of the project from the clients' perspective in this evaluation may assist others in developing similar programs.