Procedures for Evaluating the Impact of Complex Educational Interventions

The application of information and computer technology (ICT) to education, particularly when ICT is a component of a major school reform, raises difficult issues for program evaluation. In many cases even defining the program goal is difficult because so many different changes are often instituted at the same time: installation of a local area network, development of technical support, staff development for integrating ICT into the curriculum and for new teaching methods, and so on. To assist program developers and evaluators in gaining control of such complex interventions in education, two basic procedures are described: the history of the future and program logic mapping. The former is a goals and methods clarification scheme, built around a press release that describes at the projected end of the project what was achieved and how. The latter procedure draws on the goals defined in the history of the future, along with the project strategy, to make explicit all of the major links between project inputs and project outputs. This map then serves as a guide for planning an evaluation that would verify causal linkages as well as outcomes. For each type of node in such a diagram, appropriate evaluation techniques are defined.