Striking a balance: A mixed-methods approach to impact-oriented evaluation

It is commonsensical to assume that a program needs to be implemented in order to have an impact. However, most impact evaluation studies tend to pay little attention to the issue of program implementation. This paper argues for the logical priority of program implementation through a mixed-methods evaluation of a language intervention program initiated in one of the largest urban school districts in the United States. By selecting a mixed-methods evaluation approach, the evaluation yielded results that have implications for addressing implementation issues in an impact-oriented evaluation. At the local district level, the evaluation results of what works, what does not, and how the implementation varies across teachers and classrooms have a direct implication for program staff. They can use this information to monitor the program implementation and make necessary adjustments to enhance the effectiveness of the program so that the ultimate program goal can be accomplished.