The present status and problems of impact research in technology policy: A case study on the federal program for funding research and development personnel in Germany

The paper illustrates the present status and the problems of impact research through an impact research study of the program of funding research and development personnel in small and meduim-sized firms launched by the German Federal Government. This impact analysis study is one of the first of its kind to be conducted in technology policy in the Federal Republic of Germany. It is mainly devoted to the following problems: acceptance by the firms involved of the funding program; relevance of the program to internal decision-making processes; quantitative and qualitative effects on R&D personnel and R&D activities; impact on the innovation activities of firms; efficient implementation and management of the program. Impact analyses need to go beyond the framework of a shallow technocratic incidence analysis studying only effects the funding program is assumed to have. Such solidly based impact analysis is possible only if also the basic aspects underlying the funding programs can be reviewed critically and verified empirically. The practical implementation of this criterion is demonstrated by studying the effects of the funding measure on the innovative activities of firms. Typical problems associated with impact anayses are insufficient access to the data required, shortcomings in the evolution of theories and the development of methods, weaknesses inherent in the formation of indicators on a sound theoretical basis and with a sufficient degree of operationalization. It is shown how far these problems make themselves felt, and are partly overcome, in this impact analysis. In particular, the early inclusion of evaluation aspects in the conceptual phase of the funding program has established comparatively favorable conditions for this impact analysis.     

Using a Table of Specifications to improve teacher-constructed traditional tests: an experimental design

We investigated if instruction on a Table of Specifications (TOS) would influence the quality of classroom test construction. Results should prove informative for educational researchers, teacher educators, and practising teachers interested in evidenced-based strategies that may improve assessment-related practices. Fifty-three college undergraduates were randomly assigned to an experimental (exposed to the TOS strategy) and a comparison condition (no specific strategy support) and given materials for an instructional unit to use to construct a classroom test. Results of a multivariate analysis of covariance suggested that students exposed to the TOS strategy constructed a test with higher test content evidence but not response process evidence scores. Furthermore, we found that treatment participants were able to accurately complete the TOS tool and choose items that reflected the subject matter specified in the TOS tool. However, they experienced difficulty selecting items at the cognitive level specified in the TOS tool.