Report to the President on the Use of Technology to Strengthen K–12 Education in the United States: Findings Related to Research and Evaluation

The Panel on Educational Technology was organized in April 1995 under the auspices of the President's Committee of Advisers on Science and Technology (PCAST) to provide advice to the President on matters related to the application of information technologies to K–12 education in the United States. Its findings and recommendations were set forth in March 1997 in the Report to the President on the Use of Technology to Strengthen K–12 Education in the United States. This report was based on a review of the research literature and on written submissions and oral briefings from a number of academic and industrial researchers, practicing educators, software developers, governmental agencies, and professional and industry organizations involved in various ways with the application of technology to education. Its most important finding is that a large-scale program of rigorous, systematic research on education in general and educational technology in particular will ultimately prove necessary to ensure both the efficacy and cost-effectiveness of technology use within our nation's K–12 schools. Finding that less than 0.1 percent of our nation's expenditures for elementary and secondary education are currently invested to determine which educational techniques actually work, and to find ways to improve them—an extremely low level relative to comparable ratios within the private sector—the Panel recommended that this figure be increased over a period of several years to at least 0.5 percent, and sustained at that level on an ongoing basis. Further, because no one state, municipality, or private firm could hope to capture more than a small fraction of the benefits associated with a significant advance in our understanding of how best to educate K–12 students, the Panel concluded that such funding will have to be provided largely at the federal level in order to avoid a systematic underinvestment (attributable to a classical form of economic externality) relative to the level that would be optimal for the nation as a whole. This paper originally appeared as Section 8 of the report.     

Key Competencies Required of Performance Improvement Professionals

Despite the growing literature about the criticality of a performance focus in our field, the literature on empirically formulated competencies for performance improvement professionals reflecting this shift has been scarce (Dean, 1999). While this study was being conducted (2000–2001), there were no formally recognized standards to guide performance improvement professionals. What the literature did offer was standards for instructional designers, specifically, the International Board of Standards for Training, Performance and Instruction (IBSTPI) released the third edition of Instructional Design Competencies: The Standards (Ritchey et al., 2000). Around this same time, The International Society for Performance Improvement (ISPI) was simultaneously, and independently, working on developing such standards. This study pursued similar efforts through two overarching purposes. The first purpose was to identify competencies required of competent performance improvement professionals. While previous studies had generally addressed what constitutes ideal practice, no data had been collected on current practice. Thus, the second purpose was to determine how often performance improvement practitioners believed they (1) should be, and (2) are currently applying each of the identified competencies. The relationships and gaps between these two indicators were also examined.     

Planning a responsive evaluation: Establishing solid partnerships by clarifying expectations and purpose,part 2

For evaluation to be worth the resources it consumes, it must enable decision makers to make sound decisions based on relevant, reliable, and valid data that lead to improved performance. It is from here that all evaluation efforts stem. All components of the evaluation must be aligned with the objectives and expectations that the organization and its stakeholders value and the decisions that will have to be made as a result of the evaluation findings. These decisions should be concerned with how to measurably improve performance at all levels of the organization. This article, the second of a two‐part series on evaluation (see Guerra‐López, 2007b, for part 1), describes how to create a responsive evaluation through the identification of stakeholders and expectations as a first and fundamental step.     

International student mobility in a German and Portuguese university: Which factors in the host institution matter?

In a globalising world, international mobility in higher education is an important phenomenon for students and higher education institutions. It is therefore essential to understand the factors that determine international students' satisfaction with higher education institutions that serve as hosts. Through research of an exploratory, quantitative nature, this study presents results from a survey among 289 incoming international students in two universities in Germany and Portugal. Our findings show that the factor we define as educational experience was an important determinant of student satisfaction. Furthermore, the appraisal of academic reputation is dependent on the university. We conclude that satisfaction with academic factors is more important for international students than satisfaction with non-academic aspects.     

Test-retest reliability and agreement of the SPI-Questionnaire to detect symptoms of digital ischemia in elite volleyball players

The Shoulder posterior circumflex humeral artery Pathology and digital Ischemia – questionnaire (SPI-Q) has been developed to enable periodic surveillance of elite volleyball players, who are at risk for digital ischemia. Prior to implementation, assessing reliability is mandatory. Therefore, the test-retest reliability and agreement of the SPI-Q were evaluated among the population at risk.

A questionnaire survey was performed with a 2-week interval among 65 elite male volleyball players assessing symptoms of cold, pale and blue digits in the dominant hand during or after practice or competition using a 4-point Likert scale (never, sometimes, often and always). Kappa (κ) and percentage of agreement (POA) were calculated for individual symptoms, and to distinguish symptomatic and asymptomatic players. For the individual symptoms, κ ranged from “poor” (0.25) to “good” (0.63), and POA ranged from “moderate” (78%) to “good” (97%). To classify symptomatic players, the SPI-Q showed “good” reliability (κ = 0.83; 95%CI 0.69–0.97) and “good” agreement (POA = 92%). The current study has proven the SPI-Q to be reliable for detecting elite male indoor volleyball players with symptoms of digital ischemia.     

Bureaucratization in Public Research Institutions

The purpose of this paper is to analyse the nature of bureaucratization within public research bodies and its relationship to scientific performance, focusing on an Italian case-study. The main finding is that the bureaucratization of the research sector has two dimensions: public research labs have academic bureaucratization since researchers spend an increasing part of their time in administrative matters (i.e., preparing grant applications, managing grants/projects, and so on); whereas universities mainly have administrative bureaucratization generated by the increase over time of administrative staff in comparison with researchers and faculty. In addition, I show that research units with higher bureaucratization have lower scientific performance.

Effects of multiple context and cumulative stress on urban children's adjustment in elementary school

Using longitudinal data collected over 2 years on a sample of 2,745 urban elementary school children (1st-6th graders, ages 6-11 years) from economically disadvantaged communities, effects of stressful experiences within 3 contexts (school, family, neighborhood), cumulative stress, and multiple context stress on 3 indices of children's adjustment (achievement, depression, and aggression) were examined. All 3 stressor contexts were related contemporaneously and longitudinally to negative outcomes across adjustment measures, with differential paths in each predictive model. Cumulative stress was linearly related to increases in adjustment problems but multiple context stress was not related to problematic adjustment beyond effects of cumulative stress alone. The important influence of life events stress on children's adjustment in disadvantaged communities is discussed.