Designing Intervention Studies: Selected Populations,Range Restrictions,and Statistical Power

ABSTRACT

An appropriate estimate of statistical power is critical for the design of intervention studies. Although the inclusion of a pretest covariate in the test of the primary outcome can increase statistical power, samples selected on the basis of pretest performance may demonstrate range restriction on the selection measure and other correlated measures. This can result in attenuated pretest–posttest correlations, reducing the variance explained by the pretest covariate. We investigated the implications of two potential range restriction scenarios: direct truncation on a selection measure and indirect range restriction on correlated measures. Empirical and simulated data indicated that direct range restriction on the pretest covariate greatly reduced statistical power and necessitated sample size increases of 82%–155% (dependent on selection criteria) to achieve equivalent statistical power to parameters with unrestricted samples. However, measures demonstrating indirect range restriction required much smaller sample size increases (32%–71%) under equivalent scenarios. Additional analyses manipulated the correlations between measures and pretest–posttest correlations to guide planning experiments. Results highlight the need to differentiate between selection measures and potential covariates and to investigate range restriction as a factor impacting statistical power.     

试析教育评估的专家遴选机制

科学有效的专家遴选机制是评估项目有效实施的重要保障。文章主要从专家遴选原则、遴选标准、遴选组织与实施,遴选元评估四个维度,构建了适用于教育评估项目的专家遴选机制。基于合目的性、专业性和规范性原则,重点围绕学术素养、评估素养、道德素养和心理素养四个维度开展遴选。讨论了专家队伍的规模与结构、遴选信息选取与判断、轮换与回避、遴选元评估等专家遴选实施中需要重点考虑的问题。结合高校教师学术能力评议评估实践,开展案例分析,在理论研究和案例分析的基础上,进一步指出:应着重形成贯穿评估全过程的专家遴选闭环系统、统一和灵活兼顾的专家遴选标准、多元融合的专家遴选的技术与方法,并重点把握精准遴选、专家库建设等关键问题。     

Cream skimming and elitism in state/municipality-regulated enrolment: A case study of two elementary schools in Ljubljana, Slovenia

Cream skimming and elitism in public elementary education is closely related to the question of selection criteria used by the schools in their selection procedure. Although equity in access seems to be important part of any education policy, individual school's practices are often selective. In the Slovenian context, state and municipalities seemingly regulate admission and selection criteria by catchment areas, yet the case study of two Ljubljana schools reveals different practice for undersubscribed schools. The paper, therefore, describes, discusses and analyses how schools in the state regulated system and defined catchment areas choose students and, thus, operate on a quasi-market.     

Academic Criteria,Experiential Background,and Personal Interviews as Predictors of Success in a Counselor Education Program

A 4-year evaluation was conducted of a counselor education trainee selection process that places approximately equal weight on three selection criteria: academic index, experiential background, and personal interviews. Mean faculty rankings were obtained for 77 trainees on three separate evaluative areas: (a) academic success in the program, (b) overall counseling competency in the program, and (c) anticipated success in the field. Results of Pearson product-moment correlational analyses indicated that two selection criteria (Miller's Analogies Test and the Verbal portion of the Graduate Records Examination) were significantly related to faculty-ranked academic success. No single criterion, however, was significantly related with faculty rankings of either counseling competency or anticipated success in the field. The correlation between the Miller's Analogy Test and a measure of “overall success in counselor education” was in the predicted direction but failed to reach the traditional levels of statistical significance (p < .09). Implications for future research and limitations of the study are discussed.     

人物品藻与佛学对魏晋南北朝批评文体建构的影响

魏晋南北朝批评文体美轮美奂、诗意盎然,与批评家杰出的创造才华及批评策略分不开。他们受人物品藻风尚和东传佛学的影响,采用"转化"和"吸纳"策略引进品第法,并在著述体例、思维风尚、批评准则等方面,对佛学予以吸纳,此阶段批评文体的话语、结构等均受其影响。     

The Mental Model Comparison of Expert and Novice Performance Improvement Practitioners

This study investigates the mental model comparison between expert and novice performance improvement practitioners. Nonexperimental survey research design was applied. Based on specific criteria, 11 critical concepts out of 30 were selected by 23 experts. Sixteen of those 23 experts provided data about the relatedness of 55 concept‐pairs generated from the 11 concepts to develop the common mental model of experts in the Knowledge Network Organizing Tool utilizing Pathfinder scaling. Two‐hundred‐forty‐two practitioners participated in the further phases of the study. Out of those practitioners, 33 novices, who have not met any of the criteria used for expert selection, are selected. Those novices also provide similar data for the common mental model of novices. The results of the study show that the expert model has more consistent and hierarchical structure than the novice model. Discussions and further research recommendations are provided.     

The Relational Power of Education: The Immeasurability of Knowledge, Value, and Meaning

Recognizing the challenge of adequate evaluation in higher education, this essay introduces some of the critical, alternative-seeking conversation about educational measurement. The thesis is that knowledge, value, and meaning emerge in the relational dynamics of education, thus requiring complex approaches to evaluation, utilizing relational criteria. The method of the essay is to analyse two educational case studies – a travel seminar and a classroom course – in dialogue with educational literature and a process-relational philosophy of education. Building from this analysis, the essay concludes with proposals for relational criteria of evaluation: relations with self, community and culture, difference, earth, and social structures.     

Applying total quality management principles to faculty selection

This paper outlines a systematic methodology for effecting Total Quality Management in higher education decision making with particular attention to the faculty selection process. Following a brief analysis of the application of TQM to higher education in general and consideration of the contextual and power relations issues associated with implementation of TQM, a model for integrating Total Quality Management principles into the faculty recruitment, identification, and selection process is outlined. This model relates the role of vision, mission, planning and organizational design with quality concepts (understanding a system and systems theory, leadership, empowerment, client/constituet satisfaction, team building, and continuous process improvement). Drawing upon Marchese and Lawrence's (1987) work wherein they identified eight critical processes in the selection of faculty, one critical process—Defining the job is specifically analyzed using the continuous improvement methodology as an example of the specific application of TQM techniques to higher education decision making. The model and this example provide educational leaders/search committees with an enhanced framework within which they can effectively analyze and evaluate faculty selection processes, criteria for selection, and relevant contextual issues that will result in the identification and selection of the best qualified faculty available.     

Theories for teacher selection: Objective, subjective, and critical contact

Decisions made by applicants for teaching positions have generally been ignored by most teacher selection research. The purpose of this study is to obtain additional information about the teacher as a decision-maker. Contents of recruitment interviews were varied via video tape to reflect the salient factors associated with the objective, subjective, and critical contact theories. The reactions of teachers to these manipulations were assessed in a laboratory study. Teachers were more positive when the recruitment interview stressed nonpecuniary aspects of the work environment (subjective theory) than when the recruitment interview stressed either economic incentives offered by a local school district (objective theory) or specific information about the work itself (critical contact theory).     

新西兰职前教师选拔标准及其启示——基于ITE计划的分析视角

新西兰职前教师教育计划对其教师教育起着重要的引领作用。针对职前教师的选拔,它从学术入学准则、学科知识、读写及算术能力、个人和专业素质、信仰与态度等六个方面规定了统一且灵活的选拔标准。这六个方面的选拔标准为我国当前职前教师的选拔与培养提供了有益的参考。     

大学校长的遴选:实践与思考——基于美国与中国比较研究的视角

美国在长期的高等教育发展历程中积累了较为成熟的大学校长遴选机制,遴选标准客观务实,随着时代的变化和不同大学的发展特色提出不同的标准。遴选程序完全由学校董事会负责运作和管理,体现了较强的公开性与竞争性。中国大学要走向世界一流大学,必须借鉴世界各国,特别是美国大学校长的遴选机制,在角色定位理念的重构、遴选指标体系的构成以及遴选程序的设置等方面进行必要的改革,才能构建起有中国特色的大学校长遴选机制,以进一步推动我国高等教育又好又快地发展。     

Selection and Elimination of Candidates in Counselor Education Programs

This study compared actual with ideal procedures in the selection and elimination of candidates in counselor education programs. A two-factor analysis for each category (selection and elimination) resulted in some significant differences (p< .001) between the actual and ideal procedures in both selection and elimination. The criteria for actual and ideal procedures in the selection and elimination of candidates were also treated by t tests, with highly significant differences (p<.075 to p<.0005) found on some of the criteria.     

The Selection of Vice-Chancellors: The Australian Experience

This paper examines appointment procedures and selection criteria for the position of vice-chancellor in eight of Australia's nineteen universities which advertised a vacancy during 1984 and 1985. Among issues analysed are: advertisement formalities; information provisions; and selection processes and criteria including initial action taken, composition of selection committees, identification of selection criteria and selection processes employed. Ways of imprroving subsequent appointment procedures and of ensuring legitimacy for appointees are discussed.     

示范性电大分校建设及评选标准的构建

电大分校建设是整个电大系统建设的重要一环,以示范性电大分校的建设和评选带动分校建设具有重大意义。本文阐明了示范性电大分校建设应当遵循的基本原则、指标体系,并阐释了示范性电大分校建设和评选指标体系的构成要素。     

Critical Thinking, Epistemic Virtue, and the Significance of Inclusion: Reflections on Harvey Siegel's Theory of Rationality

Among proponents of critical thinking, Harvey Siegel stands out in his attempt to address fundamental epistemological issues. Siegel argues that discursive inclusion of diverse groups should not be confused with rational justification of the outcome of inquiry. He maintains that epistemic virtues such as inclusion are neither necessary nor sufficient for rational judgment, and that if we are to avoid falling prey to relativism, criteria are needed to distinguish which of these virtues are indeed rational. However, the author argues that at least some of Siegel's own rational criteria cannot pass the "necessary or sufficient" standard by which he measures epistemic virtues. Moreover, reliance upon criteria fails to settle conflict in cases of disagreement over what constitutes authoritative evidence. Jürgen Habermas's theory of communicative rationality can help us to overcome this impasse, because it provides a nonrelativistic basis for justifying inclusion and giving it a place of priority in practical reasoning.     

高中学业水平考试:功能、命题与成绩使用

高中学业水平考试是新课程改革背景下出现的新型教育考试。它考查中学生的知识掌握水平、学科素养、学习能力,应该成为高校招生的重要评价指标之一,并能为社会用人单位提供有用的信息。高中学业水平考试的命题组卷,应关注的首要指标是试题有无较全面地覆盖课程标准要求的内容,以及试题的难度范围是否足够广泛。作为高考招生的指标之一,高中学业水平考试成绩合格或达到一定等级应该成为一个关键的录用前提条件。至于考生取得哪个等级的学业水平考试成绩可以被录用,则最终取决于招生院校和专业的要求。