谈"同行评议"在科技奖励工作中的利与弊

本文通过对"同行评议"在科技奖励工作中的利与弊的分析,指出"弊"对奖励科技创新的负面影响,并针对不足,作者提出了可供参考的改进意见. "同行评议"在科技奖励工作中,尤其是对在基础研究和应用基础研究领域内取得的研究成果的推荐项目普遍采用的一种评审方法.基本做法是,管理人员根据推荐项目所推荐的学科领域内的三级学科随机遴选与该项目研究方向基本一致(不是泛泛的学科或专业相同)的5～7名专家送出通信评议,并按规定时间收回通信评审专家的评审意见(有"定性"与"定量"两种情况),以此作为评价该推荐项目的基础(称之为"第一级评审").     

国家自然科学基金项目评审专家管理办法改进研究

同行评议制度是国家自然科学基金评审体系的核心,与其他学术评价相比,其公正性受到科学界的广泛认同。但也有一些不足之处,特别是在支持非共识项目方面,由于其评审体系过于依赖评审专家,而对评审专家需求的结构、规模设计和规划以及对专家的监督与评估等方面的不足,有可能影响评审结果的公正性和科学性。因此,亟待改进评审专家管理办法,以保证评审的公开、公正、公平,提高评审专家的评审效率。     

我国高校教师评聘中同行评议机制的运行实践及权力关系探析

在“破五唯”的学术评价制度改革背景下,探索优化同行评议机制具有重要的现实意义。通过对 我国高校教师评聘中同行评议制度的运行实践和权力关系进行研究发现,评审专家所在机构的排名、学术 水平和声誉、是否有合作基础等因素是各院系选择专家的重要考量标准。受文化差异、语言障碍、工作量过 大等因素限制,国际同行评审的回收率相对较低。在候选人的参与权和知情权方面,不同规模的院系基于 人事自主权的大小演化出差异化的政策与实践。同行评审意见对候选人能否顺利晋升发挥关键影响,同行 评议的回收率本身也被视为衡量候选人学术影响力和同行认可度的重要指标。在同行评议的过程中,遵循 专业逻辑的学术权力和遵循管理逻辑的行政权力相互交织,而非学术精英个体的民主权利则相对被忽视。 尽管评价过程大体遵循政策文本所制定的正式规则,但也受到专业人际网络等非正式规则的影响     

英国“科研卓越框架”同行评议制度的改革及启示

同行评议制度是英国高校科研评价体系REF(Research Excellence Framework,科研卓越框架)的精髓。英国将要实施的REF2021吸取了先前同行评议制度经济性、公平性、可靠性不足的经验教训并进行了相应改革,提高了对评审专家的信息技术和定量数据支持,调整了专家遴选制度,完善了跨学科研究评价体系,展现了计划性、透明性、多样性和灵活性等优势。这对提高中国科研评价中同行评议制度的科学性、透明性、公平性和可靠性,以及完善专家遴选机制、接受业界内外的公众监督、突出科研成果实际价值、重视对跨学科合作研究的科研评价等方面有借鉴意义。     

瞭望

全面推行同行匿名评议制度日前有研究者对同行评议制度提出的质疑,建议请专职人员进行评审。《科学时报》对此约请知名学者发表相应观点认为,同行匿名评议制度不可轻     

学科交叉视角下同行评议的再思考

当前,学科交叉融合已成为必然趋势,科研成果的跨学科性也更加明显。为进一步改进同行评议制度,维护其科学性、公正性和严肃性,在追溯同行评议概念及其交叉学科性的基础上,对同行评议的研究历程进行系统梳理,指出交叉学科同行评议在评议质量、过程及模式上的缺点和不足,提出并论证应该重塑同行评议专家与接受评价者之间的信任关系,坚持开放性、多元化的评议原则,并建立具有包容性的公众评议制度发展不同利益相关群体之间的良序的学术评价机制,通过学者自律,凝聚学术团体力量为科学探索特别是交叉学科与跨学科研究创造足够的发展空间,以期为我国交叉学科同行评议的理论创新和实践探索提供启发。     

2002年度中科院第三批海外评审专家一览

自1998年中国科学院建立“海外评审专家”系统以来，经过评审，共有77名海外学者受聘为中科院海外评审专家。“海外评审专家”系统为鼓励海外优秀留学人员为国服务，促进我院项目立项、人才引进、重大奖励评定等评审工作的科学化、规范化发挥了重要作用。2002年度，经驻外使领馆及各单位的推荐，国内外专家评议及院终审，确定李正祥等39位海外学者为中国科学院第三批海外评审专家。现将中国科学院第三批海外评审专家名单公布如下（以所在国家/地区、姓名排序）：李正祥大地构造首席科学家澳大利亚西澳大利亚大学逯高清纳米材料教授澳大利亚…     

关于完善我国博士学位论文匿名评审制度的思考与建议

博士学位论文匿名评审制度是保障博士学位论文质量继而保障博士生培养质量的重要评价制度。在回顾我国博士学位论文评审制度发展变迁的基础上,通过调查和分析博士学位论文匿名评审制度的实施效果及利弊可知,我国博士学位论文匿名评审制度在规避非学术因素干扰、保障博士生培养质量及营造敬畏学术和学术创新的文化等方面发挥了积极作用,但也存在评审专家选择不合理、不利于博士学位论文创新、论文作者权益保护不到位等问题。为进一步完善我国博士学位论文匿名评审制度,应在坚持创新价值和知识贡献的指导思想及原则基础上,遴选具有学术造诣的“小学科同行”,完善针对评审专家的监督约束制度,建立和完善申诉及公开申辩制度与评审专家回避制度。     

2000年我校申报硕士点工作成效显著

两年一度的全国申报学位点工作已圆满结束,我校共有7个学科评审通过,获得硕士学位授予权。这样,我校硕士点数量比往年翻了将近一番,达到15个。今年是我省学位委员会首次进行增列硕士点自评的一年。全省共有9个学科申报博士学位一级授权,41个学科申报博士点,138个学科申报硕士点。其中博士学位一级学科、博士点、24个硕士点由国家学位办评审,114个硕士点由我省在41个一级学科内进行自评。我省自评的114个学科,经过省内外同行专家通讯评议,80个学科通讯评议通过,其中45个学科被山西省学位委员会批准为硕士学位授权学科。我校今年共申报17个硕…     

华中科技大学学者介绍

谭术魁男,1965年生,湖北省巴东县人,土家族,中共党员,现任华中科技大学公共管理学院土地管理系主任,教授,博士生导师。同时担任国家自然科学基金申请项目同行评议专家,国冢社会科学基金申请项目同行评议专家,教育部博士硕士学位授权点通讯评议专家,湖北省专家学者报告团成员,     

Judges' Use of Examinee Performance Data in an Angoff Standard‐Setting Exercise for a Medical Licensing Examination: An Experimental Study

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple‐choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide performance data as a fatal flaw for the procedure; others have considered it appropriate for experts to integrate performance data into their judgments but have been concerned that experts may rely too heavily on the data. There have, however, been relatively few studies examining how experts use the data. This article reports on two studies that examine how experts modify their judgments after reviewing data. In both studies, data for some items were accurate and data for other items had been manipulated. Judges in both studies substantially modified their judgments whether the data were accurate or not.     

四川省青少年田径训练管理体系现状的研究

采用文献资料法、问卷调查法、专家访谈法等研究方法,对四川省青少年田径训练管理体制和教练员情况进行分析,并借鉴国内外的训练体制管理的优劣,对四川省青少年田径项目可持续发展的制约因素从宏观上进行整体调查分析,进而根据调查结果提出相应的可持续发展对策,应对新局势发展需要的田径项目新布局,以期保持四川省田径强省地位提供理论参考依据.     

The development and validation of a framework for teaching competencies in higher education

In higher education, approaches to teaching are becoming more student-centred, which demands different teaching competencies. Therefore, it is necessary to have an adequate framework of teaching competencies that can be used for evaluation purposes. The weaknesses of the existing frameworks are that they do not pay attention to the person as teacher, they are too narrowly defined, they are not validated and they are not adjusted to modern approaches to teaching. The aim of this study was to develop and validate a framework of teaching competencies in higher education. A framework for teaching competencies was therefore constructed containing the following domains: The Person as Teacher, Expert on Content Knowledge, Facilitator of Learning Processes, Organiser and Scholar/Lifelong Learner. The framework was validated using a Delphi method. Educational experts (N = 63) were asked: ``How important are the following teaching competencies in each domain for an experienced teacher in higher education?'' A confirmatory factor analysis was conducted to assess the adequacy of the domains. After two rounds, the shift in ratings was minimal, so the results were considered stable. The response rate was 82%. From the original list with 134 items, 30 items were omitted because many experts failed to respond to them. The experts reached consensus on 61 out of the resulting 104 items (59%). A confirmatory factor analysis on the three best scoring items in each domain confirmed the model. A framework of teaching competencies was developed and validated that can be used as a starting point for teacher evaluation in higher education.     

A Note on the Application of Multiple Matrix Sampling to Standard Setting

In many of the methods currently proposed for standard setting, all experts are asked to judge all items, and the standard is taken as the mean of their judgments. When resources are limited, gathering the judgments of all experts in a single group can become impractical. Multiple matrix sampling (MMS) provides an alternative. This paper applies MMS to a variation on Angoff's method (1971) of standard setting. A pool of 36 experts and 190 items were divided randomly into 5 groups, and estimates of borderline examinee performance were acquired. Results indicated some variability in the cutting scores produced by the individual groups, but the variance components were reasonably well estimated. The standard error of the cutting score was very small, and the width of the 90% confidence interval around it was only 1.3 items. The reliability of the final cutting score was.98     

Gauging Item Alignment Through Online Systems While Controlling for Rater Effects

The alignment of test items to content standards is critical to the validity of decisions made from standards‐based tests. Generally, alignment is determined based on judgments made by a panel of content experts with either ratings averaged or via a consensus reached through discussion. When the pool of items to be reviewed is large, or the content‐matter experts are broadly distributed geographically, panel methods present significant challenges. This article illustrates the use of an online methodology for gauging item alignment that does not require that raters convene in person, reduces the overall cost of the study, increases time flexibility, and offers an efficient means for reviewing large item banks. Latent trait methods are applied to the data to control for between‐rater severity, evaluate intrarater consistency, and provide item‐level diagnostic statistics. Use of this methodology is illustrated with a large pool (1,345) of interim‐formative mathematics test items. Implications for the field and limitations of this approach are discussed.     

水族传统体育项目的现状调查分析

运用文献资料、问卷调查、专家访谈、数理统计等方法对水族传统体育项目的现状进行调查。结果显示：保持开展的水族传统体育项目在逐渐减少,全民健身和学校体育都是以现代体育为主,水族武术和一些民间技艺已濒临失传;部分水族传统体育项目变换了其内容或形式,转化为民俗活动的一部分延续下来。     

Investigating Linguistic Sources of Differential Item Functioning Using Expert Think-Aloud Protocols in Science Achievement Tests

Even if national and international assessments are designed to be comparable, subsequent psychometric analyses often reveal differential item functioning (DIF). Central to achieving comparability is to examine the presence of DIF, and if DIF is found, to investigate its sources to ensure differentially functioning items that do not lead to bias. In this study, sources of DIF were examined using think-aloud protocols. The think-aloud protocols of expert reviewers were conducted for comparing the English and French versions of 40 items previously identified as DIF (N?=?20) and non-DIF (N?=?20). Three highly trained and experienced experts in verifying and accepting/rejecting multi-lingual versions of curriculum and testing materials for government purposes participated in this study. Although there is a considerable amount of agreement in the identification of differentially functioning items, experts do not consistently identify and distinguish DIF and non-DIF items. Our analyses of the think-aloud protocols identified particular linguistic, general pedagogical, content-related, and cognitive factors related to sources of DIF. Implications are provided for the process of arriving at the identification of DIF, prior to the actual administration of tests at national and international levels.     

Allowing Students a Choice of Items on Objective Examinations

Measurement experts often advise against giving students an option of selecting which items on a test they choose to answer. A review of the related literature revealed that this caveat refers primarily to essay examinations. It was the purpose of this research to investigate the effects of permitting students the prerogative of selecting a proportion of the items on an objective test which they wished to exclude from the calculation of a grade. Findings regarding the effects of this practice on test reliability, test validity and on the students' relative test performance ranks suggest that it may not be as deleterious as we had been led to believe.     

Inclusive cultures,policies and practices in Spanish compulsory secondary education schools: teachers’ perceptions in ordinary and specific teaching contexts

This article presents a study that attempts to inquire into the indicators related with inclusive education by taking into account teachers’ responses in the ordinary and specific teaching contexts (experts in special education needs, specialists in therapeutic pedagogy and compensatory education or specific programmes organised to accommodate diversity). We used a Spanish translation of the Index for Inclusion, which has been adapted and divided into three Likert‐type scales about inclusive cultures, organisation in the teaching–learning context and inclusive practices. We did a t‐test to detect any significant differences in teachers’ responses in terms of the ordinary or specific teaching contexts in which they work. No significant differences were seen in the items related to cultures and organisation, but we found significant differences in 12 of the 36 items of the inclusive practices scale. Finally, we discuss the role of the processes for innovation and transformation in secondary schools and provide key factors to build an intercultural inclusive school.     

The Answer Key as a Source of Error in Examinations for Professionals

There has been a growing emphasis on written examinations that assess the ability of physicians and teachers to make decisions in the absence of protocols for action—a crucial aspect of professional competence. A characteristic of such tests is controversy, even among experts, about what constitutes the correct response to some of the items. This paper studied the impact of variability in answer keys, constructed using the aggregate method, on total errors of measurement. Results indicated that several scorers made a sizable contribution to reduction in measurement error and that scorers or groups of scorers who each developed the answer key for a subset of items produced better results than a single group that developed the answer key for all items. Implications of judgment tests scored using the aggregate method are described for teacher and physician certification.