Why increasing the number of raters only helps sometimes: Reliability and validity of peer assessment across tasks of different complexity

Number of raters is theoretically central to peer assessment reliability and validity, yet rarely studied. Further, requiring each student to assess more peers’ documents both increases the number of evaluations per document but also assessor workload, which can decline performance. Moreover, task complexity is likely a moderating factor, influencing both workload and validity. This study examined whether changing the number of required peer assessments per student / number of raters per document affected peer assessment reliability and validity for tasks at different levels of task complexity. 181 students completed and provided peer assessments for tasks at three levels of task complexity: low complexity (dictation), medium complexity (oral imitation), and high complexity (writing). Adequate validity of peer assessments was observed for all three task complexities at low reviewing loads. However, the impacts of increasing reviewing load varied by reliability vs. validity outcomes and by task complexity.     

实验教学中虚拟实验的有效性探究

随着虚拟实验在实验教学中的应用,对其有效性的考察显得越来越重要。本文在虚拟实验理论的基础上,介绍了该技术的分类,并从实验教学的具体实践中对其有效性进行了分析。     

提高隐式非线性微分方程的数值解算速度

非线性反馈控制系统的数学模式常常表现为隐式非线性微分方程,用通用数值积分法求解该系统的动态响应时,每步都要进行费时的矩阵求逆运算,使得解算速度大大下降.文中第一部分提出先求出隐式微分方程中时变矩阵A(x)逆的符号表达式,再用通用数值积分法求解该类系统时,每步积分不再求矩阵逆运算,从而可以大大提高解算速度.第二部分讨论了在MATLAB/Simulink新版本的各种环境下加速仿真速度的真实有效性.     

基于学校教学常规考试的试卷命制技术

学校教学中有单元测试、期中和期末考试等常规考试,用以适时检查学生达到教学目标的程度,这些常规考试属于标准参照考试。本文根据新的课程标准,综合相应的理论和实践,对学校教学常规考试从考试的设计方面进行完善,如确定考试的行为目标、不同题型的使用与编写、考试结果的分析等,以促进学校教学,促进教师专业发展。     

MHK(三级)口语考试计算机评分的效度研究

将计算机应用到考试评分中,以计算机评分代替人工评分,是一种提高评分效率、降低评分成本的有效方法。MHK(三级)口语考试的计算机评分开创了汉语口语考试计算机评分研究的先河。本研究将人工评分与计算机评分进行了比较分析,探讨了MHK(三级)口语考试计算机评分的效度。     

审计委员会失效原因及对策分析

我国审计委员会出现的时间不长,它的理论和实践都有不足,于是它就失去一部分的效应,失效的原因是多种多样的。因此针对审计委员会失效这种现象提出了一系列的应对策略,如企业应该规范审计委员会制度,提高委员会成员的质量,政府应该制定相关法律法规,同时增加理论与实务界的联系。     

蕴涵量表法在HSK阅读理解测验公平性研究中的应用

阅读理解能力测验中所选择的文章在内容方面对不同专业背景的考生亚团体是否具有公平性的问题,是测验效度高低的重要证据,也是测验效度验证（validation）的重要环节。本研究以中国语言与文学专业考生为目标组,分别将经济学专业和生物医学专业考生作为参照组,采用效标测量和蕴涵量表分析相结合的方法,对HSK（高等）阅读理解测验的文章难度对三个不同专业背景的考生组的公平性问题进行了检验。研究结果表明,两个参照组考生尽管具有各自的相对专业优势,但他们在六篇阅读材料上获得的难度排列顺序与目标组考生完全一致;虽然目标组考生不具备汉语知识以外的其他专业优势,但因为HSK考试所选择的阅读材料没有涉及语言知识本身以外的特殊专业要求,因而测验对三个不同专业背景的考生具有较高的公平性。     

A critique of the internship measurement scales in higher education

This article is a critical review of the internship measurement scales in higher education. Although the importance of internships is widely acknowledged in the literature, there is a scarcity of attempts to construct a scale to measure internship outcomes. In fact, a thorough examination of the internship literature reveals that there are only two internship measurement scales, which are relatively recent. It is a widespread practice that empirical studies in this area use job-related scales in their endeavour to measure internship-related constructs such as internship satisfaction and internship effectiveness. This is mostly due to the lack of available internship scales and partly due to the naive assumption that job-related scales can be somehow altered and may readily be adapted to internship-related contexts, disregarding the contextual and contractual differences between a job and an internship. This casts doubt on the validity of the findings in those studies. However, the valid measurement of internships is not only necessary to meet the highest academic standards but also to make evidence-based employability policies in higher education institutions. Therefore, this article aims to highlight the need to measure internship processes more effectively based on a sound method and bring this issue to the attention of the research community worldwide. To that end, the existing scales of internship are critically examined and areas for improvements are highlighted.     

Uncovering everyday dynamics in students’ perceptions of instructional quality with experience sampling

Within-student dynamics in perceptions of instructional quality have been neglected, although student states constitute a major share of these perceptions. The present study examined the structure and correlates of student state perceptions of the three basic dimensions, teacher support, cognitive activation, and classroom management. We conducted a three-week experience sampling study using state measures in four subjects (observations: n_mathematics = 2,681, n_physics = 1,555, n_German = 2,026, n_English = 1,835) and analyzed data from 372 German secondary school students (M_age = 15.3 years), conducting two-level confirmatory factor analyses. Against more parsimonious solutions, the postulated three-factor structure was confirmed within- and between-students across subjects, entailing 51% within-student variance on average. Similar to trait-like perceptions, state perceptions were positively related to grades and academic interest. Our results support the factorial and convergent validity of state student perceptions of instructional quality, expanding upon between-person-based literature and uncovering opportunities to enhance teaching effectiveness.     

Course evaluation in medical education

Course evaluation is integral to medical education. We discuss (1) distinctive features of medical education that impact on course evaluation, (2) a framework for course evaluations, (3) details that shape the evaluation process, (4) key measurement issues important to data gathering and interpretation, and (5) opportunities for expanding the scope of research regarding course evaluations. Drawing from higher education and medical education literature, a great deal is known about course evaluations. High-quality rating scales have been developed. There is evidence that ratings are valid and reproducible given sufficient ratings are gathered, but there remain many areas deserving of more research.