How to achieve accurate peer assessment for high value written assignments in a senior undergraduate course

The psychometric measures of accuracy, reliability and validity of peer assessment are critical qualities for its use as a supplement to instructor grading. In this study, we seek to determine which factors related to peer review are the most influential on these psychometric measures, with a primary focus on the accuracy of peer assessment or how closely peer-given grades match those of an instructor. We examine and rank the correlations of accuracy, reliability and validity with 17 quantitative and qualitative variables for three senior undergraduate courses that used peer assessment on high value written assignments. Based on these analyses, we altered the single most significant variable of one of the courses. We demonstrate that the number of reviews completed per reviewer has the greatest influence on the accuracy of peer assessment out of all the factors analysed. Our calculations suggest that six reviews must be completed per reviewer to achieve peer assessment that is no different from the grading of an instructor. Effective training, previous experience and strong academic abilities in the reviewers may reduce this number.     

用集合图形法检验三段论式有效性——兼论集合图形法与文恩图法的比较

集合图形法是笔者重新建构的一种检验三段论式有效性的方法。这种新的检验方法与文恩图法有很多相似之处,也有不同之处,但最后检验的结果完全一致。这对于研究三段论的有效性提供了新的研究方法。     

Evaluating the Social Validity of Project TEAM: A Problem-Solving Intervention to Teach Transition Age Youth with Developmental Disabilities to Resolve Environmental Barriers

Project TEAM teaches transition-age youth with developmental disabilities (DD) to identify physical and social environmental barriers and supports, generate solutions to barriers, and request modifications to increase participation. Establishing the social validity of this environment focused intervention with youth and their parents is critical, given the significant shift the intervention represents from rehabilitation’s more traditional focus on body structures and function. University researchers and youth research collaborators conducted a participatory evaluation of the purpose, procedures, and perceived benefits of Project TEAM. Youth with DD (n = 42) provided feedback using three methods: activity voting, a survey, and a focus group. Parents (n = 37) also provided feedback. Results suggest that both youth and parents find Project TEAM acceptable and relevant to youth’s current and future lives. The majority of youth and parents felt Project TEAM fostered independence and empowerment, although some youth and parents reported challenges with the unique environment-focused approach.     

数字加工和计算能力测验的修订及信效度检验

本研究对数字加工和计算能力测验进行了修订,修订工作主要包括三个部分:(1)测验的翻译;(2)预测验;(3)大样本测验。测验的分半信度为0.86,克伦巴赫α系数为0.93,被教师诊断为数字加工和计算障碍的学生几乎在所有的子测验成绩上都低于正常学生。这些结果表明,此测验是一个信度和效度均较高的测验。     

ON THE CURRENT RMB EXCHANGE RATE REGIME AFFECTING THE EFFECTIVENESS OF MONETARY POLICY

ＩＮＴＲＯＤＵＣＴＩＯＮＳｉｎｃｅＣｈｉｎａｓｔａｒｔｅｄｅｃｏｎｏｍｉｃｓｙｓｔｅｍｒｅｆｏｒｍｉｎ 1 978,ｍｏｎｅｔａｒｙｐｏｌｉｃｙｈａｓｂｅｅｎｆｕｌｌｙｕｓｅｄｔｏａｄｊｕｓｔｔｈｅｍａｃｒｏｅｃｏｎｏｍｙａｎｄａｃｈｉｅｖｅｄｇｏｏｄｅｆ ｆｅｃｔｓ.Ｔｉｇｈｔｍｏｎｅｔａｒｙｐｏｌｉｃｙｃｏｎｔｒｏｌｓｉｎｆｌａｔｉｏｎｅｆ ｆｅｃｔｉｖｅｌｙｗｈｉｌｅｅｘｐａｎｓｉｖｅｍｏｎｅｔａｒｙｐｏｌｉｃｙｓｔｉｍｕ ｌａｔｅｓｅｃｏｎｏｍｉｃｄｅｖｅｌｏｐｍｅｎｔｆｏｒｃｅｆｕｌｌｙ.Ｂｕｔｉ…     

论教学的有效性

研究教学的有效性是为了解决教学中大量存在的“平庸”和“低效”问题,教学有效性涉及到教学效果、教学效率和教学质量三种价值属性。在当前课堂教学中,教学主体缺乏教学有效性的观念、师生之间缺乏有效的互动以及对教学时间缺少周密的安排是三种典型的“平庸”和“低效”样态。新课程提出了建构性、生成性和多元性的教学有效性标准,建立了当前有效教学的理想模式。     

The contribution of rubrics to the validity of performance assessment: a study of the conservation–restoration and design undergraduate degrees

Rubrics have attained considerable importance in the authentic and sustainable assessment paradigm; nevertheless, few studies have examined their contribution to validity, especially outside the domain of educational studies. This empirical study used a quantitative approach to analyse the validity of a rubrics-based performance assessment. Raters evaluated the performance of 84 first-year university students producing service-learning projects for the Conservation–Restoration and Design degrees. The study data comprised the 9240 scores given by two teachers and three student tutors, who assessed the students’ projects on three occasions during the semester. Factor analyses confirmed that the students attained the expected learning outcomes and made significant learning progress. This learning progress was also corroborated by analyses of variance. The attainment of the learning goals and the evidence of learning progress demonstrated the validity of the inferences drawn from the assessment system. In addition, the results highlighted the need to consider rubrics as a first-order teaching resource and not only as a scoring tool.     

Bias hypotheses under scrutiny: investigating the validity of student assessment of university teaching by means of external observer ratings

To advance the discussion on the validity of student evaluations of university teaching, student ratings of two teaching dimensions – student involvement and rapport – were compared with corresponding observer ratings. Seven potential bias variables were tested with regard to their impact on the students’ teaching assessment: three teacher characteristics (first impression, enthusiasm, humour) and four student characteristics (prior interest, expected grades, study experience, class attendance). Bias was defined as an impediment of the students’ assessment of teaching on course level. By means of bivariate correlations with course averages and two-level latent moderated structural equations, data of 1,716 students in 80 courses were analysed. Results showed that all three teacher characteristics were genuinely connected to rapport, and even explained variance of the student-rated variable when controlling for observer-rated rapport. The assessment of student involvement was not modified by the teacher characteristics except for teacher enthusiasm, which affected the student evaluation when controlling for observed involvement and, moreover, moderated the relation between the observed and the student-rated variable. For the examined student characteristics, no biasing effects were found – neither on rapport nor on student involvement.     

测验项目编制与等值的一种有效策略——层面理论

回转翻译法关注的是“文字等价”,项目反应理论注重“统计指标等价”。层面理论项目等价注重项目的同一测量目标,即等值的项目应该在相同的条件下测试被试相同的反应。层面理论通过映射语句技术清晰地界定项目的测量目标,使得项目等值与项目编制更加科学。通过层面理论编制的项目维度结构更加清楚,测验的结构效度更有保证。将层面理论和心理计量学的其他方法结合起来,可以有效提高测验项目编制与等值的质量。     

课程知识的合法性问题--对《基础教育课程改革纲要(试行)》的社会学分析

本文通过对《基础教育课程改革纲要》中课程知识合法性的追问发现,国家、专家等角色在本次基础教育课程改革中发生了变化。“国家”角色开始隐退,“专家”由“象征性参与者”向“事实性参与者”过渡,使课程知识合法性的主体发生了变化:课程将不纯粹是“传递统治阶级意识形态,进行阶级关系再生产的工具”;“专家”与“国家”在角色、地位上既有磨合又有争夺,从而也改变了课程知识合法性的过程:课程由社会单方“控制”的中介变为多方“认同”、“妥协”的产物。