Interpretive validity theory: mapping a methodology for validating educational assessments

Background:?Validity theory has evolved significantly over the past 30 years in response to the increased use of assessments across scientific, social and educational settings. The overarching trajectory of this evolution reflects a shift from a purely quantitative, positivistic approach to a conception of validity reliant on the interpretation of multiple evidence sources integrated into validity arguments. Moreover, within contemporary validity, interpretation has been emphasised as a central process; however, despite this emphasis, there have been few explicit articulations of specific interpretive methodologies applicable to the practice of validation.

Purpose:?To link contemporary theoretical foundations in validity to practical methods and structures to help guide the collection and analysis of interpretive validity evidence. By building upon existing validity theory, this paper aims to provide greater clarity on the practice of validation and contribute toward the larger developing framework for the validation of educational assessments.

Source of evidence:?An interdisciplinary, integrative review of over 60 research articles and sources related to the theory and practice of educational validation and interpretive inquiry approaches. Sources include literature from the fields of educational assessment and more broadly social scientific research.

Main argument:?As assessments in education increasingly aim to measure complex constructs that are value-laden and socially dependant, validity theory must keep pace and evolve in ways that address the inherent complexities associated with contemporary educational assessment. Through this paper, I assert that a greater understanding of interpretive methodologies represents one of the most promising areas for development of validation theory and practice. Specifically, I argue that dialectic, hermeneutic and transgressive forms of inquiry can be integrated within current argument-based structures for the collection, analysis and representation of validity evidence in several useful ways.

Conclusions:?Interpretive inquiry processes, namely dialectic, hermeneutic and transgressive forms of interpretation, serve to expand validation practice to include diverse evidences for the generation of multiple-perspective validity arguments. The paper concludes with specific implications for future research and practice within the field of interpretive validity theory.     

Medical student use of Facebook to support preparation for anatomy assessments

The use of Facebook to support students is an emerging area of educational research. This study explored how a Facebook Page could support Year 2 medical (MBChB) students in preparation for summative anatomy assessments and alleviate test anxiety. Overall, Facebook analytics revealed that in total 49 (19.8% of entire cohort) students posted a comment in preparation for either the first (33 students) or second (34) summative anatomy assessments. 18 students commented in preparation for both. In total, 155 comments were posted, with 83 for the first and 72 for the second. Of the 83 comments, 45 related to checking anatomical information, 30 were requiring assessment information and 8 wanted general course information. For the second assessment this was 52, 14 and 6, respectively. Student perceptions on usage, and impact on learning and assessment preparation were obtained via a five‐point Likert‐style questionnaire, with 119 students confirming they accessed the Page. Generally, students believed the Page was an effective way to support their learning, and provided information which supported their preparation with increases in perceived confidence and reductions in anxiety. There was no difference between gender, except for males who appeared to be significantly less likely to ask a question as they may be perceived to lack knowledge (P < 0.05). This study suggests that Facebook can play an important role in supporting students in preparation for anatomy assessments. Anat Sci Educ 10: 205–214. © 2016 American Association of Anatomists.     

最优控制理论课程教学改革研究

最优控制这门课程着重于讨论最优控制理论在不同工程领域背景下的应用。借助于该理论,人们可以在各种控制函数中确定出最优解。本文首先概括了最优控制的理论框架,进而从教学内容、教学方法与教材等方面提出了一系列的教学改进措施。此外,并结合科研项目提高学生的认知、理解及应用能力。最后,给出了一个关于本课程改革的总结与展望。     

Oral versus written assessments: a test of student performance and attitudes

Student performance in and attitudes towards oral and written assessments were compared using quantitative and qualitative methods. Two separate cohorts of students were examined. The first larger cohort of students (n = 99) was randomly divided into ‘oral’ and ‘written’ groups, and the marks that they achieved in the same biology questions were compared. Students in the second smaller cohort (n = 29) were all examined using both written and oral questions concerning both ‘scientific’ and ‘personal development’ topics. Both cohorts showed highly significant differences in the mean marks achieved, with better performance in the oral assessment. There was no evidence of particular groups of students being disadvantaged in the oral tests. These students and also an additional cohort were asked about their attitudes to the two different assessment approaches. Although they tended to be more nervous in the face of oral assessments, many students thought oral assessments were more useful than written assessments. An important theme involved the perceived authenticity or ‘professionalism’ of an oral examination. This study suggests that oral assessments may be more inclusive than written ones and that they can act as powerful tools in helping students establish a ‘professional identity’.     

从劳动价值论到社会主义财富论

马克思的劳动价值论是在对工业化初期的资本主义英国进行批判分析过程中建立起来的，深化对马克思主义劳动价值论的研究和认识，应不拘泥于理论教条，而应当结合国情，与时俱进，建立社会主义财富论，这不会导致对劳动价值论的否定。     

Testing optimal foraging theory on the radial maze: The role of learning in patch sampling

A four-arm radial maze containing 10 feeders in each arm (patch) was used to study patch sampling in rats. In each of three experiments, rats foraged for 30 sessions. On each session, two randomly chosen patches were baited with food and the remaining two patches were empty. In Experiment 1, the number of baited feeders in baited patches (6) was varied from 1–10 over five groups of subjects. Mean visits to empty patches was an inverse function of 6, as predicted by an optimal foraging model. In Experiments 2 and 3, rats’ ability to discriminate between baited and empty patches was examined when food in baited patches was placed in fixed locations, either in clumps (Experiment 2) or distributed throughout the patch (Experiment 3). Rats in fixed-food-location conditions reliably visited fewer feeders in empty patches than did rats in randomly changing control groups. Examination of within-patch foraging patterns indicated that rats in fixed-food-location groups selectively sampled potentially baited locations and abandoned the patch if food was not found. It is suggested that processes of patch discrimination were responsible for these effects.     

Evaluating the use of computers in science assessment: Considerations and recommendations

The use of computers in science assessment is viewed through the eyes of the program evaluator by examining six considerations: (1) purpose of computer use in assessment, (2) value of program evaluation models, (3) possible harmful side effects of assessment, (4) nature of imbedded questions, (5) definition of treatment, and (6) computers and the classroom context. Following a discussion of the considerations, four recommendations are offered in regard to continuing to study the use of computers for assessment (examples provided), utilizing the ideas contained in program evaluation models, capitalizing on several interesting, novel possibilities for evaluation design, and combining qualitative and quantitative techniques (mixed methodologies) into the evaluation procedures.     

Thinking beyond the score: Multidimensional analysis of student performance to inform the next generation of science assessments

Conventional assessment analysis of student results, referred to as rubric-based assessments (RBA), has emphasized numeric scores as the primary way of communicating information to teachers about their students’ learning. In this light, rethinking and reflecting on not only how scores are generated but also what analyses are done with them to inform classroom practices is of utmost importance. Informed by Systemic Functional Linguistics and Latent Dirichlet Allocation analyses, this study utilizes an innovative bilingual (Spanish–English) constructed response assessment of science and language practices for middle and high school students to perform a multilayered analysis of student responses. We explore multiple ways of looking at students’ performance through their written assessments and discuss features of student responses that are made visible through these analyses. Findings from this study suggest that science educators would benefit from a multidimensional model which deploys complementary ways in which we can interpret student performance. This understanding leads us to think that researchers and developers in the field of assessment need to promote approaches that analyze student science performance as a multilayered phenomenon.     

Evaluating student engagement and experiential learning in global classrooms: A qualitative case study

The global classroom is an emerging technology-based pedagogy used internationally by educational institutions. To evaluate a global classroom, we conducted a qualitative study using written reflections and semi-structured interviews of global classroom participants, based on two theoretical frameworks: Kearsley and Shneiderman’s engagement theory and Kolb’s experiential learning theory. We analyzed student reflections and transcribed interviews, using the software package, NVivo, with two objectives: (1) to evaluate if global classroom is engaging and experiential to students and (2) to elucidate how student engagement is fostered in the global classroom through experiential learning. Results illustrated a complex relationship between student engagement and experiential learning. During the experiential learning cycle, engagement theory (relate-create-donate) principles contributed to student engagement at varying levels and for different purposes. Based on the results of this study, we created a framework that demonstrates the interactivity of engagement theory and experiential learning theory to describe how student engagement featured in experiential learning during this global classroom, with strategies to maximize student engagement in experiential learning.     

城市化对梅州市区鸟类取食集团的影响

2012年5月至2013年2月间,对梅州市区建筑区、城区斑块林地、周边农田和周边林地等四个不同的鸟类栖息地的鸟类取食情况进行了调查研究.结果表明:(1)梅州市区的鸟类取食集团的总数整体上随着城市化程度的提高呈减少趋势.(2)梅州市区的鸟类取食集团的物种数在总体上随城市化程度的提高而呈减少趋势.(3)某些鸟类的取食集团会随栖息地的不同而发生改变.(4)不同的鸟类取食集团的分布与不同的栖息参数有显著的回归关系.(5)食物丰富度、灌木层密度等因素是影响鸟类取食集团分布的主要因子.     

略论大学生党员理论教育的内涵

大学生党员教育是高校党建工作的重中之中,大学生肩负中国特色社会主义建设事业的重任,提高大学生党员理论教育水平是实现"两个百年"梦想的重要保障。高校要深入大学生党员的教育的研究,明确理论教育的内涵,才能有的放矢,提升党员教育工作的有效性和实效性。     

Leniency and halo bias in industry-based assessments of student competencies: a critical,sector-based analysis

Industry placements are popular means to provide students with an opportunity to apply their skills, knowledge and experience in a ‘real world’ setting. Within this context, supervisor feedback allows educators to measure students’ performance beyond academic objectives, by benchmarking it against industry expectations. However, industry assessments appear to be frequently clouded and overwhelmingly positive by nature, which questions the reliability and validity of supervisors’ judgement of competencies. Supervisor bias has been paid much attention within the context of clinical placements, as well as within the domain of social work. However, the concept has been largely ignored within business education, despite the increasing emphasis on – and deep integration of – work-integrated learning in the business curriculum. This paper sets out to address this gap by examining variances in mark distribution and apparent leniency in the context of a final-year, compulsory placement unit, based on observations and data collected over nine semesters (n?=?546). The focus of this study is on gaining an understanding of the reasons behind assessment bias and the pressures placed on industry assessors. The data indicate that different types of placement locations apply dissimilar standards when assessing student performance. The author identifies three statistically different placement types (small business, not-for-profit and professional), which influence the strength and risk of grader bias, hence ultimately the assessment outcome.     

Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice

The importance of formative assessment instudent learning is generally acknowledged, butit is not well understood across higher education.The identification of some key features offormative assessment opens the way for adiscussion of theory. It is argued that thereis a need for further theoretical developmentin respect of formative assessment, which needsto take account of disciplinary epistemology,theories of intellectual and moral development,students' stages of intellectual development,and the psychology of giving and receivingfeedback. A sketch is offered of the directionthat this development might take. It is notedthat formative assessment may be eitherconstructive or inhibitory towards learning. Suggestions are made regarding research intoformative assessment, and how research mightcontribute to the development of pedagogicpractice.     

研究生政治理论课教学方法改革探索

研究生政治理论课教学是研究生教育的重要组成部分。本文从提高任课教师要求、加强网络教学、改革教学方式、改进考核方式等方面阐述了研究生政治理论的教学改革探索。     

Measuring knowledge integration: Validation of four‐year assessments

Science education needs valid, authentic, and efficient assessments. Many typical science assessments primarily measure recall of isolated information. This paper reports on the validation of assessments that measure knowledge integration ability among middle school and high school students. The assessments were administered to 18,729 students in five states. Rasch analyses of the assessments demonstrated satisfactory item fit, item difficulty, test reliability, and person reliability. The study showed that, when appropriately designed, knowledge integration assessments can be balanced between validity and reliability, authenticity and generalizability, and instructional sensitivity and technical quality. Results also showed that, when paired with multiple‐choice items and scored with an effective scoring rubric, constructed‐response items can achieve high reliabilities. Analyses showed that English language learner status and computer use significantly impacted students' science knowledge integration abilities. Students who took the assessment online, which matched the format of content delivery, performed significantly better than students who took the paper‐and‐pencil version. Implications and future directions of research are noted, including refining curriculum materials to meet the needs of diverse students and expanding the range of topics measured by knowledge integration assessments. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 48: 1079–1107, 2011     

激励理论在高校学生干部管理中的运用

本文对如何将激励理论运用到学生干部管理工作进行了研究和阐述。     

关联理论下的英语阅读教学

Sperber和Wilson提出的关联理论被广泛应用于各个领域,对英语阅读教学有很强的指导作用。根据关联理论,阅读就是作者示意和读者推理的交际过程。在英语阅读过程中,要明确作者的真实交际意图,找到最佳关联,通过推理得出隐含语境,取得语境效果,从而达到阅读的目的。本文介绍了关联理论的含义和原则,讨论了关联理论在英语阅读教学中的重要作用。     

A plan for the co-construction and collaborative use of rubrics for student learning

Abstract

The use of assessment rubrics in the higher education sector is now widespread in a number of disciplines. Typically, these rubrics are constructed by teachers who also tend to be the main users of the rubrics throughout the grading process. In recent years, questions have been raised about this teacher-directed approach and some educators have begun to explore an alternate approach to rubric construction; that is, engaging students in collaboration with their teachers to co-construct assessment rubrics. This paper outlines the processes employed in a project that investigated the co-construction of rubrics within six different contexts. The project aimed to engage students in collaboration with their teachers to co-construct rubrics which are co-owned by teacher and student. A mixed methods approach was utilized to explore the effectiveness of the strategy. Questionnaires, interviews and focus groups were utilized to gather data from both the teacher-participants and student-participants regarding their experiences of being involved in the study. Findings are presented from the perspectives of both students and teachers, relating their views of rubrics and the activity of rubric co-construction. The paper concludes with recommendations for practical approaches to rubric co-construction and future research directions.     

关于劳动价值理论的再认识

结合党的十六大报告，对劳动价值理论从方法论、理论发展、理论内容等方面提出了新认识。指出要深入理解马克思劳动价值论的真谛，防止曲解和断章取义；要面向社会主义市场经济和现代化建设实际，丰富和发展劳动价值论，包括劳动的主体性、劳动范畴的广泛性、劳动形式的多样性、劳动价值计算的复杂性、劳动价值决定的国际性等诸多方面。