Enhancing learning in clinical placements: reflective practice,self-assessment,rubrics and scaffolding

Professional preparatory health programmes generally involve clinical placements with a focus on integration of theory into real life practice. Reflective writing is often included in the assessment requirements for clinical placement courses. However enabling students to engage in deeper levels of reflective writing in action, on action and for action requires careful scaffolding and assessment task design within university clinical placement handbooks. The purpose of this case study was to explore iterative refinement of assessment exercises within clinical placement handbook activities. Specifically, the study examined students’ responses to counselling experiences in authentic pharmacy practice, with comparative analysis undertaken for the various versions of the assessment task. Results showed that provision of iteratively developed guidelines, provided to three different groups of students, to support students in the assessment task, increased the marks achieved for individual components of the assessment. The results provide insight into how the design of these types of assessment exercises can be improved.     

The use of scoring rubrics: Reliability, validity and educational consequences

Several benefits of using scoring rubrics in performance assessments have been proposed, such as increased consistency of scoring, the possibility to facilitate valid judgment of complex competencies, and promotion of learning. This paper investigates whether evidence for these claims can be found in the research literature. Several databases were searched for empirical research on rubrics, resulting in a total of 75 studies relevant for this review. Conclusions are that: (1) the reliable scoring of performance assessments can be enhanced by the use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training; (2) rubrics do not facilitate valid judgment of performance assessments per se. However, valid assessment could be facilitated by using a more comprehensive framework of validity when validating the rubric; (3) rubrics seem to have the potential of promoting learning and/or improve instruction. The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.     

Teaching for creativity: towards sustainable and replicable pedagogical practice

This article explores the pedagogical significance of recent shifts in scholarly attention away from first generation and towards second generation understandings of creativity. First generation or big ‘C’ creativity locates the creative enterprise as a complex set of behaviours and ideas exhibited by an individual, while second generation or small ‘c’ creativity locates the creative enterprise in the processes and products of collaborative and purposeful activity. Second generation creativity is gaining importance for a number of reasons: its acknowledged significance as a driver in the new or digital economy; recent clarification of the notion of ‘creative capital’; the stated commitment of a growing number of universities to ‘more creativity’ as part of their declared vision for their staff and students; and, the recognition that the creative arts does not have a monopoly on creative capability. We argue that this shift allows more space for engaging with creativity as an outcome of pedagogical work in higher education. The article builds on the project of connecting ‘creative capital’ and university pedagogy that is already underway, assembling a number of principles from a wide range of scholarship, from computer modelling to social and cultural theorising. In doing so, it provides a framework for systematically orchestrating a ‘creativity-enhancing’ learning environment in higher education.

Attitudes of secondary school students towards doing research and design activities

Research and design activities are often employed in STEM (Science, Technology, Engineering & Mathematics) education. This study aims to examine students’ attitudes towards doing research and design activities in secondary school, among two groups of students: (1) students that take the quite recently introduced Dutch subject O&O (research & design), in which students perform authentic research and design projects related to STEM disciplines; and (2) students that do not take O&O. The subject O&O is only taught at a limited number of certified, so called ‘Technasium’, schools. A questionnaire, developed by the authors, was completed by 1625 students from Grades 8 and 11. Unlike previous studies on student attitudes, which usually use abstract concepts like ‘science’ or ‘technology’, the questionnaire used in this study contains active verbs to characterise research and design activities. The results showed that, in general, students who took the subject O&O had more positive attitudes towards doing research and design activities than regular students. Both student groups appeared to find doing design activities more enjoyable than doing research activities. The results of this study provide useful information for teachers as well as teacher educators about the existing attitudes of students, for example their preference for design projects over research projects.     

Evaluating the non-academic impact of academic research: design considerations

Evaluation of academic research plays a significant role in government efforts to steer public universities. The scope of such evaluation is now being extended to include the ‘relevance’ or ‘impact’ of academic research outside the academy. We address how evaluation of non-academic research impact can promote more such impact without undermining academic freedom and research excellence. Five questions on evaluation design are considered: (1) What should be the object of measurement? (2) What should be the timeframe? (3) How should non-academic users of research inform evaluation processes? (4) How should controversial impacts be managed? (5) When in funding cycles should impact evaluation occur? We conclude that non-academic impact should be selectively promoted and evaluated. This is how greater gains from research might be best captured without imposing misguided and onerous reporting requirements on individuals and institutions.     

标准参照评估的行动实践 ——以香港中文大学通识教育基础课程为案例

香港特别行政区高校从2015年开始，学生学业评估陆续由常模参照评估转为标准参照评估。本文以香港中文大学通识教育基础课程实施标准参照评估的行动为案例，分析了这项新的学业评估方式在实施中所遇到的问题，讨论了解决方法并提出实施模型。该模型包含评分量表的设计与开发、量表使用、教师意见反馈与量表修改，以及评分之后的集体讨论这一套实施流程。文章最后指出，要保证标准参照评估的有效实施，需建立教师实践共同体。实践共同体的行动包括：教师共同开发评估标准，通过集体讨论对评估标准的含义和使用达成共识，共同审视评估结果，根据评估结果进一步修改和完善评估标准。     

A Mastery Rubric: guiding curriculum design,admissions and development of course objectives

This article describes a ‘Mastery Rubric’ (MR) used to design both the curriculum and the assessments in a new two‐year certificate programme intended to train physicians in clinical research skills. The MR for clinical research skills is built around a set of core research skills: critical review of literature; articulation of research objective; development of research design; development of analysis plan; implementation of the study; implementation of the analysis plan and presentation of results. Four distinct levels of performance are described for each skill: beginning, novice, competent and proficient. This rubric outlines and provides a path to mastery of the clinical research skills the certificate programme was designed and funded to target. Using the rubric to design the curriculum ensures that courses will provide instruction in key domains, promotes assessment that demonstrates development in the target skills and knowledge, and encourages reflection and cognitive self‐monitoring in the students. It is a flexible, criterion‐referenced definition of ‘success’ for students as well as the programme itself. The criteria are characterised in terms of the skills, habits of mind and organisational principles that can foster excellence in clinical research, but the approach can be generalised.     

Assessment as learning: Enhancing discourse,understanding, and achievement in innovative science curricula

An assessment‐oriented design‐based research model was applied to existing inquiry‐oriented multimedia programs in astronomy, biology, and ecology. Building on emerging situative theories of assessment, the model extends prevailing views of formative assessment for learning by embedding “discursive” formative assessment more directly into the curriculum. Three twenty‐hour curricula were designed and aligned to content standards, and three levels of assessments were developed and used to assess and enhance learning for each curriculum. These assessments included three or four informal “activity‐oriented” quizzes and discursive formative feedback rubrics supporting collective discourse, a “curriculum‐oriented” examination of individual conceptual understanding, and a “standards‐oriented” test measuring aggregated achievement of targeted standards. After two design‐research cycles, worthwhile scientific argumentation and statistically significant gains were attained for two of the three packages on the exam and test. Achievement gains were comparable to or larger than those of students in comparison classrooms. Many existing innovations could be enhanced and evaluated in this fashion; designing these strategies directly into innovations could have an even greater impact on discourse, understanding, and achievement. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 1240–1270, 2012     

Putting Rubrics to the Test: The Effect of a Model, Criteria Generation, and Rubric-Referenced Self-Assessment on Elementary School Students' Writing

The purpose of this study was to investigate the effect of reading a model written assignment, generating a list of criteria for the assignment, and self-assessing according to a rubric, as well as gender, time spent writing, prior rubric use, and previous achievement on elementary school students' scores for a written assignment (N = 116). Participants were in grades 3 and 4. The treatment involved using a model paper to scaffold the process of generating a list of criteria for an effective story or essay, receiving a written rubric, and using the rubric to self-assess first drafts. The comparison condition involved generating a list of criteria for an effective story or essay, and reviewing first drafts. Findings include a main effect of treatment and of previous achievement on total writing scores, as well as main effects on scores for the individual criteria on the rubric. The results suggest that using a model to generate criteria for an assignment and using a rubric for self-assessment can help elementary school students produce more effective writing.     

The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners

Design research (DR) has been an emerging research paradigm in the field of educational technology as well as in education generally for two decades. Educational design research integrates design and research into a socially responsible approach to inquiry related to learning and teaching. Given its still relative novelty, design research requires further discussion regarding what it is and how it can be effectively executed. Instructional Systems Design (ISD) is one of the major activities carried out by educational technologists. Both ISD and design research deal with the enactment of design to improve educational practice. This paper describes the differences and similarities between these two activities and addresses the implications of these differences and similarities for educational technology researchers and practitioners.     

研究性学习活动的设计与实施

随着新课程计划的实施,作为培养创新精神和实践能力的研究性学习活动已在不少省市展开,为了避免活动的盲目性,本文就研究性学习活动设计与实施中的目标、内容、程序及评价等作必要的探讨.     

A theoretically driven teaching and research framework: learning technologies and educational practice

The emphasis of this paper is on the derivation of design principles from qualitative analysis of student reflections based on their participation in authentic, collaborative and technology-mediated activities. This paper reports on the initial phases of a design-based research project at a comprehensive university in South Africa where the authentic nature of the activities contributed to the depth of student reflections and ultimately led to meaningful design principles based entirely on the experiences of students. Students’ reflective reports were analysed for their content using simple inductive coding techniques leading to the identification of themes and the derivation of design principles from further refinement of these themes. These design principles, therefore, have their origins in the specific theories underpinning the learning activities that were used in the teaching and learning process, and contribute once again in an innovative way to this same body of knowledge. This theory-driven praxis thus makes a contribution to both educational theory and teaching practice which is applicable across a variety of contexts and sectors worldwide. More importantly, this can be seen as meaningful research that is socially responsible, with a high theoretical and practical value, and of relevance to a wider international audience.     

Moths,candles and fires: examining dance as creative practice research in a Master's degree

This article reflects on the purposes and functions of examination criteria for Masters' research degrees with creative practice components. With a particular focus on dance projects that incorporate both a performed and a written expression of the research, we consider how a rubric for creative practice research degrees might address formative and summative assessment purposes and clarify goals for learners, supervisors, examiners and academic institutions. Our discussion is situated within a South Pacific postgraduate learning context and is informed by post-colonial and post-Cartesian concerns over how dance knowledge is recognized in global academia. Much current literature has argued how particular research methods and diverse approaches to the presentation of research outputs can destabilize dominant, logocentric methods of valuing knowledge. Our queries extend this argument into the actual quantifiable measuring and evaluation of such knowledge, which Masters' grading necessitates in order to maintain its value as an institutional currency. This leads to the reflections on the rubric for Masters' degrees with Creative Practice components that is currently used within the Dance Studies programme of the University of Auckland, and how this rubric might be seen as contributing to an evolving ‘cultivated’ community of practice within postgraduate examination. The rubric continues to grow through ongoing consultation with postgraduate students, supervisory staff, internal and external examiners and international experts in creative arts academia. Our reflections extend the argument for how creative practice might be further rationalized within academia.     

Reconciling rigour and impact by collaborative research design: study of teacher agency

This paper illustrates a new way of working collaboratively on the development of a methodology for studying teacher agency for social justice. Increasing emphasis of impact on change as a purpose of social research raises questions about appropriate research designs. Large-scale quantitative research framed within externally set parameters has often been criticised for its limited potential for capturing the contexts and impacting change, while smaller, locally embedded, mostly qualitative inquiries have been questioned on the grounds of their limited generalizability and sometimes compromising research rigour. New ways of working collaboratively are increasingly explored as a way of reconciling research rigour and impact. The paper presents the procedures for designing a study that is both methodologically rigorous and potentially impactful. Twelve researchers, practitioners and policy-makers in Scotland were extensively involved in designing a mixed-method study of teacher agency for social justice. The Critical Communicative Methodology was employed to establish egalitarian dialogue between researchers and practitioners. The procedures and the resulting research tools can be used in future studies, including large-scale quantitative analysis. The paper discusses the challenges of ownership, choice of methods, and knowledge transfer, that need to be addressed in these ways of working.     

Internet-Based Proctored Assessment: Security and Fairness Issues

The COVID-19 pandemic has accelerated the shift toward online learning solutions necessitating the need for developing online assessment solutions. Vendors offer online assessment delivery systems with varying security levels designed to minimize unauthorized behaviors. Combating cheating and securing assessment content, however, is not solely the responsibility of the delivery system. Assessment design practices also effectively minimize cheating and protect content. In developing online assessment solutions, organizations also must strive to ensure that all students have the opportunity to test.     

Evaluating Students’ Performance in Responding to Art: The Development and Validation of an Art Criticism Assessment Rubric

This article reports on the development and validation of a rubric for assessing students’ written responses to artworks. Since the implementation of the Hong Kong New Senior Secondary Curriculum in 2009, art educators have seen responding to artworks as increasingly important. In this context, the Art Criticism Assessment Rubric (ACAR) was developed. On the basis of Feldman's and Geahigan's theories of art criticism, eight evaluation criteria were identified. The inter‐rater reliability (IRR) of the ACAR was examined. A preliminary IRR test was conducted and an excellent intra‐class correlation coefficient (ICC) value of .91 was obtained. For the main study, six independent raters, who were divided into three groups of two, were trained and invited to rate 87 art criticism essays written by students from eight secondary schools. Most dimensions of the ACAR achieved good ICC values. The results show that the ACAR is an acceptable rubric for providing a reliable assessment of students’ written responses to artworks. However, two dimensions, Originality and Balanced Views and Application of Aesthetic and Contextual Knowledge, obtained poor ICC values. This may be owing to the lack of consensus on the definition of originality and the raters' unfamiliarity with the concept of aesthetic knowledge. The researchers suggest that dimension‐specific samples rated from high to low scores should be provided in raters’ training.