Assessment rubrics: towards clearer and more replicable design,research and practice

‘Rubric’ is a term with a variety of meanings. As the use of rubrics has increased both in research and practice, the term has come to represent divergent practices. These range from secret scoring sheets held by teachers to holistic student-developed articulations of quality. Rubrics are evaluated, mandated, embraced and resisted based on often imprecise and inconsistent understandings of the term. This paper provides a synthesis of the diversity of rubrics, and a framework for researchers and practitioners to be clearer about what they mean when they say ‘rubric’. Fourteen design elements or decision points are identified that make one rubric different from another. This framework subsumes previous attempts to categorise rubrics, and should provide more precision to rubric discussions and debate, as well as supporting more replicable research and practice.     

Using rubrics in student self-assessment: student perceptions in the English as a foreign language writing context

The instructional value of rubrics for promoting student learning and aiding teacher feedback to student performance has been extensively researched in th educational literature. There is, nonetheless, a dearth of studies on students’ rubric use in second/foreign language contexts, and fewer studies have investigated the factors affecting rubrics’ effectiveness for promoting student learning. The paper reports a classroom-based inquiry into students’ perceptions of rubric use in self-assessment in English as a Foreign Language context and the factors moderating its effectiveness. Eighty students at a Chinese university participated in the study. The data collected included their reflective journals and six case study informants’ retrospective interviews. Results showed that the rubric was perceived as useful for fostering the students’ self-regulation by guiding them through the stages of goal-setting, planning, self-monitoring and self-reflection. Both within-rubric and rubric-user factors were identified as affecting the rubric’s effectiveness in student self-assessment. The findings are discussed with reference to the design features of rubrics. Implications are drawn for formative rubric use in student self-assessment.     

Guided portfolio writing as a scaffold for reflective learning in in-service contexts: a case study1

Language is widely recognized as an inescapable mediating tool for professional learning, and with this text we want to contribute to a better understanding of the particular role that guided writing can play in in-service professional reflective learning. We analysed one pre-school teacher’s written portfolio, the construction of which was guided to scaffold deep thinking about (and the transference of theory into) practice during participation in an in-service program about language education. Our case study shows that the writing process sustained robust learning about professional knowing, doing and learning itself: The teacher elaborated an integrative ethical understanding of the discussed theory, fully experienced newly informed practices and assessed her own learning by using theory to confront her previous knowledge and practices. Throughout the portfolio, the learning stance revealed by her voice varied accordingly. The study illustrates the potential of guided writing to scaffold reflective learning in in-service contexts.     

Scaffolding and mediating for creativity: suggestions from reflecting on practice in order to develop the teaching and learning of gymnastics

This article takes a reflective stance on the development of practice in scaffolding and mediating for creativity and potentially better performance in gymnastics. The pedagogical approach outlined illustrates how an experienced practitioner can adopt mediational (rather than meddling) and scaffolding techniques to focus on supporting the development of creativity. Various teaching tactics and their influences on the nature and direction of learning are described. Dialogic reflections from the students as well as examples of task outcomes offer insight into the impact of this approach on learning processes and performance outcomes. Reflections-in-action, on-action and post-action from this study offer suggestions about how a learning programme could be re-focused, tasks structured and mediational approaches adapted to support more creativity in the teaching and learning of gymnastics specifically and in learning in general.     

Putting Rubrics to the Test: The Effect of a Model, Criteria Generation, and Rubric-Referenced Self-Assessment on Elementary School Students' Writing

The purpose of this study was to investigate the effect of reading a model written assignment, generating a list of criteria for the assignment, and self-assessing according to a rubric, as well as gender, time spent writing, prior rubric use, and previous achievement on elementary school students' scores for a written assignment (N = 116). Participants were in grades 3 and 4. The treatment involved using a model paper to scaffold the process of generating a list of criteria for an effective story or essay, receiving a written rubric, and using the rubric to self-assess first drafts. The comparison condition involved generating a list of criteria for an effective story or essay, and reviewing first drafts. Findings include a main effect of treatment and of previous achievement on total writing scores, as well as main effects on scores for the individual criteria on the rubric. The results suggest that using a model to generate criteria for an assignment and using a rubric for self-assessment can help elementary school students produce more effective writing.     

An account of practice: employing drawings and stories to enable reflective learning

Facilitating reflection for personal growth in a business context requires careful thought and planning. As a learning process, Business-Driven Action Learning (BDAL) has two points of focus. It aims to firstly identify and make recommendations regarding a business challenge, whist secondly facilitating potentially significant individual personal learning and growth. The research reported in this account of practice focused on personal learning by exploring the potential for individuals to experience transformative learning during a BDAL process. During the research I used a number of methods including hand-drawn images and in-depth interviews to enable the writing of interpretive stories. Findings revealed that participants found reflection during and after the BDAL beneficial. They gained insights into what they learnt, what changed since the BDAL experience and also what they still need to focus on. The findings have practical implications for the action learning facilitator of BDAL during management development programmes. One of the ways action learning facilitators can initiate the reflection process of new participants is to share the experience of previous participants. ‘Sandy’s interpretive story’, contained in this article, is such an example and includes hand-drawn images of her learning journey. I argue that the use of interpretative stories helps new participants understand programme challenges and stimulates their individual learning journey.     

Bridging assessment and achievement: Repeated practice of self-assessment in college english classes in Taiwan

Abstract

Grounded on the notion of observational learning in social cognitive learning theory, this study evaluated the effectiveness of repeated self-assessment on English-as-a-foreign-language learners’ oral performance and the perceptions of the students and the instructor of this practice. Ninety-seven students from three classes in a Taiwanese college participated in this study. The classes experienced five trials of self-assessment in which they used cell phones to record their oral responses. Instead of simply rating their own recordings, the students were provided questions to guide their examination of key components of their own talk. The results show that the students’ oral performance and evaluation abilities both improved over time, and they highly valued opportunities to detect their errors and observe their real learning outcomes. The findings suggest that self-assessment bridged the gap between repeated practice and English learning by allowing the students to reflect upon their performance, find their weaknesses, adjust their following talk, and recognize their learning outcomes.     

Boosting vocabulary learning through self-assessment in an English language teaching context

This study explores the influence of self-assessment of vocabulary competence on a group of students’ oral fluency. Twenty-four young adult learners participated in a learning process that promoted their oral skills and vocabulary development. Self-assessment was mainly examined through the analysis of students’ learning logs, field notes and artefacts in the form of audio recordings. By contrasting and analysing data sources, and the results obtained from pre- and post-speaking tasks, through a mixed method design approach, it was found that when students self-assess, they are able to acknowledge their learning strengths and drawbacks. This practice enabled students to set learning commitments, use learning strategies that also allow them to raise awareness and take further actions, while self-monitoring them. Consequently, initial improvements in students’ oral fluency development were evinced through the implementation of a systematic cycle that applies self-assessment as a formative assessment source. Thus, the students entered a process in which they gradually become more able to self-monitor, judge and react towards their own gains in language and learning. Findings highlighted the value of goal setting as an essential component in self-assessment. Further longitudinal studies may well support the long-lasting effects of this strategy in similar educational contexts.     

Gender differential in self-assessment: a fact neglected in higher education peer and self-assessment techniques

This paper analyses student cross-assessment with a gender perspective in three different courses along several academic years in an engineering school in Spain. The aim of this study is to contribute to a wider understanding of the hidden effect of gender in higher education by testing if gender has any effect on the way students assess others and themselves. Previous research has demonstrated sex differences in self-image and self-confidence, which are both closely related to self-esteem and same- and other-sex esteem. It follows that gender effects might be expected in investigations on peer and self-assessment (PSA), as both techniques are closely linked to self-image and self-confidence. But a critical review of the literature on PSA with a gender perspective reveals that findings on this field are inconclusive, thus more research is needed. The analysis of the results of our study reveals that women judge themselves too harshly. As the literature on PSA does not show great concern for gender issues, some tentative suggestions are proposed to support PSA with gender-awareness sessions.     

Linking parental scaffolding with self-regulated learning in Chinese kindergarten children

The current study aimed to examine the relationships between dimensions of parental scaffolding and children's self-regulated learning (SRL). One hundred and thirty Chinese kindergarten children participated in a range of problem-solving tasks with their parents and independently. Parent-child interactions and child-alone behaviours were video-recorded for an in-depth observational analysis. Parental cognitive support, emotional support, and contingency were coded in parent-child interactions. Children's cognitive, metacognitive, and motivational strategic behaviours and task performance were coded and assessed within the context of child-alone tasks. Results showed that contingency was particularly important for children's SRL. Parental contingency was the only independent predictor of children's SRL among the three aspects of parental scaffolding and mediated the effect of parent education levels on children's SRL.     

Teaching writing: a multilayered participatory scaffolding practice

This article adds to the research on teachers' writing pedagogy. It reviews and challenges the research literature on scaffolding as an instructional practice and presents a more inclusive framework for analysis. As student participation and voice were absent from much of the literature, a participatory scaffolding framework was developed to observe, analyse and interpret how one teacher and her primary school aged students co‐constructed learning to write. The case study revealed that the scaffolding interactions were complex, recursive and responsive to students' learning. The teacher wove multiple layers of scaffolding, encouraging student talk and metacognitive awareness, thus creating a 'magic space’ where minds could meet allowing negotiation and handover.     

Using technology for enhancing reflective writing,metacognition and learning

There exists broad agreement on the value of reflective practice for personal and professional development. However, many students in higher education (HE) struggle with the concept of reflection, so they do not engage well with the process, and its full value is seldom realised. An online resource was developed to facilitate and structure the recording, storage and retrieval of reflections with the focus on facilitating reflective writing, developing metacognitive awareness and, ultimately, enhancing learning. Ten undergraduate students completed a semi-structured questionnaire prior to participating in a focus group designed to elicit a common understanding of reflective practice. They maintained reflective practice online for 6 weeks and participated in post-study individual interviews. Findings provide evidence for the positive acceptance, efficiency and effectiveness of the intervention. Using a structured approach to online reflective practice is empowering and ultimately enhances undergraduate learning through the development of metacognition.     

Assessment as learning: blurring the boundaries of assessment and learning for theory,policy and practice

This paper explores assessment and learning in a way that blurs their boundaries. The notion of assessment as learning (AaL) is offered as an aspect of formative assessment (assessment for learning). It considers how pupils self-regulate their own learning, and in so doing make complex decisions about how they use feedback and engage with the learning priorities of the classroom. Discussion is framed from a sociocultural stance, yet challenges some of the perspectives that have widely become accepted. It offers three new views to help explore the concept of AaL: understanding feedback; understanding the learning gap; and exploring vocabularies of assessment. Pragmatically, the ideas examined suggest that teachers may need to consider less about focused and directive feedback, but more about how learners interpret and understand feedback from their self-regulatory and self-productive identities and how vocabularies for assessment can be more collaboratively shared in learning contexts.     

Initiating self-assessment strategies in novice physiotherapy students: a method case study

Student self- and peer-assessment strategies ideally are instigated early in programmes for health professionals. This study presents an innovative method of stimulating critical evaluation of clinical skills learned in the practical class setting for first year physiotherapy students. Twice in the semester (beginning and end) students assessed mock clinical demonstrations using examples of two widely discrepant approaches to evaluation of clinical skills: (1) a detailed analytical rubric of listed criteria and (2) a broad holistic, competency-based proforma. Anonymously, 67 students rated the usefulness of the activity to learning. They provided estimates of individual self-efficacy to make fair and reliable judgements in grading the demonstrations using the rubric and proforma and contemporaneous levels of personal stress with the perceived stress scale_10. Students reported that the activities aided their learning of course work on both occasions. By the end of semester the change in students’ self-efficacy to grade the demonstrations with the proforma was significantly greater than that for the rubric. The changes flag a deep understanding of the analytic and holistic methods used by class examiners and future clinical educators. Practical experience with assessing clinical performances enhanced student familiarity with the practical assessment and was associated with a reduction in pre-examination personal stress levels.     

Optional self-assessment: some tensions and dilemmas

Self-assessment is promoted as an ideal form of assessment, particularly for adult learners as they need to be able to make judgements about how well they are doing something – whether it is related to learning, work or social interactions. However, there is also opposition to summative self-assessment on the grounds that students cannot validly and reliably assess their own work. At Massey University students enrolled in adult education qualifications are offered the opportunity to self or peer assess. This small-scale project investigated 472 student responses to self-assessment in this context. Findings show that 38% self-assessed by making comments on their work, and 25% awarded themselves a grade. A statistical analysis showed there was no significant difference (p > .01) between the self-assessed and teacher-assessed grades. Data also showed that higher achieving students tended to underrate and lower achieving to overrate. However, it is suggested that this may be a function of the grading scale rather than necessarily reflecting students’ capability to self-assess. Some tensions and dilemmas around optional self-assessment are identified and discussed.     

Accuracy in student self-assessment: directions and cautions for research

Student self-assessment is a central component of current conceptions of formative and classroom assessment. The research on self-assessment has focused on its efficacy in promoting both academic achievement and self-regulated learning, with little concern for issues of validity. Because reliability of testing is considered a sine qua non for the validity of assessment interpretations, and research into the human ability to self-evaluate work raises concerns about the quality of students’ judgements, it is sensible to investigate the accuracy of students’ self-assessments. This article reviews relevant literature from educational psychology and psychometrics to define the need for a better understanding of accuracy in self-assessment as well as to identify possible pitfalls in measuring accuracy that could undermine its effectiveness by, for example, trading the focus on formative feedback for summative scoring or rating. The article concludes with recommendations for the design of research on accuracy in self-assessment.     

Learning to be critically reflective: exploring fiction writing and adult learning

Many educators in adult, community and higher education contexts are concerned with fostering reflective learning amongst their students. This paper explores the concept of critical reflection and considers how engaging with fiction may be an innovative pedagogical approach to support critical learning opportunities. Drawing upon interviews with fiction writers, ways in which critical reflection may be encouraged in connection to reading and writing fiction are taken up by exploring three different thematic areas that relate to a Habermasian framework of knowledge constitutive approaches to learning. These different areas can be categorised as (a) technical-rational, (b) humanistic and (c) critical or emancipatory. The first of these considers critical reflection as a way to develop technical capacities as a creative writer. The next section takes up a humanistic framework to explore the value of individual and collective learning opportunities to enhance personal growth and critical reflection. The third area of discussion considers a deeper critical or emancipatory framework of learning through critical reflection which may lead to social change. The paper concludes by considering the value of arts-informed adult education approaches, such as those related to fiction writing, to enhance the development of critical reflection amongst adult learners.     

Teacher self-assessment of evidence-based classroom practices: preliminary findings across primary,intermediate and secondary level teachers

The limited implementation of evidence-based classroom practices and ways to provide effective professional development to address this challenge remain enduring concerns in education. Despite these concerns, there exists a well-established research literature on evidence-based practices for effective classroom management and instructional practices. One identified area of need is research investigating the extent that different classroom practices are more or less important across school and grade levels. The current study examined results from 612 general education teachers across school levels (323 primary, 208 intermediate, 81 secondary) who completed an on-line self-assessment of their use of evidence-based classroom practices. Statistical analysis found significant differences in teachers’ self-reported use of several evidence-based classroom practices across school levels (primary, intermediate, secondary). Implications are presented for using teacher self-assessment to inform professional development in schools.     

Competencies development and self-assessment in maintenance management e-training

The maintenance management function requires staff to possess a truly multidisciplinary set of skills. This includes competencies from engineering and information technology to health and safety, management and finance, while also taking into account the normative and legislative issues. This body of knowledge is rarely readily available within a single university course. The potential of e-learning in this field is significant, as it is a flexible and less costly alternative to conventional training. Furthermore, trainees can follow their own pace, as their available time is often a commodity. This article discusses the development of tools to support competencies development and self-assessment in maintenance management. Based on requirements arising from professional bodies’ guidelines and a user survey, the developed tools implement a dedicated maintenance management training curriculum. The results from pilot testing on academic and industrial user groups are discussed and user evaluations are linked with specific e-learning design issues.