Accuracy in student self-assessment: directions and cautions for research

Student self-assessment is a central component of current conceptions of formative and classroom assessment. The research on self-assessment has focused on its efficacy in promoting both academic achievement and self-regulated learning, with little concern for issues of validity. Because reliability of testing is considered a sine qua non for the validity of assessment interpretations, and research into the human ability to self-evaluate work raises concerns about the quality of students’ judgements, it is sensible to investigate the accuracy of students’ self-assessments. This article reviews relevant literature from educational psychology and psychometrics to define the need for a better understanding of accuracy in self-assessment as well as to identify possible pitfalls in measuring accuracy that could undermine its effectiveness by, for example, trading the focus on formative feedback for summative scoring or rating. The article concludes with recommendations for the design of research on accuracy in self-assessment.     

Using rubrics in student self-assessment: student perceptions in the English as a foreign language writing context

The instructional value of rubrics for promoting student learning and aiding teacher feedback to student performance has been extensively researched in th educational literature. There is, nonetheless, a dearth of studies on students’ rubric use in second/foreign language contexts, and fewer studies have investigated the factors affecting rubrics’ effectiveness for promoting student learning. The paper reports a classroom-based inquiry into students’ perceptions of rubric use in self-assessment in English as a Foreign Language context and the factors moderating its effectiveness. Eighty students at a Chinese university participated in the study. The data collected included their reflective journals and six case study informants’ retrospective interviews. Results showed that the rubric was perceived as useful for fostering the students’ self-regulation by guiding them through the stages of goal-setting, planning, self-monitoring and self-reflection. Both within-rubric and rubric-user factors were identified as affecting the rubric’s effectiveness in student self-assessment. The findings are discussed with reference to the design features of rubrics. Implications are drawn for formative rubric use in student self-assessment.     

Designing for impact: a conceptual framework for learning analytics as self-assessment tools

Abstract

Although it is frequently claimed that learning analytics can improve self-evaluation and self-regulated learning by students, most learning analytics tools appear to have been developed as a response to existing data rather than with a clear pedagogical model. As a result there is little evidence of impact on learning. Even fewer learning analytics tools seem to be informed by an understanding of the social context and social practices within which they would be used. As a result, there is very little evidence that learning analytics tools are actually impacting on practice. This paper draws on research in self-regulated learning and in the social practices of learning and assessment to clarify a series of design issues which should be considered by those seeking to develop learning analytics tools which are intended to improve student self-evaluation and self-regulation. It presents a case study of how these design issues influenced the development of a particular tool: the Learning Companion.     

Problem-based learning: student perceptions of its value in developing professional skills for engineering practice

ABSTRACT

This study explores students’ perceptions of the effectiveness of a Problem-Based Learning (PBL) design project, taken as part of a first-year engineering module, in developing professional skills needed for engineering practice. Students completed surveys before and after the PBL group project, and produced personal reflections on the process. The closed survey questions were analysed quantitatively and the main themes from the reflections outlined using General Inductive Analysis. Students rated themselves as having improved across a range of professional skills as a result of the project, with particular emphasis on teamwork, communication skills, understanding of the design process and self-directed learning. In addition, they highlighted improved confidence, as well as new friendships they developed, an important element of a module like this as they transition from secondary to higher education. They were particularly positive about the scaffolded approach taken within the PBL project in terms of its contribution to their learning.     

教育技术学专业学生的特色技能培养研究

目前,国内一些本科院校的教育技术学专业在学生专业技能培养方面缺乏特色,导致学生在就业时缺乏竞争力。提出了如何建立教育技术学专业特色技能培养体系,并结合案例阐述了实施的详细过程。     

A cyclical self-assessment process: towards a model of how students engage in self-assessment

While significant progress has been made on understanding the effects of student self-assessment, the processes by which these effects occur are much less studied. The present research identified the actions involved in a cyclical self-assessment process. In this qualitative study, 17 undergraduate students from a teacher education institute took part in in-depth interviews focusing on the common actions students normally undertake to self-assess. The findings identified the following three actions that are commonly undertaken in the self-assessment process: (1) determining the performance criteria, (2) self-directed feedback seeking and (3) self-reflection. The paper discusses the implications of the findings and concludes with suggestions for promoting student self-assessment.     

对我校体育大专生教学基本功的调查与分析

本文对我校体育大专生教学基本功调查中存在的组织教学、体育实践能力较差等问题进行了分析,并提出加强教学基本功训练的具体措施。     

The effects of peer judgements on teamwork and self-assessment ability in collaborative group work

A crucial determinant of the success or failure of collaborative group work is the effect of peer feedback interventions on learning. Research exploring such effects on developing soft skills is sparse. This study seeks to address whether peer feedback leads to enhanced teamwork behaviour and self-assessment ability, two skills highly sought after by employers. Specifically, this study examines the direct effect of formative performance rating and the mediating effect of praise and criticism in peer feedback messages on achievement in teamwork and self-assessment skills. The sample consists of quantitative and qualitative data from 98 students enrolled in business programmes using a particular form of collaborative group work. The paper finds a direct positive relationship between formative performance rating and summative self-assessment ability. It also finds that praise negatively mediates the relationship between formative performance rating and summative teamwork. Further analyses suggest that a significant proportion of comments provided is past rather than future-oriented. Potential strategies to overcome the limitations of current practices are discussed.     

Assessing heterogeneous student bodies using a methodology that encourages the acquisition of skills valued by employers

This work compares the results of three assessment systems used in two Spanish universities (the Universidad Politécnica de Madrid and the Universidad Católica de Ávila): the traditional system based on final examinations, continuous assessment with periodic tests and a proposed system (specially designed for heterogeneous student bodies) orientated towards motivating students. This third system involved dividing the syllabus into two different parts: a common core assessed by multiple choice tests, and a specialisation assessed by a literature review, the writing of an article and an oral presentation. The latter skills are highly valued by employers. The proposed system led to a greater pass rate than that achieved by students taking similar courses assessed by the more conventional systems. In addition, the results show that involving students in the assessment process increases their participation in their studies and generates a feeling of satisfaction and justice.     

Regulating the writing process in the teacher training practicum guided by feedback at the University of Barcelona

This is a longitudinal study that focuses on blog-writing for the purpose of self-assessing competencies, and which was carried out within the practicum of a degree course in Primary Education Teaching at the University of Barcelona. Over the course of three academic years, students and a teacher have used a Platform 2.0 program, specially created for the study, to reflect on their learning processes and to self-assess with respect to the benchmark competences selected. The results show, first, a significant increase in students’ levels of awareness of their own competences, thanks to the optimisation of their communicative competency to explain same, and guided by the teacher’s feedback; and second, a growing level of satisfaction that can be observed in the assessment surveys on the process implemented.     

The impact of digital skills on educational outcomes: evidence from performance tests

Digital skills are increasingly important for labour market outcomes and social participation. Do they also matter for academic performance? This paper investigates the effects of digital literacy on educational outcomes by merging data from the Italian National Assessment in secondary schools with an original data-set on performance tests of Internet skills for tenth-grade students. Our identification strategy relies on a rich set of individual, family, school and classroom control variables that are not commonly available in previous studies. The findings indicate that, overall, Internet skills have a positive impact on academic achievement. This effect is stronger for students with low academic performance or low family background. It is also stronger for students in technical or vocational schools.     

The purpose of the PhD: theorising the skills acquired by students

In the past decade there has been a marked push for the development of employability skills to be part of the PhD process. This push is generally by stakeholders from above and outside the PhD process, i.e. government and industry, who view skills as a summative product of the PhD. In contrast, our study interviewed stakeholders inside the PhD process – twenty final‐year, full‐time Australian PhD students – to provide a bottom‐up perspective into the skills question. Using grounded theory procedures we theorise the skills students develop during the PhD as a formative developmental process of acquiring intellectual virtues. Drawing on Aristotelian theory, we propose that theorising the PhD as a process of acquiring intellectual virtues offers a more robust and conceptually richer framework for understanding students’ development during the PhD than the instrumental focus on skills evident in contemporary debates.     

The assessment of learning skills in nursing student selection: A scoping review

Abstract

Higher education student selection has significant societal, institutional and individual impacts. Thousands of applicants apply only for nursing, one of the major higher education disciplines. As the nursing profession is characterised by cognitive requirements, higher education institutions assess the learning skills of nursing applicants. However, there has been no comprehensive analysis of learning skills assessment for nursing student selection. The purpose of this scoping review was to describe the assessment of learning skills in undergraduate nursing student selection. Five databases were systematically searched, and 24 studies published between 2006 and 2016 were included. Learning skills were most commonly assessed using standardised tests in the areas of language and communication, reasoning, mathematics and natural sciences. Overall scores of onsite selection methods were found to best predict future academic performance. The results indicate that higher education institutions may benefit from comprehensive assessment of learning skills in their selection processes. This assessment should focus on a wider range of cognitive aptitudes, including reasoning skills. This review focussed on nursing education, but the results may benefit other higher education disciplines due to the generic nature of learning skills and similar cognitive requirements of higher education studies. The results support the development of more comprehensive and valid methods for assessing learning skills.     

Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement

Student teachers are expected to develop their teaching skills sooner and more rapidly. However, a sound evaluation instrument that can be used to diagnose and monitor the skilfulness level to aid formative assessment of student teachers is still limited. This article is aimed to calibrate and validate a teaching skill evaluation instrument for use in secondary education. A total of 264 student teachers in the Netherlands participated in the study. Rasch and multilevel analyses were used. Results suggest that the evaluation instrument meets the restrictive assumptions of the Rasch model and has predictive value for academic engagement. This adds validation evidence and justifies the calibration of the evaluation instrument to be used for monitoring the development of teacher's teaching skills.     

Impact of interdisciplinary learning on the development of engineering students’ skills

ABSTRACT

This paper intends to present an interdisciplinary project carried out in a School of Engineering, and to refer to its effects in the development of students’ skills. This project’s main goal was to present students with an intellectually demanding challenge which implied overcoming the disciplinary barriers thus integrating knowledge to solve the problems they were challenged with. After the project had been concluded, a study was carried out using a qualitative methodology by conducting two focus groups (n?=?16). The main goal of this procedure was to get a better perception of (1) how students understood the project; (2) what kind of skills students considered to have developed throughout the project, and (3) the importance they have attributed to this kind of project. The results demonstrate that students are aware of the relevance of the project not only for their education process but also for the development of their skills. The results of the study also reveal that the students involved in the project have been capable of identifying the specific skills that the project work had intended to address and develop.     

Providing Students with Written Feedback on Their Assessment: A Collaborative Self-Study Exploring the Nexus of Research and Practice

This article is an account of a collaborative self-study of the process of providing written feedback on assessment to our teacher education students. Our five-year study grew out of concerns that written feedback might not always meet the learning needs of our students. The study was informed by on-going analysis of our reading of the relevant literature, our experience conducting a qualitative research study of students' perceptions of written feedback on assessment, and our professional conversations along the way. We became increasingly aware that our personal beliefs about learning and teaching underpin our respective approaches to providing written feedback on assessment. The process of critical reflection enabled us to achieve a congruence of professional practice that resolved our concerns about the nature of written feedback and enhanced our respective pedagogical practices.     

Exploring the moderating role of well-being on the adaptive link between self-assessment practices and learning achievement

Self-assessment practices can positively impact student learning and achievement. However, less is known about psychological moderators that can influence the impact that self-assessment practice may have on achievement. Integrating the dual processing self-regulation model and the self-assessment cycle framework, this study examined the role of students’ positive well-being in the link between self-assessment practices and domain-specific achievement scores. Hierarchical multiple regression analyses using data from 765 secondary school students revealed that self-assessment practices positively predicted achievement scores. Results also showed that positive well-being moderated the relationship between seeking internal feedback and self-reflection and achievement scores, in that the positive link between such self-assessment practices and achievement was stronger for students who had low or moderate well-being. These findings reinforce the impact of self-assessment on learning and add to the ecological validity of the dual processing self-regulation model. The discussion focuses on how these findings contribute to our understanding of how well-being could influence the adaptive role of internal and cognitive self-assessment practices in student learning. Limitations and future research directions are also discussed.     

Evaluating the impact of the Erasmus programme: skills and European identity

The primary purpose of this study lies in the evaluation of what students acquire from studies abroad in the context of credit mobility, in terms of competence development and personal growth, compared with that gained by students completing their study or internship in their home country. A pre-post test design, with both an experimental group and a control group, was employed for students participating in the Erasmus programme and those continuing their regular course of study at their home university. The findings obtained suggest that study abroad programmes have a positive impact on the perception of university students. The highest values among Erasmus students in terms of outcomes were attributed to cultural enhancement, personal development and foreign language proficiency. Study abroad has, furthermore, a Europeanising impact inasmuch as it affects the self-identity of students.     

The impact of video modelling on improving social skills in children with autism

Children with autism often show a lack of the interactive social skills that would allow them to engage with others successfully. They therefore frequently need training to aid them in successful social interaction. Video modelling is a widely used instructional technique that has been applied to teach children with developmental disabilities such as autism, and has been shown to be effective in teaching a variety of skills. The aim of this study is to evaluate the impact of video modelling on social skills development in children with autism living in the United Arab Emirates. The research was carried out by Mohammed Alzyoudi, AbedAlziz Sartawi and Osha Almuhiri, all from the United Arab Emirates University. A sample of five boys with autism was chosen to participate in this study. The boys ranged in age between five and seven years. Each child was asked to watch a videotape which presented two individuals interacting in a role‐play setting. One person portrayed a therapist giving cues, while the second portrayed a child acting appropriately and demonstrating correct social behaviours. The boys were then asked to complete a social task and their performance was compared against baseline assessment measures. The results of this study indicate that this is an effective instruction method. Results are discussed and implications for further research and practice are provided.