Does continuous assessment in higher education support student learning?

A distinction is often made in the literature about “assessment of learning” and “assessment for learning” attributing a formative function to the latter while the former takes a summative function. While there may be disagreements among researchers and educators about such categorical distinctions there is consensus that both types of assessment are often used concurrently in higher education institutions. A question that often arises when formative and summative assessment practices are used in continuous assessment is the extent to which student learning can be facilitated through feedback. The views and perceptions of students and academics from a discipline in the Humanities across seven higher education institutions were sought to examine the above question. A postal survey was completed by academics, along with a survey administered to a sample of undergraduate students and a semi-structured interview was conducted with key academics in each of the seven institutions. This comparative study highlights issues that concern both groups about the extent to which continuous assessment practices facilitate student learning and the challenges faced. The findings illustrate the need to consider more effective and efficient ways in which feedback can be better used to facilitate student learning.     

Does spatial awareness training affect anatomy learning in medical students?

Spatial ability (SA) is the cognitive capacity to understand and mentally manipulate concepts of objects, remembering relationships among their parts and those of their surroundings. Spatial ability provides a learning advantage in science and may be useful in anatomy and technical skills in health care. This study aimed to assess the relationship between SA and anatomy scores in first- and second-year medical students. The training sessions focused on the analysis of the spatial component of objects' structure and their interaction as applied to medicine; SA was tested using the Visualization of Rotation (ROT) test. The intervention group (n = 29) received training and their pre- and post-training scores for the SA tests were compared to a control group (n = 75). Both groups improved their mean scores in the follow-up SA test (P < 0.010). There was no significant difference in SA scores between the groups for either SA test (P = 0.31, P = 0.90). The SA scores for female students were significantly lower than for male students, both at baseline and follow-up (P < 0.010). Anatomy training and assessment were administered by the anatomy department of the medical school, and examination scores were not significantly different between the two groups post-intervention (P = 0.33). However, participants with scores in the bottom quartile for SA performed worse in the anatomy questions (P < 0.001). Spatial awareness training did not improve SA or anatomy scores; however, SA may identify students who may benefit from additional academic support.     

Does the extent of problem familiarity influence students’ learning in problem-based learning?

This study investigated the relationship between problem familiarity and students’ learning in a problem-based course. Problem familiarity in this study refers to the extent to which a problem fits with students’ prior knowledge and experiences. As part of regular course work, 172 students were given two problems on different occasions. These problems varied in the extent of problem familiarity. Students’ report of their learning activities and tutor’s assessment of the students’ learning were collected at the end of the problems. Results showed that both the students and tutors found the familiar problem to be better for learning. However, some elements of unfamiliarity may not be all that bad. For instance, unfamiliarity may result in more questioning, thinking and reasoning. However, this did not contribute to significant difference in terms of overall critical reasoning. Results also indicated a need to closely examine students’ critical reasoning (in terms of evaluating multiple perspectives) and students’ collaborative learning (in terms of brainstorming and discussion) when given familiar/unfamiliar problems.     

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.     

Does multimedia learning theory extend to middle-school students?

The purpose of this study is to translate principles of multimedia learning from college-age readers to middle grade students, when reading science texts with a supporting diagram. In this experimental study, sixth-grade students (n = 180) were randomly assigned to display conditions before reading. Each student read two explanatory sciences passages, a life-science and a physical science text. Passages were accompanied by either no illustrations (control), illustrations of the cycle with labels for each part (parts), illustrations of the cycle with labels for each major process (steps), or illustrations showing the labels for each part and each major process (parts and steps). Additionally, there were two text conditions in which half of the students read standard text (control) and half read texts with cues which indicated to students when to access the diagrams (cued). Through ANOVA analysis, in the life-science text students showed modest improvement (partial η² = .18) from the addition of diagrams, with the parts diagram and the steps diagram outperforming the control. In the physical science text, students did not receive benefit from the diagrams. Findings did not replicate results from college-age readers to younger readers, nor between the two texts with younger readers. These results raise concern for the application of multimedia design theory to classroom practice.     

Does peer assessment promote student learning? A meta-analysis

Abstract

In recent years, there has been an increasing use of peer assessment in classrooms and other learning settings. Despite the prevailing view that peer assessment has a positive effect on learning across empirical studies, the results reported are mixed. In this meta-analysis, we synthesised findings based on 134 effect sizes from 58 studies. Compared to students who do not participate in peer assessment, those who participate in peer assessment show a .291 standard deviation unit increase in their performance. Further, we performed a meta-regression analysis to examine the factors that are likely to influence the peer assessment effect. The most critical factor is rater training. When students receive rater training, the effect size of peer assessment is substantially larger than when students do not receive such training. Computer-mediated peer assessment is also associated with greater learning gains than the paper-based peer assessment. A few other variables (such as rating format, rating criteria and frequency of peer assessment) also show noticeable, although not statistically significant, effects. The results of the meta-analysis can be considered by researchers and teachers as a basis for determining how to make effective use of peer assessment as a learning tool.     

What is open learning?

The recent change in this journal's title, from Teaching at a Distance to Open Learning, poses obvious questions of definition. What is open learning? And how does it relate to distance education and adult education? In this article, Roger Lewis, Head of Educational Development Unit at North Staffordshire Polytechnic, makes use of his considerable and varied experience of working with the Open University, the Council for Educational Technology, the Open Tech, the National Extension College and other practitioners of open learning in providing some answers. Readers should find the analytical tools and examples included in the article of considerable use in examining their own experiences in the open learning field.     

Multimethod assessment of self-regulated learning in college students: different methods for different components?

Instructional Science - Although self-regulated learning (SRL) is seen as highly relevant for successful college learning, college students oftentimes show a lack in SRL abilities. Therefore, it...     

Does student self‐assessment empower or discipline students?

Student self‐assessment is a popular practice for enhancing student empowerment in the assessment process. However, in recent times various writers have questioned whether the practice of student self‐assessment automatically enhances student autonomy. Some writers have even warned that students' participation in the assessment process may discipline, rather than empower, students. How can student self‐assessment be practised in a way that empowers its students instead of disciplining or controlling them? It is argued that student empowerment can only be realized if the ways that power is exercised over students in self‐assessment practices are first understood. This paper examines the issues of power that underlie student self‐assessment practices and analyses how different notions of power enhance or undermine student empowerment. The notion of the teacher's unilateral power as the basis for student self‐assessment is critically examined against three contrasting notions of power in student self‐assessment: sovereign power, epistemological power and disciplinary power.     

A collaborative assessment of students’ placement learning

The research informing this paper set out to investigate the qualitative experiences of students, supervisors and tutors involved in a summative collaborative assessment of placement learning on an undergraduate professional qualifying programme in the UK. Analysis of data gathered through semi-structured interviews provides valuable insights into its use. In particular, engagement in a collaborative assessment brings a degree of exposure for all concerned. Participants need to feel ‘able’ to participate, and ability to participate is linked to notions of agency and autonomy. Collaborative assessment design needs to acknowledge power relations and include strategies to reduce imbalances. The importance of recognising and working with the complexities of assessment should not be underestimated. Criterion referencing offers a guide to support consistency but can also falsely present assessment as mechanistic and instrumental. The research shows that it is in the ‘doing’ of assessment that a ‘shared language’ is developed. It demonstrates the value of active student engagement in assessment in relation to the construction and use of theory, developing reflective practice and assessment literacy. The collaborative assessment provides students with the opportunity to experience a ‘performance of understanding’ in a community of practice that, whilst pressured, reflects ‘real’ professional life.     

Characterising programme‐level assessment environments that support learning

This article outlines a methodology for characterising features of programme‐level assessment environments so that the relationship between features of the assessment environment and students’ learning response can be studied. The methodology was developed through the detailed case study of nine undergraduate degree programmes: one in each of three contrasting discipline areas in each of three contrasting universities. Each case study involved examination of course documentation, interviews with academics and interviews with students, following which each degree programme was coded in relation to a range of features of the assessment environment, such as the proportion of marks derived from examinations and the volume and timeliness of feedback on assignments. Programmes were found to differ profoundly in terms of variables that are known to have implications for student‐learning processes. They also differed widely in the extent to which they illustrated the application of conventional wisdom about curriculum design, embodied in national quality assurance guidelines and the Bologna Agreement. Programmes were found to have either a high volume of summative assessment or a high volume of formative‐only assessment, but never both at the same time. Programmes also differed in the mechanisms used to make goals and standards clear, having either highly explicit curriculum design or high volumes of written and oral feedback, but never both at the same time. The findings suggest that there are distinctive programme‐level assessment environments that operate in quite different ways despite all programmes studied being subject to the same quality assurance code of practice.     

Comparison of learning and study strategies of traditional and open‐learning‐system students in Turkey

This study compares the learning and study strategies of the students of the Faculty of Administrative Sciences of Cukurova University taking traditional on‐campus courses and those at the Open Learning Faculty of Anadolu University following an open learning programme. The evaluation was based on the Learning and Study Strategies Inventory (LASSI), developed by Claire E. Weinstein (1987), together with a separate questionnaire to elaborate the findings.

Findings reveal that students of both groups have considerably lower scores, especially in the affective domain, in relation to the other college students answering the same items in previous evaluations. However no important difference has been found between the groups belonging to the traditional and open learning systems evaluated in this study.     

Design principles for support in developing students’ transformative inquiry skills in Web-based learning environments

The aim of this explorative study was to find the factors limiting sixth-grade learners’ outcomes in acquiring skills related to the transformative inquiry learning processes as well as to analyse the interrelations between inquiry skills in order to develop an optimal support system for designing Web-based inquiry learning environments. A Web-based learning environment ‘Young Scientist’ was developed and applied to the domain of integrated science. The skill of identifying the correct research questions appeared to be a prerequisite for formulating research questions. When students understand how to identify the research questions correctly, there is no need for supporting formulation of research questions or hypotheses. We have found that at the stages of analysing data and inferring, students have to be provided with the tools to activate their prior knowledge, assignments help to organize their work into manageable sections, and reflective support or adaptive feedback to relate the results achieved with the process of inquiry. Our findings demonstrate that the effectiveness of inquiry learning can be strongly influenced by regulative support; however, a particular level of initial inquiry knowledge in theoretical context is also needed in order to develop inquiry skills in a situational context.     

Motivating learners in open and distance learning: do we need a new theory of learner support?

This paper calls for a new theory of learner support in distance learning based on recent findings in the fields of learning and motivational psychology. It surveys some current learning motivation theories and proposes that models drawn from the relatively new field of Positive Psychology, such as the ‘Strengths Approach’, together with Dweck’s Self Theory and Anderson’s insistence on proactive support, could be developed into a ‘Proactive Motivational Support’ theory. Such a theory might enable distance educators to support learners more successfully than existing models of learning skills development and remedial support. The paper then reports on further findings from experiments (one previously described in Open Learning), using these approaches in the UK Open University. These findings confirm the previous results, showing significant increases in retention that, in the context of the Open University’s funding arrangements, appear to be at least self‐financing.     

Mapping the quality of feedback to support students’ learning in lower secondary classrooms

Feedback, talk and joint activities that build on knowledge and shared aims are powerful supports for learning. However, several studies of lower secondary students’ perceptions of teachers’ support indicate low teacher support. To investigate the quality of learning support, a content analysis instrument (CLASS) was used in analyses of 56 video-recorded lessons from 28 teachers in four lower secondary schools. This has allowed for a mapping of dimensions of quality feedback interactions, and results show significant and very strong inter-item relationships. The lessons analysed are characterized by a positive climate, and teachers emphasize encouragement, but feedback is found to be more encouraging than learning oriented. The results support previous studies based on students’ perceptions, but provide insights into practices that are necessary if a goal is to influence teachers’ professional learning.     

Does the design of learning outcomes matter from students’ perspective?

Learning outcomes have gained more attention in the development of higher education course unit programmes. This study sought to understand how the design of learning outcomes relates to students’ perceptions of their motivation, satisfaction, engagement and achievement of the learning outcomes. The learning outcomes from 78 course units were coded to reflect the level of cognitive demand according to Bloom’s Taxonomy and the attended students (n = 1329) were surveyed regarding their perceptions of their achievement of the learning outcomes. The results indicated that the lowest four levels of Bloom’s Taxonomy were most commonly used in the design of learning outcomes, the highest level was not used at all. The levels of learning outcomes related to students’ perceptions of their achievement of learning outcomes, motivation, satisfaction and engagement. The results demonstrated that students were more likely motivated, satisfied, engaged to achieving learning outcomes, which were designed at higher levels of cognitive demand.     

Teacher students’ meaningful learning in widening learning worlds

ABSTRACT

This qualitative metasynthesis investigated students’ meaningful learning experiences amid andragogical teacher education programmes. The programmes catered to the specific learning needs of adult students in the context of university-based teacher education. This study aimed to provide a wider picture of the frames of students’ meaningful learning in andragogical teacher education programmes. The findings revealed three major learning worlds of adult students’ meaningful learning, which formed a common system widening from the professional awakening to the transformative community and agency in society. Based on the findings, this study argues that in andragogical teacher education, which emphasises collaboration and networking in accordance with the current trend in higher education, teacher students may become empowered participants and active agents in society. The findings can be used in planning curricula, and developing programmes of higher education in general and teacher education in particular.     

Assessment for learning: Why assessment does not always support student teachers’ learning

Numerous studies have dealt with the importance of assessment for learning (AfL) in higher education, arguing that AfL helps to promote students’ learning and supports them in becoming self-directed learners. The majority of the recent literature focuses mainly on the role of individual aspects of AfL. Less attention has been paid to the interrelation of different aspects of AfL. The current study argues that AfL combines authentic assessment tasks given to students, feedback from teacher educators, peer assessment and self-assessment. The aim of the current study is to understand student teachers’ experiences of AfL practices in higher education, focusing on the reasons why student teachers do not always find the practices supportive. The study conducted semi-structured interviews with 20 student teachers and analysed them using an inductive content analysis method. The findings indicate that student teachers have had some experiences with individual aspects of AfL. A critical finding is that, when teacher educators continually dominate in the assessment process, summative assessment remains the principal way of assessing student teachers, thus hindering the supportive role of other AfL practices. The current study also finds that learning in higher education is still quite grade-centred, which affects student teachers’ learning.