Does continuous assessment in higher education support student learning?

A distinction is often made in the literature about “assessment of learning” and “assessment for learning” attributing a formative function to the latter while the former takes a summative function. While there may be disagreements among researchers and educators about such categorical distinctions there is consensus that both types of assessment are often used concurrently in higher education institutions. A question that often arises when formative and summative assessment practices are used in continuous assessment is the extent to which student learning can be facilitated through feedback. The views and perceptions of students and academics from a discipline in the Humanities across seven higher education institutions were sought to examine the above question. A postal survey was completed by academics, along with a survey administered to a sample of undergraduate students and a semi-structured interview was conducted with key academics in each of the seven institutions. This comparative study highlights issues that concern both groups about the extent to which continuous assessment practices facilitate student learning and the challenges faced. The findings illustrate the need to consider more effective and efficient ways in which feedback can be better used to facilitate student learning.     

Does the extent of problem familiarity influence students’ learning in problem-based learning?

This study investigated the relationship between problem familiarity and students’ learning in a problem-based course. Problem familiarity in this study refers to the extent to which a problem fits with students’ prior knowledge and experiences. As part of regular course work, 172 students were given two problems on different occasions. These problems varied in the extent of problem familiarity. Students’ report of their learning activities and tutor’s assessment of the students’ learning were collected at the end of the problems. Results showed that both the students and tutors found the familiar problem to be better for learning. However, some elements of unfamiliarity may not be all that bad. For instance, unfamiliarity may result in more questioning, thinking and reasoning. However, this did not contribute to significant difference in terms of overall critical reasoning. Results also indicated a need to closely examine students’ critical reasoning (in terms of evaluating multiple perspectives) and students’ collaborative learning (in terms of brainstorming and discussion) when given familiar/unfamiliar problems.     

Does spatial awareness training affect anatomy learning in medical students?

Spatial ability (SA) is the cognitive capacity to understand and mentally manipulate concepts of objects, remembering relationships among their parts and those of their surroundings. Spatial ability provides a learning advantage in science and may be useful in anatomy and technical skills in health care. This study aimed to assess the relationship between SA and anatomy scores in first- and second-year medical students. The training sessions focused on the analysis of the spatial component of objects' structure and their interaction as applied to medicine; SA was tested using the Visualization of Rotation (ROT) test. The intervention group (n = 29) received training and their pre- and post-training scores for the SA tests were compared to a control group (n = 75). Both groups improved their mean scores in the follow-up SA test (P < 0.010). There was no significant difference in SA scores between the groups for either SA test (P = 0.31, P = 0.90). The SA scores for female students were significantly lower than for male students, both at baseline and follow-up (P < 0.010). Anatomy training and assessment were administered by the anatomy department of the medical school, and examination scores were not significantly different between the two groups post-intervention (P = 0.33). However, participants with scores in the bottom quartile for SA performed worse in the anatomy questions (P < 0.001). Spatial awareness training did not improve SA or anatomy scores; however, SA may identify students who may benefit from additional academic support.     

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.     

Does multimedia learning theory extend to middle-school students?

The purpose of this study is to translate principles of multimedia learning from college-age readers to middle grade students, when reading science texts with a supporting diagram. In this experimental study, sixth-grade students (n = 180) were randomly assigned to display conditions before reading. Each student read two explanatory sciences passages, a life-science and a physical science text. Passages were accompanied by either no illustrations (control), illustrations of the cycle with labels for each part (parts), illustrations of the cycle with labels for each major process (steps), or illustrations showing the labels for each part and each major process (parts and steps). Additionally, there were two text conditions in which half of the students read standard text (control) and half read texts with cues which indicated to students when to access the diagrams (cued). Through ANOVA analysis, in the life-science text students showed modest improvement (partial η² = .18) from the addition of diagrams, with the parts diagram and the steps diagram outperforming the control. In the physical science text, students did not receive benefit from the diagrams. Findings did not replicate results from college-age readers to younger readers, nor between the two texts with younger readers. These results raise concern for the application of multimedia design theory to classroom practice.     

What is open learning?

The recent change in this journal's title, from Teaching at a Distance to Open Learning, poses obvious questions of definition. What is open learning? And how does it relate to distance education and adult education? In this article, Roger Lewis, Head of Educational Development Unit at North Staffordshire Polytechnic, makes use of his considerable and varied experience of working with the Open University, the Council for Educational Technology, the Open Tech, the National Extension College and other practitioners of open learning in providing some answers. Readers should find the analytical tools and examples included in the article of considerable use in examining their own experiences in the open learning field.     

Multimethod assessment of self-regulated learning in college students: different methods for different components?

Instructional Science - Although self-regulated learning (SRL) is seen as highly relevant for successful college learning, college students oftentimes show a lack in SRL abilities. Therefore, it...     

A collaborative assessment of students’ placement learning

The research informing this paper set out to investigate the qualitative experiences of students, supervisors and tutors involved in a summative collaborative assessment of placement learning on an undergraduate professional qualifying programme in the UK. Analysis of data gathered through semi-structured interviews provides valuable insights into its use. In particular, engagement in a collaborative assessment brings a degree of exposure for all concerned. Participants need to feel ‘able’ to participate, and ability to participate is linked to notions of agency and autonomy. Collaborative assessment design needs to acknowledge power relations and include strategies to reduce imbalances. The importance of recognising and working with the complexities of assessment should not be underestimated. Criterion referencing offers a guide to support consistency but can also falsely present assessment as mechanistic and instrumental. The research shows that it is in the ‘doing’ of assessment that a ‘shared language’ is developed. It demonstrates the value of active student engagement in assessment in relation to the construction and use of theory, developing reflective practice and assessment literacy. The collaborative assessment provides students with the opportunity to experience a ‘performance of understanding’ in a community of practice that, whilst pressured, reflects ‘real’ professional life.     

Characterising programme‐level assessment environments that support learning

This article outlines a methodology for characterising features of programme‐level assessment environments so that the relationship between features of the assessment environment and students’ learning response can be studied. The methodology was developed through the detailed case study of nine undergraduate degree programmes: one in each of three contrasting discipline areas in each of three contrasting universities. Each case study involved examination of course documentation, interviews with academics and interviews with students, following which each degree programme was coded in relation to a range of features of the assessment environment, such as the proportion of marks derived from examinations and the volume and timeliness of feedback on assignments. Programmes were found to differ profoundly in terms of variables that are known to have implications for student‐learning processes. They also differed widely in the extent to which they illustrated the application of conventional wisdom about curriculum design, embodied in national quality assurance guidelines and the Bologna Agreement. Programmes were found to have either a high volume of summative assessment or a high volume of formative‐only assessment, but never both at the same time. Programmes also differed in the mechanisms used to make goals and standards clear, having either highly explicit curriculum design or high volumes of written and oral feedback, but never both at the same time. The findings suggest that there are distinctive programme‐level assessment environments that operate in quite different ways despite all programmes studied being subject to the same quality assurance code of practice.     

Comparison of learning and study strategies of traditional and open‐learning‐system students in Turkey

This study compares the learning and study strategies of the students of the Faculty of Administrative Sciences of Cukurova University taking traditional on‐campus courses and those at the Open Learning Faculty of Anadolu University following an open learning programme. The evaluation was based on the Learning and Study Strategies Inventory (LASSI), developed by Claire E. Weinstein (1987), together with a separate questionnaire to elaborate the findings.

Findings reveal that students of both groups have considerably lower scores, especially in the affective domain, in relation to the other college students answering the same items in previous evaluations. However no important difference has been found between the groups belonging to the traditional and open learning systems evaluated in this study.     

Design principles for support in developing students’ transformative inquiry skills in Web-based learning environments

The aim of this explorative study was to find the factors limiting sixth-grade learners’ outcomes in acquiring skills related to the transformative inquiry learning processes as well as to analyse the interrelations between inquiry skills in order to develop an optimal support system for designing Web-based inquiry learning environments. A Web-based learning environment ‘Young Scientist’ was developed and applied to the domain of integrated science. The skill of identifying the correct research questions appeared to be a prerequisite for formulating research questions. When students understand how to identify the research questions correctly, there is no need for supporting formulation of research questions or hypotheses. We have found that at the stages of analysing data and inferring, students have to be provided with the tools to activate their prior knowledge, assignments help to organize their work into manageable sections, and reflective support or adaptive feedback to relate the results achieved with the process of inquiry. Our findings demonstrate that the effectiveness of inquiry learning can be strongly influenced by regulative support; however, a particular level of initial inquiry knowledge in theoretical context is also needed in order to develop inquiry skills in a situational context.     

Motivating learners in open and distance learning: do we need a new theory of learner support?

This paper calls for a new theory of learner support in distance learning based on recent findings in the fields of learning and motivational psychology. It surveys some current learning motivation theories and proposes that models drawn from the relatively new field of Positive Psychology, such as the ‘Strengths Approach’, together with Dweck’s Self Theory and Anderson’s insistence on proactive support, could be developed into a ‘Proactive Motivational Support’ theory. Such a theory might enable distance educators to support learners more successfully than existing models of learning skills development and remedial support. The paper then reports on further findings from experiments (one previously described in Open Learning), using these approaches in the UK Open University. These findings confirm the previous results, showing significant increases in retention that, in the context of the Open University’s funding arrangements, appear to be at least self‐financing.     

Mapping the quality of feedback to support students’ learning in lower secondary classrooms

Feedback, talk and joint activities that build on knowledge and shared aims are powerful supports for learning. However, several studies of lower secondary students’ perceptions of teachers’ support indicate low teacher support. To investigate the quality of learning support, a content analysis instrument (CLASS) was used in analyses of 56 video-recorded lessons from 28 teachers in four lower secondary schools. This has allowed for a mapping of dimensions of quality feedback interactions, and results show significant and very strong inter-item relationships. The lessons analysed are characterized by a positive climate, and teachers emphasize encouragement, but feedback is found to be more encouraging than learning oriented. The results support previous studies based on students’ perceptions, but provide insights into practices that are necessary if a goal is to influence teachers’ professional learning.     

Does the design of learning outcomes matter from students’ perspective?

Learning outcomes have gained more attention in the development of higher education course unit programmes. This study sought to understand how the design of learning outcomes relates to students’ perceptions of their motivation, satisfaction, engagement and achievement of the learning outcomes. The learning outcomes from 78 course units were coded to reflect the level of cognitive demand according to Bloom’s Taxonomy and the attended students (n = 1329) were surveyed regarding their perceptions of their achievement of the learning outcomes. The results indicated that the lowest four levels of Bloom’s Taxonomy were most commonly used in the design of learning outcomes, the highest level was not used at all. The levels of learning outcomes related to students’ perceptions of their achievement of learning outcomes, motivation, satisfaction and engagement. The results demonstrated that students were more likely motivated, satisfied, engaged to achieving learning outcomes, which were designed at higher levels of cognitive demand.     

What is assessment for learning?

The idea that assessment is intrinsic to effective instruction is traced from early experiments in the individualization of learning through the work of Benjamin Bloom to reviews of the impact of feedback on learners in classrooms. While many of these reviews detailed the adverse impact of assessment on learning, they also indicated that under certain conditions assessment had considerable potential to enhance learning. It is shown that understanding the impact that assessment has on learning requires a broader focus than the feedback intervention itself, particularly the learner's responses to the feedback, and the learning milieu in which the feedback operates. Different definitions of the terms “formative assessment” and “assessment for learning” are discussed, and subsumed within a broad definition that focuses on the extent to which instructional decisions are supported by evidence. The paper concludes by exploring some of the consequences of this definition for classroom practice.     

Assessment for learning: Why assessment does not always support student teachers’ learning

Numerous studies have dealt with the importance of assessment for learning (AfL) in higher education, arguing that AfL helps to promote students’ learning and supports them in becoming self-directed learners. The majority of the recent literature focuses mainly on the role of individual aspects of AfL. Less attention has been paid to the interrelation of different aspects of AfL. The current study argues that AfL combines authentic assessment tasks given to students, feedback from teacher educators, peer assessment and self-assessment. The aim of the current study is to understand student teachers’ experiences of AfL practices in higher education, focusing on the reasons why student teachers do not always find the practices supportive. The study conducted semi-structured interviews with 20 student teachers and analysed them using an inductive content analysis method. The findings indicate that student teachers have had some experiences with individual aspects of AfL. A critical finding is that, when teacher educators continually dominate in the assessment process, summative assessment remains the principal way of assessing student teachers, thus hindering the supportive role of other AfL practices. The current study also finds that learning in higher education is still quite grade-centred, which affects student teachers’ learning.     

Should mobile learning be compulsory for preparing students for learning in the workplace?

From the contexts of current social, educational and health policy, there appears to be an increasingly inevitable “mobilisation” of resources in medicine and health as the use mobile technology devices and applications becomes widespread and culturally “normed” in workplaces. Over the past 8 years, students from the University of Leeds Medical School have been loaned mobile devices and smartphones and been given access to mobile‐based resources to assist them with learning and assessments as part of clinical activity in placement settings. Our experiences lead us to suggest that educators should be focusing less on whether mobile learning should be implemented and more on developing mobile learning in curricula that is comprehensive, sustainable, meaningful and compulsory, in order to prepare students for accessing and using such resources in their working lives.     

Self‐assessment in university assessment of prior learning procedures

Competency‐based university education, in which lifelong learning and flexible learning are key elements, demands a renewed vision on assessment. Within this vision, Assessment of Prior Learning (APL), in which learners have to show their prior learning in order for their goals to be recognised, becomes an important element. This article focuses on a first step in APL, namely students’ self‐assessment of their prior learning before entering university education. The main aim of the presented study is to examine the suitability of the use of self‐assessment in APL. First, in an explorative study, the main sources for self‐assessment are derived and the relation between sources and domain of study is investigated. Second, in a pre‐test post‐test research design, the hypothesis that students’ self‐assessment of prior learning related to a course changes after studying a domain‐specific course is tested. Pre‐test results reveal that students indicate that they have prior knowledge related to the chosen university programme. In general, this prior learning is obtained from study experience, work experience, books, newspapers, magazines, the Internet, TV, radio, film or video. A relation is found between the type of source and the university programme. The hypothesis that students change their self‐assessment after a study period could not be confirmed. Based on these results, it is concluded that self‐assessment in APL might be a suitable tool. Implications for further research are discussed.     

Does learning in clinical context in anatomical sciences improve examination results,learning motivation,or learning orientation?

The preclinical compulsory elective course “Ready for the Operating Room (OR)!?” [in German]: “Fit für den OP (FOP)”] was implemented for students in their second year, who were simultaneously enrolled in the gross anatomy course. The objective of the study was to determine whether the direct practical application of anatomical knowledge within the surgical context of the course led to any improvement in learning motivation, learning orientation, and ultimately examination results in the gross anatomy course, as compared with a control group. Within the scope of five teaching sessions, the students learned surgical hand disinfection, suturing techniques, and the identification of commonly used surgical instruments. In addition, the students attended five surgical demonstrations performed by surgical colleagues on cadavers. Successful learning of these basic skills was then assessed based on an Objectively Structured Practical Examination. Learning motivation and learning orientation in both subgroups was determined using the SELLMO‐ST motivation test and the Approaches and Study Skills Inventory test. While a significant increase in work avoidance was identified in the control group, this was not the case for FOP participants. Similarly, an increase in the “deep approach” to learning, as well as a decrease in the “surface approach,” was able to be documented among the FOP participants following completion of the course. The results suggest that students enrolled in the gross anatomy course, who were simultaneously provided with the opportunity to learn in clinical context, were more likely to be successful at maintaining learning motivation and learning orientation required for the learning process, than students who attended the gross anatomy course alone. Anat Sci Educ. 7: 3–11. © 2013 American Association of Anatomists.